method for obtaining a data grid representative of a dynamically represented scene, device and corresponding program
The method combines static and dynamic models in a Bayesian framework to maintain dynamic object tracking and reduce computational costs, enhancing precision and responsiveness in environment modeling for autonomous agents.
Patent Information
- Application Number
- FR2022013122
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-12-09
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-12-09
AI Technical Summary
Existing methods for environment modeling in autonomous agents, such as robots or autonomous vehicles, face challenges in accurately perceiving dynamic environments due to sensor errors, occlusions, and high computational costs, particularly in maintaining the dynamics of cell occupancy over time.
A method and module for generating a state grid that combines static and dynamic models using a Bayesian framework, where probability distributions from a static grid and dynamic particles are fused to predict and evaluate cell states and speeds, allowing for independent updates and higher update frequencies.
This approach maintains the persistence of dynamic object tracking over time, reduces computational load, and enables smoother filtering with higher update frequencies, improving precision and responsiveness in dynamic environments.
Smart Images

Figure 00000025_0000 
Figure 00000026_0000 
Figure 00000026_0001
Abstract
Description
Title of the invention: method for obtaining a data grid representative of a dynamically represented scene, device and corresponding program.
[0001] Domain
[0002] The invention relates to the field of computer vision devices and autonomous devices in general that interact with their environment, such as a robot or an autonomous vehicle. For these devices, the ability to accurately perceive and model their environment in a relevant way is an essential task. Whether for navigation, collision awareness, intention planning, or mapping, this perception stage is challenging in terms of accuracy, complexity, and uncertainty management.
[0003] More specifically, the invention relates to a generic dynamic state grid tracker and a corresponding method, which filters the states of the cells and deduces the dynamics of the scene captured by one or more sensors through the interaction of a static model, based on a grid, and a dynamic model, based on particles. Prior art
[0004] Despite impressive developments and the ever-increasing use of embedded intelligence in mobile devices, the ability of an autonomous agent to accurately, robustly, and efficiently perceive its environment remains a significant challenge in robotics. The quality of environmental modeling depends not only on sensors but also on interpretation schemes that address sensor errors, occlusions, data contradictions, highly complex parameters, and so on. Probabilistic methods have been developed to formally model uncertainties and prior knowledge within these interpretation schemes.
[0005] When these interpretation schemes are confronted with moving objects, many additional problems arise. A classic way of addressing this problem is to adopt an object-based representation, which leads to tracking multiple objects. Another common approach is the domain of occupancy grids, which works on spatial occupation without higher-level segmentation. This approach has significant advantages, such as a spatially dense model with an accurate representation of free space, which is an important factor in mobile robotics. Furthermore, the critical steps of segmentation and data recognition, necessary in object-based representations, can be avoided. When applied to dynamic environments, it is generally necessary to enrich this representation by estimating velocity information.
[0006] In the case of methods specifically designed for specific sensors, velocity can be derived from sensor measurements. However, in many cases, a more generic approach is required. The Bayesian Occupancy Filter (BOF) is a generic Bayesian framework that updates a dynamic occupancy grid by filtering occupancy and velocity in each cell in parallel. The observation model is incorporated as a snapshot occupancy grid generated by mapping the sensor data onto a grid using a probabilistic sensor model. A Bayesian filtering methodology, based on a prediction-correction loop, is used to filter the distributions in each cell, using neighborhood transition histograms.This particular motion discretization design is practical for modeling cell evolution, but results in high computational costs and aliasing problems.
[0007] To reduce the dimensions of the motion field, prior map knowledge can be used, or more generally, a motion distribution sampling method, with the histogram in each cell being mostly empty. Adaptive sampling methods can significantly reduce the dimensions of the motion representation, since most cells, such as empty or static cells, do not require elaborate representations. In some processes, a variable number of samples per cell is used, depending on the estimated occupancy. The scene is represented by a set of (fictitious) moving particles having undiscretized positions and values, which corrects most aliasing problems.
[0008] Hybrid approaches exist. It is thus possible to implement interactions between grid-based and object-based representations. More recently, using a different mathematical framework, a random finite set approach and filters on belief functions, a suitable system has been proposed for accurately solving the multi-object tracking problem in occupancy grids.
[0009] The hybrid-sampling Bayesian occupancy filter, as presented in FR3022049A1, modifies the BOF structures and analyzes the scene through a static-dynamic duality. The static part is an occupancy grid, and the dynamic field is carried by (fictitious) particles moving within the occupancy grid. The two sections are generated and evaluated jointly, with their distributions on the cells being adjusted. This filter allows for more efficient computations by concentrating velocity calculations on the dynamic components, by proposing A more compact model is achieved by drastically improving the system's accuracy. Furthermore, based on the technique presented in FR3022049A1, it has been proposed to introduce formal states into the filtering process, representing static occupancy, dynamic occupancy, empty areas, and unknown areas. This formalism clarifies the overall presentation of the algorithm (for transitions, initialization, etc.), the unknown state allows for specific processing, and the particles are focused on relevant dynamic areas. While the filtered low-level representation can be used directly for mapping and short-term risk assessment, a fast dynamic occupancy grouping method is directly integrated into the process, enabling simplified analysis.
[0010] However, this evolution, like the previous technique of FR3022049A1, has the drawback of not preserving the elements, particularly dynamic ones, managed within the grid. More specifically, in the probabilistic calculation performed, since each antecedent contribution (that from the occupancy grid and that from the particles) is integrated independently, the "dynamic" occupancy predicted by particles is systematically reduced by the prediction of free or undefined space from the grid. In other words, the free space of the occupancy grid systematically influences the prediction, particularly that from the particles, reducing the weight of the "particle" prediction.
[0011] It is therefore necessary to have a solution which makes it possible in particular to solve this problem. Summary of the invention
[0012] The invention was conceived with these prior art issues in mind. More particularly, the invention relates to a method for obtaining a data grid representative of a dynamically represented scene, called a state grid, said state grid being divided into cells, a method implemented by a computer module for processing data from at least one sensor. According to the invention, this method comprises at least one iteration of the following steps: - updating probability distributions of a static grid characterizing the scene, as a function of at least one static state associated with at least one cell of the static grid, according to a static model; - updating probability distributions of a dynamic scene characterization grid, as a function of at least one particle associated with at least one cell of the dynamic grid, according to a dynamic model; - merging of probability distributions, within a resulting state grid, of the static grid and the dynamic grid, delivering, for each cell, a state prediction and a velocity prediction; - evaluation, for at least one cell of the resulting state grid, of the probability of a final state and an associated speed as a function of predictions and any data received from said at least one sensor.
[0013] According to a particular feature, the probability distribution merging step includes, for a given cell of the resulting grid, a probability calculation step p(SpVp|SgpVgpSppVpp) in which: - Sp: Predicted state of the cell at the current time step; - Vp: Predicted cell velocity at the current time step; - Sg: Predicted state of the cell at the current time step, predicted by the "static" model; - Vg^ Predicted cell speed at the current time step, predicted by the "static" model; - Sp^ Predicted state of the cell at the current time step, predicted by the "dynamic" model; - Vp^ Predicted cell velocity at the current time step, predicted by the "dynamic" model.
[0014] According to a particular feature, the step of evaluating the probability of a final state and an associated velocity includes, for a given cell of the resulting grid, a step of calculating the probability p(svjSpVpSoVo) in which: - S; Cell state at the current time step; - V; Cell speed at the current time step; - Sp: Predicted state of the cell at the current time step; - Vp: Predicted cell velocity at the current time step; - S°: Observed state of the cell at the current time step; - V°: Observed cell velocity at the current time step.
[0015] According to a particular characteristic, the update step includes: - A step of dynamic projection of particles, depending on their speeds; - a step of applying a state change to the static model; - A state transition stage, depending on the model; - A movement step, depending on the movement of the reference frame;
[0016] According to a particular feature, subsequent to the evaluation step, it includes a step of resampling the particles within the dynamic model.
[0017] According to a particular feature, the particle resampling step comprises: - A step of reallocation of particles according to the dynamic probability of each cell of the grid; - A particle generation step, based on previous particle assignments and the contribution of these previously assigned particles to the dynamic model.
[0018] According to a particular feature, the evaluation step implements an observation grid, said observation grid comprising data from said at least one sensor.
[0019] According to another aspect, the invention also relates to a module for obtaining a data grid representative of a dynamically represented scene, called a state grid, said state grid being divided into cells, a method implemented by a computer module for processing data from at least one sensor. This module includes iterative calculation means for performing: - An update of probability distributions of a static scene characterization grid, as a function of at least one static state associated with at least one cell of the static grid, according to a static model; - An update of probability distributions of a dynamic scene characterization grid, as a function of at least one particle associated with at least one cell of the dynamic grid, according to a dynamic model; - A fusion of probability distributions, within a resulting state grid, of the static grid and the dynamic grid, delivering, for each cell, a state prediction and a velocity prediction; - An evaluation, for at least one cell of the resulting state grid, of the probability of a final state and an associated speed based on predictions and any data received from said at least one sensor.
[0020] According to a preferred implementation, the various steps of the processes according to this disclosure are implemented by one or more software or computer programs, comprising software instructions intended to be executed by a data processor of an occupancy grid processing device according to this technique and designed to control the execution of the various steps of the processes, implemented at the level of a processing device, a remote server and / or a distributed system, within the framework of a distribution of the processing to be carried out and determined by a scripted source code or a compiled code.
[0021] Consequently, the present technique also relates to programs that can be executed by a computer or a data processor, these programs including instructions for ordering the execution of the process steps as mentioned above.
[0022] A program may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form.
[0023] The present technique also relates to an information carrier readable by a data processor, and comprising instructions of a program as mentioned above.
[0024] The information medium can be any entity or terminal capable of storing the program. For example, the medium can include a storage means, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or a magnetic recording means, for example a mobile medium (memory card) or a hard disk or an SSD.
[0025] On the other hand, the information medium can be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means. The program according to the present technique can, in particular, be downloaded from an Internet-type network.
[0026] Alternatively, the information carrier may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the process in question.
[0027] According to one example of implementation, the present technique is implemented using software and / or hardware components. In this context, the term "module" in this document may refer to a software component, a hardware component, or a set of hardware and software components.
[0028] A software component corresponds to one or more computer programs, one or more subroutines of a program, or more generally to any element of a program or software capable of implementing a function or set of functions, as described below for the module concerned. Such a software component is executed by a data processor of a physical entity (terminal, server, gateway, set-top box, router, etc.) and is capable of accessing the hardware resources of that physical entity (memory, storage media, communication bus, electronic input / output cards, user interfaces, etc.). Thus, it is possible to decentralize all calculations, in order to parallelize calculations, or even to exchange calculations and parameters between different devices (for example, different vehicles performing calculations for each other).
[0029] Similarly, a hardware component corresponds to any element of a hardware assembly capable of implementing a function or a set of functions, as described below for the module concerned. It may be a programmable hardware component or one with an integrated processor for executing software, for example an integrated circuit, a smart card, a memory card, an electronic card for executing firmware, etc.
[0030] Each component of the system described above naturally implements its own software modules.
[0031] The different embodiments mentioned above can be combined with each other for the implementation of the present technique. Brief description of the figures
[0032] Other objects, features and advantages of the invention will become more apparent upon reading the following description, given by way of simple illustrative, and not limiting, example, in relation to the figures, among which: - [Fig. 1] represents the Bayesian network representing the variable dependencies. The occupation O can be deduced from the state S; - [Fig.2] represents the steps of the prediction model for the grid occupation and particles; - [Fig.3] shows in general the steps implemented in the object process of the invention; - [Fig. 4] represents a simplified physical architecture of a device occupancy grid processing. Detailed description of the invention
[0033] 1. Review of principles
[0034] As a reminder, one objective of the so-called "Bayesian Occupancy Filter" techniques, to which the technique proposed here relates, is to estimate the spatial occupancy and its dynamics of an environment observed using different types of sensors. To do this, the space is divided into spatial cells, and random variables are associated with each cell. The probability distributions of these random variables are estimated recursively and used as the basis for interpreting the scene. The formal probabilistic model used, for example, in FR3022049A1, and whose formalism is integrated, can be considered similar: despite the differences in the decomposition of the joint distribution and the parametric expressions, the main improvements were not in the model equations, but in the static / dynamic distinction, the differentiated representation, and the solution of equations.In later work, a change was introduced in the model and calculations: instead of directly filtering the occupancy, some . Hidden states (s for "occupied by a static object", d for "occupied by a dynamic object", e for "empty", and u for "unknown") (a variable with four possible states, but other states are also possible) representing what is present in a cell were added. The cell occupancy distribution could then be deduced from these hidden states. In addition to presenting a clearer distinction between static and dynamic parts, the main advantage of this modification was to introduce specific treatment of dataless areas, excluding them from the velocity estimation and disabling their temporal persistence. As explained, however, this model is suboptimal in that it does not allow the evolution of predictions from the dynamic model to be preserved over time, resulting in a significant loss of information, particularly for so-called empty cells.
[0035] For this reason, in particular, the inventors have determined a new model, in which the predictions of the static and dynamic models are combined (at the end of the process) to preserve the dynamics of cell occupancy over time and allow better memorization of cell dynamics.
[0036] In this model, as explained below, grid-based (static) and particle-based (dynamic) predictions are separated and combined to handle the different cases for the final prediction (associated with a cell). This preserves the ability to track a dynamic object over time, even when no new observations are available for that object. The grid on which the iterations are performed, and which is referred to in the application, is a "semantic" occupancy grid; it includes states considered broader than simple (binary) occupancy. It can also be called a state grid.
[0037] Thus, in this new model, and its implementation, which is the subject of this document, and in the associated version of the equations, intermediate variables for the predicted states, the predicted velocity, and the observed states, the observed velocity, are added. For the predicted states and velocities, specific variables are defined for each model (variables for prediction from grids alone, variables for prediction from the particle model), the variables for predicted states and velocities being, according to the invention, the result of merging these two predictions, carried out independently of each other.
[0038] These two models are configured to represent the different parts of the captured scene and to optimize the allocation of particles only to the relevant areas, their predictions being merged accordingly with those relating to the (static) grid. The use of hidden variables in the filtering process makes it possible to handle, for example, contradictory state predictions or asynchronous sensors. (for example, when the sensor sends data at frequencies lower than that of the filtering module).
[0039] The formalism used in this description is derived from the Bayesian programming framework. Given a set of random variables, the expression of their joint probability decomposition, the desired distribution can be expressed as shown below. Specific variables are defined for each model (those for grid-based predictions, those for particle-based predictions), the predicted states and velocities being the result of merging these merged predictions with the observed states. Only the occupancy and state dynamics data are explicitly stated hereafter, but each state variable considered can include various semantic information about the cell (type of obstacle, type of navigable space, etc.).
[0040] 1.1 Definition of variables
[0041] In the context of the implementation of this technique, the variables implemented in the model are as follows: - C; Index identifying each 2D cell; - Cg: Index that identifies each antecedent of a cell, used for the "static" model, based on grids; - CP1: Index that identifies each antecedent of a cell, used for the "dynamic" model, based on particles; - S; State of the cell at the current time step. Possible states include, for example, "s" for "occupied by a static object", "d" for "occupied by a dynamic object", "e" for "empty" and "u" for "undefined"; - V; Cell speed at the current time step, in R2; - Sg: State of the cell's antecedent at the previous instant, according to the model " static” (i.e., grid-based); - Vg1: Velocity of the cell preceding the previous instant, according to the "static" model; - Sp1: State of the cell's antecedent at the previous instant, according to the "dynamic" model (i.e., based on particles); - VP1: Velocity of the cell preceding the previous instant, according to the "dynamic" model; - Sg: Predicted state of the cell at the current time step, predicted by the "static" model. Possible states include, for example, "s" for "occupied by a static object", "d" for "occupied by a dynamic object", "e" for "empty" and "u" for "undefined"; - Vg: Predicted cell velocity at the current time step, predicted by the "static" model, in R2; - Sp: Predicted state of the cell at the current time step, predicted by the "dynamic" model. Possible states include, for example, "s" for "occupied by a static object", "d" for "occupied by a dynamic object", "e" for "empty" and "u" for "undefined"; - Vp: Predicted cell velocity at the current time step, predicted by the "dynamic" model, in R2; - Sp: Predicted state of the cell at the current time step. Possible states include, for example, "s" for "occupied by a static object", "d" for "occupied by a dynamic object", "e" for "empty" and "u" for "undefined"; - Vp: Predicted cell velocity at the current time step, in R2; - S°: Observed state of the cell at the current time step. Possible states include for example "o" for "occupied", "e" for "empty" and "u" for "indefinite"; - V°: Observed cell speed at the current time step, in R2. ; - Z; Sensor measurement; - O: Cell occupancy at the current time step. Its possible values are {occ,emp}. Not directly used in inference, defined only later for applications and comparison with other methods.
[0042] The possible states are not limited to those described here. It is possible to implement more precise and numerous states, such as "occupied by a cooperative pedestrian", "occupied by a non-cooperative pedestrian", "aware of the presence of the automated entity (vehicle, robot) / not aware", etc.
[0043] 1.2 Joint probability distribution
[0044] The Bayesian network illustrating the dependence of these variables is presented in [Fig. 1]. As can be seen, unlike the prior technique (notably HBOF), for a given cell C at a given iteration, the dynamics (p) and statics (g) of the scene are evaluated differently. To do this, the static and dynamic evaluations take into account, separately, the previous static and dynamic evaluations (from the previous iteration) before being combined and mixed with the probabilistic information from the sensor(s).
[0045] Overall, the probability distribution is expressed by:
[0046] p ( CCgk^SVSgVgS^ ) =
[0047] PfCg^PCÿvÿCgO.PfSgV^^^
[0048] p( s^Vptep1 ).p( s^SpVp1 ).p(aCg'v^CpX)•
[0049] p ( sMSgV^Vp ).
[0050] P(Z).
[0051] p (S°V°IZC) J
[0052] (1)
[0053] Each expression can be interpreted as follows: - P(C„ 1 ) is the distribution over all possible antecedents of the cell (grid, static). It is chosen to be uniform because the cell is considered to be accessible from all possible antecedents with equal probability; - P(Sg”1Vg”1ICg”1) is the conditional joint distribution on the state and velocity of the antecedents. This distribution is updated at each time step (at each iteration); - P(SgPVgPISg”1Vg”1) is the prediction model, according to the “static” model, based on grids; states and velocity are inseparable, because the definition of the state is directly linked to the velocity: the velocity in the static parts is zero, while the free and undefined parts have no associated velocity; - P(Cp 1 ) is the distribution over all possible antecedents of the cell (particle, dynamic). It is chosen to be uniform because the cell is considered to be accessible from all possible antecedents with equal probability; - PIS,, 1 Vp 'lCp ') is the conditional joint distribution on the state and velocity of the antecedents. This distribution is updated at each time step; - P(SpPVpPISp 1 Vp ') is the prediction model, according to the "dynamic" model, based on particles. States and velocity are inseparable, because the definition of the state is directly linked to the velocity: the velocity in the static parts is zero, while the free and undefined parts have no associated velocity; - P(CCeVgCpVp) is the distribution that indicates whether cell c is reachable from the antecedent [C„ 1 ] with the velocity [Vgp = vgp], and from the antecedent Cp 1 with the velocity [Vpp = vpp]. This distribution is, for example, the product of two Dirac delta functions, one of which has a value of 1, only if vgp ≥ ec and c ≥ 1. This allows the method to be focus solely on the data projected by the two models of cell c, and thus reduce the computational load; - P( S^^Vg^Vp) is the prediction fusion model. The states and the speeds are inseparable, because the definition of the state is directly linked to speed: the speed in static parts is zero, while free and indefinite parts have no associated speed; - P(Z) is the probability of a given observation Z. In this model, its value will not necessarily matter, because it will be the same factor for each potential S and V, Z being provided. - P(S°V°IZC) is the inverse distribution of the sensor model, the one directly calculated and given by the other modules; - P(SVISPVPS°V°) is the probability of a pair SV (state / velocity) according to the predicted and observed states, allowing for the interaction of predictions and observations. A generation matrix, or a more complex formula, can be used.
[0054] 1.3 Symbolic expression of the problem solved
[0055] The goal of the Bayesian filtering process is to estimate the state and velocity relative to the current observation for each cell: P(SVIZC). In the system, the filtering is applied to the hidden states, and the state can then be deduced. Let O be the set of all variables: (2) < 6? pyp PP pp 2
[0056] Using a discretization of the preceding cells and velocities, the filtering equation can be written:
[0057] Next, the distribution can be expressed as:
[0058]
[0059]
[0060]
[0061]
[0062] Therefore, the cell occupancy can be deduced: F(OV^C) = y, (5) 2. Description of an example of model resolution Based on this model and its characterization, a solution method is implemented, as described herein. This method mainly comprises three steps (prediction, fusion-evaluation, and particle resampling), which are explained below. 2.1 Model Implementation As explained in the variable definitions, the system is composed of two interconnected models: a "static" model (grid-based) and a "dynamic" model (particle-based). In addition to semantic data, potential states include, for example: - The state "occupied by a static object" (s) which refers to an immobile occupation, which includes strictly speaking static objects, but also background data (buildings, etc.). - The state "occupied by a dynamic object" (d), which refers to dynamic occupation. - The "empty" state (e), which corresponds to free space. - The "indefinite" state (u), which is a state used to signify lack information, then the lack of confidence in the credibility of other states.
[0063] Other states can of course be defined (such as "occupied by a pedestrian", "occupied by a vehicle", "occupied by a cyclist", etc.), in particular according to the data provided by the sensors.
[0064] The static model consists of a grid of states (occupancy grid), each cell encoding its state distribution. The elements estimated to be static by the filter are represented in this grid of cells. The dynamic model consists of a set of moving particles, each having a real position and velocity, as well as a "weight" corresponding to the probability that the cell to which it (this particle) would belong is occupied by a dynamic object with that state and dynamics. Since only the occupancy and dynamics parts of the states are detailed here, all particles have the same state (dynamics), but more generally they can also have an associated semantic state (car, pedestrian, etc.). More precisely, at each iteration of the filter (at each time step, for example), the two models are used independently for prediction, then merged and evaluated together, and finally segmented again.While the types of states taken into account may differ, some correspond to occupied cells (pedestrian, car, building, etc.) or empty ones (road, sidewalk, etc.), and each can have a specific movement pattern.
[0065] Thus, in the dynamic model, each particle has a position and an actual velocity, as well as a "weight." This weight represents the probability that the cell (to which the particle would belong) is occupied by a dynamic object with that state and dynamics. The elements estimated to be dynamic by the filter are represented in the dynamic model (moving objects). The particles are distributed in a grid of cells, the dimensions of which are identical to those of the static model. The two grids (the two models) are, at each iteration, updated independently of each other and then merged to provide a "final" grid.
[0066] Figure 2 illustrates the general process implemented within the occupancy grid processing device. Starting with a standard occupancy grid, the process comprises: - an update step (A 10) of the static model, consisting of determining predictions of states and velocities for at least one cell of the current occupancy grid as a function of the probabilities of the static model determined at the previous iteration, delivering predictions of the static model [at time step t+1]; - an update step (A20) of the dynamic model, consisting of determining predictions of states and velocities for at least one fictitious particle of the current occupancy grid as a function of the probabilities of the dynamic model determined at the previous iteration, delivering predictions of the dynamic model [at time step t+1]; - a fusion step (A30) of the static model predictions and the dynamic model predictions, according to a predetermined fusion pattern; - an evaluation step (A40) of the fused predictions based on optional observation data from at least one sensor, for example a probabilistic sensor.
[0067] One advantage of this solution is that it is possible to update the final predictions without data from sensors: the processing of the grids can be carried out independently of the reception of data from the sensor. For example, the module implementing the technique of the invention can perform "asynchronous" updates, doing so at a frequency of 100 Hz; the sensors can transmit data at a frequency of 20 Hz, and numerous iterations of the proposed method take place without "observation" (a complete grid of "unknown" states is used in place of the data from the sensor(s)). The filtered state then relies solely on the prediction made from the previous states, separating the particle prediction from the "grid" prediction, as explained above.
[0068] 2.2 Predictions
[0069] As previously stated, the two models are projected separately, with the final prediction resulting from the merging of these predictions. In other words, two predictions are made (updating the static model and updating the dynamic model), which are predicted separately and then merged. For each model, the update steps leading to the prediction consist of: - a dynamic projection: depending on the time elapsed since the previous iteration, the particles are projected according to their velocities, and the Gaussian noise in position and velocity. The occupancy grids are defined here as static. - A state transition: Each model (static, dynamic) is associated with transition matrices, enabling state transitions. In practice, these matrices primarily consist of a slow transition to the "undefined" state, representing the loss of confidence in the estimated data over time. Occupied states (static, dynamic) are defined to be more consistent over time than free states. - a shift in the reference frame: as the system is intended to be mounted on moving (and therefore mobile) vehicles, the models must be moved according to the ego-motion. The model representation simplifies this otherwise complex operation, the operation consisting of the Transformation of particle vectors (translation and rotation for positions, rotation for velocity) and interpolation of grids. Grid areas corresponding to newly discovered regions are initialized to the undefined state (u), while deleted areas can either be forgotten (in many mobile robot applications, this data storage is not relevant) or used to generate a map based on the static part.
[0070] Figure 3 schematically illustrates the prediction process: - (i) Models at the previous time step. Both (grids and particles) are represented in superposition. - (ii) Dynamic projection (for grids and particles, according to motion models) and state transitions (according to state transition models). - (iii) Expression of the data in the new reference frame, by interpolation of the grids and translation and rotation of the particles. - (iv) Models predicted at the current time step (the two grids are shown in superposition).
[0071] The potential values of these predictions are pre-calculated (and pre-sampled), so that the values:
[0072] P(Cg-*) P(Sg-1Vg-*ICg-1) P(SgpVgpISg-1Vg-1) and P(Cp-') PtS / V / lC / ) P(SppVppISp-1Vp-1 )
[0073] are calculated independently (and these values are directly associated with the grid cells and the particles concerned).
[0074] 2.3 Fusion of predictions and evaluations
[0075] The updated distributions on the states are evaluated in parallel in each cell of the current grid. All predicted particles are reorganized according to their position in the grid, so that only the particles and data from the relevant cells are transmitted for computation at the current grid cell level (according to P(CIC„1 VgPCp1 Vpp)). In other words, only cells whose P(CIC„1 VgpCp1 Vpp) exceeds a certain threshold are transmitted for merging and evaluation a. Prediction fusion: According to the fusion model defined by P(SpVpISgp VgpSpPVpp), each static and dynamic prediction sample generates final predictions. The fusion model can be precisely designed (and could even be learned), but the main objective is to define it such that if a model predicts a dynamic state, the fusion results in a dynamic state. Otherwise, if a model predicts a static state, the fusion results in a static state. Otherwise, if a model predicts a free state, the fusion results in... to a free state. And an undefined state (u) is generated only if both models (static and dynamic) each predict an undefined state. In this way, for example, the prediction of a moving object arriving in previously free (f) or undefined (u) cells is not weakened, which guarantees the consistency of the information (over time) and thus allows the problems encountered to be solved. b. Evaluation: In a given cell "c", these final predictions are compared to the observation P(S°V°IZC). The observation model used is based on the classic probabilistic sensor model, taking into account the relevance of the sensor data (sensor confidence, impact location, impact height, vertical distribution, etc.). The observation model can be defined for any type of sensor; the distributions can be merged using probability merging rules, allowing for the integration of heterogeneous data and generic processing. The state definition used in this model can also differ from the filtered one.
[0076] For example, the observation model can be based on the classical probabilistic sensor model, with the addition of the indefinite state evaluation (u), which correlates with the relevance of the sensor data. For example, in the case of laser data, the indefinite model obtains low values before and around the impact, and high values further away (when no information is available).
[0077] To generate the final state estimate, the generative distribution P(SVISPVPS°V°) is used, defining the probability of a final state and the velocity as a function of predictions and observations. This is given in the form of a generative matrix for the states (whose parameters can or must be learned, depending on the operational implementation conditions), allowing, for example, the rapid appearance of objects in cells previously seen as free, while ensuring a slow disappearance of objects in cells now seen as free or undefined, or the absence of impact on the final state estimate of an undefined observation (outside the scope of the natural erosion of predictions over time).
[0078] Following the fusion of predictions and evaluations with regard to the data provided by the sensors, for this time step, we have a current grid in which, for each (relevant) cell, the probability of a final state and the speed as a function of the predictions and observations is obtained.
[0079] 2.4 Resampling of particles
[0080] Once the state distributions on the cells are correctly estimated (calculated), the next task, in one example embodiment, is to resampling of the particles, including for example the following two steps: a. Particle reallocation: depending on the dynamic probability of each cell of the grid, and other criteria (position, difference between predicted and observed states, velocity distribution, etc.), the overall number of particles is reallocated to the cells concerned. b. Particle generation: During the evaluation of state distributions, the weight of each existing particle is updated and normalized based on its relative contribution to the dynamics. In each cell to which particles have been associated, particles are then drawn from the existing particle distribution and an initialization distribution, according to the proportion of "dynamic appearance" (difference between predicted and observed states, velocity distribution, etc.). Finally, the cellular dynamic coefficient (i.e., that of the cell) is uniformly distributed among the particles (of that cell).
[0081] Unique markers are associated with each particle drawn from the initialization distribution and propagated to particles extracted from those already existing, allowing for rapid and ultralight pre-segmentation of objects (or at least a coherent cluster in the occupancy and dynamic spaces). This resampling can be carried out by other means. It is not mandatory to limit oneself to this method.
[0082] 3. Other features, improvements and advantages
[0083] According to the invention, it is thus possible to greatly improve the quality of predictions made by previous systems, ensuring that the predictions last longer over time. Furthermore, the invention also offers the following advantages: - Specifiable filter sensitivity: A major difference between the presented technique and the previously mentioned filters is the introduction of intermediate state variables (S°V°) to evaluate the model states, instead of direct evaluation from the data. These "observed states" can have a different number of dimensions than the estimated state, and the relationship between each potential observed and estimated state can be designed, explicitly defining the filtering sensitivity on a specific data input without affecting the temporal consistency of the filtered states. Untangling these two filter components resolves a practical problem with previous approaches, in which the rates of state appearance and disappearance were directly related. - Independent update frequency: As previously mentioned, this frequency, in conjunction with the use of a specific representation for "data deficiency" in observation states, allows the filter update frequencies to be decoupled from those of the sensors. Furthermore, in correlation with the GPU implementation, it enables update frequencies higher than the sensor frequencies, thus providing a dynamic scene estimation even when no sensor data is received. - Adaptable prediction pattern: Another major difference is the use of two distinct models for prediction: a grid-based prediction (SgpVgp) and a particle-based prediction (SPPVPP). Both are propagated independently at each iteration but merged (SPVP) and regenerated from the estimated states. This strict separation (compared to previous conditional sampling) and the configurable merging resolve another significant problem with the previous method, in which it was not possible to adapt the persistence of a state in a cell based on other predictions. A typical example of such a problem is when a dynamic sample moves over empty areas. Without new observations, the persistence of previously observed empty cells reduces the probability of the moving object being occupied, even to the point of disappearance, even without a defined temporal state transition (dynamic transition pattern).Thanks to the method of the invention, when a cell predicts a "free" cell from the first model, and a "dynamic" cell (from the projection of particles) from the second, the predicted state is correctly defined as "dynamic".
[0084] Thus, by implementing the proposed method, it is possible to obtain a filter update frequency of 100 Hz, exceeding the sensor frequencies. This higher update frequency allows for smoother filtering, more precise data integration, and a well-defined temporal confidence level, while also improving the operational reliability of the process (smooth continuation of model propagation in the event of sensor failure). Distinguishing between two models and cross-prediction ultimately resolves important problems, such as the undesirable rapid dilution of dynamic occupancy in areas previously considered free or undefined. The addition of observation states leads to greater responsiveness in the detection of specific states.Although the results are convincing, further work could focus on the quantitative validation of the approach, which remains a major challenge in the field. filtering of the dynamic occupancy grid, which lacks clear shared evaluation criteria, metrics and a reference database.
[0085] 4. Processing device
[0086] A simplified architecture of an occupancy grid processing device is presented in relation to [Fig. 4], implementing all or part of the method as previously described. Such an occupancy grid processing device comprises a memory 41, a processing unit 42 equipped, for example, with a microprocessor, and controlled by the computer program 43, implementing the method according to the invention. In at least one embodiment, the invention is implemented in the form of an application installed on a device installed within a vehicle or an autonomous agent (such as a robot). Such an occupancy grid processing device comprises, for example, all or part of the following means: - means of updating probability distributions of a static scene characterization grid, as a function of at least one static state associated with at least one cell of the static grid, according to a static model; - means of updating probability distributions of a dynamic scene characterization grid, as a function of at least one particle associated with at least one cell of the dynamic grid, according to a dynamic model; - means of merging probability distributions, within a resulting occupancy grid, of the static grid and the dynamic grid, delivering, for each cell, a state prediction and a velocity prediction; - means of evaluating, for at least one cell of the resulting occupancy grid, the probability of a final state and an associated speed based on predictions and observations.
[0087] These means take the form of a specific software application, or of dedicated hardware components built for the purpose of carrying out these functions, such as an ASIC.
Claims
1. Claims Method for obtaining a grid, called a state grid, of data representative of a dynamically represented scene intended to be used by an autonomous device interacting with its environment, captured by at least one sensor, said state grid being divided into cells, each cell being associated with a state belonging to the group comprising at least: - a state representative of an occupation by a static object; - a state representative of an occupation by a dynamic object; - an empty state; - an indefinite state; said method being implemented by a computer module for processing data from said at least one sensor, the method comprising at least one iteration of the following steps: - updating (A10) of probability distributions of a static grid for characterizing the scene, as a function of at least one static state associated with at least one cell of the static grid, according to a static model; - updating (A20) of probability distributions of a dynamic grid for characterizing the scene, as a function of at least one particle associated with at least one cell of the dynamic grid, according to a dynamic model; - merging (A30) the probability distributions, within a resulting state grid, of the static grid and the dynamic grid, delivering, for each cell, a state prediction and a speed prediction, said merging comprising the calculation of the probability P(SpVpISgPVgpSpPVpp) with: - SP: Predicted state of the cell at the current time step; - Vp: Predicted velocity of the cell at the current time step; - S: Predicted state of the cell at the current time step, predicted by the “static” model; - Vp: Predicted velocity of the cell at the current time step, predicted by the "static" model; - Sp: Predicted state of the cell at the current time step, predicted by the “dynamic” model; - Vp: Predicted velocity of the cell at the current time step, predicted by the "dynamic" model; - evaluation (A40), for at least one cell of the resulting state grid, at the current time step, of the probability of a final state and an associated velocity, as a function of said predictions of state and velocity at the current time step delivered by said merging step, and of observed state and velocity data determined from the data received from said at least one sensor at the current time step, or, in the absence of such data received from said at least one sensor at said current time step, positioned at a value representative of an undefined state and velocity, said evaluation comprising the calculation of the probability P(SVISPVPS°V°) with: - S; State of the cell at the current time step; - V; Velocity of the cell at the current time step; - Sp: Predicted state of the cell at the current time step;- Vp: Predicted velocity of the cell at the current time step; - S°: Observed state of the cell at the current time step; - V°: Observed velocity of the cell at the current time step.;
2. Method according to claim 1, characterized in that the updating step (A10, A20) comprises: - A step of dynamic projection of particles, according to their speeds; - a step of applying a state modification to the static model; - A step of state transition, according to the model; - A displacement step, according to the displacement of the reference frame.
3. Method according to claim 1, characterized in that, after the evaluation step (A40), it comprises a step of resampling the particles within the dynamic model.
4. Method according to claim 3, characterized in that the step of resampling the particles comprises: - A step of reallocation of particles according to the dynamic probability of each cell of the grid; - A particle generation step, based on previous particle assignments and the contribution of these previously assigned particles to the dynamic model.
5. Method according to claim 1, characterized in that the evaluation step (A40) implements an observation grid, said observation grid comprising data from said at least one sensor.
6. Module for obtaining a grid, called a state grid, of data representative of a dynamically represented scene intended to be used by an autonomous device interacting with its environment, captured by at least one sensor, said state grid being divided into cells, each cell being associated with a state belonging to the group comprising at least: - a state representative of an occupation by a static object; - a state representative of an occupation by a dynamic object; - an empty state; - an indefinite state; said module taking the form of a computer module for processing data from said at least one sensor, the module comprising iterative calculation means for carrying out: - An update (A10) of probability distributions of a static grid for characterizing the scene, as a function of at least one static state associated with at least one cell of the static grid, according to a static model; - An update (A20) of probability distributions of a dynamic grid for characterizing the scene, as a function of at least one particle associated with at least one cell of the dynamic grid, according to a dynamic model; - A fusion (A30) of the probability distributions, within a resulting state grid, of the static grid and the dynamic grid, delivering, for each cell, a state prediction and a speed prediction, said fusion including the calculation of the probability P(SpVpISgpVgpSppVpp) with: - Sp: Predicted state of the cell at the current time step; - Vp: Predicted speed of the cell at the current time step; - Sp: Predicted state of the cell at the current time step, predicted by the “static” model; - Vp: Predicted speed of the cell at the current time step, predicted by the “static” model; - Sp^ Predicted state of the cell at the current time step, predicted by the “dynamic” model;- An evaluation (A40), for at least one cell of the resulting state grid, at the current time step, of the probability of a final state and an associated speed, as a function of said predictions of state and speed at the current time step delivered by said fusion, and of observed state and speed data determined from the data received from said at least one sensor at the current time step, or, in the absence of such data received from said at least one sensor at said current time step, positioned at a value representative of an undefined state and speed, said evaluation comprising the calculation of the probability P(SVISPVPS°V°) with: - S; State of the cell at the current time step; - V; Speed of the cell at the current time step; - Sp: Predicted state of the cell at the current time step; - Vp: Predicted speed of the cell at the current time step;- S°: Observed state of the cell at the current time step; - V°: Observed speed of the cell at the current time step.;
7. A computer program comprising program code instructions for performing the steps of a method according to any one of claims 1 to 5 when said program is executed on a processor.
