Method for acquiring data grid containing data representative of dynamically represented scene, and corresponding device and program

Abstract

The invention relates to a method for acquiring a data grid, referred to as a state grid, containing data representative of a dynamically represented scene, said state grid being divided into units, the method being implemented by a computer module for processing data originating from at least one sensor, the method comprises at least one iteration of the following steps:-updating (A10) a probability distribution of the static mesh characterizing the scene based on at least one static state associated with at least one cell of the static mesh according to a static model; -updating (A20) a probability distribution of the dynamic grid characterizing the scene based on at least one particle associated with at least one cell of the dynamic grid according to the dynamic model; -fusing (A30) the probability distributions of the static and dynamic grids within the resulting state grid, thereby providing a state prediction and a speed prediction for each cell; -evaluating (A40), for at least one cell of the resulting state grid, a probability of a final state and a probability of an associated speed based on the prediction and any data received from the at least one sensor.

Description

Technical Field

[0001] The present invention generally relates to the field of computer vision devices and autonomous machines that interact with their environment, such as robots or autonomous vehicles. For these machines, being able to accurately perceive and model their environment in an appropriate form is a crucial task. Whether it is navigation, collision perception, intention planning, or mapping, this perception stage faces challenges in terms of accuracy, complexity, and uncertainty management.

[0002] More specifically, the present invention relates to a general dynamic state grid tracker and its corresponding method that filters the state of cells and infers the dynamics of a scene captured by one or more sensors through the interaction of a static grid-based model and a dynamic particle-based model. Background Art

[0003] Despite the remarkable progress in the on-board intelligent technology of mobile machines and the increasing level of intelligence, the ability of autonomous agents to accurately, robustly, and efficiently perceive their environment remains a major challenge in the field of robotics. The quality of environmental modeling depends not only on sensors but also on the interpretation mode for handling sensor errors, occlusions, data contradictions, highly complex parameters, etc. Probabilistic methods have been developed to formally model the uncertainty and prior knowledge in these interpretation modes.

[0004] When these interpretation modes face moving objects, many additional problems arise. The classical method to solve this problem is to adopt an object-based representation, which requires tracking multiple objects. Another common method is the occupancy grid field, which can handle space occupancy without the need for higher-level segmentation. This method has significant advantages, such as a spatially dense model that correctly represents free space, which is important data in mobile robotics. In addition, the key steps of data segmentation and data recognition required in the object-based representation can be avoided. When applied to a dynamic environment, it is usually necessary to enrich this representation by estimating velocity information.

[0005] For methods specifically designed for a particular sensor, the speed can be derived from sensor measurements. However, in many cases, a more general method is required. The Bayesian occupancy filter (BOF) is a general Bayesian framework that updates a dynamic occupancy grid by filtering occupancy and speed in each cell in parallel. The observation model is incorporated in the form of an instantaneous occupancy grid, which is generated by mapping sensor data onto the grid with the help of a probabilistic sensor model. A Bayesian filtering method based on a prediction-correction cycle is used to filter the distribution in each cell using a neighborhood transition histogram. This special design for discrete motion is practical for simulating the evolution of cells, but requires high computational costs and has an aliasing problem.

[0006] To reduce the dimensionality of the motion field, pre-existing mapping data can be used, or more generally, a motion distribution sampling method, where the histograms in each cell are mostly empty. Adaptive sampling methods can significantly reduce the dimensionality of the motion representation because most cells (such as empty or static cells) do not require a complex representation. In some processes, the variable number of samples for each cell is used as a basis for the estimated occupancy. The scene is represented by a set of (virtual) particles in motion, which have non-discretized positions and values, which can correct most aliasing problems.

[0007] There are hybrid methods. Thus, an interaction between grid-based representation and object-based representation can be achieved. Recently, systems aimed at accurately solving the problem of tracking multiple objects in an occupancy grid have been proposed by using different mathematical frameworks, namely methods using random finite sets and belief function filters.

[0008] The hybrid sampling Bayesian occupancy filter disclosed in FR3022049A1 modifies the BOF structure and analyzes the scene through static-dynamic duality. The static part is the occupancy grid, and the dynamic field is conveyed by (virtual) particles in motion in the occupancy grid. These two parts are jointly generated and evaluated, where their distributions over cells are adjusted. The filter provides a more compact model by concentrating the speed calculation on the dynamic component and brings a significant improvement in system accuracy, achieving more efficient computation. In addition, based on the method disclosed in FR3022049A1, formal states are also introduced into the filtering process, which respectively represent static occupancy, dynamic occupancy, blank areas, and unknown areas. This formalization mechanism clarifies the overall expression of the algorithm (such as transformation, initialization, etc.), and the unknown states enable specific processing and focusing of particles on relevant dynamic regions. Although the filtered low-level representation can be directly used for mapping, short-term risk assessment (fast dynamic occupancy clustering method) is directly integrated into the process, enabling simplified analysis.

[0009] However, this development method has the same drawback as the method of the previous FR3022049A1 in that it cannot provide retention of the elements (especially dynamic elements) managed in the grid. More specifically, in the probability calculation performed, since each prior contribution (the contribution from the occupancy grid and the contribution from the particles) is independently integrated, the prediction of free space or undefined space based on the grid systematically reduces the "dynamic" occupancy predicted by the particles. In other words, the free space of the occupancy grid is systematically incorporated into the prediction, especially in the particle-based prediction, which reduces the weight of the "particle" prediction.

[0010] Therefore, it is necessary to provide a solution that can solve this problem. Summary of the Invention

[0011] The present invention is developed precisely for these problems in the related art. More specifically, the present invention relates to a method for obtaining a data grid (referred to as a state grid) representing a dynamic representation of a scene, where the state grid is divided into cells, and the method is implemented by a computer module for processing data from at least one sensor. According to the present invention, the method includes at least one iteration of the following steps:

[0012] - Updating the probability distribution of the static representation grid characterizing the scene based on at least one static state associated with at least one cell of the static grid according to a static model;

[0013] - Updating the probability distribution of the dynamic representation grid characterizing the scene based on at least one particle associated with at least one cell of the dynamic grid according to a dynamic model;

[0014] - Fusing the probability distributions of the static grid and the dynamic grid in the resulting state grid to provide a state prediction and a speed prediction for each cell;

[0015] - For at least one cell of the resulting state grid, evaluating the probability of the final state and the probability of the associated speed based on the prediction and any data received from the at least one sensor.

[0016] According to a specific feature, the step of fusing the probability distributions includes: for a given cell of the resulting grid, the step of calculating the probability P(SpVp|SgpVgpSppVpp), where:

[0017] - S p : The predicted state of the cell at the current time step;

[0018] - V p : The predicted speed of the cell at the current time step;

[0019] - The predicted state of the cell at the current time step predicted by the "static" model;

[0020] - The predicted velocity of the cell at the current time step predicted by the "static" model;

[0021] - The predicted state of the cell at the current time step predicted by the "dynamic" model;

[0022] - The predicted velocity of the cell at the current time step predicted by the "dynamic" model.

[0023] The steps of evaluating the probability of the final state and the probability of the associated velocity according to specific features include: for a given cell of the result grid, the step of calculating the probability P(SV|SpVpSoVo), where:

[0024] - S: the state of the cell at the current time step;

[0025] - V: the velocity of the cell at the current time step;

[0026] - S p : the predicted state of the cell at the current time step;

[0027] - V p : the predicted velocity of the cell at the current time step;

[0028] - S O : the state of the cell observed at the current time step;

[0029] - V O : the velocity of the cell observed at the current time step.

[0030] The steps of updating according to specific features include:

[0031] - The step of dynamically projecting the particle according to the velocity of the particle;

[0032] - The step of applying state modification to the static model;

[0033] - The step of performing state transition according to the model;

[0034] - The step of shifting according to the shift of the reference frame.

[0035] According to specific features, the method includes: after the evaluation step, the step of resampling the particles in the dynamic model.

[0036] According to specific features, the steps of resampling the particles include:

[0037] - A step of reallocating particles according to the dynamic probability of each grid cell;

[0038] - A step of generating particles according to the previous allocations of the particles and the contributions of these previously allocated particles to the dynamic model.

[0039] According to a specific feature, the evaluation step uses an observation grid, wherein the observation grid includes data from the at least one sensor.

[0040] According to another aspect, the present invention also relates to a module for obtaining a data grid (referred to as a state grid) representing a dynamic representation of a scene, wherein the state grid is divided into cells, and wherein the method is implemented by a computer module for processing data from at least one sensor. The module includes iterative computing means for performing the following steps:

[0041] - Updating the probability distribution of a static representation grid characterizing the scene based on at least one static state associated with at least one cell of a static grid according to a static model;

[0042] - Updating the probability distribution of a dynamic representation grid characterizing the scene based on at least one particle associated with at least one cell of a dynamic grid according to a dynamic model;

[0043] - Fusing the probability distributions of the static grid and the dynamic grid in a resulting state grid so as to provide a state prediction and a velocity prediction for each cell;

[0044] - For at least one cell of the resulting state grid, evaluating the probability of a final state and the probability of an associated velocity based on the prediction and any data received from the at least one sensor.

[0045] According to a preferred embodiment, the different steps of the method according to the present disclosure are implemented by one or more software programs or computer programs, which software programs or computer programs include software instructions intended to be executed by a data processor of a device for processing an occupancy grid according to the present invention, and these software instructions are intended to control the execution of the different steps of the method, which steps are implemented at the level of a processing device, a remote server, and / or a distributed system, within a distribution framework of processing operations to be performed, and are determined by script source code or compiled code.

[0046] Therefore, the present invention also relates to a program capable of being executed by a computer or by a data processor, wherein these programs include instructions for controlling the execution of steps of methods such as those mentioned above.

[0047] The program can be in any programming language and can be in the form of source code, object code, or intermediate code between source code and object code (such as a partially compiled form) or any other desired form.

[0048] The invention also relates to an information carrier readable by a data processor and containing instructions of a program such as those mentioned above.

[0049] The information carrier can be any entity or terminal capable of storing the program. For example, the carrier can include a storage medium such as ROM (e.g., CD ROM or microelectronic circuit ROM), or a magnetic recording medium (e.g., a removable carrier (memory card) or a hard disk or SSD).

[0050] In addition, the information carrier can be a transmissible carrier such as an electrical signal or an optical signal, which can be transmitted via a cable, an optical fiber, radio, or other means. In particular, the program according to the invention can be downloaded from a network of the Internet type.

[0051] Alternatively, the information carrier can be an integrated circuit in which the program is incorporated, where the circuit is adapted to execute or for executing the method under discussion.

[0052] According to an example embodiment, the invention is implemented by means of software and / or hardware components. In this sense, the term "module" in this document can correspond to a software component, a hardware component, or a combination of a hardware and a software component.

[0053] A software component corresponds to one or more computer programs, one or more subprograms of a program, or more generally, to any element in a program or software program that can implement one or a set of functions (the specific functions are described according to the relevant module below). Such a software component is executed by a data processor of a physical entity (terminal, server, gateway, set-top box, router, etc.) and may access the hardware resources of the physical entity (memory, recording medium, communication bus, electronic input / output card, user interface, etc.). Therefore, the computing can be decentralized so that the computing is executed in a parallel manner, or even so that the computing and parameters can be exchanged between different devices (e.g., different vehicles that perform computing with each other).

[0054] Similarly, a hardware component corresponds to any element in a hardware group that can implement one or a set of functions, the specific functions are described according to the relevant module below. The hardware component can be a programmable hardware component, or can also be a hardware component with an integrated processor for executing software, such as an integrated circuit, a chip card, a memory card, an electronic card for executing firmware, etc.

[0055] Each component of the system described above naturally implements its own software module.

[0056] The various embodiments mentioned above can be combined with each other to implement the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Other objects, features, and advantages of the present invention will become clearer by reading the following description of the drawings (given only as a simple illustrative and non - limiting example), in which:

[0058] Figure 1 shows a Bayesian network representing variable dependencies. Occupancy O can be derived from state S;

[0059] Figure 2 shows the steps of a prediction model for occupancy grids and particles;

[0060] Figure 3 outlines the steps implemented by the method according to the present invention;

[0061] Figure 4 shows a simplified physical architecture of an occupancy grid processing device. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] 1. Detailed Principle

[0063] ​​​​As a reminder, the present invention proposed herein relates to the object known as the "Bayesian occupancy filter" technique which aims to estimate the spatial occupancy of an environment and its dynamics observed using different types of sensors. To this end, the space is divided into spatial cells and random variables are associated with each cell. The probability distributions of these random variables are estimated recursively and serve as a basis for interpreting the scene. For example, the formal probability model used in FR3022049A1 (the formalization method of which has been incorporated) can be considered similar: although there are differences in the decomposition of the joint distribution and the parameter expressions, the main improvement does not lie in the equations of the model, but in the static / dynamic distinction, the differential representation, and the solution of the equations. In subsequent work, changes were introduced in the model and the calculations: instead of directly filtering the occupancy, hidden states were added to represent what is present in the cell (s represents "occupied by a static object", d represents "occupied by a dynamic object", e represents "empty", and u represents "unknown") (a variable with four possible states, but there could also be other states). Then, the occupancy distribution of the cell can be inferred based on these hidden states. In addition to providing a clearer distinction between the static part and the dynamic part, the main purpose of this modification was to introduce a specific treatment of areas without data, exclude them from speed estimation, and deactivate their temporal persistence. However, as explained, the model is not optimal because it does not allow the retention of the evolution of the dynamic model predictions over time, resulting in a significant loss of information, especially for cells known as empty cells.

[0064] Particularly for this reason, the inventors have determined a new model in which the predictions of the static model and the dynamic model are combined (after completion) in order to retain the occupancy dynamics of the cells over time and allow for a better memory of the dynamics of the cells.

[0065] In this model, as explained hereinafter, the grid-based (static) prediction and the particle-based (dynamic) prediction are kept separate and then combined to manage the different cases for the final prediction (associated with the cell). Thus, the ability to track dynamic objects over time is retained even when no new observation data is available for the dynamic objects. The grid involved and iterated in this application is a "semantic" occupancy grid, which includes states considered to be broader than simple (binary) occupancy. It can also be called a state grid.

[0066] Thus, in the new model and its implementation, which are the subject of the present application, and in the versions of the equations associated therewith, intermediate variables for predicting the state, predicting the velocity, observing the state, and observing the velocity are added. Specific variables are defined for the predicted state and velocity of each model (prediction variables based only on the grid, prediction variables based on the particle model), where according to the invention, the predicted state and velocity variables are the result of fusing these two predictions, which are generated independently of each other.

[0067] The two models are configured to: characterize different parts of the captured scene; and optimize the allocation of particles only to relevant regions, where their predictions are then fused with the predictions associated with the (static) grid. The use of hidden variables during the filtering process can handle, for example, conflicting state predictions or asynchronous sensors (e.g., when the sensors send data at a lower frequency than the filtering module).

[0068] The formalism used in this description is derived from the Bayesian planning framework. Given a set of random variables and an expression for the joint probability decomposition thereof, the expected distribution can be represented as follows. Each model defines specific variables (prediction variables based on the grid, prediction variables based on particles), where the predicted state and velocity are the result of fusing these fused predictions with the observed state. Only the occupancy data and dynamic data of the state are explained below, but each considered state variable can include various semantic information about the cell (obstacle type, navigable space type, etc.).

[0069] 1.1 Definition of Variables

[0070] In the context of the implementation of the present invention, the variables implemented in the model are as follows:

[0071] - C: The index identifying each 2D cell;

[0072] - The index identifying each antecedent cell of the cell for the "static" grid-based model;

[0073] - The index identifying each antecedent cell of the cell for the "dynamic" particle-based model;

[0074] - S: The state of the cell at the current time step. Possible states include: for example, "s" means "occupied by a static object", "d" means "occupied by a dynamic object", "e" means "empty", and "u" means "undefined".

[0075] -- V: The velocity of the cell at the current time step, in R 2 as the unit;

[0076] - The state of the provenance cell of the cell at the previous moment according to the "static" (i.e., grid-based) model;

[0077] - The velocity of the provenance cell at the previous moment according to the "static" model;

[0078] - The state of the provenance cell of the cell at the previous moment according to the "dynamic" (i.e., particle-based) model;

[0079] - The velocity of the provenance cell at the previous moment according to the "dynamic" model;

[0080] - The predicted state of the cell at the current time step predicted by the "static" model. Possible states include, for example, "s" indicating "occupied by a static object", "d" indicating "occupied by a dynamic object", "e" indicating "empty", and "u" indicating "undefined".

[0081] - The predicted velocity of the cell at the current time step predicted by the "static" model, in R 2 as the unit;

[0082] - The predicted state of the cell at the current time step predicted by the "dynamic" model. Possible states include, for example, "s" indicating "occupied by a static object", "d" indicating "occupied by a dynamic object", "e" indicating "empty", and "u" indicating "undefined".

[0083] - The predicted velocity of the cell at the current time step predicted by the "dynamic" model, in R 2 as the unit;

[0084] -S p : The predicted state of the cell at the current time step. Possible states include, for example, "s" indicating "occupied by a static object", "d" indicating "occupied by a dynamic object",

[0085] "e" indicating "empty", and "u" indicating "undefined".

[0086] -V p : The predicted velocity of the cell at the current time step, in R 2 as the unit;

[0087] -S O : The state of the cell observed at the current time step. Possible states include, for example, "o" indicating "occupied", "e" indicating "empty", and "u" indicating "undefined".

[0088] -V O : The velocity of the cell observed at the current time step, in R 2 units;

[0089] -Z: Sensor measurement;

[0090] --O: The occupancy of the cell at the current time step. Its possible values are {occ, emp}. Not directly used for inference, only defined for subsequent applications and comparison with other methods.

[0091] The possible states are not limited to those listed here. More precise and additional states can be implemented, such as, for example, "occupied by a cooperative pedestrian", "occupied by a non - cooperative pedestrian", "aware of the presence of an automated entity (vehicle, robot) / unaware of the presence of an automated entity (vehicle, robot)", etc.

[0092] 1.2 Joint Probability Distribution

[0093] Figure 1 A Bayesian network is shown that illustrates the dependencies of these variables. It can be seen that, different from the related art (notably HBOF), for a given cell C, at a given iteration, the dynamics (p) and the static state (g) of the scene are evaluated in different ways. For this purpose, the static evaluation and the dynamic evaluation respectively consider the previous static evaluation and dynamic evaluation (from the previous iteration), and then combine and mix with the probability information from one or more sensors.

[0094] Overall, the probability distribution is represented as:

[0095]

[0096]

[0097] Each expression can be interpreted as follows:

[0098] -P(C g -1 ) is the distribution of all possible source cells of the (grid, static) cell. A uniform distribution is chosen because the cell is considered to be accessible with equal probability from all possible source cells;

[0099] -P(S g -1 V g -1 |C g -1 ) is the conditional joint distribution of the state of the source cell and the velocity of the source cell. This distribution is updated at each time step (each iteration);

[0100] -P(S g p V g p |S g -1 V g -1 ) is a prediction model based on a "static" grid-based model; state and velocity are inseparable because the definition of the state is directly related to the velocity: the velocity of the static part is zero, while the free part and the undefined part have no associated velocity;

[0101] -P(C p -1 ) is the distribution of all possible source units of the (particle, dynamic) cell. A uniform distribution is chosen because the cell is considered to be accessible from all possible source units with equal probability;

[0102] -P(S p -1 V p -1 |C p -1 ) is the conditional joint distribution of the state of the source unit and the velocity of the source unit. This distribution is updated at each time step;

[0103] -P(S p p V p p |S p -1 V p -1 ) is a prediction model based on a "dynamic" particle-based model. State and velocity are inseparable because the definition of the state is directly related to the velocity: the velocity of the static part is zero, while the free part and the undefined part have no associated velocity;

[0104] - indicates whether cell c can be reached from source unit [C g -1 at velocity [V g p = v g p , and the distribution of reaching from source unit cp -1 at velocity [Vp p = v p p . For example, this distribution is the product of two Dirac distributions, where the value of one distribution is equal to 1 only when and This enables the method to focus only on the data projected by the two models of unit c, and thus reduces the computational amount;

[0105] - is the prediction fusion model. State and velocity are inseparable because the definition of state is directly related to velocity: the velocity of the static part is zero, while the free part and the undefined part have no related velocity;

[0106] -P(Z) is the probability of the given observation Z. In this model, its value is not necessarily important because for each potential S, V, and Z provided, it will be the same factor.

[0107] -P(S o V o |ZC) is the inverse distribution of the sensor model, which is directly calculated and given by other modules;

[0108] -P(SV|S p V p S o V o ) is the probability of the SV (state / velocity) pair obtained from the predicted state and the observed state, which can realize the interaction between prediction and observation. A generation matrix or a more complex formula can be used.

[0109] 1.3 Symbolic Representation of the Problem to be Solved

[0110] The purpose of the Bayesian filtering process is to estimate the state and velocity for each unit regarding the current observation: P(SV|ZC). In the system, filtering is applied to the hidden state; then the state can be inferred. Let O be the set of all variables:

[0111]

[0112] In the case of discretizing the origin-tracing unit and velocity, the filtering equation is written as:

[0113]

[0114] Then, this distribution can be expressed as:

[0115]

[0116] Then the unit occupancy can be inferred:

[0117]

[0118] 2. Description of Exemplary Solutions of the Model

[0119] Based on this model and its characteristics, a solution method is implemented according to the present invention. This method mainly includes three steps (prediction, fusion evaluation, and particle resampling), which will be described separately below.

[0120] 2.1 Implementation of the Model

[0121] As explained in the variable definitions, the system consists of two interconnected models: a "static" (grid-based) model and a "dynamic" (particle-based) model. In addition to semantic data, the potential states are, for example:

[0122] - The state (s) of "occupied by a static object", which refers to immobile occupancy and strictly includes static objects as well as background data (such as buildings).

[0123] - The state (d) of "occupied by a dynamic object", which refers to dynamic occupancy.

[0124] -- The state (e) of "empty", which corresponds to free space.

[0125] -- The state (u) of "undefined", which is used to represent a state of lack of information and also a lack of confidence in the credibility of other states.

[0126] Of course, other states (such as "occupied by pedestrians", "occupied by vehicles", "occupied by cyclists", etc.) can be defined, especially according to the data provided by the sensors.

[0127] The static model includes a state grid (occupancy grid), where each cell encodes its state distribution. Elements estimated to be static by the filter are represented in this cell grid. The dynamic model includes a set of particles in motion, where each particle has an actual position and velocity as well as a "weight", which corresponds to the probability that the cell it is estimated to belong to is occupied by a dynamic object with that state and these dynamics. Since only the occupancy and dynamic parts of the state are explained in detail here, all particles have the same (dynamic) state, but more generally, they can also have associated semantic states (cars, pedestrians, etc.). More specifically, at each iteration of the filter (for example, at each time step), these two models are independently used for prediction, then fused and evaluated together, and finally segmented again. In cases where the types of states considered may be different, some correspond to occupied cells (pedestrians, cars, buildings, etc.) or empty cells (roads, sidewalks, etc.), and each cell can have a specific motion model.

[0128] Thus, in the dynamic model, each particle has a true position and velocity as well as a "weight". This weight represents the probability that the cell (the cell to which the estimated particle belongs) is occupied by a dynamic object with that state and those dynamics. Elements estimated to be dynamic by the filter are represented in the dynamic model (objects in motion). The particles are distributed in a cell grid, the size of which is the same as the grid of the static model. At each iteration, the two grids (the two models) are updated independently of each other and then fused to provide the "final" grid.

[0129] Figure 2 A general method implemented in the device for processing the occupancy grid is shown. Based on the current occupancy grid, the method includes:

[0130] - A step of updating (A10) the static model, which includes determining a state and velocity prediction for at least one cell in the current occupancy grid according to the probability of the static model determined at the previous iteration, so as to provide a static model prediction [at time step t+1];

[0131] - A step of updating (A20) the dynamic model, which includes determining a state and velocity prediction for at least one virtual particle in the current occupancy grid according to the probability of the dynamic model determined at the previous iteration, so as to provide a dynamic model prediction [at time step t+1];

[0132] - A step of fusing (A30) the prediction of the static model and the prediction of the dynamic model according to a predetermined fusion mode;

[0133] - A step of evaluating (A40) the fusion prediction according to optional observation data from at least one sensor (such as a probability sensor).

[0134] The advantage of this solution is that the final prediction can be updated without data from sensors: the processing of the grid can be carried out independently of the reception of data from sensors. For example, the module implementing the method of the present invention can perform an "asynchronous" update, which is executed at a frequency of 100 Hz; as for the sensor part, the sensor itself can transmit data at a frequency of 20 Hz, and the proposed method undergoes multiple iterations without "observation" (using a complete state grid with the state of "unknown" instead of data from one or more sensors); then, by separating the particle prediction from the "grid" prediction, the filtered state is only based on the prediction made according to the previous state, as explained above.

[0135] 2.2 Prediction

[0136] As previously pointed out, these two models are predicted independently, where the final prediction is the result of fusing these predictions. In other words, two predictions are performed (updates of the static model and updates of the dynamic model), which are predicted separately and then fused. For each model, the update steps leading to the prediction include:

[0137] - Dynamic projection: Depending on the time elapsed since the previous iteration, the particles are projected according to their velocity as well as Gaussian position and velocity noise. The occupancy grid here is defined as static.

[0138] - State transition: Each (static, dynamic) model is associated with a transition matrix to enable state transition. In fact, these matrices mainly include a slow transition to the "undefined"

[0139] state, which means that over time, the confidence in the estimated data gradually decreases. The occupancy (static, dynamic) state is defined to be more consistent over time than the free state.

[0140] - Shifting of the reference frame: Since the system is designed to be installed on a moving vehicle (so the vehicle is movable), the model must be shifted according to the self-motion. The representation of the model simplifies a rather complex operation, which includes the transformation of the particle vector (translation and rotation of the position, rotation of the velocity) and the interpolation of the grid. The grid regions corresponding to the newly discovered areas are initialized with the undefined state (u), and the deleted areas can be ignored (in many mobile robot applications, this data storage is irrelevant), or

[0141] used to generate a map based on the static part.

[0142] Figure 3 Schematically illustrates the prediction process:

[0143] -(i) The models at the previous time step. The two models (grid and particles) are shown in a superimposed manner.

[0144] -(ii) Dynamic projection (dynamic projection for the grid and particles according to the motion model) and state transition (according to the state transition model).

[0145] --(iii) Express the data in the new reference frame through grid interpolation and translation and rotation of the particles.

[0146] -(iv) The models predicted at the current time step (the two grids are shown in a superimposed manner).

[0147] The potential values of these predictions are pre-computed (and pre-sampled), so these values:

[0148] P(C g -1)P(S g -1 V g -1 |C g -1 )P(S g p V g p |S g -1 V g -1 ) and P(C p -1 )P(S p -1 V p -1 |C p -1 )P(S p p V p p |S p -1 V p -1 ) are calculated independently (and these values are directly associated with the relevant grid cells and particles).

[0149] 2.3 Integration of Prediction and Evaluation

[0150] In each cell of the current grid, the updated state distribution is evaluated in parallel. All predicted particles are reorganized according to their positions in the grid so that only the particles and data of the relevant cells are transmitted for the calculations regarding the current grid cell (according to P(C|C g -1 V g p C p -1 V p p ). In other words, only the cells for which P(C|C g -1 V g p C p -1 V p p ) exceeds a specific threshold are transmitted for fusion and evaluation.

[0151] a. Fusion of predictions: According to P(S p V p |S g p V g pS p p V p p ) A defined fusion model, where each static and dynamic prediction sample generates a final prediction. The fusion model can be designed to be precise (and can even learn), but the main goal is to define it in such a way that if the model predicts a dynamic state, the fusion results in a dynamic state. Or, if the model predicts a static state, the fusion results in a static state. Or, if the model predicts a free state, the fusion results in a free state. And an undefined state (u) is generated only when both (static and dynamic) models predict an undefined state. For example, in this way, the prediction of a moving object arriving at the previous free (f) unit or undefined (u) unit is not weakened, thus ensuring the persistence of information (over time), and thus allowing the problems encountered to be solved.

[0152] b. Evaluation: In the given cell "c", these final predictions are compared with the observation P(S o V o |ZC). The observation model used is based on a classical probabilistic sensor model that takes into account the correlation of sensor data (confidence of the sensor, impact location, impact height, vertical distribution, etc.). The observation model can be defined for any type of sensor, and the distributions can be fused with the help of probabilistic fusion rules to achieve the integration and general processing of heterogeneous data. The state definition used in this model can also be different from the filtered definition.

[0153] For example, the observation model can be based on a classical probabilistic sensor model and add an evaluation of the uncertain state (u) that is related to the correlation of sensor data. For example, in the case of laser data, the undefined model obtains lower values before and around the impact, and higher values in more distant areas (when there is no available information).

[0154] To generate an estimate of the final state, the generative distribution P(SV|S p V p S o V o ) is used, which defines the probability of velocity and the probability of the final state based on predictions and observations. This generative distribution is given in the form of a state generation matrix (whose parameters can or must be learned according to the implemented operating conditions). For example, this enables an object in a cell previously considered free to appear quickly, while ensuring that an object in a cell now considered free or undefined disappears slowly, or that an undefined observation does not affect the estimate of the final state (beyond the range of natural decay of the prediction over time).

[0155] Once the predictions and evaluations based on the data provided by the sensors are fused, for this time step, a current grid is provided, in which for each (relevant) cell, the probabilities of the final state based on the prediction and observation and the probability of the velocity are obtained.

[0156] 2.4 Particle Resampling

[0157] Once the state distribution of the cells is correctly estimated (calculated), in an exemplary implementation, the following tasks include resampling the particles, for example including the following two steps:

[0158] a. Reallocation of particles: According to the dynamic probability of each grid cell and other criteria (position, difference between the predicted state and the observed state, velocity distribution, etc.), the total number of particles is reallocated to the relevant cells.

[0159] b. Generation of particles: During the evaluation of the state distribution, the weight of each existing particle is updated and normalized according to its relative contribution in the dynamic part. Then, in each cell associated with the particles, according to the ratio of the "dynamic appearance" (difference between the predicted state and the observed state, velocity distribution, etc.), particles are extracted from the existing particle distribution and the initialization distribution. Finally, the cell dynamic coefficient (i.e., the dynamic coefficient of the cell) is evenly distributed among the particles (in that cell).

[0160] Each particle extracted from the initialization distribution is associated with a unique label, and the unique label is propagated to the particles extracted from the existing distribution, enabling fast and lightweight pre-segmentation of the object (or at least forming consistent clusters in the occupied space and the dynamic space). This resampling can be performed in other ways and is not limited to this way.

[0161] 3. Other Features, Improvements and Advantages

[0162] According to the present invention, by ensuring a longer prediction duration, the quality of the predictions generated by the previous systems can be greatly improved. In addition, the present invention also has the following advantages:

[0163] - Specifiable sensitivity of the filter: The first main difference between the present invention and the filters cited above is the introduction of an intermediate state variable (S o V o) to evaluate the model state rather than directly based on the data. These "observed states" can have a different number of dimensions from the estimated states, and the relationship between each observed state and the estimated latent state can be designed to explicitly define the filtering sensitivity with respect to a specific data input without affecting the temporal consistency of the filtering state. The separation of these two components of the filter addresses the practical problem in previous methods where the emergence and disappearance rates of states are directly related.

[0164] - Independent update frequency: As already mentioned, this frequency can also incorporate a way to represent "missing data" in the observed state to decouple the update frequency of the filter from that of the sensor. And, combined with a GPU implementation, it can achieve an update frequency higher than the sensor frequency, enabling dynamic estimation of the scene even when no sensor data is received.

[0165] - Adaptable prediction templates: Another major difference is the use of two different models for prediction, grid-based prediction (S g p V g p ) and particle-based prediction (S p p V p p ), both of which propagate independently at each iteration but are fused (S p V p ) based on the state estimate and regenerated. This strict separation (compared to conditional sampling to date) and configurable fusion address another important problem in previous methods, namely the impossibility of adjusting the persistence of states in a cell based on other predictions. A typical case of this problem is when a dynamic sample moves over a free area without new observations, and the persistence of previously observed free cells reduces the occupancy probability of the object in motion until it disappears, even without a defined temporal state transition (dynamic transition model). Thanks to the method of the present invention, when a cell is predicted as a "free" cell based on the first model and as a "dynamic" cell based on the second model (particle projection-based), the predicted state is correctly defined as "dynamic".

[0166] Thus, by implementing the proposed method, a filter update frequency of 100 Hz can be obtained, which is higher than the sensor frequency. This higher update frequency enables smoother filtering, more precise data integration, and a clear reduction in time confidence, while improving the operational reliability of the method (the propagation of the model remains clear in case of sensor failure). The difference and combined prediction between the two models ultimately solve important problems such as the undesired rapid dilution of dynamic occupancy in areas previously considered free or undefined. Adding the observation state leads to greater reactivity in detecting specific states. Although the results are convincing, further research can focus on the quantitative validation of the method, which remains a major challenge in the field of dynamic occupancy grid filtering, as this field lacks clear and recognized evaluation criteria, metrics, and reference databases.

[0167] 4. Processing Equipment

[0168] Figure 4 A simplified architecture of a processing device for processing an occupancy grid is shown, which implements all or part of the method presented as above. Such an occupancy grid processing device includes a memory 41 and a processing unit 42, the processing unit 42 being equipped with, for example, a microprocessor, and the processing unit 42 being controlled by a computer program 43 that implements the method according to the invention. In at least one example embodiment, the invention is implemented in the form of an application program installed on a device in a vehicle or (robot) autonomous agent. Such an occupancy grid processing device includes, for example, all or part of the following means:

[0169] - An updating means for updating the probability distribution of the static scene representation grid based on at least one static state associated with at least one cell of the static grid according to a static model;

[0170] - An updating means for updating the probability distribution of the dynamic scene representation grid based on at least one particle associated with at least one cell of the dynamic grid according to a dynamic model;

[0171] - A fusion means for fusing the probability distributions of the static grid and the dynamic grid in the resulting state grid, the fusion means providing a state prediction and a speed prediction for each cell;

[0172] -- An evaluation means for evaluating the probability of the final state and the probability of the associated speed according to the prediction and the observation for at least one cell in the resulting occupancy grid.

[0173] These devices can take the form of a specific software application or a dedicated hardware component designed to perform these functions, such as an ASIC.

Claims

1. A method for obtaining a data grid representing a dynamic representation scene, the data grid being called a state grid, the state grid being divided into cells, the method being implemented by a computer module for processing data from at least one sensor, wherein, The method includes at least one iteration of the following steps: - Updating (A10) the probability distribution of the static representation grid characterizing the scene based on a static model and at least one static state associated with at least one cell of the static grid; - Updating (A20) the probability distribution of the dynamic representation grid characterizing the scene based on a dynamic model and at least one particle associated with at least one cell of the dynamic grid; - Fusing (A30) the probability distributions of the static grid and the dynamic grid in a resulting state grid, thereby providing a state prediction and a velocity prediction for each cell; - For at least one cell of the resulting state grid, evaluating (A40) the probability of the final state and the probability of the associated velocity based on the prediction and any data received from the at least one sensor.

2. The method according to claim 1, characterized in that, The steps of fusing the probability distribution include: for a given cell of the result grid, calculating the probability P(S p V p |S g p V g p S p p V p p ), where: -S p : The predicted state of the cell at the current time step; -V p : The predicted velocity of the unit at the current time step; - The predicted state of the cell at the current time step predicted by the "static" model; - The predicted speed of the cell at the current time step predicted by the "static" model; - The predicted state of the cell at the current time step predicted by the "dynamic" model; - The predicted velocity of the cell at the current time step predicted by the "dynamic" model.

3. The method according to claim 1, wherein The steps for evaluating the probability of the final state and the probability of the associated velocity include: for a given cell of the result grid, calculating the probability P(SV|S p V p S o V o ), where: - S: The state of the cell at the current time step; - V: The velocity of the cell at the current time step; -S p : The predicted state of the cell at the current time step; -V p : The predicted velocity of the cell at the current time step; -S O : The state of the cell observed at the current time step; -V O : The velocity of the cell observed at the current time step.

4. The method according to claim 1, characterized in that, The steps of updating (A10, A20) include: - A step of dynamically projecting the particles according to the velocity of the particles; - A step of applying a state modification to the static model; - A step of performing a state transition according to the model; - A step of shifting according to the shift of the reference frame.

5. The method according to claim 1, wherein The method further includes: after the step of evaluating (A40), a step of resampling the particles in the dynamic model.

6. The method according to claim 5, characterized in that, The step of resampling the particles includes: - A step of reallocating the particles according to the dynamic probability of each grid cell; - A step of generating particles according to the previous allocation of the particles and the contribution of these previously allocated particles to the dynamic model.

7. The method according to claim 1, wherein The step of evaluating (A40) implements an observation grid, wherein the observation grid includes data from the at least one sensor.

8. A module for obtaining a data grid representing a dynamically represented scene, said data grid being called a state grid, said state grid being divided into cells, the method being implemented by a computer module for processing data from at least one sensor, wherein, The module includes iterative computing means for performing the following steps: - Updating (A10) the probability distribution of the static representation grid characterizing the scene based on a static model and at least one static state associated with at least one cell of the static grid; - Updating (A20) the probability distribution of the dynamic representation grid characterizing the scene based on a dynamic model and at least one particle associated with at least one cell of the dynamic grid; - Fusing (A30) the probability distributions of the static grid and the dynamic grid in a resulting state grid, thereby providing a state prediction and a velocity prediction for each cell; - For at least one cell of the resulting state grid, evaluating (A40) the probability of the final state and the probability of the associated velocity based on the prediction and any data received from the at least one sensor.

9. A computer program, the computer program including program code instructions for performing the steps of the method according to any one of claims 1 to 7 when the program is executed on a processor.

Images