Method for processing at least one data grid representative of a scene captured by a set of sensors, and corresponding device and program

Abstract

The invention relates to a method for processing data grids representative of a scene captured by a set of sensors, a grid being divided into cells (C1, C2, C3, Ci), each associated with a probability distribution (DE) of states for a region of the scene covered by the cell at a current time step, the probability distributions being determined on the basis of data delivered by at least one of the sensors and / or data predicted by a prediction model on the basis of data delivered by the sensor at a previous time step. The method comprises a step of determining, for at least one cell of the grid, a data item of interest (DI), representative of a degree of interest of the cell in light of predetermined criteria relating to a context in which the scene is captured, a characteristic of the captured scene, and / or an application associated with the capture, this determination step delivering an enhanced data grid (GDA).

Description

[0001] DESCRIPTION

[0002] TITLE: Process for processing at least one grid of data representative of a scene captured by a set of sensors, device and corresponding program.

[0003] technical field

[0004] The invention relates to the field of computer vision devices and autonomous devices in general that interact with their environment, such as robots or autonomous vehicles. For these devices, the ability to accurately perceive and model their environment in a relevant way is essential. Whether for navigation, collision awareness, intent planning, or mapping, this perception stage is challenging in terms of accuracy, complexity, and uncertainty management.

[0005] More specifically, the invention relates to techniques for processing one or more data grids representative of a scene captured by a set of sensors.

[0006] Previous art

[0007] Despite impressive advancements and the ever-increasing development of embedded intelligence in mobile devices, the ability of an autonomous agent to accurately, robustly, and efficiently perceive its environment remains a significant challenge in robotics. The quality of environmental modeling relies not only on sensors but also on interpretation schemes that address sensor errors, occlusions, data contradictions, highly complex parameters, and other factors. Probabilistic methods have been developed to formally model uncertainties and prior knowledge within these interpretation schemes.

[0008] When these interpretation schemes are confronted with moving objects, numerous additional problems arise. A classic approach to addressing this problem is to adopt an object-based representation, which leads to tracking multiple objects. A second common approach is to adopt a representation based on the construction of data grids, and more specifically, occupancy grids, which focuses on evaluating spatial occupancy without higher-level segmentation. More specifically, such an occupancy grid is generated at each time step by mapping sensor data onto the grid using a probabilistic sensor model. Each cell of the grid is then associated, for example, with an occupancy probability distribution.This second approach offers significant advantages, such as a spatially dense model with an accurate representation of free space, which is crucial in mobile robotics. Furthermore, the complex data segmentation and recognition steps required in object-based representations using the first approach can be avoided. When applied to dynamic environments, it is generally necessary to enrich this data grid representation by estimating velocity information in addition to occupancy estimates.

[0009] This second approach, however, also has drawbacks. For example, the data associated with each cell in a data grid is generally treated indiscriminately when the data grid is used for an application. In other words, all cells in a data grid are treated equally, including cells that do not necessarily require elaborate representation, such as cells with undefined (i.e., unknown) content, empty (i.e., unoccupied) content, or static content. This means that computing resources are not always allocated optimally. The grid itself, in most applications, must be high-dimensional (i.e.,(It must be composed of a large number of cells) in order to be usable, with the consequence that many cells are often not reachable by sensors at any given time, for example, due to obstructions. As a result, in potentially large areas, many cells are associated with data assigned exclusively on the basis of probabilities derived purely from prior knowledge (i.e., for which no data from sensors is available in the short term), and not on data estimated based on predictions made from previous and / or current sensor observations. Such a situation can lead to massive and unnecessary calculations, that is, a significant waste of computing power in irrelevant areas, compromising the optimal functioning of otherwise well-suited systems.

[0010] Another key problem with this second approach relates to the inability to differentiate between probabilities resulting solely from prior estimation (i.e., a prediction made in the absence of data reported by sensors at a given time step, thus relying on past or previous data) and those resulting from actually measured data (i.e., an observation reported by sensors at the current time step). For example, knowing the probability distribution associated with a particular cell does not allow us to determine whether this final distribution is the result of contradictory data delivered by sensors at the considered time step, or a result derived from knowledge of previous distributions in the absence of data reported by sensors at the considered time step.Put another way, estimates based on current data measured by sensors have the same status, or weight, as those based solely on predictions made on the basis of previous data.For example, if a previous distribution of cell occupancy indicates that the probability of that cell being occupied is 0.5, and a current distribution indicates that the probability of that cell being occupied is also 0.5, it is not currently possible, knowing only this value, to determine whether it results from the fact that no measured data is available for that cell at the current time step (in which case the old value, or prior estimate, of 0.5 has, for example, been retained for the current time step), or whether it results from the fact that two sensors have delivered contradictory information at the current time step regarding the probability of occupancy of that cell (in which case the fusion between these two contradictory pieces of information resulted in an occupancy probability of 0.5).It would nevertheless be desirable to be able to distinguish these two cases in order to allow for further differentiated processing (for example to carry out a more in-depth investigation in the event of obtaining contradictory data, which is not necessarily useful to carry out in the case of a pure and simple absence of data).

[0011] The limitations presented earlier can be explained by the Bayesian framework and the notational abuses commonly used on traditional occupancy grids, in which calculated probabilities are implicitly calculated within a common, unstated reference frame. In other words, all generated Bayesian probability estimates should refer to the reference frame in which they were calculated. Merging different estimates must first reconcile the reference frames in question and modify the probability values ​​accordingly before it can be performed. Within a single reference frame, relative confidence levels are implicitly carried over to each probability value, which thus incorporates prior knowledge, uncertainties related to sensors, subsequent modeling and processing, these relative confidence levels, and so on.This aggregation of these different factors into a single value is certainly the desired objective, but when done directly it no longer allows specific processing of some of these parts, for example the optimization of calculations in distributions derived solely from a priori information, or from contradictory sensor data.

[0012] Therefore, it is necessary to have a solution that allows us to resolve this problem.

[0013] Summary of the invention

[0014] The present invention proposes a solution to overcome certain drawbacks of the prior art. In one aspect, the present invention relates to a method for processing at least one data grid representing a scene captured by a set of sensors. This data grid is divided into cells, each cell being associated with a probability distribution of states of an area of ​​the scene covered by said cell at a current time step. These state probabilities are determined based on data collected at said current time step by at least one sensor of said sensor set and / or data predicted by a prediction model based on data collected at at least one previous time step by said at least one sensor.Such a process includes the determination, for at least one cell of the grid, of at least one data point of interest, representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion in relation to a context of capturing the scene, a characteristic of the captured scene, and / or an application associated with said capturing, said determination delivering a grid of data called augmented.

[0015] In a particular embodiment, said at least one predetermined criterion used to determine the degree of interest of a cell belongs to the group comprising at least:

[0016] a number and / or degree of reliability of the sensors used to determine the probability distribution of states associated with said cell at the current time step;

[0017] a nature of the data used to determine the probability distribution of states associated with said cell at the current time step, from predicted data or data reported from sensors; prior knowledge of a particular characteristic of the scene area associated with said cell at the current time step;

[0018] a prediction of a trajectory of a dynamic object belonging to the scene, according to which said object is likely to be present in the area of ​​the scene associated with said cell at the current time step;

[0019] a particular application aimed at using the probability distribution of states associated with said cell at the current time step.

[0020] In a particular embodiment, said data of interest takes the form of a probability distribution of degrees of interest.

[0021] In a particular embodiment, said method further comprises: obtaining a first augmented data grid for said scene, associated with the data provided at the current time step by a first data source;

[0022] obtaining a second augmented data grid for said scene, associated with the data provided at the current time step by a second data source distinct from the first data source;

[0023] the merging of said first and second augmented data grids, including the merging of state probability distributions and the merging of degree-of-interest probability distributions between corresponding cells of said first and second augmented data grids, said merging delivering a merged augmented data grid.

[0024] According to a particular characteristic, the said merger is carried out according to the formula:

[0025]

[0026] In another particular, alternative or complementary embodiment, said process further comprises:

[0027] obtaining a stack of augmented data grids for said scene, associated with data retrieved from the same data source at a plurality of altitudes of said scene, each augmented data grid in the stack being associated with a given altitude among said plurality of altitudes; merging the augmented data grids in said stack, including merging the state probability distributions and merging the degree of interest probability distributions between corresponding cells of the set of augmented data grids, said merging delivering a consolidated augmented data grid for said stack.

[0028] According to a particular feature of this embodiment, the merging of the augmented data grids of the stack is performed by initializing a first variable PSG = P(S0= 0 I o = 0|Z) + P(S0= 0 I o = 1 |Z), a second variable PIO = P(S0= 0 I o = 0|Z) + P(S0= 1 I o = 0|Z) and a third variable PSOIO = P(S0= 0 I o = 0|Z), then by performing a loop in which the augmented data grids of said stack are traversed sequentially to update, at each iteration of the loop, the first variable, the second variable and the third variable according to the following sequence of operations:

[0029]

[0030] c) PSOIO = PS0I0 * P(S k = 0 I k = 0|Z)

[0031] until all the grids in the stack have been traversed, then return a consolidated augmented data grid for said stack defined by the third variable PSOIO, a variable PS1I0 = PI0 — PS0I0, a variable PS0I1 = PS0 — PS0I0, and a variable PS1I1 = 1 − PS1I0 − PS0I1 − PS0I0.

[0032] Alternatively, according to another particular feature of this embodiment, the merging of the augmented data grids of the stack is performed by recursive merging, starting from a first grid Po(SI|Z) chosen equal to the augmented data grid P(So law Z) associated with the lowest altitude of the stack, the subsequent augmented data grids of the stack, via the computation of a merged augmented data grid of a later iteration Pk+i(SI\Z) from the merged augmented data grid of the previous iteration P k

[0033]

[0034] ... S k Ik ~) according to the following sequence of operations:

[0035]

[0036] I 7 fc+ 1 = 1| )

[0037] b) P k+1 (S = 0 I = 1|Z) = P k S = 0 I = 1|Z) * (P (S k+1 = 0 / k+1 = l|Z) + P(7 k+1 = O|Z) * (Æ - l) ) + P k (7 = O|Z) *P (S k+1 = 0 / k+1 = 1|Z) /

[0038] c) P k+1 (Z = 0) = 1 - P k+1 (S = 11 = 1|Z) - P k+1 (S = 01 = 1|Z), until the index k+1 denotes the index of the augmented data grid associated with the highest altitude of said stack of augmented data grids, and to return Pk+i(S=l 1=11 Z), Pk+i(S=O 1=11 Z) and Pk+i(l=O) as a consolidated augmented data grid for said stack.

[0039] According to another aspect, the present technique relates to an electronic device for processing at least one data grid representing a scene captured by a set of sensors, said data grid being divided into cells, each cell being associated with a probability distribution of states of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined as a function of data reported at said current time step by at least one sensor of said set of sensors and / or of data predicted by a prediction model as a function of data reported at at least one previous time step by said at least one sensor.Such an electronic device includes means for determining, for at least one cell of the grid, at least one data point of interest, representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion in relation to a context of capturing the scene, a characteristic of the captured scene, and / or an application associated with said capturing, said means of determination delivering a grid of data called augmented.

[0040] Such an electronic device can, of course, exhibit the various characteristics of the processing method according to the invention, which can be combined or considered individually. Thus, the characteristics and advantages of this device are the same as those of the processing method for at least one grid of data representative of a scene captured by a set of sensors, and are not described in further detail.

[0041] According to another aspect, the proposed invention also relates to a computer program product downloadable from a communication network and / or stored on a computer-readable medium and / or executable by a microprocessor, comprising program code instructions for the execution of a process for processing at least one data grid as described above in any of its embodiments, when this process is executed on a computer.

[0042] The proposed invention also relates to a computer-readable storage medium on which a computer program is stored, comprising program code instructions for executing the steps of a process as described above, in any of its embodiments. Such a storage medium can be any entity or device capable of storing the program. For example, the medium can include a storage means, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording means, for example a USB flash drive or a hard drive.

[0043] On the other hand, such a recording medium can be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means, so that the computer program it contains can be executed remotely. The program according to the invention can, in particular, be uploaded to a network, for example, the Internet.

[0044] The different embodiments mentioned above can be combined with each other for the implementation of the invention.

[0045] Figures

[0046] Other features and advantages of the invention will become more apparent upon reading the following description of a particular embodiment, given by way of simple illustrative and non-limiting example, and the accompanying drawings, among which:

[0047] [Fig 1] presents an example of an augmented data grid, in a particular embodiment of the proposed technique;

[0048] [Fig 2] schematically presents the different stages of a fusion of augmented data grids associated with different data sources, in a particular embodiment of the proposed technique;

[0049] [Fig 3] schematically presents the different stages of a fusion of augmented data grids associated with data delivered at different altitudes by the same data source, in a particular embodiment of the proposed technique;

[0050] [Fig 4] describes a simplified architecture of a device for processing at least one data grid, in a particular embodiment of the proposed technique.

[0051] Detailed description of the invention

[0052]

[0053] The invention described below addresses some of the aforementioned drawbacks. In its first aspect, the present invention relates to a method for processing at least one grid of data representing a scene captured by a set of sensors. As a reminder, one objective of Bayesian Occupancy Filter techniques, within which the proposed solution can be applied, is to estimate the spatial occupancy and dynamics of an environment observed using various types of sensors. These sensors are, for example, mounted on a potentially moving autonomous device, typically a robot or an autonomous vehicle, and they aim to provide information enabling this device to interact with its environment (e.g., following a route on a road, avoiding collisions with obstacles or other road users, complying with traffic rules, etc.).In this context, as presented in relation to prior art, one or more data grids representing the scene are repeatedly generated, for example, at regular intervals according to a predetermined time step. More specifically, each data grid is divided into cells, each cell containing state data characterizing the area of ​​the scene covered by the cell at a given time step, thus forming a mesh of the scene. Such state data includes, for example, a probability distribution of states; the number and nature of the states considered can vary depending on the type of application, the expected degree of accuracy, and so on.Thus, in some contexts, a simple two-state representation of "cell occupied by an object" versus "cell not occupied by an object" may be sufficient, while more complex representations with multiple states may be more relevant in other contexts, with, for example, states such as "cell occupied by a pedestrian," "cell occupied by a car," "open road cell," "open pedestrian zone cell," "cell with undefined content," etc. State probabilities are determined based on data collected at the current time step by at least one sensor from the sensor set used to capture the scene and / or data predicted by a prediction model based on data collected at at least one time step prior to the current time step by at least some of the sensors (prior knowledge).The data grids are updated repeatedly and continuously, thus providing the autonomous device, in real-time or near-real-time, with a set of data relating to its environment. Based on this data, a decision-making body within that device can make decisions, such as trajectory changes, braking, etc. Figure 1 illustrates an example of such a GDA data grid, divided into cells C1, C2, C3, etc., in which each cell is associated with a DE probability distribution of states (four in the example in Figure 1: S1, S2, S3, and S4) of the scene area covered or associated with the cell in question, at a current time step. For clarity, only the data from one cell C1 of the GDA data grid is shown in Figure 1.Thus, the probabilities P(S1), P(S2), P(S3) and P(S4) indicate the probability that the area of ​​the scene covered by the cell Ci is respectively in the state SI, S2, S3 or S4.

[0054] As described in relation to the prior art, knowledge of the DE probability distribution of states associated with a cell is not, however, sufficient information in itself to determine an optimal way of considering that cell (i.e. to estimate its importance), whether in terms of, for example, the interest to be given to it, the computing power to be allocated to its processing, etc.

[0055] Therefore, according to the general principle of this technique, it is proposed to associate at least one additional data point, called a data point of interest, with at least one cell in the data grid. More specifically, such a data point represents the degree of interest of the cell to which it is associated, with regard to at least one predetermined criterion related to the scene capture context, a characteristic of the captured scene, and / or an application associated with said capture. Examples of such criteria are presented later. Thus, in addition to the state data DE, cell Ci in Figure 1 (as well as other cells in the data grid GD, or even all the cells in this grid) is, for example, associated with a data point of interest DI, which demonstrates the importance and / or relevance (according to the predetermined criterion considered) that should be attributed to the probability distribution of states associated with the cell.According to a particular characteristic, the data of interest takes the form of a probability distribution between several predetermined degrees of interest (or levels of interest). Figure 1 thus illustrates a data of interest DI which takes the form of a probability distribution of degrees of interest between two predetermined levels of interest in a binary mode, namely "unimportant cell" (1=0) and "important cell" (1=1) (it being understood that such an example is given purely for illustrative purposes and is not limiting, and that the number of degrees of interest considered may be greater in order in particular to provide more nuances than a simple all-or-nothing consideration).Thus, in Figure 1, the probabilities P(l=0) and P(l=l) indicate the probability that cell Ci should be considered, respectively, as unimportant or, conversely, particularly important with regard to certain criteria. This information can then be used, for example, to determine the treatments to be applied to this cell. According to a particular characteristic, the DE probability distributions of states and the DI probability distributions of degrees of interest are not evaluated separately, but are instead estimated and stored together.

[0056] Determining the interest (e.g., the probability distribution of interest, DI) of a cell is based on one or more predetermined criteria. Depending on the selected criteria, the data of interest may, for example, represent the degree of confidence that can legitimately be placed in the state data associated with the cell in question, and / or the degree of relevance or importance of the cell in relation to a current context. Examples of such predetermined criteria are presented below; these criteria may be considered individually or in combination.

[0057] Firstly, the determination of the data of interest may take into account the number of sensors and / or the degree of reliability of the sensors used to evaluate the probability distribution of states associated with the cell under consideration at the current time step. For example, less importance may be assigned to cells whose probability distribution of states is known to have been determined based on data collected by a small number of sensors, or on data collected by sensors whose measurements are considered less reliable than other types of sensors, for example.

[0058] Secondly, the determination of the data of interest may take into account the nature (or origin) of the data used to evaluate the distribution of state probabilities, depending for example on whether this data is data resulting from measurements made by sensors at the current time step (i.e. "observed" data), or on the contrary, data not measured, but predicted or extrapolated for the current time step from, for example, older measurements.

[0059] Third, the determination of the data of interest can take into account the intended application (i.e., the purpose of the data processing) and / or the location of the cell under consideration. For example, prior knowledge of one or more specific characteristics of the geographic area of ​​the scene associated with the cell under consideration at the current time step, typically map data, may indicate that information of interest is located in that particular area, and that greater importance should therefore be given to the state data associated with that cell. Projections based on tracking the dynamics of certain moving objects present in the scene can also be taken into account when determining the data of interest.Thus, this determination can, for example, take into account the prediction of a trajectory for a dynamic object within the scene, according to which this object is likely to be present in the area associated with the cell under consideration at the current time step, thereby making this cell of interest. Similarly, the nature of the intended application can itself be a criterion for determining the data of interest; for instance, the implementation of an automatic braking system for a vehicle requires particular attention to cells that cover areas likely to be located on the vehicle's trajectory.

[0060] The examples above are of course given for illustrative purposes only and are not exhaustive; other criteria than those listed may also be taken into account, either in a complementary or alternative manner, when determining the degree of interest associated with a cell. Depending on a particular characteristic, several data points of different kinds may also be associated with a cell (for example, one data point representing the degree of reliability of the state data, another data point representing the degree of importance of the cell with regard to the intended application, etc.).

[0061] It follows from the above that, unlike the states associated with a cell, which can be considered as information of an "absolute" nature, the degree of interest associated with a cell is information that can be described as "relative", in that it depends on the application and / or the observer (i.e. the sensors providing the metrics on the basis of which the data grids are at least partially constructed).

[0062] Such data of interest associated with the cells of a data grid are interesting because they allow differentiation of the treatments to be applied to the different cells of the grid, for example by allocating more computing resources to the cells considered interesting.

[0063] The enriched data grid, incorporating this data of interest, can therefore be described as an augmented data grid. We will now examine complementary aspects of the data grid processing method using the proposed technique, focusing in particular on how different augmented data grids can be merged to obtain a consolidated, more user-friendly resulting data grid.

[0064] In this context, three specific implementation methods are presented below:

[0065] a first embodiment relating to the fusion of augmented data grids associated with different data sources (for example, different sensors or groups of sensors);

[0066] a second embodiment relating to the fusion of augmented data grids associated with data delivered at different altitudes by the same data source (for example the same sensor or group of sensors);

[0067] a third particular embodiment, combining augmented data grid fusions according to the two particular embodiments previously mentioned.

[0068] 2 - Merging Augmented Data Grids Associated with Different Data Sources The steps for merging augmented data grids associated with different data sources are presented in relation to Figure 2, in a particular embodiment of the proposed technique. For the sake of simplicity, the merging of two augmented data grids is described below, it being understood that merging a larger number of augmented data grids can always be decomposed into the implementation of a succession of mergings of two augmented data grids (for example, merging three augmented data grids can be carried out by successively merging the first two augmented data grids, then merging the resulting augmented data grid with the third augmented data grid).

[0069] Within this framework, the data grid processing method comprises a step 21 of obtaining a first augmented data grid GDA1 associated with data from a first data source Z1 (e.g., a first sensor or group of sensors), and a step 22 of obtaining a second augmented data grid GDA2 associated with data from a second data source Z2 (e.g., a second sensor or group of sensors) distinct from the first data source Z1. Augmented data grids GDA1 and GDA2 correspond to two representations of the same scene, obtained for the same current time step. Depending on the data sources Z1 and Z2 considered, the resulting augmented data grids GDA1 and GDA2 are directly of identical dimensions.Alternatively, when this is not the case, at least one of the two augmented data grids GDA1 and GDA2 undergoes preliminary processing, typically a truncation and / or completion operation, so that the augmented data grids GDA1 and GDA2 ultimately obtained at the end of steps 21 and 22 are of identical dimensions; that is, each cell in one of the two grids corresponds to a cell in the other grid associated with the same area of ​​the captured scene. More specifically, a truncation operation involves removing one or more cells from a data grid, while a completion operation involves adding one or more cells to a data grid. The cells removed during a truncation operation are, as far as possible, cells associated with representative data of low importance and / or low reliability (i.e., cells of low importance).Cells added during a completion operation are intentionally and fictitiously associated with representative data of low importance and / or low reliability (i.e., the added cells are deliberately marked as low importance). In this way, the impact of these cell additions and / or deletions, which are solely intended to obtain augmented data grids GDA1 and GDA2 of similar size, is minimized during subsequent grid merging operations.At the end of steps 21 and 22, the augmented data grids GDA1 and GDA2 therefore include - possibly after at least one truncation and / or completion operation of at least one of the two grids GDA1 and / or GDA2 as previously mentioned - the same number of cells, and corresponding cells from these two grids (i.e. cells in the same position in each of the two grids, for example the cells marked with an "x" cross in Figure 2) cover the same area of ​​the captured scene.

[0070] The merging of the two augmented data grids GDA1 and GDA2 is implemented in a step 23, and includes merging the state probability distributions and merging the degree-of-interest probability distributions between corresponding cells of said first and second augmented data grids, said merging delivering a merged augmented data grid GDF.

[0071] According to a particular characteristic, the fusion is performed by applying a Bayesian fusion mechanism, according to a formula equivalent to the formula:

[0072]

[0073] In this example, for the sake of simplicity, a two-state, two-degree-of-interest representation is adopted. Thus:

[0074] Regarding the status data, a cell can be "occupied" (S=1) or "unoccupied" (S=0);

[0075] Regarding the data of interest, a cell can be "important" (I=1) or "not important" (I=0).

[0076] Of course, this example, deliberately simplified for the sake of clarity, is not limiting, and representations with larger numbers of states (i.e. beyond two states) or degrees of interest (i.e. beyond two degrees of interest) can also be considered without going out of the scope of this technique, with a formula of the type of the one previously presented adapted to these more complex representations.

[0077] More specifically, this formula defines particularly advantageous mechanisms for propagating the data of interest I, according to which:

[0078] merging the data from corresponding cells considered very likely to be unimportant (1=0) in all augmented data grids prior to merging (i.e., both in GDA1 and GDA2) results in one cell that is marked as unimportant in the resulting merged augmented data grid GDF;

[0079] any potential interest of a cell found in at least one of the augmented data grids before merging (i.e. in GDA1 or GDA2) emerges strengthened from the merging process, in that it leads to a cell marked as even more important in the resulting merged augmented data grid GDF, in other words, P(I = 1|Z1Z2) ≥ MAX(P(I = 1|Z1), P(I = 1|Z2)).

[0080] 3 - Fusion of augmented data grids associated with data delivered at different altitudes by the same data source The steps of the fusion of augmented data grids associated with data delivered at different altitudes by the same data source are presented in relation to figure 3, in a particular embodiment of the proposed technique.

[0081] Within this framework, the data grid processing method includes a step 31 of obtaining a PL stack of augmented data grids (GDA0, GDA1, …, GDA k , …GDA n) associated with data provided by the same data source Z, representative of the scene at different altitudes h, for the same current time step. Such data can, for example, be obtained using laser rangefinder sensors. More specifically, each augmented data grid in the stack is associated with a known altitude from among a plurality of altitudes associated with the scene. In the example in Figure 2, the stack PL comprises n+1 augmented data grids (where n is an integer greater than or equal to 1), the lowest augmented data grid GDA0 in the stack being associated with data uptaken at ground level by the data source Z, and the grid GDA nThe highest cell in the stack is associated with data retrieved for the highest altitude h at which the data source Z is capable of delivering data. In the example in Figure 2, each grid in the PL stack is represented as a plane, with the altitude h corresponding, for example, to an absolute altitude relative to a reference altitude such as sea level. It should be noted, however, that in a particular embodiment (not shown), the augmented data grids in the stack are not analogous to a plane, but rather conform to the contours of the underlying ground. In other words, according to a specific characteristic, the altitude h is not an absolute altitude relative to a reference altitude, but an altitude relative to the position (i.e., the height) of the ground in the area of ​​the scene covered by each cell in the grid.Regardless of the embodiment considered (absolute or relative altitude), a stack augmented data grid remains associated with an altitude h among a plurality of altitudes. All stack augmented data grids correspond to representations of the same scene, obtained for the same current time step. Depending on the data source Z considered, the resulting stack augmented data grids PL are directly of identical dimensions. Alternatively, when this is not the case, at least one of the stack augmented data grids may undergo preprocessing, typically a truncation and / or completion operation as previously described in relation to Figure 2, so that all data grids 1.

[0082] The augmented stacks obtained at the end of step 31 must be of identical dimensions; that is, each cell in any of the stack's grids must correspond to a cell in each of the other stack grids associated with the same area of ​​the captured scene. More specifically, a truncation operation corresponds to the deletion of one or more cells from a data grid, while a completion operation corresponds to the addition of one or more cells to a data grid. The cells deleted during a truncation operation are, as far as possible, cells associated with data of interest that is representative of low importance and / or low reliability (i.e., cells of low importance).The cells added during a completion operation are intentionally associated, fictitiously, with representative data of low importance and / or low reliability (i.e., the added cells are deliberately marked as low importance). In this way, the impact of these cell additions and / or deletions, which are solely intended to obtain augmented data grids of similar size throughout the PL stack, is minimized during subsequent grid merging operations. At the end of step 31, the augmented data grids in the stack therefore contain the same number of cells, and corresponding cells in these grids (i.e., cells in the same position in each of the two grids, for example, the cells marked with an "x" in Figure 3) cover the same area of ​​the scene but at different elevations.In other words, these corresponding cells allow for the analysis of the "air column" content vertically above the associated area of ​​the scene, essentially creating a three-dimensional representation. This notably improves the detection of certain types of obstacles (such as a bridge deck, panels on a gantry, etc.) where the spatial arrangement differs between near-ground level and higher altitudes.

[0083] The merging of the augmented data grids of the stack is implemented in a step 32, and includes merging the state probability distributions and merging the degree-of-interest probability distributions between corresponding cells of the set of augmented data grids, said merging delivering a consolidated merged augmented data grid GDFC for the entire PL stack.

[0084] It is important to note that, unlike the augmented data grid fusion technique described earlier in relation to Figure 2 for data associated with different data sources, which assumes independence between data from different sources within each cell, data from the same data source at different altitudes are clearly interdependent. For example, if all the data for a cell are concentrated within a narrow range of altitudes, the probability distributions of state (i.e., occupancy) according to this data are clearly correlated with that altitude range.

[0085] Two strategies for merging grids in a stack of augmented data grids are presented below, aiming to obtain a consolidated merged augmented data grid associated with the entire stack.

[0086] Again, in the following description of these two strategies, specific implementations of which are given by way of illustration and not limitation, a two-state, two-degree-of-interest representation is adopted for the sake of simplification. Thus:

[0087] Regarding the status data, a cell can be "occupied" (S=1) or "unoccupied" (S=0);

[0088] Regarding the data of interest, a cell can be "important" (I=1) or "not important" (I=0).

[0089] Of course, this example, deliberately simplified for the sake of clarity, is not limiting, and representations with larger numbers of states (i.e. beyond two states) or degrees of interest (i.e. beyond two degrees of interest) can also be considered without going out of the scope of this technique, with mechanisms and formulas of the type presented below adapted to these more complex representations.

[0090] Considering the set of corresponding cells in the augmented data grid stack, that is, the set of cells located at different altitudes (i.e., vertically) of the same ground area of ​​the scene (for example, all cells marked with an "x" in Figure 3), these two strategies have in common that it is sufficient for a cell to be considered occupied in any of the augmented data grids in the stack (i.e., at any altitude) for the resulting merged cell (for example, the cell marked with an "r" in Figure 3, the result of merging all the cells marked with an "x") in the consolidated augmented data grid to be marked as occupied.

[0091] These two merging strategies differ, however, in how they handle the merging of data of interest, as presented below. -- First merging strategy

[0092] According to a particular embodiment corresponding to the first fusion strategy mentioned above, the fusion of the grids in the augmented data grid stack is performed by initializing a first variable PSO = P(S0= 0 I o = 0|Z) + P(S0= 0 I o = 1|Z), a second variable PIO = P(S0= 0 / 0= 0|Z) + P(S0= 1 / 0= 0|Z) and a third variable PS0I0 = P(S0= 0 I o = 0|Z), then by performing a loop in which the augmented data grids of said stack are traversed sequentially to update the first variable, the second variable, and the third variable according to the following sequence of operations, with k the index of the augmented data grid traversed at the considered iteration of the loop:

[0093]

[0094] c) PS0I0 = PS0I0 * P(S k = 0 I k = 0|Z)

[0095] Once the loop is complete, that is, once all the augmented data grids have been traversed, the probability distributions associated with the consolidated merged augmented data grid GDFC are defined by the third variable PSO / O (corresponding to the probability that a merged cell is unoccupied and unimportant), a variable PS1I0 = PIO — PS0I0 (corresponding to the probability that a merged cell is occupied and unimportant), a variable PS0I1 = PSO — PS0I0 (corresponding to the probability that a merged cell is unoccupied and important), and a variable PS1I1 = 1 — PS1I0 — PS0I1 — PS0I0 (corresponding to the probability that a merged cell is occupied and important).

[0096] According to this first merging strategy, it is sufficient for a cell to be considered important in any of the augmented data grids of the stack (i.e. at any altitude) for the resulting merged cell in the consolidated augmented data grid to be marked as important.

[0097] -- Second merger strategy

[0098] According to another specific embodiment corresponding to the second merging strategy mentioned above, the merging of grids in the augmented data grid stack is performed recursively, starting with a first grid in the stack and proceeding with successive merges two at a time until all the grids in the stack have been processed. In other words, a first grid in the stack is merged with another grid in the stack, then the resulting merged grid is itself merged with yet another grid in the stack, and so on as long as there are unmerged grids remaining in the stack. These recursive merges within the stack can be performed in any order, and any augmented data grid in the stack can be selected as the first grid.However, for the sake of clarity, a particular embodiment is described below by way of illustration only and not as a limitation, in which the merging of the stack grids is performed recursively, starting from a first grid corresponding to the GDAo augmented data grid associated with the lowest altitude, with subsequent augmented data grids in the stack, in ascending order of altitude. In other words, successive two-by-two mergings with the augmented data grid of the immediately higher altitude are performed, starting from the lowest-altitude augmented data grid, until reaching the top of the stack.

[0099] The augmented data grid GDAi is merged with the augmented data grid GDAo to deliver a first intermediate merged grid GDA 0+1 ;

[0100] The augmented data grid GDA2 is merged with the first intermediate merged grid GDA 0+1to deliver a second intermediate merged GDA grid 0+1+2 and so on, until the augmented GDA data grid n of higher altitude be merged with the n lème GDAo intermediate merged grid +i+ ... +n -i to deliver the consolidated merged augmented data grid (GDFC) for the entire PL stack.

[0101] In this context, according to a particular characteristic, the calculation of the probability distribution Pk+i(SI\Z) associated with a cell of a later iteration merged augmented data grid (i.e., of the next intermediate merged grid) from the previous iteration merged augmented data grid (i.e., of the previous intermediate merged grid)

[0102]

[0103] ... S k I k ~) is, for example, carried out according to the following sequence of operations:

[0104]

[0105] I 7 fc+ 1 = 1| )

[0106] b) P k+1 (S = 0 / = 1|Z) = P k S = 0 / = 1|Z) * (P (S k+1 = 0 / k+1 = l|Z) +

[0107] P( / k+1 = 0|Z) * (k — l) / k) + P k ( / = 0|Z) * P (S k+1 = 0 / k+1 = 1|Z) /

[0108] c) P k+1 (Z = 0) = 1 - P k+1 (S = 1 Z = 1|Z) - P k+1 (S = 01 = 1|Z),

[0109] this sequence of operations being repeated until the index k+1 is equal to n, that is to say it designates the index of the augmented data grid associated with the highest altitude of said stack of augmented data grids, the probability distributions Pk+i(S=l 1=11 Z) (corresponding to the probability that a merged cell is occupied and important), Pk+i(S=O 1=11 Z) (corresponding to the probability that a merged cell is unoccupied and important), and Pk+i(l=O) (corresponding to the probability that a merged cell is unimportant, regardless of its state of occupancy) finally obtained being those associated with the consolidated merged augmented data grid GDFC associated with the whole stack.

[0110] In this second merging strategy, the way in which the probability distributions of interest are merged differs from the first strategy. More specifically, according to this second strategy, the merging of the data of interest is defined differently depending on the occupancy state of the cells. Thus, according to this second strategy:

[0111] it is sufficient for a cell to be considered occupied and important in any of the augmented data grids of the stack (i.e. at any altitude) for the resulting merged cell in the consolidated augmented data grid to be marked as occupied and important;

[0112] If all corresponding cells are considered unoccupied in all augmented data grids of the stack, and some of these cells are considered important, the resulting merged cell in the consolidated augmented data grid is marked as unoccupied and important only in proportion to the number of augmented data grids of the stack in which the unoccupied cell is considered important.

[0113] 4 - Combined Fusion

[0114] In a particular embodiment, the fusion techniques previously described in relation to Figure 2 (fusion of augmented data grids associated with different data sources) on the one hand and with Figure 3 (fusion of augmented data grids associated with data delivered at different altitudes by the same data source) on the other hand are implemented in a complementary manner.

[0115] Thus, in the presence of different data sources (for example, ZI and Z2 data sources) each capable of delivering a stack of augmented data grids corresponding to a representation of the same scene at different altitudes, two approaches can be adopted to generate a single consolidated merged augmented data grid that is more easily usable than a multitude of grids.

[0116] According to a first approach, each stack of augmented data grids is processed using the augmented data grid fusion technique associated with data delivered at different altitudes by the same data source, as previously described in relation to Figure 3. This yields a consolidated merged augmented data grid for each data source. The augmented data grid fusion technique associated with different data sources, previously described in relation to Figure 2, is then used to merge the consolidated merged augmented data grids associated with each data source (e.g., to merge the consolidated grid obtained for data source Z1 and the one obtained for data source Z2), and generate a single merged augmented data grid.

[0117] According to a second, alternative approach, the augmented data grids associated with the same altitude in each stack are merged using the augmented data grid merging technique for data from different sources, previously described in relation to Figure 2. This results in a consolidated stack that can be considered as associated with the same sensor group Z (e.g., data source Z, composed of data sources Z1 and Z2). The augmented data grid merging technique for data delivered at different altitudes by the same data source, previously described in relation to Figure 3, is then used to process this consolidated stack and generate a single merged augmented data grid.

[0118] 5 - Devices

[0119] In another aspect, the proposed technique also relates to an electronic device for processing at least one data grid. As previously presented in relation to the processing method, said data grid is divided into cells, each cell being associated with a probability distribution of states of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined based on data reported at said current time step by at least one sensor of said sensor set and / or data predicted by a prediction model based on data reported at at least one previous time step by said at least one sensor.Such an electronic device includes means for determining, for at least one cell of the grid, at least one data point of interest, representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion in relation to a context of capturing the scene, a characteristic of the captured scene, and / or an application associated with said capturing, said means of determination delivering a grid of data called augmented.

[0120] Such an electronic device is also capable, because it has been configured in this way, of carrying out the processing method previously described in any of its embodiments.

[0121] We now present, in relation to figure 4, the simplified structure of an electronic device for processing at least one data grid, in a particular embodiment of the proposed technique.

[0122] The device, according to the proposed technique, comprises, for example, a memory 41 consisting of a buffer memory M, a processing unit 42, equipped, for example, with a microprocessor pP, and controlled by the computer program Pg 43, implementing steps of the process for processing at least one data grid, according to at least one embodiment of the invention. To this end, the electronic device also comprises, in a particular embodiment, at least one communication interface (for example, a wired or wireless communication interface) enabling it to receive and transmit data to and from other electronic equipment.Such communication interfaces allow, for example, receiving data from data sources such as different types of sensors, and sending data to actuators (for example, a braking or trajectory modification device, when the electronic device is on board an autonomous vehicle).

[0123] At initialization, the code instructions of the computer program 43 are loaded into the buffer memory before being executed by the processor of the processing unit 42. The processing unit 42 receives input E, for example, data from multiple sensors, on the basis of which at least one data grid representative of the captured scene is generated.

[0124] The microprocessor of the processing unit 42 then carries out the steps of the process of processing at least one data grid, according to the instructions of the computer program 43. More in particular, the processing unit 42 proceeds - in addition and possibly jointly with the determination of a probability distribution of states associated with an area of ​​the scene covered by a cell of the grid at a current time step - to the determination of at least one data said to be of interest for said cell, said data of interest being representative of a degree of interest of said cell with regard to at least one predetermined criterion in connection with a context of capturing the scene, a characteristic of the captured scene, and / or an application associated with said capturing.

[0125] The processed data grid is thus enriched with data of interest which complement the state data, thereby generating a data grid which is therefore described as augmented, which is delivered by the processing unit 42 at output S.

[0126] In various particular embodiments, the processing unit 42 is also configured to implement various fusion operations between data grids representing the same captured scene, according to modalities already previously presented in relation to the data grid processing method according to the present technique.

Claims

1. CLAIMS 1. A method for processing at least one data grid representing a scene captured by a set of sensors, said data grid being divided into cells (C1, C2, C3, C1), each cell being associated with a probability distribution of states (PD) of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined based on data reported at said current time step by at least one sensor of said sensor set and / or data predicted by a prediction model based on data reported at at least one previous time step by said at least one sensor, said method comprising determining, for at least one cell of the grid, at least one data point of interest (DP), representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion related to a scene capture context, a characteristic of the captured scene,and / or an application associated with said data capture, said determination delivering an augmented data grid (ADG).

2. A method according to claim 1, characterized in that said at least one predetermined criterion used to determine the degree of interest of a cell belongs to the group comprising at least:

4. a number and / or degree of reliability of the sensors used to determine the probability distribution of states associated with said cell at the current time step; 5. a nature of the data used to determine the probability distribution of states associated with said cell at the current time step, from predicted data or data reported from sensors; 6. prior knowledge of a particular characteristic of the scene area associated with said cell at the current time step; 7. a prediction of a trajectory of a dynamic object belonging to the scene, according to which said object is likely to be present in the area of ​​the scene associated with said cell at the current time step; 8. A particular application intended for the use of the probability distribution of states associated with said cell at the current time step.

3. A method according to claim 1, characterized in that said data of interest (DI) takes the form of a probability distribution of degrees of interest.

4. A method according to claim 3, characterized in that it comprises:

10. obtaining (21) a first augmented data grid (GDAI) for said scene, associated with the data provided at the current time step by a first data source (Zl); 11. obtaining (22) a second augmented data grid (GDA2) for said scene, associated with the data provided at the current time step by a second data source (Z2) distinct from the first data source (Z1); 12. the fusion (23) of said first and second augmented data grids, comprising the fusion of state probability distributions and the fusion of degree of interest probability distributions between corresponding cells of said first and second augmented data grids, said fusion delivering a fused augmented data grid (FDG).

5. A method according to claim 4, characterized in that said fusion is carried out according to the formula:

15.

6. A method according to claim 3, characterized in that it comprises:

18. Obtaining (31) a stack (PL) of augmented data grids (GDAo, GDAi, GDAk, GDA n) for said scene, associated with data retrieved from the same data source (Z) at a plurality of altitudes of said scene, each data grid augmented by the stack being associated with a given altitude (h) among said plurality of altitudes; 19. The merging (32) of the augmented data grids of said stack (PL), comprising the merging of state probability distributions and the merging of degree-of-interest probability distributions between corresponding cells of the set of augmented data grids, said merging delivering a consolidated augmented data grid (CDG) for said stack (PL).

7. A method according to claim 6, characterized in that said merging of the augmented data grids of said stack is performed by initializing a first variable PSG = P(S0= 0 I o = 0|Z) + P(S0= 0 I o = 1 \Z), a second variable PIO = P(S0= 0 I o = 0\Z) + P(S0= 1 I o= 0|Z) and a third variable PSOIO = P(S0= 0 I o = 0 |Z)7 then by performing a loop in which the augmented data grids of said stack are traversed sequentially to update, at each iteration of the loop, the first variable, the second variable and the third variable according to the following sequence of operations:

22. 24.c) PSOIO = PS0I0 * P(S k = 0 I k = 0|Z) 25. until all grids in the stack have been traversed, then return a consolidated augmented data grid for said stack defined by the third variable PSOIO, a variable PS1I0 = PI0 — PS0I0, a variable PS0I1 = PS0 — PS0I0, and a variable PS1I1 = 1 − PS1I0 − PS0I1 − PS0I0.

8. A method according to claim 6, characterized in that said merging of the augmented data grids of said stack is performed by recursive merging, starting from a first grid Po(SI | Z) chosen equal to the augmented data grid P(So law Z) associated with the lowest altitude of said stack, of the subsequent augmented data grids of said stack, via the calculation of a merged augmented data grid of a later iteration Pk+i(SI\Z) from the merged augmented data grid of the previous iteration P k (SI\Z) = 2s o ...s fc ( 5 o )| )... (5 k / k |Z)P(S / |S0 / 0... S k I k ) according to the following sequence of operations:

27.

28. 17 fc+i = 1| ) 29.b) P k+1 (S = 0 I = 1|Z) = P k S = 0 I = 1|Z) * (P (S k+1 = 0 / k+1 = l|Z) + P(7 k+1 = O|Z) * (Æ - l) ) + P k (7 = O|Z) * P (Sk+1 = 0 / k+1 = 1|Z) / 30.c) P k+1 ( / = 0) = 1 - P k+1 (S = 11 = 1|Z) - P k+1 (S = 01 = 1|Z), 31. until the index k+1 designates the index of the augmented data grid associated with the highest altitude of said stack of augmented data grids, and to return Pk+i(S=l 1=11 Z), Pk+i(S=O 1=11 Z), and Pk+i(l=O) as a consolidated augmented data grid for said stack.

9. A processing device for at least one data grid representing a scene captured by a set of sensors, said data grid being divided into cells, each cell being associated with a probability distribution of states of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined based on data reported at said current time step by at least one sensor of said sensor set and / or data predicted by a prediction model based on data reported at at least one previous time step by said at least one sensor.said device comprising means for determining, for at least one cell of the grid, at least one piece of data of interest, representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion related to a context of scene capture, a characteristic of the captured scene, and / or an application associated with said capture, said means of determination delivering a grid of data of so-called augmented data.

10. Product computer program downloadable from a communication network and / or stored on a computer-readable medium and / or executable by a microprocessor, characterized in that it includes program code instructions for the execution of a process according to any one of claims 1 to 8, when executed by a computer.

Images