Method for processing at least one grid of data representative of a scene captured by a set of sensors, device and corresponding program.

Abstract

Method for processing at least one data grid representing a scene captured by a set of sensors, device and corresponding program. The invention relates to a method for processing data grids representing a scene captured by sensors, a grid being divided into cells (C1, C2, C3, Ci) each associated with a probability distribution of states (PD) of an area of ​​the scene covered by the cell at a current time step, determined as a function of data reported by at least one of the sensors and / or data predicted by a prediction model as a function of data reported at a previous time step by said sensor.The process includes determining, for at least one cell of the grid, a data point of interest (DP) representative of the degree of interest of said cell with regard to predetermined criteria related to a scene capture context, a characteristic of the captured scene, and / or an application associated with said capture, said determination delivering an augmented data grid (ADG). Abbreviated figure: Figure 1.

Description

Title of the invention: Method for processing at least one grid of data representative of a scene captured by a set of sensors, device and corresponding program. technical field

[0001] The invention relates to the field of computer vision devices and autonomous devices in general that interact with their environment, such as a robot or an autonomous vehicle. For these devices, the ability to accurately perceive and model their environment in a relevant way is an essential task. Whether for navigation, collision awareness, intention planning, or mapping, this perception stage is challenging in terms of accuracy, complexity, and uncertainty management.

[0002] More specifically, the invention relates to techniques for processing one or more data grids representative of a scene captured by a set of sensors. Previous art

[0003] Despite impressive developments and the ever-increasing use of embedded intelligence in mobile devices, the ability of an autonomous agent to accurately, robustly, and efficiently perceive its environment remains a significant challenge in robotics. The quality of environmental modeling depends not only on sensors but also on interpretation schemes that address sensor errors, occlusions, data contradictions, highly complex parameters, and so on. Probabilistic methods have been developed to formally model uncertainties and prior knowledge within these interpretation schemes.

[0004] When these interpretation schemes are confronted with moving objects, numerous additional problems arise. A first, classic approach to addressing this problem is to adopt an object-based representation, which leads to tracking multiple objects. A second common approach is to adopt a representation based on the construction of data grids, and more specifically occupancy grids, which focuses on evaluating spatial occupancy without higher-level segmentation. More specifically, such an occupancy grid is generated at each time step by mapping sensor data onto the grid using a probabilistic sensor model, with each cell of the grid being associated, for example, after this mapping, with an occupancy probability distribution. This second approach has advantages Significant, such as a spatially dense model with an accurate representation of free space, which is important data in mobile robotics. Furthermore, the complex data segmentation and recognition steps required in object-based representations using the first approach can be avoided. When applied to dynamic environments, it is generally necessary to enrich this data grid representation by estimating velocity information in addition to occupancy estimates.

[0005] This second approach, however, also has drawbacks. For example, the data associated with each cell of a data grid is generally treated indiscriminately when the data grid is used for the needs of an application. In other words, all cells of a data grid are treated equally, including cells that do not necessarily require elaborate representation, for example, cells with undefined (i.e., unknown) content, empty (i.e., unoccupied) content, or static content, which means that computing resources are not always allocated optimally. The grid itself, in most applications, must be high-dimensional (i.e.(to be composed of a large number of cells) in order to be usable, with the consequence that many cells are often not reachable by sensors at every instant, for example due to obstructions. As a result, in potentially large areas, many cells are associated with data assigned exclusively on the basis of probabilities derived purely from prior knowledge (i.e., for which no data from sensors is available in the short term), and not on data estimated based on predictions made from previous and / or current sensor observations. Such a situation can lead to massive and unnecessary calculations, that is, a significant waste of computing power in unimportant areas, compromising the optimal functioning of otherwise well-suited systems.

[0006] Another central problem with this second approach relates to the inability to differentiate between probabilities resulting solely from a prior estimation (i.e., a prediction in the absence of data reported by the sensors at a current time step, which is therefore based on old or previous data) and those resulting from actually measured data (i.e., an observation reported by the sensors at the current time step). For example, knowing the probability distribution associated with a particular cell does not allow us to know whether this final distribution is the result of contradictory data delivered by sensors at the time step considered, or a result derived from knowledge of previous distributions in the absence of data reported by the sensors at the time step of time considered. In other words, estimates based on current data measured by sensors have the same status, or weight, as those based solely on predictions made on the basis of previous data.For example, if a previous distribution of cell occupancy indicates that the probability of that cell being occupied is 0.5, and a current distribution indicates that the probability of that cell being occupied is also 0.5, it is not currently possible, knowing only this value, to determine whether it results from the fact that no measured data is available for that cell at the current time step (in which case the old value, or prior estimate, of 0.5 has, for example, been retained for the current time step), or whether it results from the fact that two sensors have delivered contradictory information at the current time step regarding the probability of occupancy of that cell (in which case the fusion between these two contradictory pieces of information resulted in an occupancy probability of 0.5).It would nevertheless be desirable to be able to distinguish between these two cases in order to allow for differentiated subsequent processing (for example, to carry out a more in-depth investigation in the event of obtaining contradictory data, which is not necessarily useful to carry out in the case of a complete absence of data).

[0007] The limitations presented above can be explained by the Bayesian framework and the notational abuses commonly used on classical occupancy grids, in which the calculated probabilities are implicitly calculated in a common, unstated reference frame. In other words, all generated Bayesian probability estimates should refer to the reference frame in which they were calculated. The merging of different estimates must first reconcile the reference frames in question and modify the probability values ​​accordingly before it can be performed. In a single reference frame, relative confidence factors are implicitly carried over to each probability value, which thus incorporates prior knowledge, uncertainties related to sensors, subsequent modeling and processing, these relative confidence factors, and so on.While aggregating these different factors into a single value is certainly the desired objective, doing so directly no longer allows for specific processing of certain parts, for example, the optimization of calculations in distributions derived solely from a priori information, or from contradictory sensor data.

[0008] It is therefore necessary to have a solution that makes it possible in particular to solve this problem. Summary of the invention

[0009] The present invention proposes a solution to overcome certain drawbacks of the prior art. In one respect, the present invention relates to a A method for processing at least one data grid representing a scene captured by a set of sensors. Said data grid is divided into cells, each cell being associated with a probability distribution of states of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined based on data reported at said current time step by at least one sensor of said set of sensors and / or data predicted by a prediction model based on data reported at at least one previous time step by said at least one sensor.Such a process includes determining, for at least one cell of the grid, at least one data point of interest, representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion related to a scene capture context, a characteristic of the captured scene, and / or an application associated with said capture, said determination delivering a so-called augmented data grid.

[0010] In a particular embodiment, said at least one predetermined criterion used to determine the degree of interest of a cell belongs to the group comprising at least:

[0011] - a number and / or a degree of reliability of the sensors used to determine the probability distribution of states associated with said cell at the current time step;

[0012] - a nature of the data used to determine the probability distribution states associated with said cell at the current time step, among predicted data or data reported from sensors;

[0013] - prior knowledge of a particular characteristic of the scene area associated with said cell at the current time step;

[0014] - a prediction of a trajectory of a dynamic object belonging to the scene, according which said object is likely to be present in the area of ​​the scene associated with said cell at the current time step;

[0015] - a particular application intended for the use of the probability distribution of states associated with said cell at the current time step.

[0016] In a particular embodiment, said data of interest takes the form of a probability distribution of degrees of interest.

[0017] In a particular embodiment, said method further comprises:

[0018] - obtaining a first augmented data grid for said scene, associated to the data provided at the current time step by a first data source;

[0019] - obtaining a second augmented data grid for said scene, associated to the data provided at the current time step by a second data source distinct from the first data source;

[0020] - the merging of said first and second augmented data grids, comprising the merging of state probability distributions and the merging of distributions of probabilities of degree of interest between corresponding cells of said first and second augmented data grids, said fusion delivering a merged augmented data grid.

[0021] According to a particular feature, said fusion is carried out according to the formula:

[0022] ptsnzmsK^) P(SI\Z1Z2) = ----- pis=ski=ï)

[0023] In another particular, alternative or complementary embodiment, said process further comprises:

[0024] - obtaining a stack of augmented data grids for said scene, associated to data retrieved from the same data source at a plurality of altitudes of said scene, each data grid augmented by the stack being associated with a given altitude among said plurality of altitudes;

[0025] - the merging of the augmented data grids of said stack, comprising the merging state probability distributions and the merging of degree-of-interest probability distributions between corresponding cells of the set of augmented data grids, said merging delivering a consolidated augmented data grid for said stack.

[0026] According to a particular feature of this embodiment, said merging of the augmented data grids of said stack is carried out by initializing a first variable PS0 = P(S0 = 0 Zo = 0IZ) + P(SQ = 0 Zo = IIZ)' a second variable PIQ = P(So = 0 Zo = OZ) + P(S0 = s{ Zo = 1IZ) and a third variable PSQI0 = p(Sq = 0 Zo = OlZ)' Pu's by performing a loop in which the augmented data grids of said stack are traversed sequentially to update, at each iteration of the loop, the first variable, the second variable and the third variable according to the following sequence of operations:

[0027] a) Z*S0 = PS0* (P (Sk = 0 Ik = OlZ) + P (Sk = 0Ik = HZ))

[0028] b) PI0 = PI0*(P(Sk = QIk = (tZ) +P(sk=llk = $zy)

[0029] c) PS0IQ = PS0I0*P (Sk = 0Ik = $Z)

[0030] until the set of grids in the stack has been traversed, then return a consolidated augmented data grid for said stack defined by the third variable PS0I0, a variable PS1I0 - PI0 - PS0I(\ a variable PS0I1-PS0-PS0IQ, and a variable PSU1 = 1 - PSUQ - PSQIl - PS0IQ.

[0031] Alternatively, according to another particular feature of this embodiment, said merging of the augmented data grids of said stack is carried out by recursive merging, from a first grid P0(SIIZ) chosen equal to the augmented data grid P(S0 IqIZ) associated with the smallest altitude of said stack, of the following augmented data grids of said stack, via the calculation of a merged augmented data grid of a later iteration P k+1 (SI / Z) from the merged augmented data grid of the previous iteration P^SHZ) = >SP(SMZ) of the following operations:

[0032] a) pk+i(S= 11= = pk(s= 11= HZ) + ( 11= ^yyp(sk+i = 1 / Æ+1 = iiz)

[0033] b)Pk+{(S = QI= HZ) = Pk(S = 0I=U)^P(SM = ()Ik+^^)+P(Ik+^0Z)^kl) / k) +pk(i=iïz)*p (sk+l = oik+1=^ / k

[0034] c) Pk+l ( / - 0 ) = Pk+} ( S - 11 - HZ ) - = 01 = 1^)'

[0035] until the index k+1 designates the index of the augmented data grid associated with the highest altitude of said stack of augmented data grids, and to return Pk+i(S=l 1=1 | Z), Pk+i(S=0 1=1 | Z) and Pk+i(I=0) as a consolidated augmented data grid for said stack.

[0036] According to another aspect, the present technique relates to an electronic device for processing at least one data grid representing a scene captured by a set of sensors, said data grid being divided into cells, each cell being associated with a probability distribution of states of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined as a function of data reported at said current time step by at least one sensor of said set of sensors and / or of data predicted by a prediction model as a function of data reported at at least one previous time step by said at least one sensor.Such an electronic device includes means for determining, for at least one cell of the grid, at least one data point of interest, representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion related to a scene capture context, a characteristic of the captured scene, and / or an application associated with said capture, said means of determination delivering a so-called augmented data grid.

[0037] Such an electronic device may, of course, exhibit the various characteristics relating to the processing method according to the invention, which may be combined or considered separately. Thus, the characteristics and advantages of this device are the same as those of the processing method for at least one grid of data representative of a scene captured by a set of sensors, and are not described in further detail.

[0038] According to another aspect, the proposed invention also relates to a computer program product downloadable from a communication network and / or stored on a computer-readable medium and / or executable by a microprocessor, comprising program code instructions for the execution of a process for processing at least one data grid as described above in any of its embodiments, when this process is executed on a computer.

[0039] The proposed invention also relates to a computer-readable recording medium on which is recorded a computer program comprising program code instructions for executing the steps of a process as described above, in any of their embodiments.

[0040] Such a recording medium can be any entity or device capable of storing the program. For example, the medium can include a storage means, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording means, for example a USB flash drive or a hard drive.

[0041] On the other hand, such a recording medium can be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means, so that the computer program it contains is executable remotely. The program according to the invention can, in particular, be uploaded to a network, for example, the Internet.

[0042] The different embodiments mentioned above can be combined with each other for the implementation of the invention. Figures

[0043] Other features and advantages of the invention will become more apparent upon reading the following description of a particular embodiment, given by way of simple illustrative and non-limiting example, and the accompanying drawings, among which:

[0044] [Fig.1] presents an example of an augmented data grid, in a particular embodiment of the proposed technique;

[0045] [Fig.2] schematically presents the different stages of a fusion of augmented data grids associated with different data sources, in a particular embodiment of the proposed technique;

[0046] [Fig.3] schematically presents the different stages of a fusion of augmented data grids associated with data delivered at different altitudes by the same data source, in a particular embodiment of the proposed technique;

[0047] [Fig. 4] describes a simplified architecture of a device for processing at least one data grid, in a particular embodiment of the proposed technique. Detailed description of the invention 1 - General principle

[0048] The invention described below addresses some of the aforementioned drawbacks.

[0049] In a first aspect, the present invention relates to a method for processing at least one grid of data representative of a scene captured by a set of sensors. As a reminder, one objective of the so-called "Bayesian Occupancy Filter" techniques, within which the proposed solution can be applied, is to estimate the spatial occupancy and dynamics of an environment observed using different types of sensors. These sensors are, for example, mounted on a potentially moving autonomous device, typically a robot or an autonomous vehicle, and they aim to provide information enabling this device to interact with its environment (e.g., following a route on a road, avoiding collisions with obstacles or other road users, complying with traffic rules, etc.).In this context, as presented in relation to prior art, one or more data grids representing the scene are repeatedly generated, for example, at regular intervals according to a predetermined time step. More specifically, each data grid is divided into cells, each cell containing state data characterizing the area of ​​the scene covered by the cell at a given time step, thus forming a mesh of the scene. Such state data includes, for example, a probability distribution of states; the number and nature of the states considered can vary depending on the type of application, the expected degree of accuracy, and so on.Thus, in some contexts, a simple two-state representation of "cell occupied by an object" versus "cell not occupied by an object" may be sufficient, while more complex representations including multiple states may be more relevant in other contexts, with for example states of the type "cell occupied by a pedestrian", "cell occupied by a car", "free zone cell of road type", "free zone cell of pedestrian space type", "cell of undefined content". etc. State probabilities are determined based on data collected at the current time step by at least one sensor from the sensor array used to capture the scene and / or data predicted by a prediction model based on data collected at at least one time step prior to the current time step by at least some of the sensors (prior knowledge). The data grids are updated repeatedly and continuously, thus providing the autonomous device, in real-time or near-real-time, with a set of data relating to its environment, on the basis of which a decision-making body of this device can make decisions, for example, to change trajectory, brake, etc.

[0050] Figure 1 illustrates an example of such a GDA data grid, divided into cells C1, C2, C3, etc., in which each cell is associated with a DE probability distribution of states (four in the example of Figure 1: S1, S2, S3, and S4) of the scene area covered or associated with the cell in question, at a current time step. For clarity, only the data from one cell C1 of the GDA data grid are shown in Figure 1. Thus, the probabilities P(S1), P(S2), P(S3), and P(S4) indicate the probability that the scene area covered by cell C1 is in state S1, S2, S3, or S4, respectively.

[0051] As described in relation to the prior art, knowledge of the DE probability distribution of states associated with a cell is not, however, sufficient information in itself to determine an optimal way of considering this cell (i.e. to estimate its importance), whether in terms of, for example, the interest to be given to it, the computing power to be allocated to its processing, etc.

[0052] Accordingly, it is proposed, according to the general principle of the present technique, to associate at least one additional data point, called a data point of interest, with at least one cell of the data grid. More specifically, such a data point represents a degree of interest of the cell to which it is associated, with regard to at least one predetermined criterion related to a scene capture context, a characteristic of the captured scene, and / or an application associated with said capture. Examples of such criteria are presented later. Thus, in addition to the state data DE, cell Ci of [Fig. 1] (as well as other cells of the data grid GD, or even all the cells of this grid) is, for example, associated with a data point of interest DI, which indicates an importance and / or relevance (according to the predetermined criterion considered) that should be attributed to the state probability distribution associated with the cell.According to a particular characteristic, the data of interest takes the form of a probability distribution between several predetermined degrees of interest (or interest levels). Figure 1 thus illustrates a data of interest DI that takes the form of a probability distribution of degrees of interest between two predetermined interest levels in a binary mode, namely "cell not". "important" (1=0) and "important cell" (1=1) (it being understood that such an example is given purely for illustrative purposes and is not exhaustive, and that the number of degrees of interest considered can be greater, particularly to provide more nuance than a simple all-or-nothing approach). Thus, in [Fig. 1], the probabilities P(I=0) and P(I=1) indicate the probability that cell Ci should be considered, respectively, as unimportant or, conversely, particularly important with regard to certain criteria. Such information can then be used, for example, to determine the treatments to be applied to this cell. According to a particular characteristic, the DE probability distributions of states and the DI probability distributions of degrees of interest are not evaluated separately, but are instead estimated and stored jointly.

[0053] Determining the interest (e.g., the probability distribution of interest, ID) of a cell is based on one or more predetermined criteria. Depending on the selected criteria, the data of interest may, for example, represent a degree of confidence that can legitimately be placed in the state data associated with the cell in question, and / or a degree of relevance or importance of the cell with respect to a current context. Examples of such predetermined criteria are presented below; such criteria may be considered individually or in combination.

[0054] First, the determination of the data of interest may take into account the number of sensors and / or the degree of reliability of the sensors used to evaluate the probability distribution of states associated with the cell under consideration at the current time step. For example, less importance may be attributed to cells whose probability distribution of states is known to have been determined on the basis of data collected by a small number of sensors, or data collected by sensors whose measurements are considered less reliable than other types of sensors, for example.

[0055] Secondly, the determination of the data of interest may take into account the nature (or origin) of the data used to evaluate the distribution of state probabilities, depending for example on whether this data is data resulting from measurements made by sensors at the current time step (i.e. "observed" data), or on the contrary, data not measured, but predicted or extrapolated for the current time step from, for example, older measurements.

[0056] Third, the determination of the data of interest may take into account the intended application (i.e., the purpose of the data processing method) and / or the position of the cell in question. For example, prior knowledge of one or more particular characteristics of the geographical area of ​​the scene associated with the cell in question at the current time step, typically data Mapping data can reveal that information of interest is located in a particular area, and therefore, greater importance should be given to the state data associated with that cell. Projections based on tracking the dynamics of certain moving objects within the scene can also be considered when determining the data of interest. For example, this determination might take into account the predicted trajectory of a dynamic object within the scene, according to which that object is likely to be present in the area associated with the cell in question at the current time step, thus making that cell of interest.Similarly, the nature of the intended application can in itself be a criterion for determining the data of interest; for example, the implementation of an automatic braking system for a vehicle requires particular attention to cells that cover areas likely to be on the trajectory of that vehicle.

[0057] The above examples are of course given by way of illustration and are not exhaustive; other criteria than those listed may also be taken into account, in a complementary or alternative manner, when determining the degree of interest associated with a cell. Depending on a particular characteristic, several data points of interest of different kinds may also be associated with a cell (for example, a data point of interest representing a degree of reliability of the state data, another data point of interest representing a degree of importance of the cell with regard to the intended application, etc.).

[0058] It follows from the above that, unlike the states associated with a cell, which can be considered as information of an "absolute" nature, the degree of interest associated with a cell is information which can be described as "relative", in that it depends on the application and / or the observer (i.e. the sensors providing the metrics on the basis of which the data grids are at least partially constructed).

[0059] Such data of interest associated with the cells of a data grid are interesting in that they allow differentiation of the treatments to be applied to the different cells of the grid, for example by allocating more computing resources to the cells considered to be of interest.

[0060] The data grid enriched with this data of interest can therefore be described as an augmented data grid.

[0061] We are now interested in complementary aspects of the data grid processing method according to the proposed technique, in particular how different augmented data grids can be merged in order to obtain a consolidated resulting data grid that is easier to use.

[0062] In this context, three specific embodiments are presented below:

[0063] - a first embodiment relating to data grid merging augmented associated with different data sources (e.g., different sensors or groups of sensors);

[0064] - a second embodiment relating to data grid merging augmented associated with data delivered at different altitudes by the same data source (e.g., the same sensor or group of sensors);

[0065] - a third particular embodiment, combining grid fusions of data augmented according to the two particular embodiments previously mentioned.

[0066] 2 - Fusion of augmented data grids associated with data sources different

[0067] The steps for merging augmented data grids associated with different data sources are presented in relation to [Fig. 2], in a particular embodiment of the proposed technique. For the sake of simplification, the merging of two augmented data grids is described below, it being understood that merging a larger number of augmented data grids can always be decomposed into the implementation of a succession of mergings of two augmented data grids (for example, merging three augmented data grids can be carried out by successively merging the first two augmented data grids, then merging the resulting augmented data grid with the third augmented data grid).

[0068] In this framework, the data grid processing method comprises a step 21 of obtaining a first augmented data grid GDA1 associated with data from a first data source Z1 (for example, a first sensor or group of sensors), and a step 22 of obtaining a second augmented data grid GDA2 associated with data from a second data source Z2 (for example, a second sensor or group of sensors) distinct from the first data source Z1. The augmented data grids GDA1 and GDA2 correspond to two representations of the same scene, obtained for the same current time step. Depending on the data sources Z1 and Z2 considered, the augmented data grids GDA1 and GDA2 obtained are directly of identical dimensions.Alternatively, when this is not the case, at least one of the two augmented data grids GDA1 and GDA2 undergoes preliminary processing, typically a truncation and / or completion operation, so that the augmented data grids GDA1 and GDA2 ultimately obtained at the end of steps 21 and 22 are of identical dimensions; that is, each cell in one of the two grids corresponds to a cell in the other grid associated with the same area of ​​the captured scene. More specifically, a truncation operation corresponds to the deletion.

[0069]

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] A truncation operation involves removing one or more cells from a data grid, while a completion operation involves adding one or more cells to a data grid. Cells removed during a truncation operation are, whenever possible, associated with data of interest that is representative of low importance and / or low reliability (i.e., cells of low importance). Cells added during a completion operation are intentionally associated, fictitiously, with data of interest that is representative of low importance and / or low reliability (i.e., the added cells are deliberately marked as low importance). In this way, the impact of these cell additions and / or deletions, which are solely intended to obtain augmented data grids GDA1 and GDA2 of similar size, is minimized during subsequent grid merging operations.At the end of steps 21 and 22, the augmented data grids GDA1 and GDA2 therefore include - possibly after at least one truncation and / or completion operation of at least one of the two grids GDA1 and / or GDA2 as previously mentioned - the same number of cells, and corresponding cells from these two grids (i.e. cells in the same position in each of the two grids, for example the cells marked with an "x" on [Fig.2]) cover the same area of ​​the captured scene. The merging of the two augmented data grids GDA1 and GDA2 is implemented in a step 23, and includes merging the state probability distributions and merging the degree-of-interest probability distributions between corresponding cells of said first and second augmented data grids, said merging delivering a merged augmented data grid GDF. According to a particular characteristic, the fusion is performed by applying a Bayesian fusion mechanism, according to a formula equivalent to the following formula: P(SÜZIZ2)=---- ) i=o. i Hfcs.i M In this example, for the sake of simplicity, a two-state, two-degree-of-interest representation is adopted. Thus: - Regarding the state data, a cell can be "occupied" (S=1) or "unoccupied" (S=0); - Regarding the data of interest, a cell can be "important" (1=1) or "not important" (1=0). Of course, this example, deliberately simplified for clarity, is not exhaustive, and representations with larger numbers of states (i.e., beyond two states) or degrees of interest (i.e., beyond two degrees of interest) exist. can also be considered without leaving the scope of this technique, with a formula of the type previously presented adapted to these more complex representations.

[0076] More specifically, this formula defines particularly advantageous mechanisms for propagating the data of interest I, according to which:

[0077] - the merging of data from corresponding cells considered to be very probably unimportant (1=0) in all augmented data grids before merging (i.e., both in GDA1 and GDA2) results in a cell that is marked as unimportant in the resulting merged augmented data grid GDF;

[0078] - any potential interest of a cell observed in at least one of the grids of augmented data before merging (i.e. in GDA1 or GDA2) emerges strengthened from the merging process, in that it leads to a cell marked as even more important in the resulting merged augmented GDF grid, in other words, P(I = 1IZ1Z2) > MAX(P(I = 1IZ1),P(I = 1IZ2)).

[0079] 3 - Fusion of augmented data grids associated with data delivered to different altitudes from the same data source

[0080] The steps of merging augmented data grids associated with data delivered at different altitudes by the same data source are presented in relation to [Fig.3], in a particular embodiment of the proposed technique.

[0081] In this context, the data grid processing method includes a step 31 of obtaining a stack PL of augmented data grids (GDA0, GDAi, ..., GDAk, ..., GDAn) associated with data provided by the same data source Z, representative of the scene at different altitudes h, for the same current time step. Such data can, for example, be obtained via the use of laser rangefinder type sensors. More particularly, each augmented data grid in the stack is associated with a known altitude among a plurality of altitudes associated with the scene. In the example of [Fig.[2], the PL stack comprises n + 1 augmented data grids (where n is an integer greater than or equal to 1), the lowest augmented data grid GDA0 of the stack being associated with data uplifted to ground level by the data source Z, and the highest grid GDAn of the stack being associated with data uplifted to the highest altitude h at which the data source Z is capable of delivering data. In the example in [Fig. 2], each grid of the PL stack is represented as a plane, with the altitude h corresponding, for example, to an absolute altitude relative to a reference altitude such as sea level. It should be noted, however, that in a particular embodiment (not shown), the augmented data grids of the stack do not. These grids are not analogous to a plane, but rather conform to the contours of the underlying ground. In other words, according to a particular characteristic, the altitude h is not an absolute altitude relative to a reference altitude, but an altitude relative to the position (i.e., the height) of the ground within the area of ​​the scene covered by each grid cell. Regardless of the embodiment considered (absolute or relative altitude), a stack-augmented data grid remains associated with an altitude h among a plurality of altitudes. Stack-augmented data grids all correspond to representations of the same scene, obtained for the same current time step. Depending on the data source Z considered, the resulting stack-augmented data grids PL are directly of identical dimensions.Alternatively, when this is not the case, at least one of the augmented data grids in the stack may undergo preprocessing, typically a truncation and / or completion operation as previously described in relation to [Fig. 2], so that all the augmented data grids in the stack ultimately obtained at the end of step 31 are of identical dimensions; that is, each cell in any of the grids in the stack corresponds to a cell in each of the other grids in the stack associated with the same area of ​​the captured scene. More specifically, a truncation operation corresponds to the removal of one or more cells from a data grid, while a completion operation corresponds to the addition of one or more cells to a data grid.Cells removed during a truncation operation are, as far as possible, associated with representative data of low importance and / or low reliability (i.e., low-importance cells). Cells added during a completion operation are intentionally and fictitiously associated with representative data of low importance and / or low reliability (i.e., the added cells are deliberately marked as low-importance). In this way, the impact of these cell additions and / or deletions, which are solely intended to obtain augmented data grids of similar size throughout the PL stack, is minimized during subsequent grid merging operations.At the end of step 31, the augmented data grids of the stack therefore comprise the same number of cells, and corresponding cells in these grids (i.e., cells in the same position in each of the two grids, for example, cells marked with an "x" in [Fig. 3]) cover the same area of ​​the scene but at different altitudes. In other words, these corresponding cells allow for the analysis of the contents of the "air column" vertically above the associated area of ​​the scene, in other words, a three-dimensional representation. This notably improves the detection of certain types of obstacles (e.g., a bridge deck, panels on a gantry, etc.). in which the use of space is, for example, different between an altitude close to ground level and a higher altitude.

[0082] The merging of the augmented data grids of the stack is implemented in a step 32, and includes merging the probability distributions of states and merging the probability distributions of degree of interest between corresponding cells of the set of augmented data grids, said merging delivering a consolidated merged augmented data grid GDFC for the entire PL stack.

[0083] It should be noted that, unlike the augmented data grid fusion technique described earlier in relation to [Fig. 2] for data associated with different data sources, for which a basic assumption is the independence of data from different data sources within each cell, data from the same data source at different altitudes are clearly dependent. For example, if all the data for a cell are concentrated within a narrow range of altitudes, the probability distributions of state (i.e., occupancy) according to this data are clearly correlated with this range of altitudes.

[0084] Two strategies for merging grids of a stack of augmented data grids are presented below, aimed at obtaining a consolidated merged augmented data grid associated with the entire stack.

[0085] Again, in the following description of these two strategies, specific embodiments of which are given by way of illustration and not limitation, a two-state, two-degree-of-interest representation is adopted for the sake of simplification. Thus:

[0086] - Regarding state data, a cell can be "occupied" (S=l) or "not occupied » (S=0);

[0087] - with regard to the data of interest, a cell can be "important" (1=1) or "not important » (1=0).

[0088] Of course, this example, deliberately simplified for the sake of clarity, is not limiting, and representations with larger numbers of states (i.e. beyond two states) or degrees of interest (i.e. beyond two degrees of interest) can also be considered without going out of the scope of the present technique, with mechanisms and formulas of the type presented below adapted to these more complex representations.

[0089] Considering the set of corresponding cells in the augmented data grid stack, that is, the set of cells located at different altitudes (i.e., vertically) of the same ground area of ​​the scene (for example, all cells marked with an "x" in [Fig. 3]), these two strategies have in common that it suffices for a cell to be considered occupied in any one of the augmented data grids of the stack (i.e., in any altitude) so that the resulting merged cell (e.g. the cell marked with an "r" on [Fig.3], resulting from the merging of all cells marked with an "x") in the consolidated augmented data grid is marked as occupied.

[0090] These two fusion strategies differ, however, in the handling of the fusion of the data of interest, as presented below.

[0091] - First merger strategy

[0092] According to a particular embodiment corresponding to the first fusion strategy mentioned above, the fusion of the grids in the stack of augmented data grids is carried out by initializing a first variable PS® = = 0 Jo = 01Z) + p(.Sq = 0Iq= HZ)' a second variable PIQ = P(SQ = 0 IQ = OZ) + P(S0 = Si Io = ^Z) and a third variable PSQIQ = P(SQ = 0 Iq — Olz)' Pu's by performing a loop in which the augmented data grids of said stack are traversed sequentially to update the first variable, the second variable and the third variable according to the following sequence of operations, with k the index of the augmented data grid traversed at the considered iteration of the loop:

[0093] a) PSO = PSO* (P (Sk = 0 4 = OlZ) + P (Sk = 0 Ik = HZ))

[0094] b) P / 0 = P / 0* (P (Sk = 0 Ik = OlZ) + P (S^ = 1 Ik = OlZ))

[0095] c) PS0I0 = PS0I0*P (Sk = 0Ik = $Z)

[0096] Once the loop is completed, that is, once all the augmented data grids have been traversed, the probability distributions associated with the consolidated merged augmented data grid GDFC are defined by the third variable PSOZO (corresponding to the probability that a merged cell is unoccupied and unimportant), a variable PS1Z0 = PIQ-PSQIQ (corresponding to the probability that a merged cell is occupied and unimportant), a variable PS0I1 = PSO-PSQIQ (corresponding to the probability that a merged cell is unoccupied and important), and a variable PSIIl- 1-PSII0-PS0II-PSQIQ (corresponding to the probability that a merged cell is occupied and important).

[0097] According to this first merging strategy, it is sufficient for a cell to be considered important in any of the augmented data grids of the stack (i.e. at any altitude) for the resulting merged cell in the consolidated augmented data grid to be marked as important.

[0098] - Second merger strategy

[0099] According to another particular embodiment corresponding to the second merging strategy mentioned above, the merging of grids in the augmented data grid stack is performed recursively, starting with a first grid in the stack and proceeding with successive merges two by two until all the grids in the stack have been processed. In other words, a first grid in the stack is merged with another grid in the stack, then the resulting merged grid is itself merged with yet another grid in the stack, and so on as long as there are unmerged grids remaining in the stack. These recursive merges within the stack can be performed in any order, and any augmented data grid in the stack can be selected as the first grid.However, for the sake of clarity, a particular embodiment is described below by way of illustration only and not limitation, in which the merging of the stack grids is performed by recursive merging, starting from a first grid corresponding to the augmented data grid GDA0 associated with the lowest altitude, of the subsequent augmented data grids in the stack, by increasing altitude. In other words, successive two-by-two mergings with the augmented data grid of the immediately higher altitude are performed starting from the augmented data grid of the lowest altitude, until reaching the top of the stack: .

[0100] - the augmented data grid GDAi is merged with the data grid augmented GDA0 to deliver a first intermediate merged grid GDA0+i;

[0101] - the augmented data grid GDA2 is merged with the first grid intermediate fused GDA0+i to deliver a second intermediate fused grid GDA0+i +2;

[0102] - and so on, until the higher augmented GDAn data grid altitude is merged with the nth intermediate merged grid GDA0+i / to deliver the consolidated merged augmented data grid GDFC for the entire PL stack.

[0103] In this context, according to a particular feature, the calculation of the probability distribution P k+1 (SI / Z) associated with a cell of a later iteration merged augmented data grid (i.e. of the next intermediate merged grid) from the previous iteration merged augmented data grid (i.e. of the previous intermediate merged grid) P„(SBZ) = ^.SP(SMZ) ...P(S^P^,,...S„Ik) “t for example realized according to the following sequence of operations:

[0104] a) Pk+i(S = i I = i\Z) = Pk(S=]I=l\Z) + (]-Pk(S=\I=l^ = 14+1=K)

[0105] b)Pk+l(S = 0I = HZ) = pk(S=oi=$Z)*(p (SU1 = 0 4+1=iiz)+p( ik+l=azr(k- i) / k) +P*( / = OIZ)*P (Sw = QIk+l=l£) / k

[0106] c) P*+1( / = 0) = Pt^S = U=tZ)-p„Js = oi=&)’

[0107] this sequence of operations being repeated until the index k+1 is equal to n, that is to say it designates the index of the augmented data grid associated with the highest altitude of said stack of augmented data grids, the probability distributions Pk+i(S=1 1=1 | Z) (corresponding to the probability that a merged cell is occupied and important), Pk+i(S=0 1=1 | Z) (corresponding to the probability that a merged cell is unoccupied and important), and Pk+i(I=0) (corresponding to the probability that a merged cell is unimportant, regardless of its state of occupancy) finally obtained being those associated with the consolidated merged augmented data grid GDFC associated with the whole stack.

[0108] In this second merging strategy, the way in which the probability distributions of interest are merged is defined differently from the first strategy. More specifically, according to this second strategy, the merging of the data of interest is defined differently depending on the occupancy state of the cells. Thus, according to this second strategy:

[0109] - it is sufficient that a cell be considered occupied and important in one any of the augmented data grids in the stack (i.e., at any altitude) so that the resulting merged cell in the consolidated augmented data grid is marked as busy and important;

[0110] - if all corresponding cells are considered unoccupied in all augmented data grids in the stack, and some of these cells are considered important, the resulting merged cell in the consolidated augmented data grid is marked as unoccupied and important only in proportion to the number of augmented data grids in the stack in which the unoccupied cell is considered important. 4 - Combined Fusion

[0111] In a particular embodiment, the fusion techniques previously described in relation to [Fig.2] (fusion of augmented data grids associated with different data sources) on the one hand and with [Fig.3] (fusion of augmented data grids associated with data delivered at different altitudes by the same data source) on the other hand are implemented in a complementary manner.

[0112] Thus, in the presence of different data sources (for example, data sources Z1 and Z2), each capable of delivering a stack of augmented data grids corresponding to a representation of the same scene at different altitudes, two approaches can be adopted in order to generate a single augmented merged consolidated data grid that is more easily usable than a multitude of grids.

[0113] According to a first approach, each stack of augmented data grids is processed using the augmented data grid fusion technique associated with data delivered at different altitudes by the same data source, as previously described in relation to [Fig. 3]. This yields a consolidated merged augmented data grid for each data source. The augmented data grid fusion technique associated with different data sources, previously described in relation to [Fig. 2], is then used to merge the consolidated merged augmented data grids associated with each data source (e.g., to merge the consolidated grid obtained for data source Z1 and that obtained for data source Z2), and generate a single merged augmented data grid.

[0114] According to a second, alternative approach, the augmented data grids associated with the same altitude in each stack are merged using the augmented data grid merging technique associated with different data sources previously described in relation to [Fig. 2]. This results in a consolidated stack that can be considered as associated with the same sensor group Z (e.g., data source Z, composed of data sources Z1 and Z2). The augmented data grid merging technique associated with data delivered at different altitudes by the same data source, previously described in relation to [Fig. 3], is then used to process this consolidated stack and generate a single merged augmented data grid. 5 - Devices

[0115] In another aspect, the proposed technique also relates to an electronic device for processing at least one data grid. As previously presented in relation to the processing method, said data grid is divided into cells, each cell being associated with a probability distribution of states of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined based on data reported at said current time step by at least one sensor of said sensor set and / or data predicted by a prediction model based on data reported at at least one previous time step by said at least one sensor.Such an electronic device includes means for determining, for at least one cell of the grid, at least one data point of interest, representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion in relation to. a scene capture context, a characteristic of the captured scene, and / or an application associated with said capture, said means of determination delivering a so-called augmented data grid.

[0116] Such an electronic device is also capable, because it has been configured in this way, of carrying out the processing method previously described in any of its embodiments.

[0117] We now present, in relation to [Fig.4] the simplified structure of an electronic device for processing at least one data grid, in a particular embodiment of the proposed technique.

[0118] The device, according to the proposed technique, comprises, for example, a memory 41 consisting of a buffer memory M, a processing unit 42, equipped, for example, with a microprocessor qP, and controlled by the computer program Pg 43, implementing steps of the process for processing at least one data grid, according to at least one embodiment of the invention. To this end, the electronic device also comprises, in a particular embodiment, at least one communication interface (for example, a wired or wireless communication interface) enabling it to receive and transmit data to and from other electronic equipment.Such communication interfaces allow, for example, receiving data from data sources such as different types of sensors, and transmitting data to actuators (for example, a braking or trajectory modification device, when the electronic device is on board an autonomous vehicle).

[0119] At initialization, the code instructions of the computer program 43 are loaded into the buffer memory before being executed by the processor of the processing unit 42. The processing unit 42 receives as input E, for example, data from multiple sensors, on the basis of which at least one data grid representative of the captured scene is generated.

[0120] The microprocessor of the processing unit 42 then carries out the steps of the process of processing at least one data grid, according to the instructions of the computer program 43. More particularly, the processing unit 42 proceeds - in addition and possibly jointly with the determination of a probability distribution of states associated with an area of ​​the scene covered by a cell of the grid at a current time step - to the determination of at least one data said to be of interest for said cell, said data of interest being representative of a degree of interest of said cell with regard to at least one predetermined criterion in connection with a context of capturing the scene, a characteristic of the captured scene, and / or an application associated with said capturing.

[0121] The processed data grid is thus enriched with data of interest which complement the state data, thereby generating a data grid which is therefore qualified as augmented, which is delivered by the processing unit 42 at output S.

[0122] In various particular embodiments, the processing unit S is further configured to implement various fusion operations between data grids representing the same captured scene, according to modalities already previously presented in relation to the data grid processing method according to the present technique.

Claims

1.

2. Demands A method for processing at least one data grid representing a scene captured by a set of sensors, said data grid being divided into cells (C1, C2, C3, C1), each cell being associated with a probability distribution of states (PD) of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined based on data reported at said current time step by at least one sensor of said sensor set and / or data predicted by a prediction model based on data reported at at least one previous time step by said at least one sensor, said method comprising determining, for at least one cell of the grid, at least one data point of interest (DP), representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion related to a scene capture context, a characteristic of the captured scene,and / or an application associated with said data capture, said determination delivering an augmented data grid (ADG). Method according to claim 1, characterized in that said at least one predetermined criterion used to determine the degree of interest of a cell belongs to the group comprising at least: - a number and / or a degree of reliability of the sensors used to determine the probability distribution of states associated with said cell at the current time step; - the nature of the data used to determine the probability distribution of states associated with said cell at the current time step, from predicted data or data reported from sensors; - prior knowledge of a particular characteristic of the scene area associated with said cell at the current time step; - a prediction of a trajectory of a dynamic object belonging to the scene, according to which said object is likely to be present in the scene area associated with said cell at the current time step; - a particular application intended for the use of the probability distribution of states associated with said cell at the current time step.

3. Method according to claim 1, characterized in that said data of interest (DI) takes the form of a probability distribution of degrees of interest.

4. A method according to claim 3, characterized in that it comprises: - obtaining (21) a first augmented data grid (ADG1) for said scene, associated with the data provided at the current time step by a first data source (Z1); - obtaining (22) a second augmented data grid (ADG2) for said scene, associated with the data provided at the current time step by a second data source (Z2) distinct from the first data source (Z1); - merging (23) said first and second augmented data grids, comprising merging the state probability distributions and merging the degree-of-interest probability distributions between corresponding cells of said first and second augmented data grids, said merging delivering a merged augmented data grid (GDF).

5. The method according to claim 4, characterized in that said fusion is carried out according to the formula: , Pi Kl] P(SI\Z\Z2) — „ 1^.2) ] ,=ü. î p(s=sk

6. A method according to claim 3, characterized in that it comprises: - obtaining (31) a stack (PL) of augmented data grids (GDA0, GDAi, GDAk, GDAn) for said scene, associated with data brought up by the same data source (Z) at a plurality of altitudes of said scene, each augmented data grid of the stack being associated with a given altitude (h) among said plurality of altitudes; - merging (32) the augmented data grids of said stack (PL), comprising merging the probability distributions of states and merging the probability distributions of degree of interest between corresponding cells of the set of augmented data grids, said merging delivering a consolidated augmented data grid (GDFC) for said stack (PL).

7. The method according to claim 6, characterized in that said merging of the augmented data grids of said stack is performed by initializing a first variable PSO = P(S0 = 0 Zo = OlZ) + p(so = 0 Io = llz)' a second variable PIO = P(Sq = 0 Io = Oiz) + P(SQ = St Io = llz) and a third variable p$QIQ — p(,S0 = 0 Zo = ÛZ)' Pu's by performing a loop in which the data grids augmented by said stack are traversed sequentially to update, at each iteration of the loop, the first variable, the second variable and the third variable according to the following sequence of operations: a) PSO = PSO* ( P ( Sk = 0 Ik = OiZ ) + P ( Sk = 0 Ik = 11Z ) ) b) PZO = PZO* ( P ( Sk = 0 Ik = Olz ) + P ( Sk = 1 Ik = OIZ ) ) c) PSOZO = PS0Z0*P ( Sk = 0 Ik = Olz ) until all grids in the stack have been traversed, then return a consolidated augmented data grid for said stack defined by the third variable PS0I0, a variable PS 1Z0 = PZO - PSOZO, a variable PS0Z1 = PSO - PSOZO, and a variable PS 1Z1 = 1-PS1Z0-PS0Z1-PSOZO.

8. The method according to claim 6, characterized in that said merging of the augmented data grids of said stack is performed by recursive merging, starting from a first grid P0(SIIZ) chosen equal to the augmented data grid P(S0 I0IZ) associated with the lowest altitude of said stack, of the subsequent augmented data grids of said stack, via the calculation of a merged augmented data grid of a later iteration Pk(StZ) = ...PCS / ilZ)^ / ».-^) according to the following sequence of operations: a) pk^$ = । r= llz) = pk(s= i / = Hz) + (}-pk(s = ]f= iiz) y*p(sk+] = i ik+l= llz) b)P / c+1(S = 0Z=lZ) = pk(s=oi=sz)^p (sM = o= sz) +p( iM = azynk-1) / k) +pyi=az)*p (sk+, = o / t+1=iiz) ik c) Pt^7=0)= ^,(8=17=12)^(5 = 07 = 1^} until the index k+1 designates the index of the augmented data grid associated with the highest altitude of said stack of augmented data grids, and to return Pk+i(S=l 1=1 | Z), Pk+i (S=0 1=1 | Z) and Pk+i(I=0) as a consolidated augmented data grid for said stack.

9. A device for processing at least one data grid representing a scene captured by a set of sensors, said data grid being divided into cells, each cell being associated with a probability distribution of states of an area of ​​said scene covered by said cell at a current time step, said state probabilities being determined based on data reported at said current time step by at least one sensor of said sensor set and / or data predicted by a prediction model based on data reported at at least one previous time step by said at least one sensor, said device comprising means for determining, for at least one cell of the grid, at least one data point of interest, representative of a degree of interest of said at least one cell with regard to at least one predetermined criterion related to a scene capture context, a characteristic of the captured scene,and / or an application associated with said data capture, said determination methods providing an augmented data grid.

10. Product computer program downloadable from a communication network and / or stored on a computer-readable medium and / or executable by a microprocessor, characterized in that it includes program code instructions for the execution of a process according to any one of claims 1 to 8, when executed by a computer.

Images