Method for annotating a time-stamped sequence of recorded data from a vehicle's driving environment

Abstract

The invention relates to a method for annotating a time-stamped sequence of recorded data from a vehicle's driving environment in order to identify interaction between said vehicle and an agent present in said data, comprising the steps of: Determining (200) a trajectory of said vehicle; Determining (210) a trajectory of said agent; Determining (220) a series of distances, referred to as distances, such that, for each position of said vehicle, a distance is determined between the position of said vehicle and said trajectory of said agent; Determining (230) a position of said vehicle, referred to as the conflict position, based on said trajectory of said vehicle, on said distances, and on a first predetermined distance threshold, one of the distances being less than said first threshold; Annotating said sequence and / or said agent in said sequence. Figure to be published for the abstract: Figure 2

Description

Title of the invention: Method for annotating a time-stamped sequence of recorded data from a vehicle driving environment Technical field of the invention

[0001] The invention is in the field of machine learning for autonomous vehicle driving assistance system models. In particular, the invention relates to a method for annotating a time-stamped sequence of recorded data from a driving environment of a vehicle in order to identify an interactivity between said vehicle and an agent present in said data. State of the art

[0002] The term "vehicle" means any type of vehicle such as a motor vehicle, a moped, a motorcycle, a storage robot in a warehouse, etc. The term "autonomous driving" of a "vehicle" means any process capable of assisting the driving of the vehicle; the "vehicle" is then also called an "autonomous vehicle". The process may thus consist of partially or totally steering the vehicle or providing any type of assistance to a natural person driving the vehicle. The process thus covers all autonomous driving, from level 0 to level 5 in the OICA scale, for International Organization of Motor Vehicle Manufacturers.

[0003] The term "vehicle driving environment" means anything that can have an impact on the driving of the vehicle, and which is generally close to, measurable by a sensor or determinable by, the vehicle. This may be a characteristic of a road (of the roadway, the number of lanes, the road markings, etc.), a characteristic of road signs, a characteristic of an agent present in the environment of the vehicle, etc. Also, this may be information related to a state of the vehicle (position, speed, etc.) or to a function of a device of the vehicle.

[0004] The term “agent” means a device or a person evolving in the environment of the vehicle. For example, an agent is another vehicle, a pedestrian, a cyclist, a truck, etc. Among the characteristics of the agent, we can have a position, a speed, metadata, etc.

[0005] Interactivity between the vehicle and an agent is understood to mean antagonism, opposition, or priority management between the vehicle and the agent. For example, there is interactivity when the agent, like another vehicle, changes its speed to give way to the vehicle. In this example, the interactivity is strong if the agent had to brake or accelerate sharply to avoid a collision with the vehicle. In this example, the interactivity is weak if the agent had to slightly modify its speed or trajectory. to avoid a collision with the vehicle. In this example, interactivity is low if an agent trajectory does not approach a vehicle trajectory.

[0006] The term "time-stamped sequence of data recorded from a vehicle driving environment", also referred to below as "data sequence", means a set of data, over a predetermined duration, regularly or periodically, comprising information related to the vehicle driving environment. This data includes characteristics of the agents present in the vehicle environment. A predetermined duration may last 1 second, 4 seconds, 15 seconds, other values ​​are possible. A duration of a sequence may depend on the speed of the vehicle.

[0007] “Regularly or periodically” means that the information is recorded several times, at several times, over time. For example, information is recorded periodically every 1 ms (milliseconds), 10 ms, 0.1 s, other values ​​are possible. Information can be recorded not periodically, on event, but regularly at least, for example at most every second, other values ​​are possible. Timestamping allows a time (date, time, ...), an instant, to be associated with a recording.

[0008] Thus, the data sequence makes it possible to find a temporal evolution of the positions of the vehicle and the agents.

[0009] Methods capable of assisting the driving of the vehicle are also called AD AS (from the English acronym "Advanced Driver Assistance Systems"), AD AS systems or driving assistance systems. The development of these systems, in particular those based on artificial intelligence such as the construction of mathematical models based on machine learning, requires numerous data sequences, or data.

[0010] Machine learning is a general term encompassing a variety of algorithmic approaches to learning patterns from data. An artificial neural network, or neural network, is a subclass of machine learning algorithms.

[0011] The data, or data sequences, for the development may be collected from public data sets, commercial data sets, or be generated from driving or simulations. By data set is meant a plurality of data sequences.

[0012] After cleaning (removing noise, aberrant data, etc.) and formatting (normalization, expression relative to a reference point, etc.) the data, we can create data sets such as a training set (dataset), a validation set (dataset), and a test set (dataset).

[0013] One of the difficulties is to create relevant data sets (including the dif different life situations and just as necessary) depending on the purpose of the model. It is then known to annotate / label a sequence of data to be able to quickly select and build, in relation to the immense number of data, a relevant dataset. An annotation is an addition of metadata characterizing the sequence or an agent of the sequence. Unfortunately, this annotation is generally carried out manually and consumes a lot of time and resources. Summary of the invention

[0014] An object of the present invention is to remedy the aforementioned problem, in particular, faced with the multitude of data, to annotate data sequences to identify an interaction between a vehicle and an agent in order to quickly and efficiently construct and parameterize driving assistance models, these models subsequently being used in a vehicle.

[0015] To this end, a first aspect of the invention relates to a method for annotating a time-stamped sequence of recorded data from a driving environment of a vehicle in order to identify an interactivity between said vehicle and an agent present in said data, said method being implemented by a processor and comprising the steps of: • Determination of a trajectory of said vehicle from said sequence, said trajectory determining a succession of positions of said vehicle; • Determination of a trajectory of said agent from said sequence, said trajectory determining a succession of positions of said agent; • Determination of a succession of distances, called distances, such that, for each position of said vehicle, a distance is determined between the position of said vehicle and said trajectory of said agent; • Determination of a position of said vehicle, called conflict position, based on said trajectory of said vehicle, on said distances and on a first predetermined distance threshold, one of the distances being less than said first threshold; • Annotation of said sequence and / or said agent in said sequence.

[0016] Thus, from a data sequence, during the duration of the sequence, a situation in which a trajectory of the agent is close to a trajectory of the vehicle is identified. This situation is identified even if, temporally, at a given instant, the positions of the vehicle and the agent are not close. The distance is calculated relative to a position of the vehicle at a given instant and the entire trajectory of the agent. This makes it possible to identify that an action, such as a change of speed or a change of direction, past or future, relative to the position of the vehicle at a given instant, of the agent or the vehicle has been carried out to avoid, by example, to come into contact with the vehicle or the agent. In this sequence, an interactivity between the vehicle and the agent has been identified. The sequence is then annotated, it can be easily selected to build a training, testing, and / or validation dataset to build a model used by a driver assistance system.

[0017] Also, these steps allow for greater efficiency in terms of calculations and calculation times, particularly in the presence of several agents in the database. This avoids calculating trajectory extrapolations of the agents for each instant of the sequence. In the presence of several agents in the sequence, this allows for rapid selection of only the agents that have interactivity with the vehicle.

[0018] Advantageously, the determination of said conflict position further comprises the steps of: • Determination of a first position of said vehicle, called minimum position, based on a minimum among said distances; • Determination of a second position of said vehicle, said second position chronologically preceding, in said sequence, said minimum position, such that: • The distance between said second position of the vehicle and said trajectory of said agent is greater than said first threshold; • For each of the positions of said vehicle between said second position and said minimum position, the distance between the position of said vehicle and said trajectory of said agent is less than said first threshold; • If there is a distance, between a third position of said vehicle and said trajectory of said agent, greater than a second threshold, said second threshold being greater than said first threshold, said third position chronologically preceding, in the sequence, said second position, the conflict position is the second position.

[0019] Thus, first of all, in the sequence, the conflict position is more precisely determined and corresponds to the first moment when the trajectory of the agent is too close to one of the vehicle's positions. Then, the third position makes it possible not to annotate the sequence when the vehicle and the agent are traveling on the same traffic lane, one behind the other. Then, depending on the value of the third threshold, if it is large enough such as more than 3 meters, 5 meters, 9 meters, other values ​​are possible, the third position makes it possible to select only road intersections, or situations in an urban environment. This makes it possible to build models used for driver assistance systems specifically for an urban environment.

[0020] Similarly, these steps allow for greater efficiency in terms of calculations and calculation times, particularly in the presence of several agents in the database. The calculations are only made for the agents of interest. Also, the calculations are made going back in time: we start from the instant when the position is at a minimum, then we determine, going back in time, the second position and then the third position. It is not necessary to calculate trajectory extrapolations for each agent for each instant of the sequence and then to check the distances.

[0021] Advantageously, said method further comprises the steps of: • For each instant of the time-stamped sequence, when the positions of said vehicle precede said conflict position, • Determination of a speed of said vehicle and a speed of said agent; • Determination of a first time corresponding to a time taken for said vehicle to reach said conflict position from said trajectory of said vehicle and said speed of said vehicle; • Determination of a second time corresponding to a time taken for said agent to reach said conflict position from said trajectory of said agent and said speed of said agent; • Determination of an interactivity index based on the first determined times and the second determined times; • Annotation of said sequence and / or said agent in said sequence, with said determined interactivity index.

[0022] Thus, an interactivity index is determined. It is a function of the first times and the second times, which makes it possible to quantify the interactivity. Indeed, if at a given instant, upstream of the conflict position, the first time and the second time are close, this means that the vehicle with its speed at the given instant and the agent with its speed at the given instant, will arrive at approximately the same time at the conflict position. The closer one of the first times and one of the second times are, the greater the interactivity. The annotation of this index to the sequence or to the agent makes it possible to give a score on the situation encountered in the data set. The score makes it possible to quickly filter data sets between them to select only particular data sets or those useful for the model's learning objective.

[0023] Advantageously, for each instant, a time difference, in absolute value, between the first and the second time is determined, a minimum difference is calculated from the time differences, and for which the determination of the interactivity index is based on said minimum difference, the smaller said minimum difference, the greater the interactivity index.

[0024] Advantageously, at a given instant of said sequence, the first time is equal to a distance between the conflict position and the position of said vehicle divided by said speed of said vehicle, and wherein the second time is equal to a distance between the conflict position and the position of said agent divided by said speed of said agent.

[0025] Thus it is a fairly simple and quick way to calculate an approximation of the first times and second times without taking into account a lateral movement of the vehicle and / or the agent.

[0026] Advantageously, in which the method further comprises a step of constructing a kd tree based on the positions of the agent, said kd tree being used for determining the distances and / or determining said conflict position and / or determining said minimum position.

[0027] Thus, the calculation time is even more optimized, faster, because the distances are calculated more quickly. The kd tree is used as a database. Each vehicle position and a threshold are used as an input query to the database. The database then returns, for each position, the agents verifying the distance condition. This is particularly effective in the case of the presence of several agents or in the case of long recorded sequences, for example greater than 4 seconds.

[0028] A second aspect of the invention relates to a data processing device comprising a memory associated with at least one processor configured to implement the method according to the first aspect of the invention.

[0029] A third aspect of the invention relates to a computer program comprising instructions which, when the program is executed by the device according to the second aspect of the invention, cause the latter to implement the method according to the first aspect of the invention.

[0030] A fourth aspect of the invention relates to a vehicle driving assistance system comprising a model trained from at least one annotated sequence according to the first aspect of the invention, and from the device according to the second aspect of the invention.

[0031] The invention also relates to a vehicle comprising the driving assistance system according to the fourth aspect of the invention. Brief description of the figures

[0032] Other characteristics and advantages of the invention will emerge from the description of the non-limiting embodiments of the invention below, with reference to the appended figures, in which:

[0033] [Fig-1] schematically illustrates a device, according to a particular example of rea lization of the present invention.

[0034] [Fig.2] schematically illustrates a method for annotating a time-stamped sequence of recorded data of a driving environment of a vehicle, according to a particular exemplary embodiment of the present invention.

[0035] [Fig.3] schematically illustrates a life situation of interactivity between a vehicle and an agent in a roundabout, according to a second particular example of embodiment of the present invention.

[0036] [Fig.4] schematically illustrates an example of distance between each position of a vehicle with a trajectory of an agent, according to a third particular example of embodiment of the present invention. Detailed description of the invention

[0037] [Fig. 1] represents an example of a device 101 included in the vehicle, in a network (“cloud”) or in a server. This device 101 can be used as a centralized device in charge of at least certain steps of the method described below with reference to [Fig. 2]. In one embodiment, it corresponds to a computer on board a vehicle.

[0038] This device 101 can take the form of a box comprising printed circuits, any type of computer or even a mobile telephone (“smartphone”).

[0039] The device 101 comprises a random access memory 102 for storing instructions for the implementation by a processor 103 of at least one step of the method as described below. The device also comprises a mass memory 104 for storing data intended to be retained after the implementation of the method.

[0040] The device 101 may further comprise a digital signal processor (DSP) 105. This DSP 105 receives data to format, demodulate and amplify, in a manner known per se, this data.

[0041] The device 101 also comprises an input interface 106 for receiving the data implemented by the method according to the invention and an output interface 107 for transmitting the data implemented by the method according to the invention.

[0042] For example, the input interface 106 can receive the following data: time-stamped sequence of recorded data of a vehicle driving environment, position or geographical location of the vehicle, speed and / or acceleration of the vehicle, set or predetermined positions / speeds / accelerations, engine speed, position and / or travel of the clutch, brake and / or acceleration pedal, detection of other vehicles, objects or agents, position or geographical location of the other vehicles, objects or agents detected, speed and / or acceleration of the other vehicles, objects or agents detected, operating states of sensors, confidence index of data originating from or processed by sensors and / or devices similar to the device 101.

[0043] For example, the input interface 106 may also receive the data following of a data sequence: metadata allowing to identify the vehicle and agents, trajectory of the vehicle, positions of the vehicle, trajectory of the agent(s), positions of the agent(s), distances in each position of the vehicle and a trajectory of an agent, times of reaching of a vehicle or an agent to reach a position to another position ... And also, the input interface can receive predetermined thresholds, parameters of adjustments or configuration of the implemented process, a kd tree, ...

[0044] For example, the output interface 107 can transmit data similar to the data received by the input interface 106. Also, the output interface 107 can transmit: a conflict position, metadata linked to a sequence of data and / or to agents of the sequence, an annotated sequence, a sequence with annotated agents, a minimum position, positions linked to distance comparisons, an index of interactivity between the vehicle and an agent, a learned model, ...

[0045] [Fig.3] schematically illustrates a life situation of interactivity between a vehicle 310 and an agent 320 in a roundabout 300, according to a second particular embodiment of the present invention. The roundabout has 3 entry and exit lanes, 301, 302, and 303. A trajectory of the vehicle 310 over the duration of a time-stamped sequence of recorded data is represented by the dotted line 311. A trajectory of the agent 320 over the duration of the sequence is represented by the dotted line 321.

[0046] The trajectories 311 and 321 intersect at a point 331. This point 331 is a position of the vehicle 310 at a first given instant, for example 4 seconds after the start of the recording of the sequence. This point is called the first position and is also called the minimum position. This position is defined as being the minimum distance, on the sequence, between each position of the vehicle 310 with the trajectory 321.

[0047] Point 332 is a second position of the vehicle 310 at a second given time, for example 3.5 seconds after the start of the recording of the sequence. At this position 332, the distance from the trajectory 321 is greater than a first predetermined threshold. It is said that position 332 precedes position 331 because, chronologically, the second given time is before the first given time. If, for each position between position 332 and position 331, the distance from the agent's trajectory 321 is less than the first threshold, then the second position is called a conflict position. This corresponds to the first time in the recording where the agent 320 may be too close to the vehicle 310. There is then a risk of an accident if the vehicle 310 and the agent 320 are at this position at the same time.

[0048] At the time, here 3.5 seconds after the start of the recording of the sequence, when the vehicle 310 is at position 332, the agent may be at another position, for example position 342. In this example, the agent 320 then gives way When moving to vehicle 310, he had to stop or slow down significantly, especially if the speed of the agent at position 344 was high, for example 50 km / h. Without braking and stopping, the position of agent 320 would certainly be at position 331 at the instant 4 seconds after the start of recording the sequence.

[0049] Point 333 is a third position of the vehicle 310 at a third given time, for example 3 seconds after the start of the recording. At this position 333, the distance with the trajectory 321 of the agent is greater than a second predetermined threshold. The second threshold is greater than the first threshold. We say that position 333 precedes position 332 and position 331 because, chronologically, the third given time is before the second given time and before the first given time.

[0050] [Fig.4] schematically illustrates an example of distance between each position of a vehicle with a trajectory of an agent, according to a third particular example of embodiment of the present invention. [Fig.4] comprises an abscissa axis 401 corresponding to positions of the vehicle or to instants of a recorded sequence. [Fig.4] comprises an ordinate axis 402 which is a distance.

[0051] A distance is a positive measurement between two objects, for example, two points, a point and a curve, ... In the case of a distance measurement between a point (or position), and a curve (or trajectory), we generally take the minimum distance between the point and all the points of the curve. In [Fig.4], the curve 403 represents a distance between each position of the vehicle and the trajectory of the agent.

[0052] Line 411, in dash format, represents a first threshold. Line 412, in dash format, represents a second threshold. The second threshold is greater than the first threshold. The first and second thresholds may be predetermined.

[0053] Three positions 431, 432 and 433 of the vehicle are represented. The first position 431, called the minimum position, is a position of the vehicle for which the distance between each position of the vehicle and the trajectory of the agent is minimal. It corresponds to the minimum of the curve 403. The second position 432, called the conflict position, is the position of the vehicle such that the distance is greater than the first threshold 411 and such that each position between the second position and the first position is less than the first threshold. The third position 433 is the position of the vehicle such that the distance is greater than the second threshold 412 and such that each position between the third position and the second position is less than the second threshold. Position 433 precedes position 432 which precedes position 431.

[0054] [Fig.2] schematically illustrates a method for annotating a time-stamped sequence of recorded data of a driving environment of a vehicle 310, according to a particular exemplary embodiment of the present invention. The objective is to identify an interactivity between said vehicle 310 and an agent 320 present in said data. The method is implemented by a processor and comprises several steps.

[0055] In this particular example, the data sequence comprises periodic information on the vehicle 310 and the environment of the vehicle, including in particular an agent 320.

[0056] Step 200, TrajV, is a step of determining a trajectory 311 of said vehicle from said sequence, said trajectory 311 determining a succession of positions, for example 331, 332 and 333, of said vehicle. It should not be understood from [Fig. 3] that the trajectory 311 only comprises 3 positions. For example, if the sequence lasts 6 seconds and a sampling period of the recordings is 0.01 seconds, then the trajectory 311 comprises 600 positions. A position is for example a coordinate in two or three dimensions relative to a predetermined reference frame.

[0057] In one operating mode, the trajectory and positions of the vehicle have been precalculated and then received by the input interface 106. In another operating mode, the trajectory is conventionally determined from each position of the vehicle.

[0058] Step 210, TrajA, is a step of determining a trajectory 321 of said agent 320 from said sequence, said trajectory determining a succession of positions, for example 341, 342 and 344, of said agent 320. It should not be understood from [Fig. 3] that the trajectory 321 only comprises 3 positions. For example, if the sequence lasts 6 seconds and a sampling period of the recordings is 0.01 seconds, then the trajectory 321 can comprise up to 600 positions. It may be, for an agent, that there are fewer positions than the vehicle 310. For example, at the start of recording, the agent 320 is too far away to be in the environment close to the vehicle, an environment captured by sensors.

[0059] In one operating mode, the trajectory and the positions of the agent have been precalculated then received by the input interface 106. In another operating mode, the trajectory is conventionally determined from each position of the agent.

[0060] Advantageously, the method further comprises a step of constructing a kd tree based on the positions of the agent, said kd tree being used for determining the distances 403 and / or determining said conflict position 332, 432 and / or determining said minimum position 331, 341.

[0061] Step 220, Dist, and a step of determining a succession of distances 403, called distances, such that, for each position of said vehicle 310, a distance is determined between the position of said vehicle and said trajectory 321 of said agent.

[0062] In one mode of operation, the kd tree is used to determine the distances 403 by basing the query on the successive positions of the vehicle 310.

[0063] Step 230, PosConf, is a step of determining a position 332, 422 of said vehicle, called conflict position, based on said trajectory 321 of said vehicle, on said distances 403 and on a first predetermined distance threshold 411, one of the distances being less than said first threshold. For example, the first threshold can be equal to 2 meters, to a length proportional to a width of the car, to a length proportional to a width of the agent, to a length based on the widths of the car, or of the agent, other values ​​are possible.

[0064] In one mode of operation, the kd tree is used to determine which distances 403 are greater than the first threshold by basing the query on the successive positions of the vehicle 310 and on the first threshold.

[0065] Advantageously, the method further comprises a step 232, Posl, for determining a first position 331, 341 of said vehicle, called minimum position, based on a minimum among said distances. Having the distances 403 for each position of the vehicle 310, it is then simple to determine the minimum distance.

[0066] In one mode of operation, the kd tree is used to determine what the minimum distance is by basing the query on the successive positions of the vehicle 310. This is particularly efficient in terms of computation time in the presence of several agents and / or a large number of data.

[0067] Advantageously, step 234, Pos, is a step of determining a second position 332, 432 of said vehicle, said second position chronologically preceding, in said sequence, said minimum position 331, 431. The distance, between said second position 332 of the vehicle and said trajectory 321 of said agent, is greater than said first threshold. For each of the positions of said vehicle between said second position 332, 432 and said minimum position 331, 431, the distance, between the position of said vehicle and said trajectory of said agent, is less than said first threshold.

[0068] In one mode of operation, the kd tree is used to determine what is the first, for a vehicle position preceding the minimum position, distance greater than the first minimum threshold by basing the query on the vehicle positions 310 preceding the minimum position and on the first threshold. This is particularly efficient in terms of computation time in the presence of several agents and / or a large number of data.

[0069] Advantageously, in step 236, Pos3, if there is a distance, between a third position 333, 433 of said vehicle and said trajectory 321 of said agent, greater than a second threshold 412, said second threshold 412 being greater than said first threshold 411, said third position 333, 433 chronologically preceding, in the sequence, said second position 332, 432, said conflict position 332, 432 is the second position. This amounts to confirming that the second position is a conflict position if there exists, in the sequence, prior to the instant when the vehicle reaches the conflict position or at the instant when the agent reaches the conflict position, a sufficiently large distance between the vehicle and the agent. If this is not the case, then the vehicle and the agent travel on the same lane or on a lane close to each other. This step allows only interactivity to be kept during an intersection.

[0070] Advantageously, the second threshold is predetermined according to a type of road, such as an urban road, a country road, a motorway, etc., in order to annotate the interactivity context more precisely. For example, on an urban road, the second threshold is between 4 and 12 meters, for example 9 meters, but other values ​​are possible.

[0071] In one mode of operation, the kd tree is used to determine whether the third position exists by basing the query on the successive positions of the vehicle 310 and on the second threshold. This is particularly efficient in terms of computation time in the presence of several agents and / or a large number of data.

[0072] The sequence of steps 232, 234 and 236, determining the positions from the minimum distance makes it possible to be even more efficient in terms of calculation time, particularly in the presence of several agents in the sequence. Indeed, this makes it possible to select, very quickly, the agents who can have an interaction with the vehicle. There is no need to study chronologically all the positions of each agent.

[0073] Advantageously, the method further comprises a determination of an interactivity index based on the time that the vehicle and the agent would take to reach the conflict position.

[0074] Advantageously, the method further comprises the steps of: • For each instant of the time-stamped sequence, when the positions of said vehicle preceding said conflict position 332, 432, • Determination of a speed of said vehicle 310 and a speed of said agent 320; • Determination of a first time corresponding to a time taken to reach said conflict position 332,432 by said vehicle 310 from said trajectory 311 of said vehicle and said speed of said vehicle; • Determination of a second time corresponding to a time taken to reach said conflict position 332, 342 by said agent 320 from said trajectory 321 of said agent and said speed of said agent; • Determination of an interactivity index based on the first determined times and the second determined times; and • Annotation of said sequence and / or said agent in said sequence, with said determined interactivity index.

[0075] Advantageously, for each instant, a time difference, in absolute value, between the first and the second time is determined, a minimum difference is calculated from the time differences, and for which the determination of the interactivity index is based on said minimum difference, the smaller said minimum difference, the greater the interactivity index.

[0076] Advantageously, at a given instant of said sequence, the first time is equal to a distance between the conflict position and the position of said vehicle divided by said speed of said vehicle, and in which the second time is equal to a distance between the conflict position and the position of said agent divided by said speed of said agent.

[0077] In order to perfect models, for use by driver assistance systems, trained from a data set it is necessary to have data sets with different interactivities. An annotation with an interactivity index allows for better classification of sequences.

[0078] In the case of [Fig.3], the vehicle 310 can accelerate, brake, remain at a constant speed, just like the agent 320. For example, if the agent 320 passes just in front of the vehicle 10 without giving way, the vehicle will have to brake quite abruptly. In this case there is a strong interactivity, otherwise, at the conflict position, the vehicle could be too close to the agent. In this same case, if the vehicle 310 does not change its speed, because the vehicle is still far from the conflict position, the interactivity is weak or even absent. There is no interactivity if the agent has passed through the conflict point for enough time, for example 1 second, 2 seconds or more, before the vehicle arrives at the conflict position.

[0079] It is known to determine speeds, accelerations, and jerks of a vehicle or an agent from a sequence. The sequence can also contain this information. Thus, for each instant of the sequence, for each position of the vehicle and the agent, we have a speed of the vehicle and the agent.

[0080] The distance to be traveled between a position of the vehicle or the agent and the conflict position can be determined from the sequence. Thus, at a given instant, therefore at a given position of the vehicle and at a given position of the agent, the time to travel the distance to reach the conflict position is calculated from the speed of the vehicle or the agent at the given instant. The calculation is carried out for each instant before the instant when the vehicle or the agent reaches the conflict position. There is then a multitude of times, for the vehicle and for the agent, to reach the conflict position.

[0081] In one operating mode, among the multitude of times, we take the minimum time of the vehicle and the minimum time of the agent. We construct an interactivity index which is based on this time difference. The smaller the time difference, the greater the interactivity index, and therefore the interactivity, is judged to be. For example, if the time difference is equal to zero, then the vehicle and the agent will be in the conflict position at the same time if the vehicle and the agent do not change their speeds (brake, accelerate). In this example, the interactivity is maximum.

[0082] Step 240 is a step of annotating said sequence and / or said agent in said sequence. Having determined a conflict position, there is an interaction between the vehicle and the agent. The sequence and / or the agent in the sequence is then annotated. In a preferred operating mode, the sequence is annotated, which makes it possible to find the sequence more simply to construct data sets. The agent is also annotated, which makes it possible to identify in a sequence an agent, among the other possible agents in the sequence, which has an interactivity with the vehicle. This makes it possible to extract, for example, a sub-sequence useful for learning a driving assistance model.

[0083] In one operating mode, said sequence and / or said agent in said sequence are annotated with said determined interactivity index.

[0084] The present invention is not limited to the embodiments described above as examples: it extends to other variants.

[0085] The present invention is also suitable when there are several agents in the same sequence. The method is iterated for each agent. Certain steps can also be carried out jointly or in parallel, for example, when using the kd tree.

[0086] The present invention is also suitable for the presence of long sequences, for example sequences lasting 10 to 20 seconds, several minutes or more, or when several relative minimum positions are present at different times, either with the same agent or with another agent. In this case, it is possible to iterate the method several times and proceed through time windows, between two predetermined times.

[0087] Advantageously, the invention can also be used to create new shorter sequences, and therefore new data sets, with a limited number of interactivities, for example only one. For example, at each determined minimum position, the sub-sequence will only take into account a predetermined duration preceding the instant of a detected minimum position.

[0088] Also, the method has been described according to a sequence of steps. Certain steps can be carried out in parallel or according to another sequence.

[0089] Equations and calculations have further been detailed. The invention is not limited to the form of these equations and calculations, and extends to any type of other mathematically equivalent form.

Claims

Claims

1. A method for annotating a time-stamped sequence of recorded data of a driving environment of a vehicle (310) in order to identify an interactivity between said vehicle (310) and an agent (320) present in said data, said method being implemented by a processor (103) and comprising the steps of: • Determination (200) of a trajectory (311) of said vehicle (310) from said sequence, said trajectory (311) determining a succession of positions (331, 332, 333) of said vehicle; • Determination (210) of a trajectory (321) of said agent from said sequence, said trajectory determining a succession of positions (341, 342, 344) of said agent; • Determination (220) of a succession of distances, called distances, such that, for each position of said vehicle (310), a distance is determined between the position of said vehicle and said trajectory (321) of said agent; • Determination (230) of a position (332) of said vehicle, called conflict position, based on said trajectory (321) of said vehicle, on said distances (403) and on a first predetermined distance threshold (411), one of the distances being less than said first threshold; • Annotation (240) of said sequence and / or said agent in said sequence.

2. The method of claim 1, wherein for determining said conflict position (332, 432) further comprises the steps of: • Determination (232) of a first position (331, 341, 431) of said vehicle, called minimum position, based on a minimum among said distances; • Determination (234) of a second position (332) of said vehicle, said second position (332, 432) chronologically preceding, in said sequence, said minimum position (331, 341, 431), such that: • The distance between said second position (332, 432) of the vehicle and said trajectory (321) of said agent, i.e. greater than said first threshold; • For each of the positions of said vehicle between said second position and said minimum position, the distance between the position of said vehicle and said trajectory of said agent is less than said first threshold; • If there is a distance, between a third position (333, 433) of said vehicle and said trajectory (321) of said agent, greater than a second threshold, said second threshold being greater than said first threshold, said third position (333, 433) chronologically preceding, in the sequence, said second position (332), said conflict position (332, 432) is the second position.

3. A method according to any preceding claim, wherein said method further comprises the steps of: • For each instant of the time-stamped sequence, when the positions of said vehicle precede said conflict position (332, 432), • Determination of a speed of said vehicle and a speed of said agent; • Determination of a first time corresponding to a time taken for said vehicle to reach said conflict position from said trajectory of said vehicle and said speed of said vehicle; • Determination of a second time corresponding to a time taken for said agent to reach said conflict position from said trajectory of said agent and said speed of said agent; • Determination of an interactivity index based on the first determined times and the second determined times; • Annotation of said sequence and / or said agent in said sequence, with said determined interactivity index.

4. Method according to the preceding claim, in which, for each instant, a time difference, in absolute value, between the first and second times is determined, a minimum difference is calculated from the time differences, and for which the determination of the interactivity index is based on said minimum difference, the smaller said minimum difference, the greater the interactivity index.

5. A method according to claim 3 or 4, wherein, at a given time in said sequence, the first time is equal to a distance between the conflict position and the position of said vehicle divided by said speed of said vehicle, and wherein the second time is equal to a distance between the conflict position and the position of said agent divided by said speed of said agent.

6. Method according to one of the preceding claims, in which the method further comprises a step of constructing a kd tree based on the positions of the agent, said kd tree being used for determining the distances, and / or determining the point of conflict and / or determining said conflict position (332) and / or determining said minimum position (331, 341).

7. Data processing device (101) comprising a memory (102) associated with at least one processor (103) configured to implement the method according to one of the preceding claims.

8. A computer program comprising instructions which, when the program is executed by a device (101) according to claim 7, cause the latter to implement the method according to one of claims 1 to 6.

9. Vehicle driving assistance system comprising a model trained from at least one sequence annotated according to the method of one of claims 1 to 6, and from the device according to claim 7.

10. Vehicle comprising the driving assistance system according to the preceding claim.

Images