Method for detecting dynamic objects

Abstract

A method for detecting dynamic objects in the environment of a vehicle is proposed and described, comprising: receiving (100) vehicle environment data from successive acquisitions of an on-board sensor, detecting (200), from data from a first acquisition, a discontinuity in a road boundary line, determining (300) a first position of at least one point in the interval, detecting (400), from data from a second acquisition at a later time, the discontinuity in the road boundary line, and determining (500) a second position of the corresponding point in the interval, in the reference frame associated with the vehicle, calculating (600), from the positions of the point in the interval, a speed of movement of the interval, and detecting (700) a dynamic object (O) in the environment of the vehicle, if the speed of movement of the interval is non-zero.Figure from the summary: Figure 2.

Description

Title of the invention: Method for detecting dynamic objects Technical field

[0001] The present disclosure relates to a method and a device for detecting dynamic objects in the environment of a vehicle, in particular a motor vehicle, for driving assistance applications or for the development of autonomous vehicles. The disclosure applies in particular to detections of dynamic objects such as vulnerable road users (also referred to by the English acronym VRU for Vulnerable Road Users), such as cyclists, scooters, pedestrians, scooter users, etc. Prior art

[0002] Land vehicles are increasingly equipped with sensors, such as cameras, radars, and lidars, to enable observation and analysis of the vehicle's environment, with a view to analyzing the road, detecting obstacles, and assisting, or even gradually replacing, the driver. The data acquired by different types of sensors can in certain applications be merged to enable a more in-depth or more complete analysis of the environment.

[0003] In the field of object detection in the vehicle environment, many current techniques are based on the application of trained classifiers to recognize shapes specific to certain classes of known objects. For example, a trained classifier can recognize with a high degree of precision road signs, street furniture elements, or even the front or rear face of various types of vehicles.

[0004] However, particularly in the case of low-speed vehicles of small dimensions, such as typically bicycles or scooters, methods based on classifiers have limited performance. It is therefore possible not to detect a vehicle of this type, traveling for example on a reduced width of the road or on a cycle path, at low speed. The absence of detection of this type of vehicle can cause serious problems. Indeed, in the case of non-autonomous vehicles, driving assistance systems include detectors of objects in the environment of the vehicle, and the absence of detection of a vehicle of this type, particularly when it is in the blind spot of the vehicle, means that the driver is not warned of the presence of the vehicle and may also not see it.In the case of autonomous or semi-autonomous vehicles, determining the vehicle's trajectory must take into account the presence of this vehicle to avoid any risk of collision.

[0005] There is therefore a need for an object detection solution that can complement, or replace, the shortcomings of prior art solutions, in particular solutions based on classifiers. Summary

[0006] The present disclosure improves the situation.

[0007] A method is proposed for detecting dynamic objects in the environment of a vehicle traveling on a road, the vehicle comprising at least one on-board sensor adapted to acquire data relating to the environment of the vehicle, the sensor being oriented towards the road, the method being implemented by a device comprising a computer and a memory, and comprising: - the reception of vehicle environment data corresponding to a plurality of successive acquisitions from the on-board sensor, - The detection, from the data corresponding to a first acquisition, of a discontinuity in a road boundary line, the discontinuity being formed by an interval separating two consecutive segments of the road boundary line, - The determination of a first position of at least one point of the interval in a reference system associated with the vehicle, - The detection, from the data corresponding to a second acquisition at a later time, of the discontinuity of the road boundary line, and the determination of a second position of the point of the corresponding interval, in the reference system associated with the vehicle, - Calculation, from the first and second positions of the point of the interval, of a speed of movement of the interval, and - the detection of a dynamic object in the environment of the vehicle, if a set of conditions relating to the detected interval is respected, the set of conditions including at least the fact that the speed of movement of the interval is non-zero.

[0008] In embodiments, the steps of detecting a discontinuity of a roadway boundary line are implemented by a line detection algorithm.

[0009] In embodiments, the method further comprises estimating a future position of the dynamic object from initial velocity and position data of the moving object and movement information of the vehicle.

[0010] In embodiments, the method further comprises an evaluation of the criticality of the dynamic object and, if the dynamic object is considered critical, the generation of a visual and / or audible alert, or the generation of a command relating to the trajectory of the vehicle.

[0011] In embodiments, the criticality assessment of the dynamic object O) comprises the calculation of a collision time between the dynamic object and the vehicle, and the dynamic object is considered critical if the collision time is less than a determined threshold.

[0012] In embodiments, the criticality assessment of the dynamic object further takes into account a relative position of the dynamic object with respect to the vehicle and / or a trajectory of the vehicle.

[0013] In embodiments, the point of the interval is a point located in the middle of the interval or a point located at one end of one of the two marking segments separated by the interval.

[0014] In embodiments, the method further comprises estimating, from the positions of the point of the interval on the two images and a movement of the vehicle, the size of the interval on each image.

[0015] According to another object, a computer program product is described, comprising code instructions for implementing the method according to the preceding description, when it is executed by a computer.

[0016] According to another object, a device for analyzing the environment of a vehicle is described, comprising at least one computer and a memory, configured for implementing the method according to the preceding description.

[0017] According to another object, a vehicle is described, comprising at least one on-board sensor oriented towards the road, and an environment analysis device as described above.

[0018] The proposed method makes it possible to detect dynamic objects in the environment of a vehicle, in particular vulnerable road users, by detecting and analyzing intervals, in other words holes, or obstructions formed by the object in the field of view of the sensor, in boundary lines of the roadway, such lines being able to be formed by road marking lines, or road edge elements which may be raised above the roadway, such as sidewalks or crash barriers.

[0019] The proposed method can therefore be used autonomously to detect objects, or be combined with prior art solutions, for example based on classifiers, to make these solutions more robust.

[0020] This method is agnostic, that is to say that it is not based on a priori knowledge of the object to be detected, as is the case with trained classifiers. In addition, it uses line detection algorithms, which are known and widely mastered. In addition, the method is not very resource-intensive and requires little computing power.

[0021] The proposed method can advantageously be combined with trajectory determination algorithms for an autonomous vehicle, or with trajectory assistance algorithms. conduct. Brief description of the drawings

[0022] Other characteristics, details and advantages will appear on reading the detailed description below, and on analyzing the attached drawings, in which:

[0023] [Fig-1] [Fig.l] schematically represents an example of a context of implementation implementation of a method according to an embodiment.

[0024] [Fig.2] [Fig.2] schematically represents the main steps of a method according to one embodiment of the invention.

[0025] [Fig.3] [Fig.3] schematically represents a discontinuity detected in a road boundary line.

[0026] [Fig.4] [Fig.4] schematically represents a case of application of the method according to one embodiment. Description of the embodiments

[0027] Reference is now made to [Fig.l], in which an example of a context in which a method for detecting dynamic objects in the environment of a vehicle is implemented is shown. [Fig.l] represents a road on which a vehicle V is traveling, hereinafter referred to as the ego-vehicle since it is the reference vehicle for which the method is implemented.

[0028] The road may comprise one or more main traffic lanes for road vehicles such as cars, trucks, etc., the traffic lanes being in the same direction of travel or in opposite directions. The road may also comprise one or more secondary lanes corresponding to cycle paths, but which in fact can also be used by different types of vehicles: bicycles, scooters, electric scooters, gyroscopic wheels or any other light mobility device. Finally, the road may also be bordered by a sidewalk, on one side or on both sides of the roadway.

[0029] The road may also include ground marking elements, which may be continuous or discontinuous marking lines, painted on the ground or, for temporary marking, formed by adhesive tape stuck to the ground. These marking lines may in particular be edge lines, which separate the outermost traffic lanes of the roadway from the shoulder or the emergency lane. In the case of a discontinuous line, the ground marking is formed from a set of segments aligned in the same longitudinal direction and having a regular spacing pattern between successive segments, typically a constant spacing between successive segments as shown in the example of [Fig.l].

[0030] The ego-vehicle V comprises at least one on-board sensor C oriented towards the road, the sensor C being adapted to acquire information relating to the environment of the vehicle. The fact that the sensor C is oriented towards the road allows it to acquire data representative of a portion of the road on which the ego-vehicle is traveling. The on-board sensor may be a visual sensor adapted to acquire images on which the portion of the road is visible, for example a camera or a lidar sensor. Alternatively or additionally, the sensor may also be a radar sensor, which can detect data relating to edges of the roadway such as crash barriers. In embodiments, the ego-vehicle V may comprise several sensors C of different types, the data of which complement each other and can be merged.

[0031] The visual sensor, or sensors where appropriate, may for example, and as shown in Figures 1 and 3, be mounted at the rear of the vehicle so as to capture a portion of the road located behind the vehicle. Alternatively, it may also be mounted at the front, or laterally relative to the ego-vehicle. The sensor C may be oriented so as to have a field of vision including at least one traffic lane adjacent to the traffic lane in which the vehicle is traveling.

[0032] In embodiments, the ego-vehicle V also comprises other types of sensors (not shown), for example a GNSS receiver making it possible to obtain the absolute position of the vehicle, i.e. its longitude, latitude and altitude, an inertial unit, odometric sensors for determining the rotation speed of the wheels, and therefore the speed of movement of the vehicle, etc.

[0033] The method described below is implemented by a device 1 comprising at least one computer 10, for example an electronic control unit ECU of a vehicle, a processor, a microprocessor, a central processing unit CPU, a graphics processor GPU, etc. The device also comprises a memory 11, typically a non-volatile memory. The method is implemented by the computer 10, by executing code instructions stored for this purpose in the memory 11. In embodiments, the device can be embedded in the ego-vehicle V, and in communication with at least the embedded sensor C to receive the data acquired by it and implement the method. Alternatively, the device can be remote from the ego-vehicle.For example, it may be a remote server, receiving data acquired by sensor C, and possibly by the other sensors mentioned above, and implementing the processing described below, before returning information relating to the object O possibly detected to the vehicle.

[0034] With reference to [Fig.2], the main steps of a method for detecting dynamic objects according to one embodiment will be described.

[0035] During a step 100, the device 1 receives a plurality of data sets corresponding to successive acquisitions of the sensor C. In the case where the sensor is a visual sensor of the camera or lidar type, a data set corresponding to an ac acquisition is typically an image. In the case where the sensor is a radar, the data set corresponding to an acquisition is a radar acquisition. The sensor can carry out acquisitions at a determined acquisition frequency, and transmit the acquired data, or a sub-sampling of the acquisitions, as they are acquired, to the device 1.

[0036] Then, the method comprises a detection 200, from the data corresponding to the first acquisition of the sensor(s) C, at a first acquisition time tb of a discontinuity D in a line L delimiting the roadway, this discontinuity being formed by an interval separating two segments aligned in a longitudinal direction and belonging to the same line delimiting the roadway. A “line delimiting the roadway” may correspond to a ground marking line, delimiting a traffic lane of the road, but it may also correspond to a sidewalk which borders the road and therefore also delimits the roadway, or even a safety barrier, etc. In [Fig.l], it can be seen that the object O represented behind the ego-vehicle does not obscure a ground marking line but partially obscures a sidewalk which borders the roadway.

[0037] The detected discontinuity D may be an intrinsic discontinuity of a discontinuous ground marking line, formed by a succession of segments separated from each other by a constant interval. However, the discontinuity may also correspond to a part of a continuous marking line, of a sidewalk, or of a continuous segment of a discontinuous marking line, which is obstructed by an object present in the field of vision of the visual sensor C. With reference to [Fig. 3], an example has thus been schematically represented in which a detected discontinuity D corresponds to a part of a segment of a ground marking line, obscured by an object present in the environment of the ego-vehicle.

[0038] The detection of the discontinuity is implemented by a line detection algorithm. Such algorithms may include a step of detecting contours or contrasts in an image. Then, the different detected ground markings belonging to the same ground marking line are identified. This step therefore makes it possible to detect a set of points belonging to the same line, and the detected points can thus be grouped into segments whose intervals between each, i.e. the discontinuities of the line, are calculated. Then, a model making it possible to determine the coordinates of the corresponding line in a reference frame of the vehicle is established. This makes it possible, when the vehicle moves, to estimate a possible position of the ground marking line from the movement of the vehicle and to update the model from the acquired image.More detailed examples of line detection methods are for example given in the publication “Lane Detection (Part I): Mono-Vision Based Method”, Hao Li, Fawzi Nashashibi, Technical Report. No. 433, INRIA, January 2013, or in document FR3107763.

[0039] In embodiments, step 200 comprises detecting a plurality of intervals separating pairs of successive segments. In embodiments, the method may comprise calculating the length of each interval and selecting, for implementing the remainder of the method, certain intervals from among those detected on the basis of the calculated lengths. For example, intervals of constant length, which may correspond to discontinuities of discontinuous marking lines, may be detected, and only the intervals of different length are selected for the remainder of the method. Alternatively, since the different possible distances between two segments of discontinuous lines are known from the highway code, the selected intervals may be those having a length different from these distances.Alternatively, and for increased robustness, the rest of the method can be implemented on all the intervals separating two successive segments that have been detected. The rest of the method is described below for a given interval.

[0040] The method then comprises determining 300 a first position of at least one point of the interval detected between the two aligned segments, in a reference system associated with the vehicle. In embodiments, the point of the interval whose position is determined is one of the ends of the interval, which also corresponds to one end of the two consecutive segments which are separated by the interval. The ends P1 and P2 of the interval D are shown in [Fig. 3]. In embodiments, the position of the two ends of the interval is determined during this step. Alternatively, the point of the interval whose position is determined may be a point located in the middle of the interval, considered longitudinally.

[0041] In a frame of reference centered on the visual sensor C of the ego-vehicle, whose x axis is for example parallel to the direction of movement of the vehicle and the y axis orthogonal to the y axis, we note the coordinates of the points PI and P2 at the instant ti (xi(ti),yi(ti)) and (x2(ti), y2(ti)) respectively.

[0042] Optionally, in the case where the positions of the two end points Pi and P2 are determined, step 300 can also comprise the calculation of the length of the interval at time tb which is equal to x2-xi.

[0043] The method then comprises a step 400 comprising the detection, from data from the sensor(s) C corresponding to a second acquisition, at a later time t2, of the discontinuity D detected in step 200, still formed by at least two successive segments separated by an interval. This step also comprises the implementation of a line detection algorithm as previously for step 200. The correspondence between the discontinuity detected in step 200 and that detected in step 400 can be obtained by implementing the line detection algorithm. line detection, when the algorithm used makes it possible to estimate the future position of the detected lines taking into account the movement of the vehicle. Alternatively, this correspondence can be determined by applying algorithms known to those skilled in the art for image matching, including algorithms based on Harris detectors, SIFT (Scale-Invariant Feature Transform) type transformations, or on trained models.

[0044] The position of the point(s) of the interval in the reference system associated with the vehicle, at time t2, is then determined during a step 500. In the example mentioned above in which the positions of two end points Pi and P2 are determined, these positions are noted (xi(t2), yi(t2)) and (x2(t2), y2(t2)) respectively.

[0045] The method then comprises the calculation 600 of a speed of movement of the interval considered, from the positions of the point of the interval in the reference frame of the vehicle at times b and t2, calculated as follows: _ .y;( r2 ) -Jû ,(?; ) v ~ hh

[0046] Since this speed is calculated in the frame of reference of the ego-vehicle, it is a relative speed of the interval with respect to the ego-vehicle.

[0047] In embodiments, in the case where steps 300 and 500 comprise determining the position of two end points of the interval, they may also comprise calculating a length of the interval, i.e. the gap xrx2 , at time ti and at time t2, and step 600 may further comprise comparing the length of the interval between times ti and t2.

[0048] The method then comprises the detection 700 of a dynamic object O in the environment of the ego-vehicle, if a set of conditions relating to the detected interval is respected, comprising at least the fact that the speed of movement of the interval is non-zero. That is to say, the minimum condition for detecting a dynamic object is to determine v non-zero in step 600.

[0049] Once the dynamic object has been detected, various actions can be implemented. Firstly, the dynamic object can be tracked or pursued, i.e. an estimation of its future position based on its position at time t2, its speed, and information on the movement of the ego-vehicle (position, speed, acceleration), in order to be able to detect this object in images or data sets acquired subsequently.

[0050] The dynamic object may also be taken into account in driver assistance applications, or, in the case where the ego-vehicle is an autonomous vehicle, in vehicle trajectory control functions. Alternatively, the detected dynamic object may also be merged with existing object detection data from other algorithms, for example based on classifiers, to make these algorithms more robust and / or consolidate existing detection data associated with a low level of confidence.

[0051] In embodiments, the method then comprises an evaluation 800 of the criticality of the dynamic object, that is to say a determination of the critical nature or not of the detected object and, if the detected object is considered critical, the generation 900 of a command relating to this detection of critical object. The command may aim to emit an audible or visual alert, this alert being able to be emitted inside the vehicle, intended for the driver of the vehicle, or outside the vehicle, for the attention of the detected dynamic object in order to warn him of the danger of its non-detection and therefore of the uncertainty about the behavior of the ego-vehicle.According to another variant, in the case where the vehicle is capable of implementing communication protocols with its environment of the V2X type, in particular of the V2I type for communication with the infrastructure, the command 900 can be generated for an infrastructure element close to the vehicle, so that the latter displays an audible or luminous alert. For example, a connected traffic light capable of communicating with vehicles by V2I type communication can also be configured to display alert signals for pedestrians or vulnerable road users, and display such an alert upon receipt of a command 900 from the vehicle.

[0052] Alternatively or cumulatively, the command may aim to modify the trajectory of the ego-vehicle, or to prevent certain changes in trajectory of the vehicle.

[0053] The criticality evaluation 800 advantageously comprises the calculation of a collision time between the ego-vehicle and the object O, calculated as the distance between the ego-vehicle and the object, divided by the relative speed v calculated above. The distance between the ego-vehicle and the object is calculated as follows: . Here we have considered the point Pi closest to the ego-vehicle, in the case where the coordinates of two points are acquired at steps 300 and 500, but in the case where the coordinates of a single point are determined, we can calculate the distance on the basis of the coordinates of this point.

[0054] If the obtained collision time is less than a predetermined threshold, then the object is considered critical.

[0055] In embodiments, other conditions may be taken into account to evaluate the criticality of the object, which may be cumulative, or alternative, to the condition on the collision time.

[0056] For example, and as shown in [Fig. 3], a condition may relate to the position of the object relative to the ego-vehicle. If the object is located in, or in the vicinity of, an area determined to be critical, for example a blind spot area of the vehicle, then the object can also be considered critical.

[0057] According to yet another example, a condition may concern both the position of the object relative to the ego-vehicle and a trajectory of the vehicle. Thus, if the position of the object at the time of implementation of the method (for example at time t2), or an estimated future position of the object, is located on the trajectory of the vehicle, then the object may be considered critical. This is for example the case of [Fig.l], where the vehicle is located at an intersection towards the right, and an object O is detected which arrives at the rear of the vehicle, on its right, and may be on the trajectory of the vehicle at the moment when the latter turns. In this case, the object O is considered critical and an action 900 of alerting the driver is triggered, or of controlling the trajectory of the vehicle for example so that the vehicle waits to detect the object O in front of it before making the turn.

[0058] [Fig. 4] shows schematically another example, applicable in particular to an autonomous vehicle, in which an emergency vehicle Ve arrives at the rear of the ego-vehicle V, this vehicle Ve being for example detected by sound sensors (if the vehicle has activated a siren) or visual sensors (recognition of the vehicle and / or flashing lights). In this case, the vehicle can determine a free location L to move to in order to let the emergency vehicle pass, and the choice of the free location (corresponding to a trajectory control step 900) can take into account the presence of an object considered to be critical.

Claims

Claims

1. Method for detecting dynamic objects in the environment of a vehicle traveling on a road, the vehicle (V) comprising at least one on-board sensor adapted to acquire data relating to the environment of the vehicle, the sensor being oriented towards the road, the method being implemented by a device (1) comprising a computer (10) and a memory (11), and comprising: - The reception (100) of environment data of the vehicle corresponding to a plurality of successive acquisitions of the on-board sensor, - The detection (200), from the data corresponding to a first acquisition, of a discontinuity of a boundary line of the roadway, the discontinuity being formed by an interval separating two consecutive segments of the boundary line of the roadway, - The determination (300) of a first position of at least one point of the interval in a reference system associated with the vehicle, - The detection (400),from the data corresponding to a second acquisition at a later time, of the discontinuity of the boundary line of the roadway, and the determination (500) of a second position of the point of the corresponding interval, in the reference system associated with the vehicle, - The calculation (600), from the first and second positions of the point of the interval, of a speed of movement of the interval, and - the detection (700) of a dynamic object (O) in the environment of the vehicle, if a set of conditions relating to the detected interval is respected, the set of conditions comprising at least the fact that the speed of movement of the interval is non-zero.,

2. The method of claim 1, wherein the steps of detecting a discontinuity (200, 400) of a road boundary line are implemented by a line detection algorithm.

3. A method according to any one of claims 1 or 2, comprising further estimating a future position of the dynamic object from initial speed and position data of the moving object and vehicle movement information.

4. Method according to any one of claims 1 to 3, further comprising a criticality assessment (800) of the dynamic object (0) and, if the dynamic object is considered critical, the generation (900) of a visual and / or audible alert, or the generation (900) of a command relating to the trajectory of the vehicle.

5. Method according to claim 4, in which the criticality evaluation (800) of the dynamic object (O) comprises the calculation of a collision time between the dynamic object and the vehicle, and the dynamic object is considered critical if the collision time is less than a determined threshold.

6. Method according to claim 4 or 5, wherein the criticality assessment (800) of the dynamic object further takes into account a relative position of the dynamic object with respect to the vehicle and / or a trajectory of the vehicle.

7. A method according to any preceding claim, wherein the point in the interval is a point in the middle of the interval or a point at one end of one of the two marking segments separated by the interval.

8. Method according to the preceding claim, further comprising an estimation, from the positions of the point of the interval on the two images and a movement of the vehicle, of the size of the interval on each image.

9. Computer program product, comprising code instructions for implementing the method according to any one of the preceding claims, when executed by a computer (10).

10. Device for analyzing the environment of a vehicle, comprising at least one computer and a memory, configured for implementing the method according to any one of claims 1 to 8.

11. Vehicle (V), comprising at least one on-board sensor (1) oriented towards the road, and a device (1) according to the preceding claim.

Images