METHOD AND DEVICE FOR DETERMINING SPATIAL POSITIONS AND KINEMATICS PARAMETERS OF OBJECTS IN THE ENVIRONMENT OF A VEHICLE

Description

1. Domaine de l'invention

[0001] The invention relates to the field of determining spatial positions and kinematic parameters of objects in the environment of a vehicle. It relates more particularly to a method for determining spatial positions and kinematic parameters of objects in the environment of a motor vehicle, said vehicle being equipped with at least one camera for capturing images of its environment. It also relates to a device for determining spatial positions and kinematic parameters of objects in the environment of a motor vehicle on board said vehicle, as well as a motor vehicle. 2. Art antérieur

[0002] Detection devices for determining the distance and speed of a target object on the road have become standard equipment in today's vehicles. These devices typically include a position sensor, such as a radar or camera. They also include a distance and speed calculation unit to generate information needed for security checks.

[0003] Radar devices have the following limitation: their range is limited to 250 meters. This range is too short to prevent dangers such as a risky overtaking maneuver.

[0004] For devices using a camera, the computing unit implements computing techniques based on the change in the dimensions of the object. Using homography, the variation in the size of the object and the route gives a position of the object whose accuracy depends on the quality of the camera. Examples of this type of device can be found in documents US3120421 or US2015 / 354976.

[0005] Devices with cameras have the following limitation: cameras with very good resolution are expensive and conventional cameras have insufficient resolution in some cases to provide the computing unit with a series of usable images, especially when the image of the target object is very small. In this case, the computing unit cannot determine with sufficient precision the distance and speed of a target object that may constitute a danger.

[0006] Other causes may make the image series unusable by the computing unit, such as incorrect adjustment of the optical devices or reduced brightness. 3. Objectifs de l'invention

[0007] The present invention proposes a solution aimed at overcoming the aforementioned drawbacks. One objective of the invention is to enable efficient determination of spatial positions and kinematic parameters of mobile or static objects located in the environment of a vehicle using one or more cameras. 4. Résumé de l'invention

[0008] The principle of the invention is to use substantially identical elements regularly spaced in the environment of the vehicle in order to estimate more precisely spatial positions and kinematic parameters of target objects. The increase in the performance of this determination of spatial positions and kinematic parameters is therefore based on the positioning of target objects relative to substantially identical elements regularly spaced.

[0009] The subject of the invention is a method for determining at least one spatial position or one kinematic parameter according to claim 1.

[0010] "Substantially identical and regularly spaced elements" means elements of the same type and for which the distance calculated between two consecutive elements is the same. For example, a row of ten trees planted every fifty meters along the road is considered to be a set of substantially identical, regularly spaced elements.

[0011] A "class" is a set of elements of the same type. For example, dotted lines on the ground belong to the same class.

[0012] According to a particular embodiment, the detection of at least three substantially identical elements and a target object contained in said image and the grouping of said at least three substantially identical elements into at least one class comprises a detection of at least one pattern in said image.

[0013] According to a particular embodiment, the determination of the position in space of at least three substantially identical elements in said at least one class comprises a determination of at least three clear elements among said at least three substantially identical elements.

[0014] A bright element is an element whose number of pixels in the image is greater than a threshold value.

[0015] According to a particular embodiment, said at least one image of the environment of the vehicle comprising a road, the determination of the position in space of at least three substantially identical elements in said at least one class comprises a projection of said at least three substantially identical elements onto the plane of said road.

[0016] According to a particular embodiment, the determination of at least one class comprising at least three substantially identical elements regularly spaced among said at least one class comprises a determination of the spacing between said at least three clear elements among said at least three substantially identical elements.

[0017] According to a particular embodiment, the determination of at least one spatial position or a kinematic parameter of said target object from said distribution in space of at least three substantially identical and regularly spaced elements comprises a projection of the target object onto the plane of said route.

[0018] The invention also relates to a device on board a motor vehicle for determining spatial positions and kinematic parameters of objects in the environment of said vehicle, said vehicle being equipped with at least one camera for capturing images of the environment of the vehicle, which device comprises a calculation unit capable of implementing the method as defined previously.

[0019] The invention also relates to a motor vehicle equipped with at least one camera for capturing images of its surroundings, comprising said device.

[0020] According to a particular embodiment, said vehicle is equipped with a radar, said radar being interfaced with said on-board device. 5. Liste des figures

[0021] Other innovative features and advantages will emerge from the following description, provided for informational purposes and in no way limiting, with reference to the attached drawings, in which: There figure 1 represents a flowchart in accordance with the method of the invention; The figure 2 represents a schematic view of a first embodiment of a device according to the invention; and The figure 3 represents a schematic view of a second embodiment of a device according to the invention; and The figure 4 illustrates the grouping of substantially identical elements into classes. 6. Description détaillée

[0022] The invention proposes a determination of spatial positions and kinematic parameters such as the speed, acceleration or trajectory of target objects in the environment of a vehicle from repeatable elements of this environment.

[0023] An example of application of the invention is given in figure 4 The method of the invention allows a vehicle V1 traveling along a row of trees B1-B6 planted at regular intervals, for example every fifty meters, to know the position and / or speed of a target object, in this case, a truck V2 arriving in the opposite direction and located between the fifth tree B5 and the sixth tree B6. To do this, the vehicle V1 proceeds as follows: the vehicle V1 determines a target object V2 (the truck) located on the road R, the vehicle V1 determines that a series of substantially identical and repeatable elements B1-B6 (the trees) is located along the road R, the vehicle V1 determines the interval between the successive substantially identical elements, here fifty meters, the vehicle V1 determines that it is located at the first tree B1, the vehicle V1 positions the target object V2 between the 5th tree B5 and the 6th tree B6, the vehicle V1 deduces that it is located at a distance from the target object V2 of, according to the example proposed, between 200 meters and 250 meters, the vehicle V1 calculates that the target object V2 passes from the 6th tree B6 to the 5th tree B5 in 2 seconds, the vehicle V1 deduces that the target object V2 is moving at a speed of 25m / s (90km / h). The application of the invention therefore allowed the vehicle V1 to determine that the truck V2 arriving on the lane face is at least 200 meters away and moving at 90km / h.

[0024] A flowchart of the method for determining spatial positions and kinematic parameters according to the invention is illustrated in figure 1 This process involves a succession of steps which are repeated continuously and at a predetermined frequency as long as the determination is active.

[0025] In a first step E1, the vehicle camera captures images of the vehicle's surroundings. The images are captured at regular time intervals, for example every 40ms (25 images per second).

[0026] According to a step E2, the captured images are processed so as to determine a spatial distribution of substantially identical and regularly spaced elements such as trees, lampposts or road marking strips and to identify target objects on the road. At the end of this step, the substantially identical and regularly spaced elements are detected and their respective positions are known. Step E2 comprises, for example, the three sub-steps E21, E22 and E23.

[0027] According to step E21, the captured images are processed so as to extract a list of elements corresponding to elements detected in the environment of the vehicle. The extraction of lists of elements is similar to a detection of areas of interest in the captured images. The processing applied to the images also makes it possible to detect substantially identical elements and to group them into classes of substantially identical elements. The processing allowing the detection of substantially identical elements and the grouping of substantially identical elements is for example carried out by a pattern detection algorithm. Such processing can be carried out according to the methods and models proposed in the literature on this subject, see for example the following documentations: Detecting, localisation and grouping repetead scene element from an image, Thomas Leung and Jitendra Malik (http: / / www.eecs.berkeley.edu / Research / Projects / CS / vision / shape / papers / leung-ECCV96.pdf); Detecting and Matching Repeated Patterns for Automatic Geo-tagging in Urban Environments, Grant Schindler, Panchapagesan Krishnamurthy, Roberto Lublinerman2, Yanxi Liu, Frank Dellaert (https: / / smartech.gatech.edu / bitstream / handle / 1853 / 38342 / Schindler08cvpr.pdf); A Computational Model for Repeated Pattern Perception using Frieze and Wallpaper Groups, Yanxi Liu and Robert T. Collins (https: / / www.ri.cmu.edu / pub_files / pub2 / liu_yanxi_2 000_2 / liu_yanxi_2000_2.pdf); Automatic Kronecker Product Model Based Detection of Repeated Patterns in 2D Urban Images (http: / / www.cv-foundation.org / openaccess / content_iccv_2013 / papers / Liu_Automatic_Kronecker_Product_2013_ICCV_paper.p df).

[0028] The grouping of substantially identical elements is illustrated in figure 4 : A1, A2...An, are the repeatable elements of class A ( Fig.4 : A1, A2...A7 are for example broken white lines of road markings), B1, B2...Bη, are the repeatable elements of class B ( Fig.4 : B1, B2...B5 are for example trees), C1, C2...Cm, are the repeatable elements of class C ( Fig.4 : C1, C2, C3 are for example traffic signs), ... It should be noted that at this stage of the process, each class contains substantially identical elements but these substantially identical elements are not necessarily regularly spaced. For example, the elements of class C of the figure 4 are not regularly spaced.

[0029] Furthermore, the elements located on the road are identified as target objects. At the end of this step, a list of elements is obtained, the substantially identical elements being associated with a class and the target objects being identified as such.

[0030] According to step E22, the position of the elements of each class of substantially identical elements is estimated. This position estimation is advantageously carried out by combining the following methods: homography which makes it possible to determine a distance from the element to the camera by projecting said element onto the road plane and calculating the width of the road at the location of the projection; and SfM image processing (for “Structure from Motion” in Anglo-Saxon literature) which makes it possible to determine the position of elements by comparing several successive images.

[0031] According to a particular embodiment, not all the elements of each class are positioned but only the elements whose resolution is sufficient to achieve acceptable precision during their positioning. Thus, only the clear elements having sufficient resolution, i.e. greater than a predetermined threshold, are positioned. At the end of this step, a correspondence list associating a position with each of the clear elements of the image is obtained.

[0032] According to step E23, the different classes of elements are analyzed using the position data obtained in step E22 to determine which classes actually contain substantially identical elements that are regularly spaced.

[0033] According to a particular embodiment, for each pair of successive identical clear elements, a spacing is calculated from the positions obtained in E22. For each class containing at least three identical clear elements, at least two spacings are obtained. When the difference between these at least two spacings within the same class is less than a certain threshold, it is considered that the elements of the class are regularly spaced. At the end of this step, a list of classes is obtained whose elements are regularly spaced, said list associating with each class a spacing between two consecutive elements. Thus, by combining the information obtained in steps E21, E22 and E23, a distribution in space of the regularly spaced elements is obtained.

[0034] According to step E3, spatial positions or kinematic parameters of the target objects identified in E21 are determined. The spatial positions and kinematic parameters are deduced from the information from step E2 concerning the identical and regularly spaced objects.

[0035] According to a particular embodiment, the target objects are projected onto the road plane and the position of this projection is compared with the positions of the substantially identical regularly spaced elements determined in E23. The position of the target objects can then be deduced from the position of the substantially identical regularly spaced elements. The positions of the target objects during successive iterations then make it possible to determine the other kinematic parameters such as the trajectory, the speed or the acceleration. Indeed, the successive images make it possible to position the target object over time. The successive positions of the target object over time draw the trajectory of the target object. By deriving the trajectory thus determined with respect to time, the speed of the target object is obtained. By deriving the speed thus obtained with respect to time, the acceleration of the target object is obtained.

[0036] According to a particular embodiment, the processing also takes into account the data of the vehicle model to integrate the movement of the vehicle into the calculations. This vehicle model can in particular be based on odometry to calculate the distance traveled by the vehicle from the rotation of the wheels. At the end of this step, a list of target objects is obtained, said list associating their spatial positions and their kinematic parameters with the target objects.

[0037] A device for determining spatial positions and kinematic parameters capable of implementing the method of the invention is shown schematically in figure 2 .

[0038] The determination device is mounted in a vehicle equipped with at least one camera 10.

[0039] The camera 10 has the function of capturing images of the vehicle's surroundings. It is, for example, positioned in the front bumper, the vehicle's grille, at the top of the windshield, in the driver's rearview mirror or in any location that ensures observation of the scene surrounding the vehicle with a wide field of vision.

[0040] The detection device comprises a calculation module 11, typically a microprocessor. Said module 11 receives the images captured by the camera 10 and processes them in accordance with the method mentioned in step E2 to determine a spatial distribution of the substantially identical and regularly spaced elements and to identify target objects on the road.

[0041] The spatial distribution of the substantially identical and regularly spaced elements and the list of target objects are stored in memory in an internal or external storage module 12 of the detection device.

[0042] The device further comprises a module for determining spatial positions and kinematic parameters of target objects. Advantageously, said module is integrated into the calculation module 11. This module processes the information from the storage module containing the spatial distribution of the substantially identical and regularly spaced elements and the list of target objects in accordance with the method described in step E3. . The spatial positions and kinematic parameters of the target objects are also stored in memory.

[0043] An improved device for determining spatial positions and kinematic parameters capable of implementing the method of the invention is shown schematically in figure 3 .

[0044] According to one embodiment, the module 11 receives information from the vehicle's on-board computer from the vehicle model in order to integrate kinematic data from said vehicle into the calculations carried out in step E3. .

[0045] Advantageously, the module 11 is interfaced with another detection device 13 such as a radar in order to detect any inconsistencies between the two devices.

[0046] The method of the invention can be coupled with alarm management. The information resulting from the determination of spatial positions and kinematic parameters of target objects is processed in order to detect a danger such as a risk of collision, an overtaking maneuver in the presence of a vehicle in front or an inconsistency within said information.

[0047] A risk of collision between a vehicle and a target object can be estimated from the respective trajectories of said vehicle and said target object, their relative speed and their distance. When said trajectories intersect and the distance between said target object and the car is less than a threshold depending on their relative speed, a risk of collision is detected.

[0048] Furthermore, if the device for determining spatial positions and kinematic parameters is interfaced with other detection devices such as a radar and their data for the same kinematic parameter of the same object differ by a difference greater than a certain threshold, an inconsistency is detected. Following the detection of hazards, the alarms corresponding to the identified hazards are issued.

[0049] The method of the invention can be implemented in a device interfaced with a driving automaton making it possible to provide the driver of the vehicle with assisted driving. The information resulting from the determination of spatial positions and kinematic parameters of target objects is processed in order to determine the optimal behavior of the vehicle, for example, to avoid a detected danger in a manner similar to that described above for alarm management. Said optimal behavior of the vehicle comprises, for example, the trajectory and the deceleration that the vehicle must have to avoid the danger. From said optimal behavior of the vehicle, driving instructions are determined and transmitted to the driving automaton so that it assists the driver in his driving. For example, the automaton regulates a speed making it possible to avoid a collision.Conventionally, the action of a driving automaton is under the control of the driver, who can take control when desired by performing a certain manipulation. In this sense, the driving automaton is not usually considered as operating automatic driving but as providing driving assistance. In this case, the driver is assisted in his driving according to the spatial positions and kinematic parameters of objects in the vehicle's environment.

[0050] The method of the invention can be implemented by the on-board computer of a vehicle equipped with a camera and optionally a radar by integrating modules 11 and 12 into the on-board computer.

[0051] The invention is described in the foregoing by way of example. It is understood that the person skilled in the art is able to carry out different variant embodiments of the invention, for example by combining the different characteristics above taken alone or in combination, without departing from the scope of the invention, as defined by the appended claims.

Claims

1. Method for determining at least one spatial position or kinematic parameter, such as the trajectory, speed or acceleration, of at least one target object in the environment of a vehicle, said vehicle being equipped with at least one camera for capturing images of the environment of said vehicle, comprising: - a step of capturing (E1) at least one image of the environment of the vehicle comprising the road on which the vehicle and the target object are travelling, characterized in that it furthermore comprises the following steps: - determining (E2), in said at least one image, the spatial distribution of at least three substantially identical and regularly spaced elements, and determining at least one target object; and - determining (E3) at least one spatial position or kinematic parameter of said target object based on said spatial distribution of at least three substantially identical and regularly spaced elements, in that the step of determining (E2) the spatial distribution of at least three substantially identical and regularly spaced elements and of determining at least one target object contained in said at least one image comprises the following steps: - E21: detecting at least three substantially identical elements and a target object that are contained in said image, and grouping said at least three substantially identical elements into at least one class, - E22: determining the spatial position of at least three substantially identical elements in said at least one class, - E23: determining at least one class comprising at least three substantially identical elements that are regularly spaced among said at least one class, and in that, during the step of determining (E3) at least one spatial position or kinematic parameter of said target object, said target object is projected onto the plane of the road and the position of this projection is compared with the positions of the substantially identical regularly spaced elements determined in step E23.

2. Method according to the preceding claim, characterized in that the step of detecting (E21) at least three substantially identical elements and a target object that are contained in said image and of grouping said at least three substantially identical elements into at least one class comprises detecting at least one pattern in said image.

3. Method according to Claim 1, characterized in that said determining (E22) the spatial position of at least three substantially identical elements in said at least one class comprises projecting said at least three substantially identical elements onto the plane of said road.

4. Method according to Claim 1, characterized in that said determining (E23) at least one class comprising at least three substantially identical elements that are regularly spaced among said at least one class comprises determining the spacing between said at least three clear elements among said at least three substantially identical elements.

5. Method according to Claim 1, characterized in that said determining (E3) at least one spatial position or kinematic parameter of said target object based on said spatial distribution of at least three substantially identical and regularly spaced elements comprises projecting the target object onto the plane of said road.

6. On-board device on board a motor vehicle for determining spatial positions and kinematic parameters of objects in the environment of said vehicle, said vehicle being equipped with at least one camera (10) for capturing images of the environment of the vehicle, characterized in that it comprises a computing unit (11) able to implement the method according to any one of Claims 1 to 5.

7. Motor vehicle equipped with at least one camera for capturing images of its environment, characterized in that it furthermore comprises a device according to Claim 6.

8. Vehicle according to Claim 7, furthermore equipped with a radar, characterized in that said radar is interfaced with said on-board device.