Segment detection

Abstract

A method for detecting a neighboring vehicle in the vicinity of a primary vehicle, using a driver assistance device connected to a camera mounted on the primary vehicle capable of acquiring images within a field of view and at acquisition times; obtaining, for a pair of successively acquired images, at least one lateral segment and an associated time data point; determining a lateral distance associated with the lateral segment from at least one measurement taken on one image of said pair of images; determining a speed associated with the lateral segment from the lateral distance and the time data point; determining, from at least the speed associated with the lateral segment and a speed of the primary vehicle, a plausibility score associated with the lateral segment; and, if the plausibility score is greater than at least one threshold, detecting the presence of the neighboring vehicle. Abstract Figure: Figure 2

Description

Title of the invention: Segment detection Technical field

[0001] The present disclosure relates to the field of driver assistance systems and more particularly to the automatic detection of vehicles in areas of interest, in particular lateral areas. Prior art

[0002] The rise of Intelligent Transport Systems (or ITS) has led to the development of numerous systems embedded in vehicles, particularly road transport. Such embedded systems include driver assistance systems and autonomous driving. In particular, the detection of objects, including vehicles, plays an important role in issues such as traffic flow, road safety and road infrastructure management (for example, variable message signs or automatic speed cameras).

[0003] In the context of systems embedded in autonomous or semi-autonomous vehicles traveling in road traffic (for example, on a highway), the detection of objects surrounding a vehicle concerned is particularly important in order to avoid collisions between the vehicle concerned and other vehicles, for example. In particular, in lane change assistance, automatic lane change or blind spot detection systems, precise and anticipated detection of obstacles such as a central reservation or another vehicle located in a lane adjacent to the vehicle concerned is required. Indeed, if the autonomous vehicle concerned does not detect (or does not detect sufficiently early) another vehicle located in a neighboring lane and moves out of the way, a collision may occur between the two vehicles.

[0004] In such a context, most existing vehicle detection methods are based on classification algorithms for identifying vehicles from image sequences or video streams acquired by vision sensors (typically, a camera). Such vision sensors are generally arranged at the front and / or rear of the vehicle concerned, so as to acquire images and / or image sequences of a front and / or rear field of view (FoV) of the vehicle concerned. The classification algorithms can then identify one or more vehicles neighboring the vehicle concerned on the basis of frontal recognition of the neighboring vehicles (i.e. by identifying and classifying vehicles detected in the field of view from their front or rear face).

[0005] However, such detection methods using vehicle classifiers prove ineffective when a front face of the neighboring vehicles is not yet detectable in the field of view, even if such neighboring vehicles are partially visible in the field of view, for example via a partial side face. Thus, if the vehicle concerned is located on a motorway lane and a neighboring vehicle is located relatively level with (or slightly behind) the vehicle concerned on an adjacent lane, a camera placed at the rear of the vehicle concerned would capture a rear lateral portion of the neighboring vehicle. In such a situation, the front face of the neighboring vehicle is not yet visible to the camera placed at the rear of the vehicle concerned, for example until the speed differential between the vehicle concerned and the neighboring vehicle is large enough for the vehicle concerned to completely overtake the neighboring vehicle.Until such a complete overtaking occurs, the neighbouring vehicle is therefore located substantially at the same level as the vehicle concerned and a lane change of the vehicle concerned into the neighbouring lane would result in a collision with the neighbouring vehicle. Such a criticality situation is, however, not detectable by existing classification methods.

[0006] There is therefore a need to secure the decision-making of driver assistance and / or autonomous driving systems, in particular in such a context of lane change decision. In particular, there is a need for early detection of vehicles surrounding a vehicle in question even before a frontal view (front or rear) of such surrounding vehicles is visible in a field of view of a vision sensor of the vehicle in question - and therefore that a classification and conventional tracking of such surrounding vehicles by existing classifiers is possible. Abstract

[0007] The present disclosure improves such a situation.

[0008] A method is proposed for detecting at least one neighboring vehicle present in an environment of a main vehicle, said neighboring vehicle and said main vehicle being motor vehicles, the method being implemented by a device configured to provide a driving assistance function for the main vehicle, said device being connected to at least one camera on board the main vehicle and capable of acquiring images of a scene surrounding the main vehicle according to at least one field of view and at acquisition times, the method comprising the following steps: - obtain, for at least one pair of successively acquired images: * at least one lateral segment corresponding to a lateral coplanar set of optical flows detected in the field of view, between the respective acquisition times of the two images of said pair of images, and * temporal data linked to a collision time associated with said lateral segment, - determine, according to at least one predefined coordinate system, a value relating to a lateral distance associated with the lateral segment, said lateral distance being determined from at least one measurement carried out on one image among the two images of said pair of images, - determine a speed associated with the lateral segment from the lateral distance and the collision time, - determining, from at least the speed associated with the lateral segment and a known speed of the main vehicle, a plausibility score associated with the lateral segment, and - if the plausibility score is greater than at least a predetermined threshold, detecting the presence of the neighboring vehicle in the scene.

[0009] Consequently, the proposed method allows the implementation of an early detection of a vehicle neighboring the main neighbor, namely as soon as such a neighboring vehicle enters, even partially, into the field of view of a camera on board the main vehicle. In other words, the proposed method makes it possible to detect a vehicle neighboring the main vehicle despite the absence of visibility of a front or rear frontal face of the neighboring vehicle on which most existing classifiers rely for vehicle detection.

[0010] Thus, the method allows a main vehicle to benefit from a driving assistance function in potentially critical moments, for example when the main vehicle wishes to change lanes or overtake and neighboring vehicles are located almost at the same level as the main vehicle, or even in the latter's blind spot.

[0011] By a driving assistance function of the main vehicle, reference is made to a functionality of a system on board the main vehicle making it possible to assist, guide or even decide on guidance of the main vehicle. Such a driving assistance function can be implemented in the context of a semi-autonomous or autonomous vehicle for example, with or without a driver. A driving assistance function can include assistance with driving, changing lanes, overtaking another vehicle, or even parking for example.

[0012] By a field of view, reference is made to a portion of the surroundings of the main vehicle covered by the camera's field of view. Such a field may for example be in front and / or behind the main vehicle.

[0013] By images acquired at acquisition times, reference is made to a continuous or discretized succession of images (i.e., in two dimensions) of the portion of the environment included in the field of view, associated with a succession of instants at which these images were acquired. In particular, the movement of the elements surrounding the main vehicle over time implies a field of view covering an evolving environment: the vehicles and elements neighboring the main vehicle exhibit relative motion to the main vehicle (as the main vehicle moves) and / or absolute motion in the environment.

[0014] By a lateral segment corresponding to a lateral coplanar set of optical flows detected in the field of view, between the respective acquisition times of the two images of said pair of images, reference is made to obtaining a pixel displacement field observed between two acquired images. An optical flow can then be likened to a bipoint (or vector) representing a displacement in space. A set of coplanar optical flows reflecting a common movement can then be associated with a segment. Three different types of segments can then be distinguished in a three-dimensional environment, each type of segment being parallel to a plane formed by two axes of a Cartesian reference system associated with the three-dimensional environment. A lateral segment then corresponds to a segment parallel to a plane called lateral or oblique with respect to a main axis of circulation of the main vehicle (eg, on a motorway lane).

[0015] By a temporal data linked to a collision time associated with said lateral segment, reference is made to a value reflecting a time before collision between the element associated with the lateral segment and the main vehicle observing such a lateral segment. Such a collision time can be deduced for example from the calculation of an optical flow, a set of coplanar optical flows forming a lateral segment having the same collision time. A collision time can then be associated with a lateral segment.

[0016] By a lateral distance, reference is made to a lateral or bias gap between the element associated with the detected lateral segment and the main vehicle. For example, in the case of a main vehicle and a neighboring vehicle traveling on two parallel traffic lanes (e.g., two highway lanes), the lateral distance corresponds to the lateral gap separating the lanes occupied by the main and neighboring vehicles. Such a lateral distance thus reflects a gap in the environment between the main and neighboring vehicles.

[0017] A plausibility score refers to a value reflecting a probability (or plausibility) that the detected lateral segment corresponds to a vehicle neighboring the main vehicle. Such a plausibility score can then be determined from parameters associated with the lateral segment (e.g., its relative speed with respect to the main vehicle, its absolute speed in the environment, its position in the environment, etc.) and one or more predetermined tolerance thresholds, which make it possible to delimit the values ​​of said parameters as plausibly assimilable to a vehicle. Thus, the higher the plausibility score, the more the object associated with the lateral segment can be plausibly assimilated to a vehicle.

[0018] According to another aspect, there is provided a device configured to provide a function for assisting in driving a main vehicle, said device being connected to at least one camera on board the main vehicle and capable of acquiring images of a scene surrounding the main vehicle according to at least one field of view and at acquisition times, the device comprising at least one processing circuit configured to implement a method for detecting at least one neighboring vehicle present in an environment of the main vehicle as proposed.

[0019] According to another aspect, there is provided a computer program comprising instructions for implementing the method as proposed when this program is executed by a processor.

[0020] According to another aspect, there is provided a non-transitory recording medium readable by a computer on which is recorded a program for implementing the method as proposed when this program is executed by a processor.

[0021] The features set out in the following paragraphs may, optionally, be implemented, independently of one another or in combination with one another:

[0022] According to one embodiment, the lateral distance associated with the lateral segment is determined from an optical flow belonging to the chosen lateral segment so that a minimum angle is measured between said chosen optical flow and a vertical axis defined according to the predefined coordinate system.

[0023] Consequently, the proposed method makes it possible to determine a lateral distance effectively separating the main vehicle from the object associated with the lateral segment in the environment (which is not yet identified by the device as being a vehicle or not at this stage). Such a lateral distance can in particular be obtained by carrying out measurements on the image presenting the lateral segment. The minimum angle determined then makes it possible to ensure that the optical flow chosen to determine the lateral distance is the closest to the ground among all the optical flows forming the lateral segment.

[0024] According to one embodiment, the known speed of the main vehicle and the speed associated with the lateral segment each being composed of a lateral component and a longitudinal component according to a predefined coordinate system, the plausibility score varies inversely with a difference between the longitudinal component of the speed associated with the lateral segment and the longitudinal component of the known speed of the main vehicle.

[0025] Consequently, the plausibility score may depend on a difference in forward speed (or movement) between the main vehicle and the object associated with the lateral segment. The plausibility score then makes it possible to assimilate to a vehicle the lateral segments having a longitudinal movement speed close to the known speed of the main vehicle.

[0026] By the longitudinal component of a speed, reference is made to the component of the speed along the main direction of travel of the vehicle. For example, in the case of a vehicle moving on a motorway lane, the longitudinal component of the speed corresponds to the speed component along the axis of the road (i.e., of the motorway lane). By the lateral component of a speed, reference is made to the component of the speed along a biased or lateral direction relative to the main direction of travel of the vehicle. For example, in the case of a vehicle moving on a motorway lane, the lateral component of the speed corresponds to the speed component along an axis on the ground perpendicular to the axis of the road, i.e., the axis of travel of a vehicle moving from one motorway lane to another, for example.

[0027] Thus, the difference between the two longitudinal components of two speeds makes it possible to quantify a differential in travel speed between the main vehicle and the element associated with the lateral segment. For example, if the element associated with the lateral segment is a sign or a tree leaf, the differential in longitudinal speed components of such an element and the main vehicle will exceed a predetermined tolerance threshold, which will reduce the plausibility score.

[0028] According to one embodiment, the known speed of the main vehicle and the speed associated with the lateral segment each being composed of a lateral component and a longitudinal component in a predefined coordinate system, the plausibility score varies inversely with an increasing value of the lateral component of the speed associated with the lateral segment.

[0029] Therefore, the plausibility score makes it possible to evaluate whether the element associated with the lateral segment moves in a direction substantially similar to a direction of movement of the main vehicle. Indeed, the greater the lateral component of the speed associated with the lateral segment, the more it reflects a significant lateral displacement of the element associated with the lateral segment.

[0030] According to one embodiment, the method further comprises: - a determination, according to the predefined coordinate system, of a value relative to a height associated with the lateral segment, and in which the plausibility score depends on the height associated with the lateral segment.

[0031] Therefore, the plausibility score takes into account a dimension associated with the lateral segment, such a dimension being able to reflect a size or a height of the element associated with the lateral segment. The plausibility score therefore makes it possible to verify that the element associated with the lateral segment has a height comparable to a vehicle.

[0032] Thus, an insufficient height (eg, less than a minimum height threshold predefined) may indicate that the element associated with the lateral segment is "too low" to be considered a vehicle (e.g., a central reservation separating two traffic lanes). Conversely, too great a height (e.g., greater than a predefined maximum height threshold) may indicate that the element associated with the lateral segment is "too high" to be considered a vehicle (e.g., a bird flying above the vehicles).

[0033] According to one embodiment, the method further comprises, after detecting the presence state of the neighboring vehicle corresponding to a presence of the neighboring vehicle,: - a determination, according to the predefined coordinate system, of a position of at least one reference point associated with the lateral segment, said position of the reference point being determined from at least the lateral distance associated with the lateral segment, and - creation of an object associated with the lateral segment in the camera's field of view from at least the position of the reference point.

[0034] Therefore, the proposed method makes it possible to detect the presence of a neighboring vehicle in the environment of the main vehicle, as well as to estimate a position of such a vehicle, once detected, in the environment. The method also makes it possible to create a virtual object associated with the lateral segment from such a position, in order to mark the presence of a detected neighboring vehicle.

[0035] By a reference point associated with the lateral segment, reference is made to a point belonging to the element associated with the lateral segment, which is, at this stage, identified as being a neighboring vehicle. Such a reference point can for example be assimilated to a front end of the neighboring vehicle. Such a reference point can for example be a point whose position marks a presumed position of a front face of the vehicle.

[0036] By a created object associated with the lateral segment in the field of view of the camera, reference is made to a virtual object superimposable on an image acquired by the camera. Such a created virtual object is further associated with the lateral segment in the sense that the object is estimated so as to be superimposed on a location presumed to be occupied by the detected neighboring vehicle. Such a virtual object is in particular characterized by a position in the environment and / or in the image. Such a virtual object may also be characterized by dimensions so as to reflect presumed dimensions of the detected vehicle.

[0037] Thus, the virtual object created can be used to provide a driving assistance function to the main vehicle, for example by displaying such a virtual object on a field of view displayable on a dashboard of the main vehicle for example. The virtual object thus created then makes it possible to represent the space of the environment occupied by the neighboring vehicle, so that a signal or an alert can be triggered when it is detected that the space occupied by the main vehicle is approaching and / or substantially coincides with the space occupied by the virtual object.

[0038] According to one embodiment, the object associated with the lateral segment is further created at from predetermined dimensions associated with a type of motor vehicle.

[0039] Consequently, the created virtual object may include a modeling of the neighboring vehicle by representing the presumed space occupied by such a vehicle. The predetermined dimensions may for example correspond to average dimensions of vehicles traveling on such a portion of the environment, or even to maximum dimensions of a vehicle, so as to assist the driving of the main vehicle in an “unfavorable” configuration (where the space of the environment occupied by the main vehicle can coincide more quickly with the space occupied by the neighboring vehicle compared to a smaller vehicle). Brief description of the drawings

[0040] Other characteristics, details and advantages will appear on reading the detailed description below, and on analyzing the attached drawings, in which:

[0041] [Fig-1] [Fig.l] shows a diagram of a main vehicle according to a mode of realization.

[0042] [Fig.2] [Fig.2] shows an aerial view of a scene surrounding the main vehicle according to one embodiment.

[0043] [Fig.3] [Fig.3] shows a diagram of a driving assistance device according to one embodiment.

[0044] [Fig.4] [Fig.4] shows steps of a method for early detection of a vehicle according to one embodiment.

[0045] [Fig.5] [Fig.5] shows steps of a method for early detection of a vehicle according to one embodiment.

[0046] [Fig.6] [Fig.6] shows a shot of a scene surrounding the main vehicle according to one embodiment.

[0047] [Fig.7] [Fig.7] shows a shot of a scene surrounding the main vehicle according to one embodiment.

[0048] [Fig.8] [Fig.8] shows a shot of a scene surrounding the main vehicle according to one embodiment.

[0049] [Fig.9] [Fig.9] shows a shot of a scene surrounding the main vehicle according to one embodiment.

[0050] [Fig. 10] [Fig. 10] shows an aerial view of a scene surrounding the main vehicle according to one embodiment.

[0051] [Fig. 11] [Fig. 11] shows a modeling of a point of a scene surrounding the main vehicle according to one embodiment.

[0052] [Fig. 12] [Fig. 12] shows a virtual object associated with a lateral segment according to one embodiment. Description of the embodiments

[0053] Reference is made to [Fig.l]. [Fig.l] shows a diagram of a main vehicle VP. The main vehicle VP may be a motor vehicle. No limitation is imposed on the type of vehicle to which the main vehicle VP belongs. The main vehicle VP may, for example, be a private, utility, or industrial vehicle, and may, for example, correspond to a car, a van, a two-wheeler, a truck, or even a bus. The main vehicle VP may also be a towed vehicle towing, for example, a trailer, a semi-trailer, or even a caravan. The dimensions of the main vehicle VP may be between 2 meters and 20 meters in length, between 0.5 meters and 5 meters in width, and between 1 meter and 5 meters in height.

[0054] The main vehicle VP is equipped with at least one on-board system providing a plurality of functions or applications of the main vehicle VP. Such functions may, for example, correspond to cruise control, power steering, automated airbag deployment, automatic headlight adjustment, etc.

[0055] The main vehicle VP is in particular equipped with an on-board system allowing detection of objects surrounding the main vehicle VP, for example in the context of a driving assistance function of the obstacle detection type, lane change assistance or automatic lane change. For this, the on-board system of the main vehicle VP comprises a driving assistance device 2. The device 2 is capable of providing a driving assistance function on the basis of a plurality of shots (or images) of the environment of the main vehicle VP. For this, the device 2 is in particular connected to one or more vision sensors such as a camera or a camera 1 (the vision sensor will be considered to be a camera 1 in the remainder of the description). The device 2 may also be capable of transmitting or communicating data, for example relating to the driving assistance function provided.For this, the device 2 can be connected to a communication interface 50. In a particular embodiment, the interface 50 can be integrated into the device 2. Such a communication interface 50 can for example be a human-machine interface. The interface 50 can include a display screen, a touch screen, a dashboard, and / or even a speaker. The data transmitted by the device 2 to the interface 50 can for example correspond to assistance information indicating to the driver of the main vehicle VP whether or not he can change lanes.

[0056] Reference is now made to [Fig.2]. [Fig.2] illustrates an environment (or scene) ENV, in which the main vehicle VP is located. [Fig.2] is an aerial view of such an ENV scene.

[0057] The ENV environment can be defined in a three-dimensional frame (X,Y,Z), called a “world frame” as illustrated in Figures 1 and 2. The origin of such a world reference (X,Y,Z) is predefined and fixed in the ENV environment.

[0058] The main vehicle VP is considered to be moving in the scene ENV. Such a scene ENV corresponds for example to a road or a motorway made up of several traffic lanes. These traffic lanes may in particular be parallel to each other, as represented by the dotted vertical lines in [Fig.2]. [Fig.2] illustrates for example three traffic lanes, the main vehicle VP being located on the middle traffic lane. The main vehicle VP is considered to be moving in the main direction X of the reference frame (X,Y,Z), such a main direction being called the longitudinal direction. Such a movement is shown diagrammatically in Figures 1 and 2 by an arrow attached to the main vehicle VP. The main vehicle VP may also have a movement in the direction Y of the reference frame (X,Y,Z), called the lateral direction.Such a movement in the Y direction, called lateral movement (or displacement), can for example take place when the vehicle changes traffic lane. A movement in the Z direction, that is to say in height, of the main vehicle VP is not considered. The main vehicle VP is therefore considered to be kept on the ground and therefore has a height in the Z direction that is substantially constant, corresponding approximately to a predefined dimension of the main vehicle VP. In one embodiment, such a height in the Z direction of the main vehicle VP can vary by the order of a centimeter or a decimeter, such a variation in height in the Z direction being for example linked to shock absorbers of the main vehicle VP and / or to reliefs or roughness present in the ENV environment (in particular on the traffic lane of the main vehicle VP).In the remainder of the description, such a height in the Z direction of the main vehicle VP is considered known.

[0059] The scene ENV surrounding the main vehicle VP may also comprise other elements OV and vehicles VV1, VV2, VV3. The vehicles VV1, VV2, VV3 are neighboring vehicles of the main vehicle VP. Such neighboring vehicles VV1, VV2, VV3 are, like the main vehicle VP, motor vehicles moving in the scene ENV. Such neighboring vehicles VV1, VV2, VV3 may in particular have dimensions and movement characteristics (in terms of speed or acceleration for example) similar to or different from the main vehicle VP. For example, [Fig.2] may represent a main vehicle VP moving on a motorway lane and neighboring vehicles VV1, VV2, VV3 traveling on the motorway lanes neighboring the lane taken by the main vehicle VP. OV objects are neighboring elements of the main vehicle VP and can refer to any distinct element of a neighboring vehicle VV1, VV2, VV3.A neighboring element OV can for example correspond to an obstacle located in the scene such as a central reservation separating two traffic lanes, an indication sign or . another bird flying in the ENV scene.

[0060] As illustrated in [Fig. 2], the main vehicle VP is considered to be equipped with a camera 1 positioned at the rear of the main vehicle VP. In another embodiment (not shown in [Fig. 2]), the camera 1 can be positioned at the front of the main vehicle VP or several cameras 1 can be positioned both at the front and at the rear of the main vehicle VP. The camera 1 makes it possible to capture images (or shots) of the scene ENV of the main vehicle VP according to a field of view FOV (or in English "Field of View"). Such a field of view FOV depends in particular on the type of camera 1 used and the positioning of the camera 1 in (or on) the main vehicle VP. With reference to [Fig. 2], the field of view FOV covers a part of the scene ENV, so that a limited portion of the scene ENV is captured according to the field of view FOV. Thus, in [Fig.2], the field of view FOV represented covers a portion of the traffic lane in which the main vehicle VP is located and respective portions of the neighboring traffic lanes. In particular, the field of view FOV covers a portion of the scene ENV in which the neighboring vehicle VV2 is entirely located and a portion of the scene ENV in which the neighboring vehicle VV1 is partially located. For example, the portion of the scene ENV in which the neighboring vehicle VV1 is partially located may comprise a point A, called the first reference point A (having coordinates in the world reference frame (X,Y,Z)), positioned on the neighboring vehicle VV1, corresponding for example to a reference point placed at the rear-left of the neighboring vehicle VV1 at the height of the neighboring vehicle VV1.Unlike the first landmark A, a second landmark W, for example located at the middle of the front face of the vehicle closest to the ground, is not in the field of view FOV in [Fig. 2]. The neighboring vehicle VV3 is not visible in the field of view FOV of the main vehicle VP. A portion of the neighboring vehicle VV1 is not visible in the field of view FOV of the main vehicle VP. The non-visible parts of the neighboring vehicles VV1, VV3 in the field of view FOV are represented in [Fig. 2] by striped areas. A neighboring object OV, for example a bird, may be visible in the field of view FOV.

[0061] In the context of an ENV scene as represented in Figure 2 according to the reference frame (X,Y,Z), it is for example considered that the main vehicle VP moves at a known main speed y^. To facilitate the rest of the description, such a speed main vehicle is considered to have a constant VVp standard. The neighboring vehicle VV1 is considered to be moving at a neighboring speed y^ unknown to device 2 driving assistance for the main vehicle VP. In the embodiment described subsequently, it can be considered that the VVp standard of the neighboring speed V is in- lower than the VVp standard of the main speed ^VP Under such a hypothesis and under the assumption that the neighboring speed y^ of the neighboring vehicle VV1 remains substantially constant over a considered time interval, like the main speed y^ of the main vehicle VP, the distance difference in the longitudinal direction X between the main vehicle VP and the neighboring vehicle VV1 will increase over time, so that at a future time, the overtaking of the main vehicle VP on the neighboring vehicle VV 1 will be such that the neighboring vehicle VV 1 will be entirely included in the (rear) field of view FOV of the main vehicle VP, as is the initial case of the neighboring vehicle VV2.

[0062] Such an evolution of the membership of the neighboring vehicle VV1 in the field of view FOV of the main vehicle VP over time is illustrated in Figures 6, 7 and 8.

[0063] However, in another embodiment, the norm VVp of the neighboring speed y^ may be greater than the norm VVp of the main speed y^. Under such an assumption and under the assumption that the neighboring speed y^ of the neighboring vehicle VV1 remains substantially constant over a given time interval, like speed main y^ of the main vehicle VP, the distance difference according to the longitudinal direction tudinal X between the main vehicle VP and the neighboring vehicle VV 1 will increase over time, so that at a future time, the overtaking of the neighboring vehicle on the main vehicle VP will be such that the neighboring vehicle VV 1 will be entirely included in a field of view (front) of the main vehicle VP, for a camera 1 placed in front of the main vehicle VP (case not shown in the figures).

[0064] In the remainder of the description, the case of the main vehicle VP overtaking the neighboring vehicle VV1 is considered, so that the norm VVp of the neighboring speed y^ is considered to be lower than the norm VVp of the main speed . It is also considered, as shown in [Fig.2], that the camera 1 is placed at the rear of the main vehicle VP, so that the field of view FOV considered is a rear field of view relative to the main vehicle VP.

[0065] Reference is made to Figures 6, 7 and 8. Figures 6, 7 and 8 diagrammatically show shots of the scene ENV surrounding the main vehicle VP according to the (rear) field of view FOV of the camera 1 of the main vehicle VP, as positioned in [Fig. 2]. In order to facilitate the readability of Figures 6, 7 and 8, only the portion of the neighboring vehicle VV1 visible in the field of view FOV is shown in Figures 6, 7 and 8, the neighboring vehicle VV2 and the neighboring element OV of [Fig. 2] included in the field of view FOV are not shown in the shots of Figures 6, 7 and 8.

[0066] Figures 6, 7 and 8 correspond to images successively acquired by the camera 1 at successive instants (or acquisition times) T1, T2 and T3. Each of the instants T1, T2 and T3 may for example be spaced apart by one or more milli seconds in time. Each of the images corresponds to a set of pixels with definable coordinates in a two-dimensional (y,z) frame of reference, called the "image frame", as shown in Figures 6, 7 and 8. For example, the bottom-most and left-most pixel of each image acquired by the camera 1 may correspond to the origin of the two-dimensional (y,z) frame of reference. In other embodiments, the central pixel of the image acquired by the camera, the pixel corresponding to a vanishing point C' of the acquired image (these two pixels being the same in [Fig.6]) or the pixel with coordinates (yC', zO) as shown in [Fig.6] may correspond to the origin of the (y,z) frame of reference. The resolution of the acquired images (and therefore the pixels) is the same for all the acquired images and depends in particular on the properties of the camera 1.

[0067] The portion of the neighboring vehicle VV1 visible in the field of view FOV of the main vehicle VP over time is highlighted in Figures 6, 7 and 8, for example by observing the respective projections A', A”, A”' of the first reference point A mentioned above on the images acquired in Figures 6, 7 and 8.

[0068] [Fig. 6] schematizes an image acquired by the camera 1 at time T1, corresponding for example to a configuration of the scene ENV and the field of view FOV illustrated in [Fig. 2]. Thus, only a first, lateral portion of the neighboring vehicle VV1 is visible in the image of [Fig. 6], such a first lateral portion including in particular the left rear wheel of the neighboring vehicle VV 1 and the portion of the neighboring vehicle VV 1 on which the first reference point A is located. In particular, such a first reference point A is positioned on a pixel A' of the image of [Fig. 6]. Such a pixel A' then corresponds to the projection of the first reference point A in the image reference frame (y, z) at time T1 and has coordinates (yA', zA').

[0069] [Fig.7] schematizes an image acquired by the camera 1 at time T2 following time TL. At such a time T2, a second lateral portion of the neighboring vehicle VV1 larger than the first lateral portion is visible in the image of [Fig.7], such a second lateral portion including in particular the left rear wheel, partially the left front wheel of the neighboring vehicle VV 1 and the portion of the neighboring vehicle VV 1 on which the first reference point A is located. In particular, such a first reference point A is positioned on a pixel A” of the image of [Fig.7] different from the pixel A' of the image of [Fig.6]. Such a pixel A” then corresponds to the projection of the first reference point A in the image reference frame (y,z) at time T2 and has coordinates (yA”,zA”).In other words, the speed differential between the main vehicle VP and the neighboring vehicle VV1 between times T1 and T2 is such that the neighboring vehicle VV1 “enters more and more” into the field of view FOV of camera 1 between times T1 and T2.

[0070] [Fig.8] shows schematically an image acquired by camera 1 at time T3 following time T2. At such a time T3, the entirety of the neighboring vehicle VV1 is visible on the image of [Fig.8] and in particular, the portion of the neighboring vehicle VV1 on which the first reference point A is located as well as a portion of the neighboring vehicle VV1 on which the second reference point W shown in [Fig.2] is located and positioned on the front end of the neighboring vehicle VV1 are therefore visible. In particular, the first reference point A is positioned on a pixel A”' of the image of [Fig.8], different from the pixels A' and A” of the images of figures 6 and 7. Such a pixel A'” then corresponds to the projection of the first reference point A in the image frame (y,z) at time T3 and has coordinates (yA'”,zA”'). The second reference point W is positioned on a pixel W”' of the image of [Fig.8]. Such a pixel W'” then corresponds to the projection of the second reference point W in the image frame (y,z).In other words, the speed differential between the main vehicle VP and the neighboring vehicle VV 1 between times T2 and T3 is such that the neighboring vehicle VV 1 “entered completely” into the field of view FOV of camera 1 between times T2 and T3.

[0071] In the context of a driving assistance function provided by the device 2, for example aiming to assist the main vehicle VP in a lane change maneuver, a process for detecting vehicles surrounding the main vehicle VP is required, so that the main vehicle VP does not move onto a neighboring traffic lane if it risks colliding with one of the surrounding vehicles. For example, in the context of [Fig. 2], the device 2 may aim to assist the main vehicle VP in its lane change to the traffic lane of the neighboring vehicle VV1, so that the main vehicle VP does not collide with the neighboring vehicle VV1.Existing techniques for detecting vehicles surrounding the main vehicle VP rely in particular on the use of classifiers, based for example on convolutional neural network (CNN) methods, K-Nearest Neighbors (KNN) or support vector machines (SVM). Such classifiers allowing the detection of vehicles surrounding the main vehicle VP rely in particular on the detection and classification of a frontal (front or rear) view (or face) of the vehicles surrounding the main vehicle VP. The use of such classifiers may in particular comprise a learning phase based on a plurality of images representing frontal views of various types of vehicles. Thus, with reference to [Fig.2], a front frontal view of the neighboring vehicle VV2 belonging to the field of view FOV of camera 1, the detection of the neighboring vehicle VV2 can be implemented based on existing classification techniques. The same applies to the neighboring vehicle VV 1 in the image of [Fig.8] at time T3. .

[0072] However, the positions of the neighboring vehicle VV1 shown in [Fig.2] and Figures 6 and 7 correspond to situations in which existing classifiers fail to detect or track the neighboring vehicle VV1 effectively, or in any case, not without generating significant costs and / or computational time, due to the absence of a visible frontal view of the neighboring vehicle VV1 in the shots of Figures 6 and 7 (the second reference point W of [Fig. 2] positioned on the front frontal face of the neighboring vehicle VV1 not yet being visible in the field of view FOV of camera 1 at these stages). These situations, however, correspond to critical situations during which the main vehicle VP could collide with the neighboring vehicle VV1 if the main vehicle VP changes position and moves into the lane of the neighboring vehicle VV1, in the absence of lane change assistance information indicating that the neighboring vehicle VV1 is detected near such a position.

[0073] A method for early detection of the neighboring vehicle VV 1 is then proposed and described in FIGS. 4, 5, 9, 10 and 11 for detecting the neighboring vehicle VV1, and in particular at the stage of the situations represented in FIGS. 2, 6 and 7. Such a detection method is described as early in that it is implemented before the visibility of a front or rear face of the neighboring vehicle VV 1 in the field of view FOV. Such a method for early detection of the neighboring vehicle VV1 is implemented by a driving assistance system for the main vehicle VP represented in [Fig.3].

[0074] Reference is now made to [Fig. 3]. [Fig. 3] represents a diagram of an on-board system of a main vehicle VP. In particular, such an on-board system corresponds to a driving assistance system for the main vehicle VP. The driving assistance system of the main vehicle VP makes it possible in particular to provide a lane change assistance or automatic lane change function for the main vehicle VP, when the main vehicle VP is moving on a traffic lane as represented for example in [Fig. 2].

[0075] The system firstly comprises the driving assistance device 2. The device 2 may in particular comprise a unit 20 for detecting optical flows, a unit 30 for detecting an object associated with the detected vehicle and a unit 40 for creating such an object associated with the detected vehicle.

[0076] The driving assistance device 2 is further connected, on the one hand, to a vision sensor of the camera or camera type 1. The device 2 comprises an input unit (not shown in [Fig. 3]) allowing the device 2 to receive in substantially real time a data stream from the camera 1. Such a data stream corresponds to a discrete or continuous succession of images (or shots) of the scene ENV according to the field of view FOV of the camera 1. Each image received is associated with an acquisition time of the image by the camera 1. Each image can be time-stamped. Examples of such images transmitted by the camera 1 are shown in Figures 6, 7 and 8.

[0077] On the other hand, the driving assistance device 2 can be connected to an interface 50 communication interface. Such an interface 50 may correspond to a human-machine interface integrated into the on-board system of the main vehicle VP. Such an interface 50 may also be integrated into the device 2. The communication interface 50 may also be a remote interface. The communication interface 50 may for example comprise a display screen, a touch screen, a dashboard or even a speaker, making it possible to transmit, for example by visual, haptic and / or audible information, indications relating to assistance in driving the main vehicle VP. In particular, the communication interface 50 may make it possible to transmit a detection status of a vehicle neighboring the main vehicle VP or even an indication relating to a possibility of changing lanes of the main vehicle VP.Such information transmitted by the interface 50 may for example correspond to an estimated aerial visual representation of the respective positions of the main vehicle VP and the neighboring elements belonging to the field of view FOV in real time (eg, as represented in [Fig.2]), or even an audio stimulus alerting of a risk of collision.

[0078] The optical flow detection unit 20 is capable of receiving and processing the data stream received from the camera 1. In particular, the unit 20 can, from the succession of images acquired by the camera 1 at successive acquisition times, obtain optical flows over given time intervals. Such a unit 20 can also implement a step of segmenting the determined optical flows, so as to obtain one or more segments associated with each acquired image (or segmented images), a segment then being associated with a set of optical flows on a given image. For this, the unit 20 can comprise a processing circuit composed of at least one processor 21 and at least one memory unit 22 capable of determining optical flows and / or segments associated with temporal data from the acquired images.

[0079] The unit 30 for detecting an object associated with the detected vehicle may be capable, for each segment determined by the unit 20, of implementing a plausibility test to associate this segment with a detected vehicle. For this, the unit 30 may comprise a processing circuit composed of at least one processor 31 and at least one memory unit 32, such a processing circuit being capable of implementing the plausibility test and, for example, of determining a plausibility score or a probability of presence of a vehicle obtained at the end of such a test.

[0080] The unit 40 for creating an object associated with the detected vehicle may be capable, for each vehicle detected by the unit 20 and according to the plausibility score determined by the unit 30, of creating a virtual object associated with the detected vehicle corresponding to the optical flows (or segments) detected as belonging to such a vehicle. For this, the unit 40 may comprise a processing circuit composed of at least one processor 41 and at least one memory unit 42, the processing circuit of the unit 40 being capable of taking as input data from the units 20 and 30 and of determining initial conditions (e.g., initial position and / or speed data, vehicle dimensioning data, etc.) so as to create a virtual object associated with the vehicle detected by the unit 30. The virtually created object and / or its initial conditions may then be transmitted for communication to the user of the device 2 (e.g., the driver of the main vehicle VP) via the communication interface 50.

[0081] Reference is now made to [Fig. 4]. [Fig. 4] illustrates a succession of steps for implementing a method for early detection of a neighboring vehicle VV1 by a system including a device 2 for assisting the driving of a main vehicle VP as shown in Figures 1 and 3.

[0082] In a first step 400, the device 2 receives a plurality of images (or shots) associated with respective acquisition times of the images from the camera 1. Such images may in particular be received continuously, for example via a video stream. In such a case, in step 400, the unit 20 of the device 2 may discretize the received video stream so as to obtain a set of discrete images associated with respective acquisition times. From at least two successively acquired images, the unit 20 of the device 2 may proceed, in step 400, to image processing, for example by homographies, so as to identify one or more related sets of relatively homogeneous speed on the images. Such image processing then consists of determining a displacement of pixels at a substantially common speed from one image to another, or optical flow.Each optical flow can be determined by matching points between several successive images, for example by the Lucas-Kanade method. Each optical flow is then determined on at least one of the images of a pair of images and is associated with iTTC temporal data. In particular, such iTTC temporal data relates to an inverse magnitude of a collision time associated with the optical flow. The determined optical flows can in particular, in step 400, be segmented into one or more segments having a common point or vanishing line and belonging to planes in the field of view FOV. In other words, each acquired image can be segmented into one or more segments, each segment defining a grouping of optical flows (and therefore a grouping of pixels having substantially the same speed and describing the same movement in a given time interval).Step 400 can then comprise, after determining such optical flows, the generation of hypotheses of movement of the objects and vehicles of the ENV scene, for example on the basis of an affine movement model of the objects and vehicles of the ENV scene, and to classify (or partition) the optical flows into segments. A segment can then be. defined as a set of optical flows having a substantially common motion. In particular, all the optical flows of a segment have a substantially common collision time value iTTC: we therefore consider that a collision time associated with a lateral segment SL is equivalent to a collision time associated with an optical flow belonging to this lateral segment SL. In addition, a segment can be defined by a coplanar set of optical flows. It is thus possible to distinguish three types of segments in a segmented image:

[0083] - so-called “horizontal” segments, corresponding to coplanar sets of optical flows belonging to planes parallel to the (X, Y) plane in the world frame (X, Y, Z). For example, a horizontal segment can correspond to the plane formed by the road in images 6 to 9.

[0084] - so-called “vertical” segments, corresponding to coplanar sets of flows optics belonging to planes parallel to the plane (Y,Z) in the world frame (X,Y,Z). For example, a vertical segment can correspond to the plane of camera 1.

[0085] - so-called “lateral” segments, corresponding to coplanar sets of flows optics belonging to planes parallel to the plane (X, Z) in the world frame (X, Y, Z). For example, a lateral segment SL is represented in [Fig.9].

[0086] Obtaining such optical flows and such segments from successively acquired images as well as iTTC temporal data associated with each optical flow are notably described in prior documents WO 2018059629, WO 2018059631 and WO 2018059632. For the remainder of the description, it is considered that at least one segment is determined in step 400. In particular, for the remainder of the description, it is considered that at least one lateral segment is determined from several coplanar optical flows in a detected lateral plane and the remainder of the method is implemented for each lateral segment determined in a segmented image.

[0087] Such a step 400 can be implemented concomitantly with the following steps of the method, in particular when the camera sends a stream of images in real time and continuously (non-punctual). The device 2 can implement the rest of the steps (410, 420, 430 and 440) of the method on a lateral segment considered and continue to detect other optical flows (and / or other segments) in step 400.

[0088] In a step 410, a lateral distance d is first determined for at least one lateral segment SL. Reference is made for this to [Fig. 9]. [Fig. 9] corresponds to the image acquired by the camera 1 at time T1 (corresponding to the image of [Fig. 6]) on which is represented a lateral segment SL determined by the unit 20 in step 400. The lateral segment SL comprises in particular a set of optical flows represented schematically by points in [Fig. 9]. In another embodiment, the optical flows can be represented schematically by bipoints reflecting a displacement between two successively acquired images.

[0089] A vertical axis VL of the acquired image is considered. A vanishing point of the acquired image is also considered, corresponding to the vanishing point of the optical flows in the image. Such a vanishing point is represented by the point C' with coordinates (yC',zC') in [Fig.9].

[0090] In order to determine the lateral distance d associated with the detected lateral segment SL, the angle formed by each of the straight lines formed between each of the optical flows of the lateral segment SL and the vanishing point C' with the vertical axis VL is considered. An optical flow B' of the lateral segment SL is then chosen so that the angle formed 0min between such a chosen optical flow B' and the vanishing point C' with the vertical VL is minimal among the angles 0 respectively formed between each of the optical flows of the lateral segment SL and the vanishing point C' with the vertical VL. With reference to [Fig.9], an optical flow thus chosen corresponds to the optical flow closest to the ground among the optical flows of the lateral segment SL.

[0091] The chosen optical flow is associated with a pixel B' with coordinates (yB',zB') on the acquired image and corresponding to a third reference point B of the vehicle detected in the scene ENV (not shown in the figures, corresponding for example to a reference point located on the left rear wheel of the neighboring vehicle VV1). In the remainder of the description, reference will be made to the optical flow B' corresponding to the optical flow of the lateral segment SL being closest to the ground thus determined.

[0092] The chosen optical flow B' then corresponds to a pixel B' on one of the acquired images having coordinates (yB',zB') in the image frame (y,z). From such coordinates (yB',zB') in the image frame (y,z), the coordinates of the third reference point B are determined, in step 410, in the world frame (X,Y,Z). In particular, such coordinates of the third reference point B in the world frame (X,Y,Z) can be determined, from the coordinates (yB',zB') and the predefined characteristics of the camera 1, on the basis of the pinhole model (or in English, "pinhole model"). The application of the pinhole model in order to obtain the coordinates of the third reference point B in the world reference frame (X,Y,Z) from the coordinates (yB',zB') of the chosen optical flow B' is based in particular on several calculation hypotheses linked to the properties of camera 1 as well as to the ENV environment associated with the world reference frame (X,Y,Z).For this, conventional extrinsic parameters of the camera 1 can be considered. A “Bird View” type camera 1 can be used. In addition, a flat world assumption (or in English “fiat world assumption”) can be considered. Otherwise, georeferencing of the ENV environment by ground marking or other topographic surveys can also be considered. In the context of the applied pinhole model, the geometric distortions possibly induced by the optical system of the camera 1 can be neglected. In another embodiment, step 410 of determining the coordinates of the third reference point B. in the world frame (X,Y,Z) may include a correction for the distortion of the camera lens 1.

[0093] During step 410, it is then possible to obtain a position according to the world reference frame (X,Y,Z) of the third reference point B corresponding to the chosen optical flow B' in the environment ENV. A lateral distance d (real, expressed in meters) of the third reference point B can then be directly deduced from the coordinate along the Y axis of the position according to the world reference frame (X,Y,Z) of the third reference point B corresponding to the chosen optical flow B' in the environment ENV. The lateral distance d can in particular be obtained by calculating the difference between the coordinate along the Y axis of the position according to the world reference frame (X,Y,Z) of the third reference point B in the environment ENV and the coordinate along the Y axis of a point of origin of the camera 1 of the main vehicle VP, for example coincident with a vanishing point of the camera 1.The lateral distance d estimated at step 410 then corresponds to an estimate of a lateral gap separating the main vehicle VP from the object to which the detected lateral segment SL belongs (at the stage of step 410, the system 2 has not yet determined that it is a vehicle, namely the neighboring vehicle VV1).

[0094] A lateral distance d' (on the acquired image, expressed in pixels) of the optical flow B' can also be obtained from the coordinate yB' of the optical flow B' along the y axis in the image reference frame (y,z) as well as from a coordinate yC' of the origin (or central) point of the camera 1 of the main vehicle VP, for example coincident with the vanishing point C'. Such a central point can for example correspond to the optical center of the acquired image or to a point associated with the location of the camera 1 in the environment ENV. Such a lateral distance d' is for example represented in [Fig.9].

[0095] Therefore, the lateral distance d associated with a detected lateral segment SL corresponds to the lateral distance d associated with the optical flow B' detected in the lateral segment SL as being closest to the ground.

[0096] In the remainder of the method described and unless explicitly stated otherwise, any reference to an optical flow of the lateral segment SL corresponds to the chosen optical flow B' belonging to the lateral segment SL and having made it possible to determine the lateral distance d (or equivalently d') associated with the lateral segment SL.

[0097] At the end of step 410, a value of a speed associated ^7. with the lateral segment SL can then be estimated:

[0098] Math.l vSL-d * iTTCbù : - vsl is a relative speed associated with the lateral segment SL (expressed in meters per second), - d is the lateral distance associated with the lateral segment SL (expressed in meters), and - iTTC is the time data associated with the lateral segment SL (expressed as the inverse of a time).

[0099] The speed vsl is then an estimated relative speed in the scene ENV of the element corresponding to the detected lateral segment SL (such an element not yet being identified as being a vehicle or not at this stage). Such a speed vsl is relative to a known defined speed VVp of the main vehicle VP.

[0100] In a step 420, the lateral segment SL detected in step 400 and characterized in step 410 by a lateral distance d and in particular, by a relative speed vsl associated with the lateral segment SL, is tested according to a plausibility test, in order to identify whether such a lateral segment SL is likely to correspond to a neighboring vehicle VV1.

[0101] For this, a plausibility test implemented in step 420 can be based on one or more plausibility (or likelihood) criteria, each criterion being able for example to be quantified by a plausibility sub-score.

[0102] Reference is made to [Fig.5] detailing sub-steps of step 420 of a plausibility test implemented for a lateral segment SL detected in step 400. Such sub-steps are detailed in a non-limiting manner and the person skilled in the art will easily understand that it is possible to calculate a plurality of plausibility sub-scores on the basis of the position, speed, acceleration, dimensions, timestamp data of the acquired images or even lateral distance associated with a detected lateral segment.

[0103] In a step 421, a value relating to a pixel height associated with the lateral segment SL is estimated, from the acquired image as represented in [Fig.6]. The projection of the point (or pixel) A' of [Fig.6] onto the z axis makes it possible to obtain the coordinate zA', which corresponds, in pixels, to a height associated with the lateral segment SL. In step 310, a first sub-score sh can be determined:

[0104] Math.2 fl si zA S ilA, where: 0 if zA < hths -shQs\. a first plausibility sub-score, - zA' is the height (in pixels) associated with the lateral segment SL on an acquired image, - hths is a predefined height threshold (in pixels or px).

[0105] In other words, the first plausibility sub-score sh verifies that a height in pixels of the lateral segment SL in the acquired image exceeds a certain height threshold which depends on the resolution of the acquired image. Such a height threshold hths can for example be predefined at 200 px. Such a sub-score then makes it possible to assign a first sub-score sh zero if the height of the lateral segment SL is considered “too close to the ground” (i.e. height zA' below the height threshold htlw). Indeed, if the detected SL lateral segment is in this sense too close to the ground, the plausibility test considers that it is not plausible that such a SL lateral segment is associated with a vehicle.

[0106] At a step 422, an absolute speed y^ associated with the lateral segment SL can be determined:

[0107] Math.3 V sl ~ vp + Usz, where: - vsl is the relative speed associated with the lateral segment SL, -Vsl is an absolute speed associated with the lateral segment SL, and - VVP is the known speed of movement of the main vehicle VP and therefore of movement of camera 1.

[0108] Such an absolute speed y^ associated with the lateral segment SL includes in particular a lateral component, noted yç^? of norm VSL}at along the Y axis and a longitudinal component, noted VsLhng norm ^SLJcmg along the X axis. For example, if the element associated with the lateral segment SL has a perfectly rectilinear trajectory along the X direction, then V iat = 0.

[0109] At step 422, a second sub-score siat can then be determined on the basis of the norm V SL lut of the lateral component y 7“ of the absolute speed y associated with the lateral segment SL:

[0110] Math.4 s / at = 1 - min(Ksr / «f*«i) where: - if "f" is a second plausibility sub-score, - VSL^lt is the norm of the lateral component of the absolute velocity associated with the lateral segment SL, and - is a first predefined adjustment parameter (homogeneous to the inverse of a speed). Such a first adjustment parameter can be relative to a maximum lateral speed tolerated for a vehicle along the Y axis.

[0111] In other words, the second plausibility sub-score si«t makes it possible to verify that the lateral segment SL belongs to an element which moves in the same direction as the main vehicle VP, namely substantially along the X axis.

[0112] At a step 423, a third sub-score can be determined on the basis of the norm V sl. along the longitudinal component y„. of the absolute speed y associated with the lateral segment SL:

[0113] Math.5 |<a2 où: $ long— 1 ] 1 rSUong\ > a2 23 - skmg is a third plausibility sub-score, ■ VSLkmg is the norm of the longitudinal component of the absolute velocity associated with the lateral segment SL, and - is a second predefined adjustment parameter (homogeneous to a speed). Such a second adjustment parameter can be relative to a minimum longitudinal displacement speed tolerated for a vehicle along the X axis, so as to exclude (i.e., assign a third plausibility sub-score^Ln# zero) lateral segments that are substantially immobile or considered not or insufficiently mobile to be associated with vehicles. The lower the value of the second adjustment parameter ^2, the more false vehicle detections (i.e., false positives) can be avoided.

[0114] In other words, the third plausibility sub-score slong makes it possible to verify that the difference between the respective longitudinal components of the main vehicle VP and the element corresponding to the lateral segment SL is small (i.e. less than a2). Thus, the third plausibility sub-score slong allows us to rule out the hypothesis that the element corresponding to the lateral segment SL is a vehicle, if a significant differential (greater than ra2) in speed exists between the main vehicle VP and the element corresponding to the lateral segment SL (such an element could be another object close to the main vehicle VP such as a sign, a bird or a leaf for example).

[0115] At a step 424, the total plausibility score s, can then be determined by:

[0116] Math.6 ç — ç Jç, + played again by: *' ^long) r

[0117] Math.7 — 4" + SlongOÙ: [Math.6] sh is the first plausibility sub-score, [Math.6] '^iat is the second plausibility subscore, and [Math.6] is the third plausibility subscore.

[0118] The plausibility score formula can be chosen between the expressions Math. 6 and Math. 7 depending on whether it is considered that the height criterion in pixels of the side segment SL is respectively a required or not preliminary condition (binary way).

[0119] Alternatively to the expressions Math. 6 and Math. 7, the plausibility score s may be based on one or a portion of the plausibility subscores.

[0120] In other embodiments, one or more other plausibility sub-scores may be determined and adapted in the formula Math. 6 or Math. 7. For example, a fourth plausibility sub-score Sans may be related to the lateral distance d associated with the detected lateral segment SL. Such a fourth plausibility sub-score Sanz may for example be defined by:

[0121] Math.8 blood = min (1, tan *a3) °where: - Sans is a fourth plausibility subscore, - d is the lateral distance associated with the lateral segment SL, - D is a predefined Euclidean distance, and - as is a third predefined adjustment parameter.

[0122] In other embodiments, other plausibility sub-scores may include an estimation of the height of the element associated with the lateral segment SL from the pixel B', comparing ratios of lateral and / or longitudinal speeds respectively associated with the main vehicle VP and the lateral segment SL with respect to predefined thresholds for example.

[0123] In step 424, the plausibility score can then be compared to a plausibility threshold sths comprised for example between 1 and 2, or between 0 and 1. If, at the end of such a comparison in step 424, the plausibility score s is lower than the plausibility threshold s^, then, a step 430 of detecting a presence state (eg, by a Boolean value of plausibility or non-plausibility of presence) of a vehicle represented in [Fig.4] consists of considering that it is not plausible that the detected lateral segment SL corresponds to a neighboring vehicle VV1 and concludes that there is no vehicle corresponding to the lateral segment SL. The early detection method then stops, as shown in a sub-step 4242 of [Fig.5], for the lateral segment SL considered and is possibly implemented for another lateral segment detected in step 400.On the contrary, if the plausibility score s is greater than the plausibility threshold s^, then step 430 of detecting a state of presence of a vehicle consists of considering that it is plausible that the lateral segment SL is associated with a neighboring vehicle VV1 (sub-step 4241 of [Fig.5]) and concludes that there is a vehicle corresponding to the lateral segment SL.

[0124] Depending on the outcome of step 430 of [Fig.4] (and therefore of step 424 of [Fig.5]) of detecting a state of presence of a vehicle associated with the lateral segment SL, a step 440 of creating an object associated with the lateral segment SL may optionally be implementation, as shown in [Fig.4].

[0125] If at step 424, the plausibility score s is strictly lower than the plausibility threshold sths, then the device 2 determines that the detected lateral segment SL does not belong to a potential neighboring vehicle (sub-step 4242). No virtual object is then created in correspondence with the detected lateral segment SL and the method represented in [Fig.4] ends without creation of an object associated with the detected lateral segment SL at step 440. The method can then be repeated for another lateral segment detected at step 400.

[0126] If, in step 424, the plausibility score s is greater than the plausibility threshold sths, then a presence of a vehicle is detected (substep 4241) in correspondence with the lateral segment SL considered. In other words, the neighboring vehicle VV1 has been detected in an anticipated manner on the basis of a lateral portion of the neighboring vehicle VV1 visible in the field of view FOV (resulting in the lateral segment SL detected on the acquired image).

[0127] In this case 4241, the device 2, and more precisely the unit 40 of the device 2, can determine a (plausible) presence of a vehicle associated with the initial conditions (i.e., speed, lateral distance, etc.) of the lateral segment SL detected in step 430. The device 2 can then emit a signal, via the communication interface 50, informing of the detection of a neighboring vehicle VV1 to the main vehicle VP and / or provide data relating to a positioning of such a vehicle detected in the environment ENV (e.g., by providing one or more elements among the lateral distance d, the position of the point B, the speed associated with the lateral segment SL, etc.). Optionally, the device 2 can also proceed, in a step 440, to the creation of an object associated with the lateral segment SL identified as corresponding to a neighboring vehicle VV1.

[0128] Reference is now made to Figures 10, 11 and 12. [Fig. 10] represents an aerial view of the scene ENV at an acquisition time preceding the instant T3, and the unhatched grayed surface has been identified as corresponding, as one or more detected optical flows (or segments), to a neighboring vehicle VV1. [Fig. 11] corresponds to a schematization of the second reference point W according to the pinhole model. [Fig. 12] corresponds to an image acquired at an acquisition time upstream of T3, on which a virtual object associated with the detected lateral segment SL is created by the device 2.

[0129] In the embodiment described subsequently in Figures 10 to 12, it is considered that a neighboring vehicle VV1 has been detected in step 430 as being associated with the lateral segment SL. In a step 440, the unit 40 can estimate a position (X_W, Y_W, Z_W) of a second reference point, or reference point, W belonging to the neighboring vehicle VV1, for example placed at the lower part in the center of the front face of the neighboring vehicle VV1 detected, as illustrated in Figures 1, 2 and 10.

[0130] For this, the unit 40 can use predetermined dimensions of a type of vehicle similar to the neighboring vehicle VV1, for example a truck with dimensions L1 in length, L2 in width and H in height and make the assumption that these dimensions correspond to those of the neighboring vehicle VV1.

[0131] From the coordinates (X_C, Y_C) of the central point C of the camera 1 in the world reference frame (X,Y,Z) corresponding for example to a point associated with the location of the camera 1 or of a sensor of the camera 1 in the environment ENV, of predetermined dimensions L1, L2 of the neighboring vehicle VV1, of the determined real lateral distance d, of the focal length f of the camera 1, and of the size of the sensor sz_capt, the unit 40 is able to determine, by applying the pinhole model as well as Thales' theorem:

[0132] Math.9 =r c ± (|rf|+H2) x „,=x c - : - W) are coordinates of the reference point W in the frame world (X,Y,Z), - (X_C, Y_C) are coordinates of the central point C of camera 1 in the world frame (X,Y,Z), - d is the real lateral distance associated with the lateral segment SL, - f is the X-axis component of the focal length of camera 1 - sensor_size is the actual size of the sensor, and - L2 is a predefined width dimension of the neighboring vehicle VV1.

[0133] Such coordinates of the reference point W are determined by considering that the optical center C (or central point) is coincident with the center of the acquired image. Optical deformations can be considered neglected. Furthermore, the focal length f is considered identical on the two axes - vertical VL and horizontal - of the image.

[0134] Indeed, such coordinates (X_W, Y_W) are determined by applying the pinhole model as illustrated in [Fig. 11], on which the origin of the world reference frame (X, Y, Z) corresponds to the central point C of camera 1. In [Fig. 10], the origin point O of the world reference frame (X, Y, Z) differs from the central point C of camera 1, which then has coordinates (X_C, Y_C, Z_C) in the world reference frame (X, Y, Z). In particular, the formulas Math. 9 depend on the relative positioning of the neighboring vehicle VV1 with respect to the main vehicle VP.

[0135] In particular, the ZW coordinate of the reference point W along the Z axis in the world frame is determined as corresponding to the coordinate along the Z axis of the chosen optical flow B to determine the lateral distance d, in the world frame. In other words, the reference point W is placed, in height along the Z axis, at the level of the optical flow of the lateral segment SL being closest to the ground.

[0136] Thus, during step 440, the device 2 can obtain coordinates (X_W, Y_W, Z_W) in the world reference frame (X,Y,Z) of a reference point W located on the front face of the detected neighboring vehicle VV1. The device 2 can then be configured to transmit such position data via the interface 50 in order to indicate a front position of the neighboring vehicle VV1. The device 2 can also transmit data making it possible to estimate an occupation space of the environment of the detected neighboring vehicle VV 1, for example by transmitting the coordinates (X_W, Y_W, Z_W), the predefined dimensions L1, L2, H. The device 2 can also transmit a detection or alert signal when the main vehicle VP is estimated as approaching such an occupation space of the neighboring vehicle VV1 (for example, by comparing the coordinates of the neighboring vehicle VV 1 with positioning coordinates of the main vehicle VP).

[0137] In one embodiment, the device 2 may also be configured to create a virtual object associated with the detected neighboring vehicle VV1 and / or display such a virtual object on an image transmitted via the communication interface 50. For this, the unit 40 may, from the coordinates (X_W, Y_W, Z_W) of the reference point W determined and using the camera model, place a virtual reference point W' having coordinates (yW',zW') in the image frame (y,z) on an acquired image, for example the image on which the lateral segment SL was detected. In particular, the virtual point W' is positioned in the image frame (y,z) under several hypotheses:

[0138] - the detected lateral segment SL corresponds to a rear portion of the neighboring vehicle VV 1 (eg, it includes a rear wheel of the neighboring vehicle VV 1),

[0139] - the virtual reference point W' is considered positioned on a lateral border of the acquired image. In other words, the lateral portion visible on the acquired image is considered to represent the entire length of the neighboring vehicle VV1 in the state considered on the acquired image and the reference point W is considered to be associated with the front of the neighboring vehicle VV1.

[0140] In another embodiment, the virtual reference point W' can be placed elsewhere than on the lateral border of the image, for example inside the image or outside the image.

[0141] Thus, if the point, noted O' in figure 12 (corresponding to the pixel located at the bottom left of the acquired image) is considered to be the origin of the image reference (y,z) and the neighboring vehicle VV 1 is detected as being on the right lateral side of the main vehicle VP (as illustrated in figures 6 to 10 and 12), we have y_M / , = 0, as illustrated in [Fig. 12].

[0142] In addition, the coordinate zW' of the reference point W' along the z axis is determined at from the coordinate along the z axis of the chosen optical flow B' to determine the lateral distance d', in the image reference frame. In other words, the reference point W' is placed, in height along the z axis, at the level of the optical flow of the lateral segment SL being closest to the ground. In one embodiment, the height of the ground corresponding to the height zW' of the reference point W' along the z axis can be predefined and stored by the device 2. In one embodiment, the height zW' can also differ from the height of the optical flow of the lateral segment SL.

[0143] Such a virtual reference point W' is for example represented in [Fig. 12].

[0144] From such a reference point W' positioned on the acquired image and the predefined dimensions L1, L2, H associated with the neighboring vehicle VV1, a new virtual object can be created by the unit 40 of the device 2, in the two-dimensional space (y,z) and substantially superimposed on the lateral segment SL identified as corresponding to the neighboring vehicle VV1 on the acquired image. An example of such a virtual object is represented by the object OV in [Fig. 12].

[0145] Such a virtual object associated with the detected lateral segment SL can then be displayed on the communication interface 50 corresponding to a display screen, for example by superimposing the created object on a display of the image stream coming from the camera 1, as illustrated in [Fig. 11].

[0146] In particular, at the end of step 440, a virtual object having an initial position (for example, obtained via the coordinates (X_W, Y_W, Z_W) and the dimensions L1, L2, H) and a speed corresponding to VVv = VSLest is determined.

[0147] Therefore, the method allows early detection of a neighboring vehicle from a detected lateral segment SL. In particular, such detection does not require that a frontal face (front or rear) be included in the field of view of the camera 1.

[0148] The method also makes it possible, once such a neighboring vehicle has been detected, to create a virtual object associated with such a lateral segment SL, such a virtual object being created with initial conditions of position, speed and associated dimensions.

Claims

Claims

1. Method for detecting at least one neighboring vehicle (VV1) present in an environment (ENV) of a main vehicle (VP), said neighboring vehicle (VV1) and said main vehicle (VP) being motor vehicles, the method being implemented by a device (2) configured to provide a driving assistance function for the main vehicle (VP), said device (2) being connected to at least one camera (1) on board the main vehicle (VP) and capable of acquiring images of a scene surrounding the main vehicle (VP) according to at least one field of view (FOV) and at acquisition times, the method comprising the following steps: - obtaining, for at least one pair of successively acquired images,: * at least one lateral segment (SL) corresponding to a lateral coplanar set of optical flows detected in the field of view (FOV), between the respective acquisition times of the two images of said pair of images,and *a temporal data linked to a collision time associated with said lateral segment (SL), - determining, according to at least one predefined coordinate system, a value relating to a lateral distance (d) associated with the lateral segment (SL), said lateral distance (d) being determined from at least one measurement carried out on an image among the two images of said pair of images, - determining a speed (VSl) associated with the lateral segment (SL) from the lateral distance (d) and the collision time, - determining, from at least the speed (VSl) associated with the lateral segment (SL) and a known speed (VVp) of the main vehicle (VP), a plausibility score (s) associated with the lateral segment (SL), and - if the plausibility score (s) is greater than at least one predetermined threshold, detecting the presence of the neighboring vehicle (VV1) in the scene.,

2. Method according to claim 1, wherein the lateral distance (d) associated with the lateral segment (SL) is determined from an optical flow belonging to the chosen lateral segment (SL) so that a minimum angle (0min) is measured between said chosen optical flow and a vertical axis (VL) defined according to the predefined coordinate system.

3. Method according to any one of the preceding claims, the known speed (VVp) of the main vehicle (VP) and the speed (VSL) associated with the lateral segment (SL) each being composed of a lateral component and a longitudinal component (Vvpjong, VSL,iong) according to a predefined coordinate system, and in which the plausibility score (s) varies inversely with a difference between the longitudinal component (VsLjong) of the speed (VSL) associated with the lateral segment (SL) and the longitudinal component (VVp,iong) of the known speed (VVp) of the main vehicle (VP).

4. Method according to any one of the preceding claims, the known speed (VVp) of the main vehicle (VP) and the speed (VSL) associated with the lateral segment (SL) each being composed of a lateral component (VSL,iat) and a longitudinal component (VVp,iong, VsLjong) in a predefined coordinate system, and in which the plausibility score (s) varies inversely with an increasing value of the lateral component (VSL,iat) of the speed (VSl) associated with the lateral segment (SL).

5. Method according to any one of the preceding claims further comprising: - a determination, according to the predefined coordinate system, of a value relative to a height associated with the lateral segment (SL), and in which the plausibility score (s) depends on the height associated with the lateral segment (SL).

6. Method according to any one of the preceding claims further comprising, if the presence of the neighboring vehicle (VV1) is detected: - a determination, according to the predefined coordinate system, of a position of at least one reference point (W) associated with the lateral segment (SL), said position of the reference point (W) being determined from at least the lateral distance (d) associated with the lateral segment (SL), and - a creation of an object (OV) associated with the lateral segment (SL) in the field of view (FOV) of the camera (1) from at least the position of the reference point (W).

7. Method according to claim 6, wherein the object (OV) associated with the lateral segment (SL) is further created from predetermined dimensions (L1, L2, H) associated with a type of motor vehicle.

8. Device (2) configured to provide a driving assistance function for a main vehicle (VP), said device (2) being connected to at least one camera (1) on board the main vehicle (VP) and capable of acquiring images of a scene surrounding the main vehicle (VP).

9.

10. according to at least one field of view (FOV) and at acquisition times, wherein the device (2) comprises at least one processing circuit (20, 30, 40, 50) configured to implement a method for detecting at least one neighboring vehicle (VV1) present in an environment (ENV) of the main vehicle (VP) according to one of claims 1 to 7. Computer program comprising instructions for implementing the method according to one of the preceding claims when this program is executed by at least one processor (21, 31, 41). Non-transitory recording medium readable by a computer on which is recorded a program for implementing the method according to one of claims 1 to 7 when this program is executed by at least one processor (21, 31, 41).

Images