Detection of segments

Abstract

The invention relates to a method for detecting a neighbouring vehicle in an environment of a main vehicle by means of a driver assistance device connected to a camera on board the main vehicle that is capable of acquiring images within a field of view and at acquisition times, the method comprising: - obtaining, for a pair of consecutively acquired images, at least one lateral segment and an associated item of time data; - determining a lateral distance associated with the lateral segment from at least one measurement taken from one of the images in the pair of images; - determining a speed associated with the lateral segment from the lateral distance and the item of time data; - determining, from at least the speed associated with the lateral segment and a speed of the main vehicle, a plausibility score associated with the lateral segment; and - if the plausibility score exceeds at least one threshold, detecting the presence of the neighbouring vehicle.

Description

Description Title: Segment Detection Technical field [1] This disclosure relates to the field of driver assistance systems and more particularly to the automatic detection of vehicles in areas of interest, in particular lateral areas. Prior art [2] The rise of Intelligent Transport Systems (or ITS) has led to the development of numerous systems embedded in vehicles, particularly road transport. Such embedded systems include driver assistance systems and autonomous driving. In particular, the detection of objects, including vehicles, plays an important role in issues such as traffic flow, road safety and road infrastructure management (for example, variable message signs or automatic speed cameras). [3] In the context of systems embedded in autonomous or semi-autonomous vehicles traveling in road traffic (for example, on a motorway), the detection of objects surrounding a vehicle in question is particularly important in order to avoid collisions between the vehicle in question and other vehicles, for example. In particular, in lane change assistance, automatic lane change or blind spot detection systems, precise and anticipated detection of obstacles such as a central reservation or another vehicle located in a lane adjacent to the vehicle in question is required. Indeed, if the autonomous vehicle in question does not detect (or does not detect sufficiently early) another vehicle located in a neighboring lane and moves out of the way, a collision may occur between the two vehicles. [4] In such a context, most existing vehicle detection methods are based on classification algorithms for identifying vehicles from image sequences or video streams acquired by vision sensors (typically, a camera). Such vision sensors are generally arranged at the front and / or rear of the vehicle concerned, so as to acquire images and / or image sequences of a front and / or rear field of view (Field of View) of the vehicle concerned. The classification algorithms can then identify one or more vehicles neighboring the vehicle concerned on the basis of frontal recognition of the neighboring vehicles (i.e. by identifying and classifying vehicles detected in the field of view from their front or rear face). [5] However, such vehicle classifier-based detection methods prove ineffective when the frontal face of neighboring vehicles is not yet detectable within the field of view, even though such neighboring vehicles may be partially visible within the field of view, for example via a partial side face. Thus, if the vehicle in question is on a highway lane and a neighboring vehicle is located relatively at the level of (or slightly behind) the vehicle in question on an adjacent lane, a camera placed at the rear of the vehicle in question would capture a rear lateral portion of the neighboring vehicle. In such a situation, the front face of the neighboring vehicle is not yet visible to the camera placed at the rear of the vehicle in question, for example until the speed differential between the vehicle in question and the neighboring vehicle is large enough for the vehicle in question to completely overtake the neighboring vehicle.Until such a complete overtaking occurs, the neighbouring vehicle is therefore located substantially at the same level as the vehicle concerned and a lane change of the vehicle concerned into the neighbouring lane would result in a collision with the neighbouring vehicle. However, such a criticality situation is not detectable by existing classification methods. [6] There is therefore a need to secure the decision-making of driver assistance and / or autonomous driving systems, particularly in such a context of lane change decision. In particular, there is a need for early detection of vehicles surrounding a vehicle in question even before a frontal view (front or rear) of such surrounding vehicles is visible in a field of view of a vision sensor of the vehicle in question - and therefore for classification and conventional tracking of such surrounding vehicles by existing classifiers to be possible. Summary [7] This disclosure improves such a situation. [8] A method is proposed for detecting at least one neighboring vehicle present in an environment of a main vehicle, said neighboring vehicle and said main vehicle being motor vehicles, the method being implemented by a device configured to provide a driving assistance function for the main vehicle, said device being connected to at least one camera on board the main vehicle and capable of acquiring images of a scene surrounding the main vehicle according to at least one field of view and at acquisition times, the method comprising the following steps: - obtain, for at least one pair of successively acquired images: * at least one lateral segment corresponding to a lateral coplanar set of flows optical detected in the field of view, between the respective acquisition times of the two images of said pair of images, and * time data linked to a collision time associated with said lateral segment, - determine, according to at least one predefined coordinate system, a value relative to a lateral distance associated with the lateral segment, said lateral distance being determined from at least one measurement carried out on an image among the two images of said pair of images, - determine a speed associated with the lateral segment from the lateral distance and the collision time, - determine, from at least the speed associated with the lateral segment and a known speed of the main vehicle, a plausibility score associated with the lateral segment, and - if the plausibility score is greater than at least a predetermined threshold, detect the presence of the neighboring vehicle in the scene. [9] Therefore, the proposed method allows the implementation of an early detection of a vehicle neighboring the main neighbor, namely as soon as such a neighboring vehicle enters, even partially, into the field of view of a camera on board the main vehicle. In other words, the proposed method makes it possible to detect a vehicle neighboring the main vehicle despite the absence of visibility of a front or rear frontal face of the neighboring vehicle on which most existing classifiers rely for vehicle detection.

[0010] Thus, the method allows a main vehicle to benefit from a driving assistance function in potentially critical moments, for example when the main vehicle wishes to change lanes or overtake and neighboring vehicles are located almost at the same level as the main vehicle, or in the latter's blind spot.

[0011] A main vehicle driving assistance function refers to a functionality of a system embedded in the main vehicle that can assist, guide or even decide on the guidance of the main vehicle. Such a driving assistance function can be implemented in the context of a semi-autonomous or autonomous vehicle, for example, with or without a driver. A driving assistance function can include assistance with driving, changing lanes, overtaking another vehicle, or even parking, for example.

[0012] A field of view refers to a portion of the main vehicle's surroundings covered by the camera's field of view. Such a field may, for example, be in front and / or behind the main vehicle.

[0013] By images acquired at acquisition times, we mean a continuous or discretized succession of images (i.e., in two dimensions) of the portion of the environment included in the field of view, associated with a succession of instants at which these images were acquired. In particular, the movement of the elements surrounding the main vehicle over time implies a field of view covering an evolving environment: the vehicles and elements neighboring the main vehicle exhibit a relative movement with respect to the main vehicle (as the main vehicle moves) and / or an absolute movement in the environment.

[0014] By a lateral segment corresponding to a lateral coplanar set of optical flows detected in the field of view, between the respective acquisition times of the two images of said pair of images, reference is made to obtaining a pixel displacement field observed between two acquired images. An optical flow can then be assimilated to a bipoint (or vector) representing a displacement in space. A set of coplanar optical flows reflecting a common movement can then be associated with a segment. We can then distinguish three different types of segments in a three-dimensional environment, each type of segment being parallel to a plane formed by two axes of a Cartesian reference system associated with the three-dimensional environment. A lateral segment then corresponds to a segment parallel to a plane called lateral or oblique with respect to a main axis of circulation of the main vehicle (eg, on a motorway lane).

[0015] By a time data linked to a collision time associated with said lateral segment, reference is made to a value reflecting a time before collision between the element associated with the lateral segment and the main vehicle observing such a lateral segment. Such a collision time can be deduced for example from the calculation of an optical flow, a set of coplanar optical flows forming a lateral segment having the same collision time. A collision time can then be associated with a lateral segment.

[0016] By lateral distance, we mean a lateral or bias gap between the element associated with the detected lateral segment and the main vehicle. For example, in the case of a main vehicle and a neighboring vehicle traveling on two parallel traffic lanes (e.g., two highway lanes), the lateral distance corresponds to the lateral gap separating the lanes occupied by the main and neighboring vehicles. Such a lateral distance thus reflects a gap in the environment between the main and neighboring vehicles.

[0017] By a plausibility score, we refer to a value reflecting a probability (or plausibility) that the detected side segment corresponds to a vehicle neighboring the main vehicle. Such a plausibility score can then be determined from parameters associated with the lateral segment (eg, its relative speed with respect to the main vehicle, its absolute speed in the environment, its position in the environment, etc.) and one or more predetermined tolerance thresholds, which make it possible to delimit the values ​​of said parameters as plausibly assimilated to a vehicle. Thus, the higher the plausibility score, the more the object associated with the lateral segment can be plausibly assimilated to a vehicle.

[0018] According to another aspect, a device is proposed configured to provide a driving assistance function for a main vehicle, said device being connected to at least one camera on board the main vehicle and capable of acquiring images of a scene surrounding the main vehicle according to at least one field of view and at acquisition times, the device comprising at least one processing circuit configured to implement a method for detecting at least one neighboring vehicle present in an environment of the main vehicle as proposed.

[0019] According to another aspect, there is provided a computer program comprising instructions for implementing the method as proposed when this program is executed by a processor.

[0020] According to another aspect, there is provided a non-transitory recording medium readable by a computer on which is recorded a program for implementing the method as proposed when this program is executed by a processor.

[0021] The features set out in the following paragraphs may, optionally, be implemented, independently of each other or in combination with each other:

[0022] According to one embodiment, the lateral distance associated with the lateral segment is determined from an optical flow belonging to the chosen lateral segment so that a minimum angle is measured between said chosen optical flow and a vertical axis defined according to the predefined coordinate system.

[0023] Consequently, the proposed method allows to determine a lateral offset effectively separating the main vehicle from the object associated with the lateral segment in the environment (which is not yet identified by the device as being a vehicle or not at this stage). Such a lateral distance can notably be obtained by performing measurements on the image showing the lateral segment. The determined minimum angle then allows to ensure that the optical flow chosen to determine the lateral distance is the closest to the ground among the set of optical flows forming the lateral segment.

[0024] According to one embodiment, the known speed of the main vehicle and the speed associated with the lateral segment each being composed of a lateral component and a longitudinal component according to a predefined coordinate system, the plausibility score varies inversely with a deviation between the longitudinal component of the speed associated with the lateral segment and the longitudinal component of the known speed of the main vehicle.

[0025] Therefore, the plausibility score may depend on a difference in forward speed (or movement) between the main vehicle and the object associated with the lateral segment. The plausibility score then makes it possible to assimilate to a vehicle the lateral segments having a longitudinal movement speed close to the known speed of the main vehicle.

[0026] The longitudinal component of a speed refers to the component of the speed along the vehicle's main direction of travel. For example, in the case of a vehicle moving on a motorway lane, the longitudinal component of the speed corresponds to the speed component along the axis of the road (i.e., the motorway lane). The lateral component of a speed refers to the speed component along a biased or lateral direction relative to the vehicle's main direction of travel. For example, in the case of a vehicle moving on a motorway lane, the lateral component of the speed corresponds to the speed component along an axis on the ground perpendicular to the axis of the road, i.e., the axis of travel of a vehicle moving from one motorway lane to another, for example.

[0027] Thus, the difference between the two longitudinal components of two speeds makes it possible to quantify a differential in travel speed between the main vehicle and the element associated with the lateral segment. For example, if the element associated with the lateral segment happens to be a sign or a tree leaf, the differential in longitudinal speed components of such an element and the main vehicle will exceed a predetermined tolerance threshold, which will reduce the plausibility score.

[0028] According to one embodiment, the known speed of the main vehicle and the speed associated with the lateral segment each being composed of a lateral component and a longitudinal component in a predefined coordinate system, the plausibility score varies inversely with an increasing value of the lateral component of the speed associated with the lateral segment.

[0029] Therefore, the plausibility score allows us to assess whether the element associated with the lateral segment is moving in a direction that is substantially similar to the direction of travel of the main vehicle. Indeed, the greater the lateral component of the speed associated with the lateral segment, the more it reflects a significant lateral displacement of the element associated with the lateral segment.

[0030] According to one embodiment, the method further comprises: - a determination, according to the predefined coordinate system, of a value relative to a height associated with the lateral segment, and in which the plausibility score depends on the height associated with the lateral segment.

[0031] Therefore, the plausibility score takes into account a dimension associated with the side segment, such a dimension can reflect a size or a height of the element associated with the side segment. The plausibility score therefore makes it possible to verify that the element associated with the side segment has a height comparable to a vehicle.

[0032] Thus, an insufficient height (e.g., less than a predefined minimum height threshold) may indicate that the element associated with the lateral segment is "too low" to be considered a vehicle (e.g., a central reservation separating two traffic lanes). Conversely, an excessively high height (e.g., greater than a predefined maximum height threshold) may indicate that the element associated with the lateral segment is "too high" to be considered a vehicle (e.g., a bird flying above the vehicles).

[0033] According to one embodiment, the method further comprises, after detecting the presence state of the neighboring vehicle corresponding to a presence of the neighboring vehicle,: - a determination, according to the predefined coordinate system, of a position of at least one reference point associated with the lateral segment, said position of the reference point being determined from at least the lateral distance associated with the lateral segment, and - creation of an object associated with the lateral segment in the camera's field of view from at least the position of the reference point.

[0034] Therefore, the proposed method makes it possible to detect the presence of a neighboring vehicle in the environment of the main vehicle, as well as to estimate a position of such a vehicle, once detected, in the environment. The method also makes it possible to create a virtual object associated with the lateral segment from such a position, in order to mark the presence of a detected neighboring vehicle.

[0035] By a landmark associated with the side segment, reference is made to a point belonging to the element associated with the side segment, which is, at this stage, identified as being a neighboring vehicle. Such a landmark may for example be assimilated to a front end of the neighboring vehicle. Such a landmark may for example be a point whose position marks a presumed position of a front face of the vehicle.

[0036] By a created object associated with the side segment in the camera's field of view, reference is made to a virtual object superimposable on an image acquired by the camera. Such a created virtual object is further associated with the side segment in the sense that the object is estimated to be superimposed on a location presumed to be occupied by the neighboring vehicle detected. Such a virtual object is notably characterized by a position in the environment and / or in the image. Such a virtual object can also be characterized by dimensions so as to reflect presumed dimensions of the detected vehicle.

[0037] Thus, the virtual object created can be used to provide a driving assistance function to the main vehicle, for example by displaying such a virtual object on a field of view displayable on a dashboard of the main vehicle for example. The virtual object thus created then makes it possible to represent the space of the environment occupied by the neighboring vehicle, so that a signal or an alert can be triggered when it is detected that the space occupied by the main vehicle is approaching and / or substantially coincides with the space occupied by the virtual object.

[0038] According to one embodiment, the object associated with the side segment is further created from predetermined dimensions associated with a type of motor vehicle.

[0039] Consequently, the created virtual object can include a modeling of the neighboring vehicle by representing the presumed space occupied by such a vehicle. The predetermined dimensions can for example correspond to average dimensions of vehicles circulating on such a portion of the environment, or to maximum dimensions of a vehicle, so as to assist the driving of the main vehicle in an "unfavorable" configuration (where the space of the environment occupied by the main vehicle can coincide more quickly with the space occupied by the neighboring vehicle compared to a smaller vehicle). Brief description of the drawings

[0040] Other characteristics, details and advantages will appear on reading the following detailed description, and on analysis of the attached drawings, in which:

[0041] [Fig. 1] Figure 1 shows a schematic of a main vehicle according to an embodiment.

[0042] [Fig. 2] Figure 2 shows an aerial view of a scene surrounding the main vehicle according to one embodiment.

[0043] [Fig. 3] Figure 3 shows a diagram of a driving assistance device according to one embodiment.

[0044] [Fig. 4] Figure 4 shows steps of a method for early detection of a vehicle according to one embodiment.

[0045] [Fig. 5] Figure 5 shows steps of a method for early detection of a vehicle according to one embodiment.

[0046] [Fig. 6] Figure 6 shows a shot of a scene surrounding the main vehicle according to one embodiment.

[0047] [Fig. 7] Figure 7 shows a shot of a scene surrounding the main vehicle according to one embodiment.

[0048] [Fig. 8] Figure 8 shows a shot of a scene surrounding the main vehicle according to one embodiment.

[0049] [Fig. 9] Figure 9 shows a shot of a scene surrounding the main vehicle according to one embodiment.

[0050] [Fig. 10] Figure 10 shows an aerial view of a scene surrounding the main vehicle according to one embodiment.

[0051] [Fig. 11] Figure 11 shows a modeling of a point of a scene surrounding the main vehicle according to one embodiment.

[0052] [Fig. 12] Figure 12 shows a virtual object associated with a lateral segment according to one embodiment. Description of the embodiments

[0053] Reference is made to Figure 1. Figure 1 shows a diagram of a main passenger vehicle (PC). The main passenger vehicle (PC) may be a motor vehicle. There is no limitation on the type of vehicle to which the main passenger vehicle (PC) belongs. The main passenger vehicle (PC) may, for example, be a private, utility, or industrial vehicle, and may, for example, correspond to a car, a van, a two-wheeler, a truck, or a bus. The main passenger vehicle may also be a towed vehicle, for example, towing a trailer, a semi-trailer, or a caravan. The dimensions of the main passenger vehicle may be between 2 meters and 20 meters in length, between 0.5 meters and 5 meters in width, and between 1 meter and 5 meters in height.

[0054] The main vehicle (VP) is equipped with at least one on-board system providing a plurality of functions or applications of the main vehicle (VP). Such functions may, for example, correspond to cruise control, power steering, automated airbag deployment, automatic headlight adjustment, etc.

[0055] The main vehicle VP is in particular equipped with an on-board system allowing detection of objects surrounding the main vehicle VP, for example as part of a driving assistance function such as obstacle detection, lane change assistance or automatic lane change. For this, the on-board system of the main vehicle VP comprises a driving assistance device 2. The device 2 is capable of providing a driving assistance function on the basis of a plurality of shots (or images) of the environment of the main vehicle VP. For this, the device 2 is in particular connected to one or more vision sensors such as a camera or a camera 1 (the vision sensor will be considered to be a camera 1 in the remainder of the description). The device 2 may also be capable of transmitting or communicating data, for example relating to the driving assistance function provided. For this, the device 2 may be connected to a communication interface 50. In a particular embodiment, the interface 50 may be integrated into the device 2. Such a communication interface 50 may for example be a human-machine interface. The interface 50 may include a display screen, a touch screen, a dashboard, and / or a loudspeaker.The data transmitted by the device 2 to the interface 50 may, for example, correspond to assistance information indicating to the driver of the main vehicle VP whether or not he can change lanes.

[0056] Reference is now made to Figure 2. Figure 2 illustrates an ENV environment (or scene), in which the main vehicle VP is located. Figure 2 is an aerial view of such an ENV scene.

[0057] The ENV environment can be defined in a three-dimensional frame (X,Y,Z), called a "world frame" as illustrated in Figures 1 and 2. The origin of such a world frame (X,Y,Z) is predefined and fixed in the ENV environment.

[0058] The main vehicle VP is considered to be moving in the ENV scene. Such an ENV scene corresponds, for example, to a road or a motorway consisting of several traffic lanes. These traffic lanes may in particular be parallel to each other, as represented by the linked dotted vertical lines in Figure 2. Figure 2 illustrates, for example, three traffic lanes, with the main vehicle VP being located on the middle traffic lane. The main vehicle VP is considered to be moving in the main direction X of the (X,Y,Z) frame, such a main direction being called the longitudinal direction. Such a movement is shown diagrammatically in Figures 1 and 2 by an arrow attached to the main vehicle VP. The main vehicle VP may also have a movement in the Y direction of the (X,Y,Z) frame, called the lateral direction.Such a movement in the Y direction, called lateral movement (or displacement), can for example take place when the vehicle changes traffic lane. A movement in the Z direction, that is to say in height, of the main vehicle VP is not considered. The main vehicle VP is therefore considered to be kept on the ground and therefore has a height in the Z direction that is substantially constant, corresponding approximately to a predefined dimension of the main vehicle VP. In one embodiment, such a height in the Z direction of the main vehicle VP can vary by the order of a centimeter or a decimeter, such a variation in height in the Z direction being for example linked to. shock absorbers of the main vehicle VP and / or to reliefs or roughness present in the ENV environment (in particular on the traffic lane of the main vehicle VP). In the remainder of the description, such a height in the Z direction of the main vehicle VP is considered known.

[0059] The scene ENV surrounding the main vehicle VP may also include other OV elements and vehicles VV1, VV2, VV3. The vehicles VV1, VV2, VV3 are neighboring vehicles of the main vehicle VP. Such neighboring vehicles VV1, VV2, VV3 are, like the main vehicle VP, motor vehicles moving in the scene ENV. Such neighboring vehicles VV1, VV2, VV3 may in particular have dimensions and movement characteristics (in terms of speed or acceleration for example) similar to or different from the main vehicle VP. For example, figure 2 may represent a main vehicle VP moving on a highway lane and neighboring vehicles VV1, VV2, VV3 traveling on the highway lanes neighboring the lane taken by the main vehicle VP. OV objects are neighboring elements of the main vehicle VP and can refer to any distinct element of a neighboring vehicle VV1, VV2, VV3.A neighboring element OV can, for example, correspond to an obstacle located in the scene such as a central reservation separating two traffic lanes, an indication sign or even a bird flying in the ENV scene.

[0060] As illustrated in Figure 2, the main vehicle VP is considered to be equipped with a camera 1 positioned at the rear of the main vehicle VP. In another embodiment (not shown in Figure 2), the camera 1 can be positioned at the front of the main vehicle VP or several cameras 1 can be positioned both at the front and at the rear of the main vehicle VP. The camera 1 makes it possible to capture images (or shots) of the ENV scene of the main vehicle VP according to a field of view FOV (or in English “Field of View”). Such a field of view FOV depends in particular on the type of camera 1 used and the positioning of the camera 1 in (or on) the main vehicle VP. With reference to Figure 2, the field of view FOV covers a part of the ENV scene, so that a limited portion of the ENV scene is captured according to the field of view FOV.Thus, in Figure 2, the field of view FOV represented covers a portion of the traffic lane in which the main vehicle VP is located and respective portions of the neighboring traffic lanes. In particular, the field of view FOV covers a portion of the scene ENV in which the neighboring vehicle VV2 is entirely located and a portion of the scene ENV in which the neighboring vehicle VV1 is partially located. For example, the portion of the scene ENV in which the neighboring vehicle VV1 is partially located may comprise a point A, called the first reference point A (having coordinates in the world reference frame (X,Y,Z)), positioned on the neighboring vehicle VV1, corresponding for example to a placed reference point. at the rear-left of the neighboring vehicle VV1 at the height of the neighboring vehicle VV1. Unlike the first reference point A, a second reference point W, for example located at the middle of the front face of the vehicle closest to the ground, is not in the field of view FOV in Figure 2. The neighboring vehicle VV3 is not visible in the field of view FOV of the main vehicle VP. A portion of the neighboring vehicle VV1 is not visible in the field of view FOV of the main vehicle VP. The non-visible parts of the neighboring vehicles VV1, VV3 in the field of view FOV are represented in Figure 2 by striped areas. A neighboring object OV, for example a bird, may be visible in the field of view FOV.

[0061] In the context of an ENV scene as represented in figure 2 according to the (X,Y,Z) reference frame, it is for example considered that the main vehicle VP moves at a main speed known. To facilitate the rest of the description, such a main speed is considered to be of constant VVP standard. The neighboring vehicle VV1 is considered to be moving at a neighboring speed unknown to the driving assistance device 2 of the main vehicle VP. In the embodiment described below, it can be considered that the VVP standard of the neighboring speed is lower than the VVP standard of the main speed V^. Under such an assumption and under the assumption that the neighboring speed of the neighboring vehicle VV1 remains substantially constant over a considered time interval, like the main speed of the main vehicle VP, the distance difference in the longitudinal direction X between the main vehicle VP and the neighboring vehicle VV1 will increase over time, so that at a future time, the overtaking of the main vehicle VP on the neighboring vehicle VV1 will be such that the neighboring vehicle VV1 will be entirely included in the (rear) field of view FOV of the main vehicle VP, as is the initial case of the neighboring vehicle VV2.

[0062] Such an evolution of the membership of the neighboring vehicle VV1 to the field of view FOV of the main vehicle VP over time is illustrated in Figures 6, 7 and 8.

[0063] However, in another embodiment, the VVP norm of the neighboring speed may be greater than the VVP norm of the main speed V^. Under such an assumption and under the assumption that the neighboring speed of the neighboring vehicle VV1 remains substantially constant over a considered time interval, like the main speed of the main vehicle VP, the distance difference in the longitudinal direction X between the main vehicle VP and the neighboring vehicle VV1 will increase over time, so that at a future time, the overtaking of the neighboring vehicle on the main vehicle VP will be such that the neighboring vehicle VV1 will be entirely included in a field of view (front) of the main vehicle VP, for a camera 1 placed at the front of the main vehicle VP (case not shown in the figures).

[0064] In the remainder of the description, the case of the main vehicle VP overtaking the neighboring vehicle VV1 is considered, so that the VVP norm of the neighboring speed is considered to be lower than the Vvp norm of the main speed ^. It is also considered, as shown in Figure 2, that the camera 1 is placed at the rear of the main vehicle VP, so that the field of view FOV considered is a rear field of view relative to the main vehicle VP.

[0065] Reference is made to Figures 6, 7 and 8. Figures 6, 7 and 8 diagrammatically show shots of the scene ENV surrounding the main vehicle VP according to the (rear) field of view FOV of camera 1 of the main vehicle VP, as positioned in Figure 2. In order to facilitate the readability of Figures 6, 7 and 8, only the portion of the neighboring vehicle VV1 visible in the field of view FOV is shown in Figures 6, 7 and 8, the neighboring vehicle VV2 and the neighboring element OV of Figure 2 included in the field of view FOV are not shown in the shots of Figures 6, 7 and 8.

[0066] Figures 6, 7 and 8 correspond to images successively acquired by the camera 1 at successive instants (or acquisition times) T1, T2 and T3. Each of the instants T1, T2 and T3 may for example be spaced one or more milliseconds apart in time. Each of the images corresponds to a set of pixels with definable coordinates in a two-dimensional (y,z) frame of reference, called the “image frame”, as shown in Figures 6, 7 and 8. For example, the lowest and leftmost pixel of each image acquired by the camera 1 may correspond to the origin of the two-dimensional (y,z) frame of reference. In other embodiments, the central pixel of the image acquired by the camera, the pixel corresponding to a vanishing point C' of the acquired image (these two pixels being merged in figure 6) or even the pixel of coordinates (yC', zO) as represented in figure 6 can correspond to the origin of the reference (y,z).The resolution of the acquired images (and therefore the pixels) is the same for all acquired images and depends in particular on the properties of camera 1.

[0067] The portion of the neighboring vehicle VV1 visible in the field of view FOV of the main vehicle VP over time is highlighted in Figures 6, 7 and 8, for example by observing the respective projections A', A”, A'” of the first reference point A mentioned above on the images acquired in Figures 6, 7 and 8.

[0068] Figure 6 shows a diagram of an image acquired by camera 1 at time T 1 , corresponding for example to a configuration of the scene ENV and the field of view FOV illustrated in figure 2. Thus, only a first, lateral, portion of the neighboring vehicle VV1 is visible on the image of figure 6, such a first lateral portion including in particular the left rear wheel of the neighboring vehicle VV1 and the portion of the neighboring vehicle VV1 on which the first reference point A is located. In particular, such a first reference point A is positioned on a pixel A' of the image of figure 6. Such a pixel A' then corresponds to the projection of the first reference point A in the image reference frame (y,z) at time T1 and has the coordinates (yA'.zA').

[0069] Figure 7 shows a diagram of an image acquired by the camera 1 at time T2 following time T1. At such a time T2, a second lateral portion of the neighboring vehicle VV1 larger than the first lateral portion is visible in the image of Figure 7, such a second lateral portion including in particular the left rear wheel, partially the left front wheel of the neighboring vehicle VV1 and the portion of the neighboring vehicle VV1 on which the first reference point A is located. In particular, such a first reference point A is positioned on a pixel A” of the image of Figure 7 different from the pixel A' of the image of Figure 6. Such a pixel A” then corresponds to the projection of the first reference point A in the image reference frame (y,z) at time T2 and has the coordinates (yA”,zA”).In other words, the speed differential between the main vehicle VP and the neighboring vehicle VV1 between times T1 and T2 is such that the neighboring vehicle VV1 “enters more and more” into the field of view FOV of camera 1 between times T1 and T2.

[0070] Figure 8 shows a diagram of an image acquired by the camera 1 at time T3 following time T2. At such a time T3, the entirety of the neighboring vehicle VV1 is visible in the image of Figure 8 and in particular, the portion of the neighboring vehicle VV1 on which the first reference point A is located as well as a portion of the neighboring vehicle VV1 on which the second reference point W shown in Figure 2 and positioned on the front end of the neighboring vehicle VV1 is located are therefore visible. In particular, the first reference point A is positioned on a pixel A'” of the image of Figure 8, different from the pixels A' and A” of the images of Figures 6 and 7. Such a pixel A'” then corresponds to the projection of the first reference point A in the image reference frame (y,z) at time T3 and has coordinates (yA'”,zA'”). The second reference point W is positioned on a pixel W'” of the image of Figure 8.Such a pixel W'” then corresponds to the projection of the second reference point W in the image reference frame (y,z). In other words, the speed differential between the main vehicle VP and the neighboring vehicle VV1 between times T2 and T3 is such that the neighboring vehicle VV1 “entered completely” into the field of view FOV of camera 1 between times T2 and T3.

[0071] In the context of a driving assistance function provided by the device 2, for example aimed at assisting the main vehicle VP in a lane change maneuver, a process for detecting vehicles surrounding the main vehicle VP is required, so that the main vehicle VP does not move onto a neighboring traffic lane if it risks colliding with one of the surrounding vehicles. example, in the context of figure 2, the device 2 may aim to assist the main vehicle VP in its lane change towards the traffic lane of the neighboring vehicle VV1, so that the main vehicle VP does not collide with the neighboring vehicle VV1. Existing techniques for detecting vehicles surrounding the main vehicle VP rely in particular on the use of classifiers, based for example on convolutional neural network (CNN) methods, K-nearest neighbors (KNN) or support vector machines (SVM). Such classifiers allowing the detection of vehicles surrounding the main vehicle VP rely in particular on the detection and classification of a frontal (front or rear) view (or face) of the vehicles surrounding the main vehicle VP.The use of such classifiers may in particular comprise a learning phase based on a plurality of images representing frontal views of various types of vehicles. Thus, with reference to FIG. 2, a front frontal view of the neighboring vehicle VV2 belonging to the field of view FOV of the camera 1, the detection of the neighboring vehicle VV2 may be implemented on the basis of existing classification techniques. The same applies to the neighboring vehicle VV1 in the image of FIG. 8 at time T3.

[0072] However, the positions of the neighboring vehicle VV1 shown in Figure 2 and Figures 6 and 7 correspond to situations in which the existing classifiers fail to detect or track the neighboring vehicle VV1 effectively, or in any case, not without generating significant costs and / or computational time, due to the absence of a visible frontal view of the neighboring vehicle VV1 in the shots of Figures 6 and 7 (the second reference point W of Figure 2 positioned on the front frontal face of the neighboring vehicle VV1 not yet being visible in the field of view FOV of camera 1 at these stages).These situations, however, correspond to critical situations during which the main vehicle VP could collide with the neighboring vehicle VV1 if the main vehicle VP changes position and moves into the lane of the neighboring vehicle VV1, in the absence of lane change assistance information indicating that the neighboring vehicle VV1 is detected near such a position.

[0073] A method for early detection of the neighboring vehicle VV1 is then proposed and described by figures 4, 5, 9, 10 and 11 to detect the neighboring vehicle VV1, and in particular at the stage of the situations represented in figures 2, 6 and 7. Such a detection method is described as early in that it is implemented before the visibility of a front or rear face of the neighboring vehicle VV1 in the field of view FOV. Such a method for early detection of the neighboring vehicle VV1 is implemented by a driving assistance system for the main vehicle VP represented in figure 3.

[0074] Reference is now made to Figure 3. Figure 3 represents a diagram of an on-board system of a main vehicle VP. In particular, such an on-board system corresponds to a driving assistance system for the main vehicle VP. The driving assistance system of the main vehicle VP makes it possible in particular to provide a lane change assistance or automatic lane change function for the main vehicle VP, when the main vehicle VP is moving on a traffic lane as shown for example in Figure 2.

[0075] The system firstly comprises the driving assistance device 2. The device 2 may in particular comprise a unit 20 for detecting optical flows, a unit 30 for detecting an object associated with the detected vehicle and a unit 40 for creating such an object associated with the detected vehicle.

[0076] The driving assistance device 2 is further connected, on the one hand, to a vision sensor of the camera or camera type 1. The device 2 comprises an input unit (not shown in FIG. 3) allowing the device 2 to receive in substantially real time a data stream from the camera 1. Such a data stream corresponds to a discrete or continuous succession of images (or shots) of the scene ENV according to the field of view FOV of the camera 1. Each image received is associated with an acquisition time of the image by the camera 1. Each image can be time-stamped. Examples of such images transmitted by the camera 1 are shown in FIGS. 6, 7 and 8.

[0077] Furthermore, the driving assistance device 2 may be connected to a communication interface 50. Such an interface 50 may correspond to a human-machine interface integrated into the on-board system of the main vehicle VP. Such an interface 50 may also be integrated into the device 2. The communication interface 50 may also be a remote interface. The communication interface 50 may for example comprise a display screen, a touch screen, a dashboard or even a speaker, making it possible to transmit, for example by visual, haptic and / or audible information, indications relating to assistance in driving the main vehicle VP. In particular, the communication interface 50 may make it possible to transmit a detection status of a vehicle neighboring the main vehicle VP or even an indication relating to a possibility of changing lane of the main vehicle VP.Such information transmitted by the interface 50 may for example correspond to an estimated aerial visual representation of the respective positions of the main vehicle VP and the neighboring elements belonging to the field of view FOV in real time (eg, as represented in FIG. 2), or even an audio stimulus alerting of a risk of collision.

[0078] The optical flow detection unit 20 is capable of receiving and processing the data stream received from the camera 1. In particular, the unit 20 can, from the succession of images acquired by the camera 1 at successive acquisition times, obtain optical flows over given time intervals. Such a unit 20 can also implement a step of segmenting the determined optical flows, so as to obtain one or more segments associated with each acquired image (or segmented images), a segment then being associated with a set of optical flows on a given image. For this, the unit 20 can comprise a processing circuit composed of at least one processor 21 and at least one memory unit 22 capable of determining optical flows and / or segments associated with temporal data from the acquired images.

[0079] The unit 30 for detecting an object associated with the detected vehicle may be capable, for each segment determined by the unit 20, of implementing a plausibility test to associate this segment with a detected vehicle. For this, the unit 30 may comprise a processing circuit composed of at least one processor 31 and at least one memory unit 32, such a processing circuit being capable of implementing the plausibility test and, for example, of determining a plausibility score or a probability of presence of a vehicle obtained at the end of such a test.

[0080] The unit 40 for creating an object associated with the detected vehicle may be capable, for each vehicle detected by the unit 20 and according to the plausibility score determined by the unit 30, of creating a virtual object associated with the detected vehicle corresponding to the optical flows (or segments) detected as belonging to such a vehicle. For this, the unit 40 may comprise a processing circuit composed of at least one processor 41 and at least one memory unit 42, the processing circuit of the unit 40 being capable of taking as input data from the units 20 and 30 and of determining initial conditions (e.g., initial position and / or speed data, vehicle dimensioning data, etc.) so as to create a virtual object associated with the vehicle detected by the unit 30. The virtually created object and / or its initial conditions may then be transmitted for communication to the user of the device 2 (e.g., the driver of the main vehicle VP) via the communication interface 50.

[0081] Reference is now made to Figure 4. Figure 4 illustrates a succession of steps for implementing a method for early detection of a neighboring vehicle VV1 by a system including a device 2 for assisting the driving of a main vehicle VP as shown in Figures 1 and 3.

[0082] In a first step 400, the device 2 receives a plurality of images (or shots) associated with respective acquisition times of the images from the camera 1. Such images may in particular be received continuously, for example via a video stream. In such a case, in step 400, the unit 20 of the device 2 can discretize the received video stream so as to obtain a set of discrete images associated with respective acquisition times. From at least two successively acquired images, the unit 20 of the device 2 can carry out, in step 400, image processing, for example by homographies, so as to identify one or more related sets of relatively homogeneous speed on the images. Such image processing then consists of determining a displacement of pixels at a substantially common speed from one image to another, or optical flow.Each optical flow can be determined by matching points between several successive images, for example by the Lucas-Kanade method. Each optical flow is then determined on at least one of the images of a pair of images and is associated with iTTC temporal data. In particular, such iTTC temporal data relates to an inverse magnitude of a collision time associated with the optical flow. The determined optical flows can in particular, at step 400, be segmented into one or more segments having a common point or vanishing line and belonging to planes in the field of view FOV. In other words, each acquired image can be segmented into one or more segments, each segment defining a grouping of optical flows (and therefore a grouping of pixels having substantially the same speed and describing the same movement in a given time interval).Step 400 can then comprise, after determining such optical flows, the generation of hypotheses of movement of the objects and vehicles of the ENV scene, for example on the basis of an affine movement model of the objects and vehicles of the ENV scene, and to classify (or partition) the optical flows into segments. A segment can then be defined as a set of optical flows having a substantially common movement. In particular, all the optical flows of a segment have a substantially common collision time value iTTC: it is therefore considered that a collision time associated with a lateral segment SL is equivalent to a collision time associated with an optical flow belonging to this lateral segment SL. In addition, a segment can be defined by a coplanar set of optical flows. It is thus possible to distinguish three types of segments in a segmented image:.

[0083] - so-called "horizontal" segments, corresponding to coplanar sets of optical flows belonging to planes parallel to the (X, Y) plane in the world frame (X, Y, Z). For example, a horizontal segment can correspond to the plane formed by the road in images 6 to 9.

[0084] - so-called “vertical” segments, corresponding to coplanar sets of optical flows belonging to planes parallel to the (Y, Z) plane in the world frame (X, Y, Z). For example, a vertical segment can correspond to the plane of camera 1.

[0085] - so-called “lateral” segments, corresponding to coplanar sets of optical flows belonging to planes parallel to the plane (X, Z) in the world frame (X, Y, Z). For example, a lateral segment SL is represented in figure 9.

[0086] Obtaining such optical flows and such segments from successively acquired images as well as iTTC temporal data associated with each optical flow are notably described in prior documents WO 2018059629, WO 2018059631 and WO 2018059632. For the remainder of the description, it is considered that at least one segment is determined in step 400. In particular, for the remainder of the description, it is considered that at least one lateral segment is determined from several coplanar optical flows in a detected lateral plane and the remainder of the method is implemented for each lateral segment determined in a segmented image.

[0087] Such a step 400 can be implemented concomitantly with the following steps of the method, in particular when the camera sends a stream of images in real time and continuously (non-punctual). The device 2 can implement the rest of the steps (410, 420, 430 and 440) of the method on a lateral segment considered and continue to detect other optical flows (and / or other segments) in step 400.

[0088] In a step 410, a lateral distance d is first determined for at least one lateral segment SL. Reference is made for this to FIG. 9. FIG. 9 corresponds to the image acquired by the camera 1 at time T1 (corresponding to the image of FIG. 6) on which is represented a lateral segment SL determined by the unit 20 in step 400. The lateral segment SL comprises in particular a set of optical flows represented schematically by points in FIG. 9. In another embodiment, the optical flows can be represented schematically by bipoints reflecting a displacement between two successively acquired images.

[0089] A vertical axis VL of the acquired image is considered. A vanishing point of the acquired image is also considered, corresponding to the vanishing point of the optical flows in the image. Such a vanishing point is represented by the point C' with coordinates (yC',zC') in Figure 9.

[0090] In order to determine the lateral distance d associated with the detected lateral segment SL, the angle formed by each of the straight lines formed between each of the optical flows of the lateral segment SL and the vanishing point C' with the vertical axis VL is considered. An optical flow B' of the lateral segment SL is then chosen so that the angle formed 0 m in between such an optical flow B' chosen and the vanishing point C' with the vertical VL is minimal among the angles 0 respectively formed between each of the optical flows of the lateral segment SL and the vanishing point C' with the vertical VL. With reference to figure 9, an optical flow thus chosen corresponds to the optical flow closest to the ground among the optical flows of the lateral segment SL

[0091] The chosen optical flow is associated with a pixel B' with coordinates (yB'.zB') on the acquired image and corresponding to a third reference point B of the vehicle detected in the scene ENV (not shown in the figures, corresponding for example to a reference point located on the left rear wheel of the neighboring vehicle VV1). In the remainder of the description, reference will be made to the optical flow B' corresponding to the optical flow of the lateral segment SL being closest to the ground thus determined.

[0092] The chosen optical flow B' then corresponds to a pixel B' on one of the acquired images having coordinates (yB',zB') in the image frame (y,z). From such coordinates (yB',zB') in the image frame (y,z), the coordinates of the third reference point B are determined, in step 410, in the world frame (X,Y,Z). In particular, such coordinates of the third reference point B in the world frame (X,Y,Z) can be determined, from the coordinates (yB',zB') and the predefined characteristics of the camera 1, on the basis of the pinhole model. The application of the pinhole model in order to obtain the coordinates of the third reference point B in the world reference frame (X,Y,Z) from the coordinates (yB',zB') of the chosen optical flow B' is based in particular on several calculation hypotheses linked to the properties of camera 1 as well as to the ENV environment associated with the world reference frame (X,Y,Z).For this, conventional extrinsic parameters of the camera 1 can be considered. A “Bird View” type camera 1 can be used. In addition, a flat world assumption (or in English “fiat world assumption”) can be considered. Otherwise, georeferencing of the ENV environment by ground marking or other topographic surveys can also be considered. In the context of the applied pinhole model, the geometric distortions possibly induced by the optical system of the camera 1 can be neglected. In another embodiment, the step 410 of determining the coordinates of the third reference point B in the world reference frame (X,Y,Z) can comprise a correction of the distortion of the lens of the camera 1.

[0093] In step 410, it is then possible to obtain a position according to the world reference frame (X,Y,Z) of the third reference point B corresponding to the chosen optical flow B' in the environment ENV. A lateral distance d (real, expressed in meters) of the third reference point B can then be directly deduced from the coordinate along the Y axis of the position according to the world reference frame (X,Y,Z) of the third reference point B corresponding to the chosen optical flow B' in the environment ENV. chosen optical flow B' in the environment ENV. The lateral distance d can in particular be obtained by calculating the difference between the coordinate along the Y axis of the position according to the world reference frame (X,Y,Z) of the third reference point B in the environment ENV and the coordinate along the Y axis of a point of origin of the camera 1 of the main vehicle VP, for example coincident with a vanishing point of the camera 1. The lateral distance d estimated at step 410 then corresponds to an estimate of a lateral gap separating the main vehicle VP from the object to which the detected lateral segment SL belongs (at the stage of step 410, the system 2 has not yet determined that it is a vehicle, namely the neighboring vehicle VV1).

[0094] A lateral distance d' (on the acquired image, expressed in pixels) of the optical flow B' can also be obtained from the coordinate yB' of the optical flow B' along the y axis in the image frame (y,z) as well as from a coordinate yC' of the origin (or central) point of the camera 1 of the main vehicle VP, for example coincident with the vanishing point C'. Such a central point can for example correspond to the optical center of the acquired image or to a point associated with the location of the camera 1 in the environment ENV. Such a lateral distance d' is for example represented in figure 9.

[0095] Therefore, the lateral distance d associated with a detected SL lateral segment corresponds to the lateral distance d' associated with the optical flow B' detected in the SL lateral segment as being closest to the ground.

[0096] In the remainder of the method described and unless explicitly stated otherwise, any reference to an optical flow of the lateral segment SL corresponds to the chosen optical flow B' belonging to the lateral segment SL and having made it possible to determine the lateral distance d (or equivalently d') associated with the lateral segment SL.

[0097] At the end of step 410, a value of an associated speed v SL to the lateral segment SL can then be estimated:

[0098] Math.1 v SL = dx ITTC where: - v SL is a relative speed associated with the lateral segment SL (expressed in meters per second), - d is the lateral distance associated with the lateral segment SL (expressed in meters), and - iTTC is the time data associated with the lateral segment SL (expressed as the inverse of a time).

[0099] The speed v SLis then an estimated relative speed in the ENV scene of the element corresponding to the detected lateral SL segment (such an element not yet being identified as being a vehicle or not at this stage). Such a speed v SL is relative to a known defined speed VVP of the main vehicle VP.

[0100] At a step 420, the lateral segment SL detected at step 400 and characterized at step 410 by a lateral distance d and in particular, by a relative speed v SL associated with the lateral segment SL, is tested according to a plausibility test, in order to identify whether such a lateral segment SL is likely to correspond to a neighboring vehicle VV1.

[0101] For this, a plausibility test implemented in step 420 can be based on one or more plausibility (or likelihood) criteria, each criterion being able for example to be quantified by a plausibility sub-score.

[0102] Reference is made to Figure 5 detailing sub-steps of step 420 of a plausibility test implemented for a lateral segment SL detected in step 400. Such sub-steps are detailed in a non-limiting manner and the person skilled in the art will easily understand that it is possible to calculate a plurality of plausibility sub-scores on the basis of the position, speed, acceleration, dimensions, timestamp of the acquired images or even lateral distance data associated with a detected lateral segment.

[0103] In a step 421, a value relating to a pixel height associated with the lateral segment SL is estimated, from the acquired image as represented in FIG. 6. The projection of the point (or pixel) A' of FIG. 6 onto the z axis makes it possible to obtain the coordinate zA', which corresponds, in pixels, to a height associated with the lateral segment SL. In step 310, a first sub-score Sh can be determined:

[0104] Math.2 where: s h is a first plausibility sub-score, - zA' is the height (in pixels) associated with the lateral segment SL on an acquired image, - h ths is a predefined height threshold (in pixels or px).

[0105] In other words, the first plausibility sub-score s h checks that a pixel height of the lateral segment SL in the acquired image exceeds a certain height threshold h tfls , which depends on the resolution of the acquired image. Such a height threshold h ths can for example be predefined at 200 px. Such a sub-score then allows to assign a first sub-score s h zero if the height of the lateral segment SL is considered “too close to the ground” (i.e. height zA' below the height threshold h tfls). Indeed, if the detected SL lateral segment is in this sense too close to the ground, the plausibility test considers that it is not plausible that such a SL lateral segment is associated with a vehicle.

[0106] At a step 422, an absolute speed V^ L associated with the lateral segment SL can be determined:

[0107] Math.3 . L = ^VP + V SL where: v SL is the relative speed associated with the lateral segment SL, - V SL is an absolute velocity associated with the lateral segment SL, and - V vp is the known speed of movement of the main vehicle VP and therefore of movement of camera 1.

[0108] Such an absolute speed V^ L associated with the lateral segment SL includes in particular a lateral component, noted V SL i M of standard V SL i at along the Y axis and a longitudinal component, noted V SLiiong of standard V SL iongalong the X axis. For example, if the element associated with the lateral segment SL has a perfectly rectilinear trajectory along the X direction, then V SL lat = 0.

[0109] At step 422, a second sub-score s (at can then be determined on the basis of the V standard SL i at of the lateral component V SL i M of the absolute speed V^ L associated with the lateral segment SL:

[0110] Math.4 where: if at is a second plausibility sub-score, - v sL,iat is the norm of the lateral component of the absolute velocity associated with the lateral segment SL, and - a is a first predefined adjustment parameter (homogeneous to the inverse of a speed). Such a first adjustment parameter can be relative to a maximum lateral speed tolerated for a vehicle along the Y axis.

[0111] In other words, the second plausibility subscore s lat allows us to verify that the lateral segment SL belongs to an element which moves in the same direction as the main vehicle VP, namely substantially along the X axis.

[0112] At step 423, a third sub-score if ong can be determined on the basis of standard V SL i ong of the longitudinal component V SLiiong of the absolute speed V^ L associated with the lateral segment SL:

[0113] Math.5 where: siong is un third plausibility sub-score, - v sL,iong est the norm of the longitudinal component of the absolute velocity associated with the lateral segment SL, and - has 2is a second predefined adjustment parameter (akin to a speed). Such a second adjustment parameter can be related to a minimum tolerated longitudinal displacement speed for a vehicle along the X axis, so as to exclude (i.e., assign a third plausibility sub-score if ong null) the substantially immobile or considered not or insufficiently mobile lateral segments to be associated with vehicles. The lower the value of the second adjustment parameter 2 is, the more false vehicle detections (i.e., false-positives) can be avoided.

[0114] That is to say, the third plausibility sub-score if ong is used to verify that the difference between the respective longitudinal components of the main vehicle VP and the element corresponding to the lateral segment SL is small (i.e., less than a 2 ). Thus, the third plausibility sub-score if ongallows to rule out the hypothesis that the element corresponding to the lateral segment SL is a vehicle, if a significant differential (greater than a 2 ) of speed exists between the main vehicle VP and the element corresponding to the lateral segment SL (such an element could be another object close to the main vehicle VP such as a sign, a bird or a leaf for example).

[0115] At a step 424, the total plausibility score s, can then be determined by:

[0116] Math.6 h (lat + iong) or by:

[0117] Math.7 / i + S a t + S ong where: s h is the first plausibility sub-score, if at is the second plausibility subscore, and - s iong is I e third plausibility sub-score.

[0118] The plausibility score formula can be chosen between the expressions Math. 6 and Math. 7 depending on whether the pixel height criterion of the side segment SL is considered a required preliminary condition or not (binary).

[0119] As a variation of the expressions Math. 6 and Math. 7, the plausibility score s can be based on one or some of the plausibility subscores.

[0120] In other embodiments, one or more other plausibility sub-scores may be determined and accommodated in the Math. 6 or Math. 7 formula. For example, a fourth plausibility sub-score s ang can be related to the lateral distance d associated with the detected lateral segment SL. Such a fourth plausibility subscore s ang can for example be defined by:

[0121] Math.8 Blood ®^t A fourth plausibility subscore, d is the lateral distance associated with the lateral segment SL, D is a predefined Euclidean distance, and - has 3 is a third predefined adjustment parameter.

[0122] In other embodiments, other plausibility sub-scores may include an estimation of the height of the element associated with the lateral segment SL from the pixel B', comparing ratios of lateral and / or longitudinal speeds respectively associated with the main vehicle VP and the lateral segment SL with respect to predefined thresholds for example.

[0123] In step 424, the plausibility score can then be compared to a plausibility threshold Sths comprised for example between 1 and 2, or between 0 and 1. If, at the end of such a comparison in step 424, the plausibility score s is lower than the plausibility threshold s ths, then, a step 430 of detecting a presence state (eg, by a Boolean value of plausibility or non-plausibility of presence) of a vehicle represented in FIG. 4 consists in considering that it is not plausible that the detected lateral segment SL corresponds to a neighboring vehicle VV1 and concludes that there is no vehicle corresponding to the lateral segment SL. The anticipated detection method then stops, as represented in a substep 4242 of FIG. 5, for the lateral segment SL considered and is possibly implemented for another lateral segment detected in step 400. On the contrary, if the plausibility score s is greater than the plausibility threshold Sths, then, the step 430 of detecting a presence state of a vehicle consists in considering that it is plausible that the segment lateral SL is associated with a neighboring vehicle VV1 (sub-step 4241 of figure 5) and concludes that there is a vehicle corresponding to the lateral segment SL.

[0124] Depending on the outcome of step 430 of FIG. 4 (and therefore of step 424 of FIG. 5) of detecting a state of presence of a vehicle associated with the lateral segment SL, a step 440 of creating an object associated with the lateral segment SL can optionally be implemented, as shown in FIG. 4.

[0125] If at step 424, the plausibility score s is strictly lower than the plausibility threshold s t hs, then the device 2 determines that the detected lateral segment SL does not belong to a potential neighboring vehicle (substep 4242). No virtual object is then created in correspondence with the detected lateral segment SL and the method represented in FIG. 4 ends without creation of an object associated with the detected lateral segment SL in step 440. The method can then be repeated for another lateral segment detected in step 400.

[0126] If, at step 424, the plausibility score s is greater than the plausibility threshold s t hs, then a presence of a vehicle is detected (sub-step 4241) in correspondence with the lateral segment SL considered. In other words, the neighboring vehicle VV1 was detected in an anticipated manner on the basis of a lateral portion of the neighboring vehicle VV1 visible in the field of view FOV (resulting in the lateral segment SL detected on the acquired image).

[0127] In this case 4241, the device 2, and more precisely the unit 40 of the device 2, can determine a (plausible) presence of a vehicle associated with the initial conditions (i.e., speed, lateral distance, etc.) of the lateral segment SL detected in step 430. The device 2 can then emit a signal, via the communication interface 50, informing of the detection of a neighboring vehicle VV1 to the main vehicle VP and / or provide data relating to a positioning of such a vehicle detected in the environment ENV (e.g., by providing one or more elements among the lateral distance d, the position of the point B, the speed associated with the lateral segment SL, etc.). Optionally, the device 2 can also proceed, in a step 440, to the creation of an object associated with the lateral segment SL identified as corresponding to a neighboring vehicle VV1.

[0128] Reference is now made to Figures 10, 11 and 12. Figure 10 represents an aerial view of the scene ENV at an acquisition time preceding the instant T3, and the unhatched grayed surface has been identified as corresponding, as one or more detected optical flows (or segments), to a neighboring vehicle VV1. Figure 11 corresponds to a schematization of the second reference point W according to the pinhole model. Figure 12 corresponds to an image acquired at an acquisition time upstream of T3, on which a virtual object associated with the detected lateral segment SL is created by the device 2.

[0129] In the embodiment described subsequently in Figures 10 to 12, it is considered that a neighboring vehicle VV1 has been detected in step 430 as being associated with the lateral segment SL. In a step 440, the unit 40 can estimate a position (X_W, Y_W, Z_W) of a second reference point, or reference point, W belonging to the neighboring vehicle VV1, for example placed at the lower part in the center of the front face of the detected neighboring vehicle VV1, as illustrated in Figures 1, 2 and 10.

[0130] For this, the unit 40 can use predetermined dimensions of a type of vehicle similar to the neighboring vehicle VV1, for example a truck with dimensions L1 in length, L2 in width and H in height and assume that these dimensions correspond to those of the neighboring vehicle VV1.

[0131] From the coordinates (X_C, Y_C) of the central point C of the camera 1 in the world frame (X,Y,Z) corresponding for example to a point associated with the location of the camera 1 or of a sensor of the camera 1 in the environment ENV, of predetermined dimensions L1, L2 of the neighboring vehicle VV1, of the determined real lateral distance d, of the focal length / of the camera 1, and of the size of the sensor sz_capt, the unit 40 is able to determine, by applying the pinhole model as well as Thales' theorem:

[0132] Math.9 Or : - Y_W) are coordinates of the reference point W in the world frame (X,Y,Z), - (X_C, Y_C) are coordinates of the central point C of camera 1 in the world frame (X,Y,Z), - d is the real lateral distance associated with the lateral segment SL, - / is the X-axis component of the camera focal length 1 sensor_size is the actual sensor size, and - L2 is a predefined width dimension of the neighboring vehicle VV1.

[0133] Such coordinates of the reference point W are determined by considering that the optical center C (or central point) is coincident with the center of the acquired image. Optical distortions can be considered neglected. Moreover, the focal length f is considered identical on both axes - vertical VL and horizontal - of the image.

[0134] Indeed, such coordinates (X_W, Y_W) are determined by applying the pinhole model as illustrated in figure 11, on which the origin of the world reference (X,Y,Z) corresponds to the central point C of camera 1. In Figure 10, the origin point O of the world frame (X,Y,Z) differs from the central point C of camera 1, which then has coordinates (X_C, Y_C, Z_C) in the world frame (X,Y,Z). In particular, the formulas Math. 9 depend on the relative positioning of the neighboring vehicle VV1 with respect to the main vehicle VP.

[0135] In particular, the Z_W coordinate of the reference point W along the Z axis in the world frame is determined as corresponding to the coordinate along the Z axis of the chosen optical flow B to determine the lateral distance d, in the world frame. In other words, the reference point W is placed, in height along the Z axis, at the level of the optical flow of the lateral segment SL being closest to the ground.

[0136] Thus, during step 440, the device 2 can obtain coordinates (X_W, Y_W, Z_W) in the world reference frame (X,Y,Z) of a reference point W located on the front face of the detected neighboring vehicle VV1. The device 2 can then be configured to transmit such position data via the interface 50 in order to indicate a front position of the neighboring vehicle VV1. The device 2 can also transmit data making it possible to estimate an occupation space of the environment of the detected neighboring vehicle VV1, for example by transmitting the coordinates (X_W, Y_W, Z_W), the predefined dimensions L1, L2, H. The device 2 can also transmit a detection or alert signal when the main vehicle VP is estimated as approaching such an occupation space of the neighboring vehicle VV1 (for example, by comparing the coordinates of the neighboring vehicle VV1 with positioning coordinates of the main vehicle VP).

[0137] In one embodiment, the device 2 may also be configured to create a virtual object associated with the detected neighboring vehicle VV1 and / or display such a virtual object on an image transmitted via the communication interface 50. For this, the unit 40 may, from the determined coordinates (X_W, Y_W, Z_W) of the reference point W and using the camera model, place a virtual reference point W having coordinates (yW'.zW') in the image frame (y,z) on an acquired image, for example the image on which the lateral segment SL was detected. In particular, the virtual point W is positioned in the image frame (y,z) under several hypotheses:

[0138] - the detected lateral segment SL corresponds to a rear portion of the neighboring vehicle VV1 (eg, it includes a rear wheel of the neighboring vehicle VV1),

[0139] - the virtual reference point W is considered positioned on a lateral edge of the acquired image. In other words, the lateral portion visible on the acquired image is considered to represent the entire length of the neighboring vehicle VV1 in the state considered on the acquired image and the reference point W is considered to be associated with the front of the neighboring vehicle VV1.

[0140] In another embodiment, the virtual reference point W may be placed elsewhere than on the lateral border of the image, for example inside the image or outside the image.

[0141] Thus, if the point, noted O' in figure 12 (corresponding to the pixel located at the bottom left of the acquired image) is considered to be the origin of the image reference (y,z) and the neighboring vehicle VV1 is detected as being on the right lateral side of the main vehicle VP (as illustrated in figures 6 to 10 and 12), we have W' = 0, as illustrated in figure 12.

[0142] In addition, the coordinate zW of the reference point W' along the z axis is determined from the coordinate along the z axis of the chosen optical flow B' to determine the lateral distance d', in the image reference frame. In other words, the reference point W is placed, in height along the z axis, at the level of the optical flow of the lateral segment SL being closest to the ground. In one embodiment, the height of the ground corresponding to the height zW of the reference point W' along the z axis can be predefined and stored by the device 2. In one embodiment, the height zW can also differ from the height of the optical flow of the lateral segment SL.

[0143] Such a virtual reference point W is for example shown in Figure 12.

[0144] From such a reference point W positioned on the acquired image and the predefined dimensions L1, L2, H associated with the neighboring vehicle VV1, a new virtual object can be created by the unit 40 of the device 2, in the two-dimensional space (y,z) and substantially superimposed on the lateral segment SL identified as corresponding to the neighboring vehicle VV1 on the acquired image. An example of such a virtual object is represented by the object OV in FIG. 12.

[0145] Such a virtual object associated with the detected lateral segment SL can then be displayed on the communication interface 50 corresponding to a display screen, for example by superimposing the created object on a display of the image stream coming from the camera 1, as illustrated in FIG. 11.

[0146] In particular, at the end of step 440, a virtual object having an initial position (for example, obtained via the coordinates (X_W, Y_W, Z_W) and the dimensions L1, L2, H) and a speed corresponding to Vw = Vsi_ is determined.

[0147] Therefore, the method allows early detection of a neighboring vehicle from a detected lateral segment SL. In particular, such detection does not require that a frontal face (front or rear) be included in the field of view of the camera 1. -SO- 1148] The method also makes it possible, once such a neighboring vehicle has been detected, to create a virtual object associated with such a lateral segment SL, such a virtual object being created with initial conditions of position, speed and associated dimensions.

Claims

Claims

1. Method for detecting at least one neighboring vehicle (VV1) present in an environment (ENV) of a main vehicle (VP), said neighboring vehicle (VV1) and said main vehicle (VP) being motor vehicles, the method being implemented by a device (2) configured to provide a driving assistance function for the main vehicle (VP), said device (2) being connected to at least one camera (1) on board the main vehicle (VP) and capable of acquiring images of a scene surrounding the main vehicle (VP) according to at least one field of view (FOV) and at acquisition times, the method comprising the following steps: - obtain, for at least one pair of successively acquired images: * at least one lateral segment (SL) corresponding to a lateral coplanar set of optical flows detected in the field of view (FOV), between the respective acquisition times of the two images of said pair of images, and *temporal data linked to a collision time associated with said lateral segment (SL), - determine, according to at least one predefined coordinate system, a value relative to a lateral distance (d) associated with the lateral segment (SL), said lateral distance (d) being determined from at least one measurement carried out on an image among the two images of said pair of images, - determine a speed (VSL) associated with the lateral segment (SL) from the lateral distance (d) and the collision time, - determine, from at least the speed (VSL) associated with the lateral segment (SL) and a known speed (VVP) of the main vehicle (VP), a plausibility score (s) associated with the lateral segment (SL), and - if the plausibility score (s) is greater than at least a predetermined threshold, detect the presence of the neighboring vehicle (VV1) in the scene.

2. Method according to claim 1, in which the lateral distance (d) associated with the lateral segment (SL) is determined from an optical flow belonging to the lateral segment (SL) chosen so that a minimum angle (0 m in) is measured between said chosen optical flow and a vertical axis (VL) defined according to the predefined coordinate system.

3. Method according to any one of the preceding claims, the known speed (VVP) of the main vehicle (VP) and the speed (VSL) associated with the lateral segment (SL) each being composed of a lateral component and a longitudinal component (VVP, long, VSL, long) according to a predefined coordinate system, and in which the plausibility score (s) varies inversely with a difference between the longitudinal component (VsL ong) of the speed (VSL) associated with the lateral segment (SL) and the longitudinal component (Vvp.iong) of the known speed (VVP) of the main vehicle (VP).

4. Method according to any one of the preceding claims, the known speed (VVP) of the main vehicle (VP) and the speed (VSL) associated with the lateral segment (SL) each being composed of a lateral component (VsL.iat) and a longitudinal component (VVP, long, VSL, long) in a predefined coordinate system, and in which the plausibility score (s) varies inversely with an increasing value of the lateral component (VsL.iat) of the speed (VSL) associated with the lateral segment (SL).

5. A method according to any preceding claim further comprising: - a determination, according to the predefined coordinate system, of a value relative to a height associated with the lateral segment (SL), and in which the plausibility score (s) depends on the height associated with the lateral segment (SL).

6. Method according to any one of the preceding claims further comprising, if the presence of the neighboring vehicle (VV1) is detected: - a determination, according to the predefined coordinate system, of a position of at least one reference point (W) associated with the lateral segment (SL), said position of the reference point (W) being determined from at least the lateral distance (d) associated with the lateral segment (SL), and - a creation of an object (OV) associated with the lateral segment (SL) in the field of view (FOV) of the camera (1) from at least the position of the reference point (W).

7. Method according to claim 6, wherein the object (OV) associated with the lateral segment (SL) is further created from predetermined dimensions (L1, L2, H) associated with a type of motor vehicle.

8. Device (2) configured to provide a driving assistance function for a main vehicle (VP), said device (2) being connected to at least one camera (1) on board the main vehicle (VP) and capable of acquiring images of a scene surrounding the main vehicle (VP) according to at least one field of view (FOV) and at acquisition times, in which the device (2) comprises at least one processing circuit (20, 30, 40, 50) configured to implement a method for detecting at least one neighboring vehicle (VV1) present in an environment (ENV) of the main vehicle (VP) according to one of claims 1 to 7.

9. Computer program comprising instructions for implementing the method according to one of the preceding claims when this program is executed by at least one processor (21, 31, 41).

10. Non-transitory recording medium readable by a computer on which is recorded a program for implementing the method according to one of claims 1 to 7 when this program is executed by at least one processor (21, 31, 41).

Images