Method and device for learning a depth prediction model associated with a stereoscopic vision system by comparing the positions of points in a three-dimensional scene.

Abstract

A method or device for learning a depth prediction model implemented by a neural network associated with a vision system. Specifically, the method comprises the generation (32) of depth maps associated with images acquired by each camera of the vision system and the generation (34) of additional images and depth maps from the received images and the depth maps obtained by changing the reference frame. A visibility mask is determined (35) from the generated images, and a zero value is assigned (36) to the pixels of the generated images not included in the visibility mask. The depth prediction model is then learned (38) by minimizing a loss error determined by comparing the positions of points projected (33, 37) into the three-dimensional scene from the different depth maps, the points corresponding to pixels. Figure 3 (for the abstract)

Description

Title of the invention: Method and device for learning a depth prediction model associated with a stereoscopic vision system by comparing the positions of points in a three-dimensional scene. technical field

[0001] The present invention relates to methods and devices for determining depth using a vision system mounted in a vehicle, for example, in a motor vehicle. The present invention also relates to a method and device for measuring the distance between an object and a vehicle equipped with a vision system. Technological background

[0002] Many modern vehicles are equipped with Advanced Driver-Assistance Systems (ADAS). Such ADAS systems are passive and active safety systems designed to eliminate human error in driving all types of vehicles. ADAS systems use advanced technologies to assist the driver while driving and thus improve performance. ADAS systems use a combination of sensor technologies to perceive the environment around a vehicle and then provide information to the driver or act on certain vehicle systems.

[0003] There are several levels of ADAS, such as reversing cameras and blind spot sensors, lane departure warning systems, adaptive cruise control or automatic parking systems.

[0004] The AD AS systems embedded in a vehicle are powered by data obtained one or more onboard sensors, such as cameras. These cameras make it possible to detect and locate other road users or potential obstacles around a vehicle in order to, for example: • to adapt the vehicle's lighting according to the presence of other road users; • to automatically regulate the vehicle's speed; • to act on the braking system in case of risk of impact with an object.

[0005] Predicting the distance between an object and a vehicle, that is, the depth associated with a pixel in an image acquired by a camera, the pixel representing the object, requires high precision to feed ADAS. Predicting the depth associated with a pixel relies in particular on a model of Depth prediction involves adapting the depth prediction model to the vehicle's onboard vision system. Such a prediction model is commonly implemented using a neural network, which is trained to predict accurate depths based on the vehicle's vision system and the type of environment in which it operates. This training phase requires data, such as annotated data from libraries or data from more precise onboard systems like LiDAR®. However, this type of training requires a massive amount of data or the presence of this other onboard system, which the vehicle may not always have.It is then possible to carry out a learning phase of the depth prediction model from data acquired by the on-board vision system, but existing solutions are not adapted to the configuration of every on-board vision system in a vehicle.

[0006] Furthermore, objects present in a three-dimensional scene observed by a stereoscopic vision system, that is, a vision system comprising at least two cameras, may not be visible in some images acquired by the stereoscopic vision system, these objects being occluded from the point of view of one camera of the stereoscopic vision system. The presence of occluded objects is then likely to generate errors and distort the learning of a depth prediction model. Summary of the present invention

[0007] One object of the present invention is to solve at least one of the problems of the technological background described above.

[0008] Another object of the present invention is to improve the quality of data from the processing of an image acquired by a vision system, in particular by a depth prediction model implemented by a neural network associated with a stereoscopic vision system.

[0009] Another object of the present invention is to improve road safety, in particular by improving the reliability of AD AS systems powered by data obtained from a camera, in particular a wide-angle camera.

[0010] According to a first aspect, the present invention relates to a method for learning a depth prediction model implemented by a convolutional neural network associated with a stereoscopic vision system embedded in a vehicle, the stereoscopic vision system comprising a first camera and a second camera arranged so as to each acquire an image of a three-dimensional scene from a different point of view, said process being implemented by at least one processor, and being characterized in which includes the following steps: - reception of a first image and a second image acquired by the first camera and the second camera respectively at the same time instant of acquisition; - generation of a first depth map including depths associated with a set of pixels from the first image and a second depth map including depths associated with a set of pixels from the second image, the depths being predicted with the depth prediction model from the first and second images; - determination of three-dimensional positions of points from a first set of points from the first depth map, each point of the first set of points corresponding to a pixel of the first depth map, and determination of three-dimensional positions of points from a second set of points from the second depth map, each point of the second set of points corresponding to a pixel of the second depth map; - generation of a third image and a third depth map from the first image, the first depth map and extrinsic parameters of the stereoscopic vision system, the third depth map including depths associated with a set of pixels of the third image, and generation of a fourth image and a fourth depth map from the second image, the second depth map and extrinsic parameters of the stereoscopic vision system, the fourth depth map including depths associated with a set of pixels of the fourth image; - determination of a visibility mask as the union of a first visibility mask and a second visibility mask, each pixel of the third image resulting from at least one pixel of the first image belonging to the first visibility mask and each pixel of the fourth image resulting from at least one pixel of the second image belonging to the second visibility mask; - assigning a null value to the pixels of the third and fourth images not included in the visibility mask; - determination of three-dimensional positions of points from a third set of points from the third depth map, each point of the third set of points corresponding to a pixel of the third depth map, and determination of three-dimensional positions of points from a fourth set of points from the fourth depth map, each point of the fourth set of points corresponding to a pixel of the fourth depth map; - learning the depth prediction model by minimizing a loss error, the loss error being determined from: • a first consistency error determined by comparing the three-dimensional positions of points from the first and fourth sets of points, • a second consistency error determined by comparing the three-dimensional positions of points from the second and third sets of points, and • a photometric error determined by comparing the first and fourth images and by comparing the second and third images.

[0011] According to one variant, the first consistency error and the second consistency error are determined respectively by the following function: l^p)=^(**(p)-^(p) )2+ (y^p) )2+ Mp)r With : • Lc^p) corresponding to the first consistency error Lc^p) for a pixel P of the first depth map, respectively the second consistency error For a pixel of the second depth map, a pixel P being defined by coordinates in two dimensions; • p) a component along a first axis of a three-dimensional position of a point of the first set of points corresponding to pixel P of the first depth map denoted xf ( p ), respectively of the second set of points corresponding to pixel P of the second depth map denoted x2 ( p ); • x'4 p) a component along a first axis of a three-dimensional position of a point of the fourth set of points corresponding to the pixel P of the fourth depth map denoted x\(p), respectively of the third set of points corresponding to the pixel P of the third depth map denoted x \ ( p ); • y^p) a component along a second axis of a three-dimensional position from a point in the first set of points corresponding to pixel P of the first map of depths denoted y ( p ), respectively of the second set of points cor corresponding to pixel P of the second depth map denoted y (p}; * y*(p) a component along a second axis of a three-dimensional position of a point from the fourth set of points corresponding to pixel P of the fourth depth map denoted y' (p), respectively from the third set of points corresponding to pixel P of the third depth map denoted y ( p ); • zUp) a component along a third axis of a three-dimensional position of a point from the first set of points corresponding to pixel P of the first depth map denoted z^p), respectively of the second set of points corresponding to pixel P of the second depth map denoted z7(p); and * Z'(p) a component along a third axis of a three-dimensional position of a point in the fourth set of points corresponding to pixel P of the fourth depth map denoted z'(p), respectively of the third set of points corresponding to pixel P of the third depth map denoted z'2(p) ■

[0012] According to another variant, the third set of pixels comprises only pixels from the third depth map included in the visibility mask and the fourth set of pixels comprises only pixels from the fourth depth map included in the visibility mask.

[0013] According to yet another variant, the loss error is determined by the following function: L = L^min(Lcï(p), Lc2(p))+ min(Lpl(p), Lp2(P) )) With : • The loss error, • Lpi(jp) is the first photometric error for a pixel P defined by its two-dimensional coordinates. • Lpi, the second photometric error for pixel P. • Lcl the first consistency error for pixel P, and • Lc2 the second consistency error for pixel P.

[0014] According to a further variant, the determination of a three-dimensional position of a point corresponding to a pixel is obtained by the following function: P3o=^(pF3-d(p,)) With : • Three-dimensional position of a point in the first, second, third, and fourth sets of points, respectively • K' is a direction prediction model associated with the first camera, respectively with the second camera, • 0 a projection function in the three-dimensional scene of a pixel P, as a function of its coordinates and the depth p^p ) associated with it in the first, second, third and respectively fourth depth map.

[0015] According to another variant, the direction prediction model is implemented by a neural network.

[0016] According to yet another variant, the third and fourth depth maps are generated by the following function: Ps = T^p^K^, D(pf) ) ] ) With : • Ps the coordinates of a pixel from the third depth map, respectively from the fourth depth map, • a function to convert from homogeneous coordinates to pixel coordinates by removing one dimension from a vector, • K is a direction prediction model associated with the second camera, respectively with the first camera, • K' is a direction prediction model associated with the first camera, respectively with the second camera, • T is an extrinsic matrix of the stereoscopic vision system, and • 0 a projection function in the three-dimensional scene of a pixel Pt as a function of its coordinates in the first image, respectively second image, and of the depth j^pj which is associated with it in the first depth map, respectively second depth map.

[0017] According to a second aspect, the present invention relates to a device configured to learn a depth prediction model by a vision system embedded in a vehicle, the device comprising a memory associated with at least one processor configured for the implementation of the steps of the process according to the first aspect of the present invention.

[0018] According to a third aspect, the present invention relates to a vehicle, for example of the automobile type, comprising a device as described above according to the second aspect of the present invention.

[0019] According to a fourth aspect, the present invention relates to a computer program which includes instructions adapted for carrying out the steps of the process according to the first aspect of the present invention, in particular when the computer program is executed by at least one processor.

[0020] Such a computer program may use any programming language and be in the form of source code, object code, or an intermediate form between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0021] According to a fifth aspect, the present invention relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the process according to the first aspect of the present invention.

[0022] On the one hand, the recording medium can be any entity or device capable of storing the program. For example, the medium can include a storage means, such as a ROM, a CD-ROM or a microelectronic circuit-type ROM, or a magnetic recording means or a hard disk drive.

[0023] On the other hand, this recording medium can also be a trans medium A signal such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by conventional or radio frequency, by self-directing laser beam, or by other means. The computer program according to the present invention can, in particular, be downloaded from an Internet-type network.

[0024] Alternatively, the recording medium may be an integrated circuit in which the computer program is incorporated, the integrated circuit being adapted to execute or to be used in the execution of the process in question. Brief description of the figures

[0025] Other features and advantages of the present invention will become apparent from the description of the particular and non-limiting embodiments of the present invention below, with reference to the attached Figures 1 to 4, in which:

[0026] [Fig-1] schematically illustrates a vision system equipping a vehicle, according to a a particular and non-limiting example of a realization of the present invention;

[0027] [Fig.2] illustrates a flowchart of the different stages of a determination process mination of a depth of one pixel of an image by a depth prediction model associated with a vision system embedded in the vehicle of the [Fig.1], according to a particular and non-limiting embodiment of the present invention;

[0028] [Fig.3] illustrates a flowchart of the different stages of a learning process of the depth prediction model used in the process of [Fig. 2], according to a particular and non-limiting embodiment of the present invention; and

[0029] [Fig.4] schematically illustrates a device configured to learn a model depth prediction by a vision system embedded in the vehicle of [Fig.1], according to a particular and non-limiting embodiment of the present invention. Description of examples of achievements

[0030] A method and device for learning a depth prediction model implemented by a convolutional neural network associated with a stereoscopic vision system embedded in a vehicle will now be described in what follows with joint reference to Figures 1 to 4. The same elements are identified with the same reference signs throughout the description that follows.

[0031] The terms "first," "second" (or "firsts," "seconds"), etc., are used in this document by arbitrary convention to allow for the identification and distinction of different elements (such as operations, means, etc.) implemented in the embodiments described below. Such elements may be distinct or correspond to a single element, depending on the embodiment.

[0032] For the purposes of this description, the reception of an image is understood to mean the reception of representative image data. Similarly, image generation refers to the generation of representative image data, and depth map generation refers to the generation of representative depth map data. These abbreviations are intended solely to simplify the description; however, since the processes are implemented by one or more processors, it is clear that the input and output data of the various stages of a process are computer data.

[0033] According to a particular and non-limiting embodiment of the present invention, a depth prediction model associated with a stereoscopic vision system comprising two cameras is learned in a learning phase.

[0034] Indeed, the method relating to the learning phase includes the generation of depth maps associated with images acquired by each camera of the vision system and the generation of other images and other depth maps from the received images and depth maps by change of reference frame.

[0035] A visibility mask is determined from the generated images and a null value is assigned to the pixels of the generated images not included in the visibility mask.

[0036] The depth prediction model is then learned by minimizing a loss error determined by comparing the positions of points projected into the three-dimensional scene from the different depth maps, the points corresponding to pixels.

[0037] Fig. 1 schematically illustrates a vision system equipping a vehicle, according to a particular and non-limiting embodiment of the present invention.

[0038] Such an environment 1 corresponds, for example, to a road environment consisting of a network of roads accessible to the vehicle 10.

[0039] In this example, vehicle 10 corresponds to a vehicle with an internal combustion engine, an electric motor(s), or a hybrid vehicle with an internal combustion engine and one or more electric motors. Vehicle 10 thus corresponds, for example, to a land vehicle such as a car, a truck, a bus, or a motorcycle. Finally, vehicle 10 corresponds to an autonomous or non-autonomous vehicle, that is to say, a vehicle operating according to a predetermined level of autonomy or under the total supervision of the driver.

[0040] The vehicle 10 advantageously comprises at least two on-board cameras, a first camera 11 and a second camera 12, configured to acquire images of a three-dimensional scene unfolding in the environment of the vehicle 10 from distinct observation positions. The first camera 11 and the second camera 12 form a stereoscopic vision system when used together, as illustrated in [Fig. 1]. The first camera 11 forms a monoscopic vision system when used alone, and similarly, the second camera 12 forms another monoscopic vision system when used alone. However, the present invention extends to any vision system comprising at least two cameras, for example 2, 3 or 5 cameras.

[0041] The intrinsic parameters of the first camera 11 characterize the transformation which associates, for an image point, hereafter called a "point", its three-dimensional coordinates in the reference frame of the first camera 11 with the pixel coordinates in an image acquired by the first camera 11. These parameters do not change if the first camera 11 is moved. The intrinsic parameters of the first camera 11 include in particular a first focal length fl associated with the first camera 11.

[0042] The intrinsic parameters of the second camera 12 characterize, for their part, the transformation which associates, for an image point, its three-dimensional coordinates in the reference frame of the second camera 12 with the pixel coordinates in an image acquired by the second camera 12. These parameters do not change if the second camera 12 is moved. The intrinsic parameters of the second camera 12 include in particular a second focal length f2 associated with the second camera 12.

[0043] Distortions, which are due to imperfections in the optical system such as defects in the shape and positioning of camera lenses, will deflect the light beams and thus induce a positioning error for the projected point relative to an ideal model. It is then possible to complete the camera model by introducing the three distortions that generate the most significant effects, namely radial, decentering, and prismatic distortions, induced by defects in lens curvature, parallelism, and coaxiality of the optical axes. In this example, the cameras are assumed to be perfect, meaning that distortions are not taken into account, and their correction is addressed during image acquisition or calibration.

[0044] These two cameras 11, 12 are arranged so that each acquires an image of a scene from a different viewpoint. The first viewpoint is, for example, located on or in the left-hand rearview mirror of the vehicle 10 or at the top of the windshield of the vehicle 10. The second viewpoint is, for example, located on or in the right-hand rearview mirror of the vehicle 10 or at the top of the windshield of the vehicle 10. If both cameras are located at the top of the windshield of the vehicle, they are then positioned at a certain distance. In this example, the first camera 11 is located at the top of the windshield of the vehicle 10, and the second camera 12 is located in the right-hand rearview mirror of the vehicle 10.

[0045] A first marker is associated with the first camera 11: - The direction of the x-axis is defined as horizontal and normal to the optical axis Cl of the first camera 11. The distance B separating the optical center of the first camera 11 the projection of the optical center of the second camera 12 onto the horizontal plane passing through the optical center of the first camera 11 is called the reference base (in English "baseline"); - the direction of the y-axis is defined as vertical and normal to the optical axis Cl of the first camera 11; - The direction of the z-axis is defined as orthogonal to the directions of the x and y axes. The three axes x, y and z thus form an orthonormal coordinate system.

[0046] The optical axis Cl of the first camera 11 and the optical axis C2 of the second camera 12 are not necessarily parallel or even included in the same plane.

[0047] According to one variant, the optical axes of the first and second cameras are not coplanar

[0048] The extrinsic parameters related to the position of cameras 11, 12 are the following parameters: - three translations in the x, y, and z directions: Tx, Ty, and Tz, constituting the translation vector T; and - three rotations in the x, y and z directions: 0x, 0y and 0z.

[0049] An extrinsic matrix of the vision system then includes the extrinsic parameters previously defined.

[0050] The extrinsic parameters are determined, for example, during a calibration phase of the stereoscopic vision system comprising the first camera 11 and the second camera 12.

[0051] A key constraint of stereoscopic vision systems used in automobiles is, for example, the large distance between the two cameras. Indeed, to cover a measurement range of 200 meters, the reference base must be 60 cm for cameras commonly used in this field.

[0052] The two cameras 11, 12 acquire images of a scene located in front of the vehicle 10, the first camera 11 alone covering a first acquisition field 13, the second camera 12 alone covering a second acquisition field 14 and the two cameras 11, 12 both covering a third acquisition field 15. The first and third acquisition fields 13, 15 thus allow a monoscopic view of the scene by the first camera 11, the second and third acquisition fields 14, 15 allow a monoscopic view of the scene by the second camera 12 and the third acquisition field 15 allows a stereoscopic view of the scene by the stereoscopic vision system composed of the two cameras 11, 12.

[0053] An obstacle 18 is placed in the acquisition field of the cameras, for example in the third acquisition field 15. The presence of the obstacle 18 defines an occlusion field for the stereoscopic vision system composed here of the three fields 16, 17 and 19.

[0054] Among these three fields, field 16 is visible from the second camera 12. The part of the scene present in this field 16 is therefore observable using the monoscopic vision system comprising the second camera 12.

[0055] The field 17 is visible from the first camera 11. The part of the scene present in this field 17 is therefore observable using the monoscopic vision system comprising the first camera 11.

[0056] Finally, field 19 is not visible to any of the cameras. The part of the scene present in this field 19 is therefore not observable.

[0057] According to one particular embodiment, the field of view of the second camera 12 covers at least half of the field of view of the first camera 11.

[0058] It is evident that it is possible to use such a stereoscopic vision system to take images of scenes located on the sides or behind the vehicle 10 by equipping it with cameras placed and oriented differently.

[0059] The images acquired by the cameras 11, 12 at a given acquisition time are in the form of data representing pixels characterized by: - coordinates in each image; and - data relating to the colours and brightness of objects in the observed scene in the form of, for example, RGB colourimetric coordinates (from the English "Red Green Blue", in French "Rouge Vert Bleu") or HSL (Tone, Saturation, Luminosity).

[0060] Each pixel of the acquired image represents an object in the three-dimensional scene present in the camera's field of view. Indeed, a pixel of the acquired image is the smallest visible unit and corresponds to a point of light resulting from the emission or reflection of light by a physical object present in the three-dimensional scene. When light strikes this object, photons are emitted or reflected, which are captured by a photosensitive sensor of the camera after passing through its lens. This sensor divides the three-dimensional scene into a grid of pixels. Each pixel records the light intensity at a specific location, thus capturing visual details. The combination of millions of pixels creates an image that faithfully represents the physical object observed by the camera. An image point described above is therefore a point on the surface of an object in the three-dimensional scene.

[0061] The images acquired by cameras 11, 12 represent views of the same scene taken from different viewpoints, the camera positions being distinct. For example, this scene includes: - buildings; - road infrastructure; - other stationary users, for example a parked vehicle; and / or - other mobile users, for example another vehicle, a cyclist or a moving pedestrian.

[0062] According to a particular embodiment, the first camera 11 and / or the second camera 12 is of the "wide-angle" type, a wide-angle camera being, for example, equipped with a lens designed to acquire a representative image of a three-dimensional scene seen over a wider field of view than a standard camera, also sometimes called a panoramic lens. In other words, a wide-angle lens makes it possible to capture a larger portion of the three-dimensional scene unfolding in front of or around the wide-angle camera, which is particularly useful in situations where it is necessary to include more elements in the frame of the image acquired by this camera. The angle α of the field of view of the wide-angle camera is, for example, equal to 120°, 145°, 180° or 360°, whereas a standard camera offers, for example, a field of view open at an angle of 45° or less.A wide-angle camera, for example, is a camera equipped with mirrors or a fisheye camera. Wide-angle lenses have a shorter focal length compared to standard lenses, making them suitable for capturing images of landscapes, architecture, road intersections, or any other subject requiring a wide perspective. Wide-angle cameras are used, for example, to capture immersive and dynamic images with an extended depth of field.

[0063] According to a particular embodiment, an image acquired by the first camera 11 and / or an image acquired by the second camera 12 includes a distortion equal to 0.5%, 0.8%, or greater than 1%. The measurement of such distortion corresponds to determining a ratio between: - the maximum spacing of a pixel in the image of a straight line in the first three-dimensional scene whose image is a line touching the longest edge of the first image, either at the center of the image edge or at the corners of the image edge, and - the length of this edge.

[0064] In the world of photography, distortion is commonly considered to be: • negligible if it is less than 0.3%, • not very sensitive if it is between 0.3% or 0.4%, • sensitive if it is between 0.5% and 0.6%, • very sensitive if it is between 0.7% and 0.9%, and • bothersome if it is greater than or equal to 1% or more.

[0065] Barrel distortion is characterized by a positive percentage, while crescent distortion is characterized by a negative percentage.

[0066] According to a particular embodiment, a field of vision of the first Camera 11 covers at least half of the field of view of the second camera 12, and the field of view of the second camera 12 covers at least half of the field of view of the first camera 11. In other words, more than half of the pixels in an image acquired by the first camera 11 correspond to an object in the three-dimensional scene seen by the second camera 12, and pixels in an image acquired by the second camera 12 also correspond to this object in the three-dimensional scene. Similarly, more than half of the pixels in an image acquired by the second camera 12 correspond to an object in the three-dimensional scene seen by the first camera 11, and pixels in an image acquired by the first camera 11 also correspond to this object in the three-dimensional scene.

[0067] The images acquired by the first camera 11 and by the second camera 12 are sent to a computer of a device equipping the vehicle 10 or stored in a memory of a device accessible to a computer of a device equipping the vehicle 10.

[0068] A method for determining depth by a vision system on board the vehicle 10 is advantageously implemented by the vehicle 10, i.e. by a processor, a computer or a combination of computers of the on-board system of the vehicle 10, for example by the computer or computers in charge of the vision system of the vehicle 10.

[0069] Figure 2 illustrates a flowchart of the different steps of a method 2 for determining the depth of a pixel in an image by means of a depth prediction model implemented by a convolutional neural network associated with a vision system embedded in a vehicle, for example in vehicle 10 of Figure 1, according to a particular and non-limiting embodiment of the present invention. Method 2 is implemented, for example, by a device of the vision system embedded in vehicle 10 or by device 4 of Figure 4.

[0070] In a step 21, representative data of an image acquired by the first camera 11 and of an image acquired by the second camera 12 are received.

[0071] In a step 22, depths associated with a set of pixels of one of the received images are determined by the depth prediction model from the two received images.

[0072] Each determined depth then corresponds to a distance separating the vehicle 10 or a part of the vehicle 10 from an object in the three-dimensional scene to which a pixel is associated, the determination of a depth of a pixel then corresponding to a measurement of a distance separating an object from the vehicle carrying the vision system.

[0073] If the ADAS uses these depths or distances as input data to determine the distance between a part of the vehicle 10, for example the front bumper, and another road user present on the road, the ADAS is then able to determine this distance precisely. For example, if the ADAS's function is to act on the braking system of vehicle 10 in the event of a risk of collision with another road user and the distance separating vehicle 10 from that same road user decreases sharply, then the ADAS is able to detect this sudden approach and act on the braking system of vehicle 10 to avoid a possible accident.

[0074] Figure 3 illustrates a flowchart of the different stages of a method for learning the depth prediction model used in a method for determining the depth of a pixel of an image, for example in method 2 of Figure 2, according to a particular and non-limiting embodiment of the present invention.

[0075] The learning process 3 is for example implemented by the device on board the vehicle 10 implementing the process of determining a depth by the vision system on board a vehicle or by the device 4 of the [Fig.4].

[0076] In a step 31, a first image and a second image are received, the first image being acquired by the first camera 11 at an acquisition time instant and the second image being acquired by the second camera at the same acquisition time instant.

[0077] According to a particular embodiment, the first image and second image are of the same definition, that is to say they have the same number of pixels, have the same number of pixels according to their height and the same number of pixels according to their width.

[0078] According to another particular embodiment, the first and second images are not of the same resolution. An additional step then consists of resizing or cropping them to obtain a first and second image of the same resolution.

[0079] In step 32, a first depth map comprising depths associated with a set of pixels from the first image is determined. The depths associated with the pixels of the first image are predicted using the depth prediction model from the first and second images.

[0080] Similarly, a second depth map comprising depths associated with a set of pixels from the second image is determined. The depths associated with the pixels of the second image are also predicted using the depth prediction model from the first and second images.

[0081] The depth prediction model, implemented by a convolutional neural network, is known to those skilled in the art and is presented, for example, in the document "Unifying Flow, Stereo and Depth Estimation" written by Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao and Andreas Geiger, published in July 2023, adapted for any stereoscopic vision system including those with cameras whose optical axes are not in the same plane.

[0082] Thus, a predicted depth is associated with each pixel of a set of pixels of the first image, the first depth map comprising the coordinates of each pixel of the set of pixels of the first image and a depth associated with each pixel recorded for example in a lookup table, for example in a memory accessible to the processor implementing this learning process 3. This lookup table then contains pairs (coordinates of a pixel of the image 1; predicted depth for this pixel).

[0083] According to another particular embodiment, the depths are recorded in an additional channel associated with each image, so a depth map and an image are the same object, each pixel having coordinates in two dimensions, one or more values ​​(image) and a depth (depth map).

[0084] Similarly, a predicted depth is associated with each pixel of a set of pixels of the second image, the second depth map comprising the coordinates of each pixel of the set of pixels of the second image and a depth associated with that each pixel recorded for example in a lookup table, for example in a memory accessible to the processor implementing this learning process 3. This lookup table then contains pairs (coordinates of a pixel of the image 2; predicted depth for that pixel).

[0085] In a step 33, three-dimensional positions of points of a first set of points, which is a virtual point, are determined from the first depth map, each point of the first set of points corresponding to a pixel of the first depth map, and three-dimensional positions of points of a second set of points are determined from the second depth map, each point of the second set of points corresponding to a pixel of the second depth map.

[0086] The three-dimensional position of a point in the first set of points defines spatial coordinates in the three-dimensional scene, in a reference frame associated with the first camera 11, of a point corresponding to a first pixel of the first depth map, based on the coordinates of the first pixel in the first depth map, i.e., in the first image, and the depth predicted for this first pixel, its coordinates and depth being recorded in the first depth map. It should be noted that the coordinates of the first pixel in the first depth map, as well as in the first image, comprise two components while the spatial coordinates of the point associated with the first pixel comprise three components.

[0087] This step 33 amounts to reprojecting into the three-dimensional scene a pixel from the first image and / or the first depth map.

[0088] According to a particular embodiment, the determination of a three-dimensional position of a point in the first set of points corresponding to a pixel of the first depth map is obtained by the following function:

[0089] [Math.l] P3d=^.p^PdM)

[0090] With: • P3D the three-dimensional position of a point in the first set of points, • K' is a direction prediction model associated with the first camera 11, • 0 a projection function in the three-dimensional scene of a pixel Pt as a function of its coordinates and the depth [^p ) which is associated with it in the first depth map.

[0091] If the depths predicted by the depth prediction model are accurate and the direction prediction model is also accurate, then the position in the scene of this virtual point is coincident with that of a point of an object in the scene, this point of the object in the scene then corresponding to the pixel of the first image of which the virtual point is the image.

[0092] According to a first embodiment, the direction prediction model associated with the first camera 11 is an intrinsic matrix of the first camera 11. Such an embodiment is particularly applicable in the case of pinhole cameras generating images with little distortion. An intrinsic matrix is ​​notably determined during an upstream calibration phase of the first camera 11 or of the cameras 11, 12 of the stereoscopic vision system.

[0093] According to a second variant, the direction prediction model is implemented by a neural network, this direction prediction model being better suited to wide-angle cameras generating highly distorted images. Such a direction prediction model is known to those skilled in the art; it is notably presented in the paper "Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion" by Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Wolfram Burgard, Greg Shakhnarovich, and Adrien Gaidon, published in August 2020. The projections and reprojections are then inverse functions obtained from the direction prediction model and are a function of depth and pixel coordinates.

[0094] Similarly, the three-dimensional position of a point of the second set The point definition defines spatial coordinates in the three-dimensional scene, in a reference frame associated with the second camera 12, of a point corresponding to a second pixel of the second depth map. This is based on the coordinates of the second pixel in the second depth map (i.e., in the second image) and the predicted depth for this second pixel, its coordinates and depth being recorded in the second depth map. It should be noted that the coordinates of the second pixel in both the second depth map and the second image comprise two components, while the spatial coordinates of the point associated with the second pixel comprise three components.

[0095] This step 33 amounts to reprojecting into the three-dimensional scene a pixel from the second image and / or the second depth map.

[0096] According to a particular embodiment, the determination of a three-dimensional position of a point in the second set of points corresponding to a pixel of the second depth map is obtained by the following function:

[0097] [Math.2] P3D=^P^DM)

[0098] With: • P^d is the three-dimensional position of a point in the second set of points, • K' is a direction prediction model associated with the second camera 12, • 0 a projection function in the three-dimensional scene of a pixel Pt as a function of its coordinates and the depth associated with it in the second depth map.

[0099] If the depths predicted by the depth prediction model are accurate and the direction prediction model is also accurate, then the position in the scene of this virtual point is coincident with that of a point of an object in the scene, this point of the object in the scene then corresponding to the pixel of the second image of which the virtual point is the image.

[0100] As before, according to the first variant, the direction prediction model associated with the second camera 12 is an intrinsic matrix of the second camera 12 or, according to a second variant, the direction prediction model is implemented by a neural network.

[0101] In a step 34, a third image and a third depth map are generated from the first image, the first depth map and extrinsic parameters of the stereoscopic vision system, the third depth map comprising depths associated with a set of pixels of the third image.

[0102] The generation of the third depth map from the first image, of The first depth and extrinsic parameter map of the stereoscopic vision system consists of: • the determination of spatial coordinates in the three-dimensional scene, in a reference frame associated with the first camera 11, of a point corresponding to a first pixel of the first depth map, from the coordinates of the first pixel in the first image and the depth associated with this first pixel, its coordinates and depth being recorded in the first depth map. It should be noted that the coordinates of the first pixel in the first image comprise two components while the spatial coordinates of the point associated with the first pixel comprise three components; • the determination of spatial coordinates in the three-dimensional scene of the point associated with the first pixel in a reference frame associated with the second camera 12, in other words a change of reference frame from that associated with the first camera 11 to that associated with the second camera 12, using the extrinsic parameters of the stereoscopic vision system; and • The projection of the point associated with the pixel onto the image plane of the second camera 12, this image plane corresponding to that of the second image, allows us to determine the arrival coordinates of the first pixel of the first image in an image as the second camera 12 would have acquired it. The image plane of a camera corresponds to a plane defined in the camera's frame of reference, normal to the optical axis of the camera and located at the first focal length of the camera. The third depth map then includes the arrival coordinates of the first pixels and the depth associated with the first pixel.

[0103] The arrival coordinates of a pixel in the third depth map are thus determined and are the same as the arrival coordinates of a pixel in the third image. In the third depth map, the depth of a pixel corresponds to a depth determined from the pixel depths of the first depth map, while in the third image, the value of a pixel corresponds to a value determined from the pixel values ​​of the first image.

[0104] Similarly, a fourth image and a fourth depth map are generated from the second image, the second depth map, and the extrinsic parameters of the stereoscopic vision system, the fourth depth map comprising depths associated with a set of pixels from the fourth image. The fourth depth map then includes the arrival coordinates of the second pixels from the second depth map and the depth associated with the second pixel.

[0105] According to a particular embodiment, the third depth map is generated by the following function: [Math.3] ps = 7r(K[T^(p)iK'-\ D(pf) ) ] )

[0106] With: • Ps the coordinates of a pixel on the third depth map, •77 a function to convert from homogeneous coordinates to pixel coordinates by removing one dimension from a vector, • K' a direction prediction model associated with the first camera 11, • K a direction prediction model associated with the second camera 12, • T an extrinsic matrix of the stereoscopic vision system, and • 0 a projection function in the three-dimensional scene of a first pixel Pt as a function of its coordinates in the first image and the depth L^p^ associated with it in the first depth map. This function thus determines the arrival coordinates of a pixel of an image acquired by the first camera 11 in an image as it would be acquired by the second camera 12.

[0107] Determining the fourth depth map is similar to determining the third depth map, with the cameras being reversed. The arrival coordinates of a pixel in the fourth depth map are thus determined and are the same as the arrival coordinates of a pixel in the fourth image. In the fourth depth map, the depth of a pixel corresponds to a depth determined from the pixel depths of the second depth map, while in the fourth image, the value of a pixel corresponds to a pixel value determined from the pixel values ​​of the second image.

[0108] Thus, the previous function for determining the arrival coordinates of a first pixel of the first depth map is applicable for determining the arrival coordinates of a second pixel of the second depth map, the function being adapted as follows: [Math.4] p =PK[T^p^~\D(Pl) ) ])

[0109] With: • Ps the coordinates of a pixel on the fourth depth map, • 77 a function to convert from homogeneous coordinates to pixel coordinates by removing one dimension from a vector, • K' a direction prediction model associated with the second camera 12, • K a direction prediction model associated with the first camera 11, • T an extrinsic matrix of the stereoscopic vision system, and • 0 a projection function in the three-dimensional scene of a second pixel Pt as a function of its coordinates in the second image and the depth which is associated with it in the second depth map. This function thus determines the arrival coordinates of a pixel of an image acquired by the second camera 12 in an image as it would be acquired by the first camera 11.

[0110] According to one embodiment, values ​​associated with pixels of the third and fourth images, i.e. colorimetric values ​​of the pixels of the third and fourth images, are obtained by interpolating the values ​​of the pixels of the first image and respectively of the second image using a function such as torch.nn.functional.grid_sample() in Python® language which requires as arguments the arrival coordinates of the pixels of the third and fourth images and the values ​​of the preceding pixels in the first image, respectively second image.

[0111] In a step 35, a visibility mask is determined. This visibility mask is the union of a first visibility mask and a second visibility mask, each pixel of the third image resulting from at least one pixel of the first image belonging to the first visibility mask, and each pixel of the fourth image resulting from at least one pixel of the second image belonging to the second visibility mask. In other words, the first visibility mask includes the set of pixels of the third image that have a predecessor in the first image during the generation step of the third image; that is, the set of pixels of the third image whose coordinates are the destination coordinates of a pixel of the first image.Similarly, the second visibility mask includes the set of pixels in the fourth image that have a predecessor in the second image during the generation step of the fourth image; that is, the set of pixels in the fourth image whose coordinates are the destination coordinates of a pixel in the second image.

[0112] Conversely, pixels not included in the first visibility mask are pixels that have no antecedent in the first image acquired by the first camera 11, they then correspond to occluded objects from the point of view of the second camera 12 and pixels not included in the second visibility mask are pixels that have no antecedent in the second image acquired by the second camera 12, they then correspond to occluded objects from the point of view of the first camera 11.

[0113] The determination of the visibility mask is then the union of the first and second visibility masks, that is to say, coordinates corresponding to those of a A pixel in the third image that has no corresponding pixel in the first image, and which also corresponds to a pixel in the fourth image that has no corresponding pixel in the second image, defines the coordinates of a pixel not included in the visibility mask. Conversely, if coordinates correspond to those of a pixel in the third image that has a corresponding pixel in the first image and / or to those of a pixel in the fourth image that has a corresponding pixel in the second image, then these coordinates define the coordinates of a pixel included in the visibility mask.

[0114] The determination of the visibility mask is obtained for example by using the function torch.nn.functional.grid_sample() in Python® language which requires as arguments the coordinates of the pixels of the third and fourth depth maps and the depths determined in the second and first depth maps.

[0115] In step 36, a value of zero is assigned to the pixels of the third and fourth images not included in the visibility mask. These pixels of the third and fourth images have no antecedent in the first image, respectively in the second image. A pixel with a value of zero corresponds, for example, to a black pixel. In the case of multiple channels, each channel associated with the pixel is assigned a value of zero.

[0116] In a step 37, three-dimensional positions of points of a third set of points are determined from the third depth map, each point of the third set of points corresponding to a pixel of the third depth map, and three-dimensional positions of points of a fourth set of points are determined from the fourth depth map, each point of the fourth set of points corresponding to a pixel of the fourth depth map.

[0117] According to a variant, similarly to step 33, the determination of a three-dimensional position of a point in the third set of points corresponding to a pixel of the third depth map is obtained by the following function:

[0118] [Math.5] P3d = ^P^PD(p,))

[0119] With: • the three-dimensional position of a point in the third set of points, • K' is a direction prediction model associated with the second camera 12, • 0 a projection function in the three-dimensional scene of a pixel Pt as a function of its coordinates and the depth associated with it in the third depth map.

[0120] Similarly, determining a three-dimensional position of a point in the fourth set of points corresponding to a pixel of the fourth pro- map foundry workers is obtained using the following function:

[0121] [Math.6] p}d=^p^\d(pi))

[0122] With: • Pio, the three-dimensional position of a point in the fourth set of points, • K' is a direction prediction model associated with the first camera 11, • 0 a projection function in the three-dimensional scene of a pixel Pt as a function of its coordinates and the depth p^pj associated with it in the fourth depth map.

[0123] In a step 38, the depth prediction model is learned by minimizing a loss error, the loss error being determined from: • a first consistency error determined by comparing the three-dimensional positions of points from the first and fourth sets of points, • a second consistency error determined by comparing the three-dimensional positions of points from the second and third sets of points, and • a photometric error determined by comparing the first and fourth images and by comparing the second and third images.

[0124] According to a particular embodiment, the first consistency error is determined by the following function: [Math.7]

[0125] With: • LcÂp) corresponding to the first consistency error Lci(p) for a pixel P of the first depth map, a pixel P being defined by two-dimensional coordinates; • X*(p) a component along a first axis of a three-dimensional position of a point from the first set of points corresponding to pixel P of the first depth map denoted X^p); * x'*(p) a component along a first axis of a three-dimensional position of a point of the fourth set of points corresponding to pixel P of the fourth depth map denoted x\(p); * (P) a component along a second axis of a three-dimensional position from a point in the first set of points corresponding to pixel P of the first depth map denoted y ( p ); * y' -(p) a component along a second axis of a three-dimensional position from a point in the fourth set of points corresponding to pixel P of the fourth depth map denoted y' (p); * Z*(p) is a component along a third axis of a three-dimensional position of a point in the first set of points corresponding to pixel P of the first depth map, denoted z^(p); and • z'*(p) a component along a third axis of a three-dimensional position of a point in the fourth set of points corresponding to pixel P of the fourth depth map denoted z\( p) ■

[0126] It should be noted that the first consistency error is based on the distance separating a point in the three-dimensional scene obtained from the first depth map from a point in the three-dimensional scene obtained from the fourth depth map.

[0127] According to one variant, the visibility mask is used to determine the first consistency error. Indeed, according to this variant, only points associated with visible pixels, that is, pixels included in the visibility mask, of the fourth depth map are taken into account in this calculation. In other words, the fourth set of pixels comprises only pixels of the fourth depth map included in said visibility mask.

[0128] Similarly, the second consistency error is determined by the following function: [Math. 8] Lc{p) =

[0129] With: • L corresponding to the second consistency error L^Çp) for a pixel P of the second depth map, a pixel P being defined by coordinates in two dimensions; • X*( p) a component along a first axis of a three-dimensional position of a point of the second set of points corresponding to pixel P of the second depth map denoted x^(p); • x'*( p) a component along a first axis of a three-dimensional position of a point of the third set of points corresponding to pixel P of the third depth map denoted x'2 ( / ?) ; • y J p) a component along a second axis of a three-dimensional position of a point of the second set of points corresponding to the pixel P of the second depth map denoted y^ ( p ) ; * y'Ap) a component along a second axis of a three-dimensional position from a point in the third set of points corresponding to pixel P of the third depth map denoted y' (p); * Z*(p) is a component along a third axis of a three-dimensional position of a point in the second set of points corresponding to pixel P of the second depth map, denoted z2(p); and • z'*(p) a component along a third axis of a three-dimensional position of a point in the third set of points corresponding to pixel P of the third depth map denoted z'2(p)-

[0130] It should be noted that the second consistency error is based on the distance separating a point in the three-dimensional scene obtained from the second depth map from a point in the three-dimensional scene obtained from the third depth map.

[0131] According to one embodiment, the visibility mask is used to determine the second consistency error. Indeed, according to this embodiment, only points associated with visible pixels, that is, pixels included in the visibility mask, of the third depth map are taken into account in this calculation. In other words, the third set of pixels comprises only pixels of the third depth map included in the visibility mask.

[0132] According to a first particular embodiment, the photometric error is determined from a first photometric error by comparing pixel values ​​of the first and fourth images and from a second photometric error determined by comparing pixel values ​​of the second and third images, the loss error being determined from the first and second photometric errors.

[0133] A photometric error is, for example, presented in the document "Digging Into Self-Supervised Monocular Depth Estimation" by Clément Godard, Oisin Mac Aodha, Michael Firman and Gabriel Brostow published in August 2019 and is determined by the following function:

[0134] [Math.9] LX / 7 ) ~ ( 1- ^) ■ U(p)-Hp) I +«' ) )

[0135] With: • Lp«(p) the first photometric error denoted Lpi(p), respectively the second photometric error denoted L^p), P being a pixel defined by its coordinates in an image, • Kp) a value of pixel P in the first image, respectively second image, • a value of pixel P in the fourth image, respectively third picture, • SSIM is a function that takes into account a local structure, and • has a weighting factor depending in particular on the type of environment.

[0136] According to this particular embodiment, the loss error is determined by the following function: [Math. 10] L = Yp(min(^ Lc2(p)) + min(Lpl(p), Lp2(p)))

[0137] With: • The loss error, • Lp^p) the first photometric error for a pixel P, P being a pixel defined by its coordinates in an image, • Lp2, the second photometric error for pixel P. • Lcl the first consistency error for pixel P, and • ^c2 the second consistency error for pixel P.

[0138] According to a second particular embodiment, the loss error further includes a construction error, each component of which is determined by the following function: [Math. 11]

[0139] With: • L(p) the component for a pixel p of the third image, respec tively the component L 4( p) ​​for a pixel p of the fourth image , • D(p) is a depth of a pixel P obtained from the third depth map, respectively obtained from the fourth depth map; • W is a parameter matrix; • 0 is the order of a smoothing gradient; • an L1 norm of the second order depth gradients is calculated with W = 1, et0 = 2; • x and y are the dimensions of the images; • / i is a hyperparameter dependent on the environment in which the vehicle operates; and • It{p ) is a value of pixel P in the third image, respectively fourth image.

[0140] This function is generally used to deal with the discontinuity at the edge of objects (in English "edge aware smoothness").

[0141] The loss error includes, for example, the consistency errors, photometric errors and construction error determined above: [Math. 12] L = £p(min(Lcl(p), Lc2(p) ) + min(Lp}(p), Lp2(p) ) + Ls3(p) + Ls4(p))

[0142] With: • The loss error, • Lp](p) the first photometric error for a pixel P, P being a pixel defined by its coordinates in an image, • Lp2, the second photometric error for pixel P. • Lc} the first consistency error for pixel P, • Lc2, the second consistency error for pixel P, • Ls3 is the component in the third image for pixel P, • Lx4 the component in the fourth image for pixel P.

[0143] It should be noted that the pixels compared are those with similar or equal coordinates in both the acquired and generated images.

[0144] It should be noted that taking into account the consistency error also makes it possible to learn the depth prediction model if necessary.

[0145] Furthermore, the visibility mask used for determining the first and second consistency errors (depending on the variant) may be inaccurate at the beginning of training. Therefore, in order to make it more important as training progresses, for example as a function of the number of iterations of the learning process 3, also called the number of epochs, the first and second consistency errors are, for example, weighted by a factor [3] that increases as a function of the number of iterations. Thus, according to another particular embodiment, the loss error includes, for example, the weighted consistency errors, the photometric errors, and the construction error determined above: [Math. 13] L^^min^ Up)) + mm(Lpï(p), Lp2(p))+Ls3(p)+Ls4(p))

[0146] With: • The loss error, • Lpx[p] is the first photometric error for a pixel P, where P is a pixel defined by its coordinates in an image. • Lp2, the second photometric error for pixel P. • a weighting factor between 0 and 1, • Lci the first consistency error for pixel P, • Lc2 the second consistency error for pixel P, • Ls3 the component in the third image for pixel P, • the component in the fourth image for pixel P.

[0147] Learning the depth prediction model consists of adjusting input parameters of the convolutional neural network in order to minimize the previously calculated loss error.

[0148] Furthermore, an object that is occluded or not visible in the field of view of one camera and masked in the field of view of the other camera does not affect the loss error, as the use of a visibility mask makes this learning process insensitive to occlusions. Thus, the depth prediction model used for predicting the depth of a pixel in an image acquired by one of the cameras of the stereoscopic vision system is made more reliable thanks to this learning process.

[0149] This learning process is performed using data acquired by the on-board vision system and therefore does not require data annotated by another on-board system or the storage of a training image library. Furthermore, the training data is representative of the data received when the system is in operation or in production; indeed, the training data is representative of real-world environments in which the vehicle carrying the stereoscopic vision system operates or moves, making this training data particularly relevant.

[0150] Figure 4 schematically illustrates a device 4 configured to learn a depth prediction model from a vehicle-mounted vision system, according to a particular, non-limiting embodiment of the present invention. The device 4 corresponds, for example, to a device mounted in the first vehicle 10, for example, a computer associated with the stereoscopic vision system.

[0151] Device 4 is, for example, configured to carry out the operations described opposite Figures 1 and 4 and / or the steps described opposite Figures 2 and 3. Examples of such a device 4 include, but are not limited to, embedded electronic equipment such as a vehicle's on-board computer, an electronic control unit such as an ECU (Electronic Control Unit), a smartphone, a tablet, or a laptop computer. The elements of device 4, individually or in combination, may be integrated into a single integrated circuit, into several integrated circuits, and / or into discrete components. Device 4 may be implemented in the form of electronic circuits or software (or computer) modules, or a combination of electronic circuits and software modules.

[0152] The device 4 comprises one (or more) processor(s) 40 configured to execute instructions for carrying out the steps of the process and / or for executing instructions from the software embedded in the device 4. The processor 40 may include integrated memory, an input / output interface, and various circuits known to those skilled in the art. The device 4 further comprises at least one memory 41 corresponding for example to volatile and / or non-volatile memory and / or includes a memory storage device which may include volatile and / or non-volatile memory, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic or optical disk.

[0153] The computer code of the embedded software(s) including the instructions to be loaded and executed by the processor is for example stored on memory 41.

[0154] According to various particular and non-limiting embodiments, the device 4 is coupled in communication with other similar devices or systems (for example other computers) and / or with communication devices, for example a TCU (Telematic Control Unit), for example via a communication bus or through dedicated input / output ports.

[0155] According to a particular and non-limiting embodiment, the device 4 includes a block 42 of interface elements for communicating with external devices. The interface elements of the block 42 include one or more of the following interfaces: - radio frequency RF interface, for example of the Wi-Fi® type (according to IEEE 802.11), for example in the 2.4 or 5 GHz frequency bands, or of the Bluetooth® type (according to IEEE 802.15.1), in the 2.4 GHz frequency band, or of the Sigfox type using UBN (Ultra Narrow Band) radio technology, or LoRa in the 868 MHz frequency band, LTE (Long-Term Evolution), LTE-Advanced; - USB interface (from the English "Universal Serial Bus" or "Universal Serial Bus" in French); HDMI interface (from the English "High Definition Multimedia Interface", or "High Definition Multimedia Interface" in French); - LIN interface (from the English "Local Interconnect Network", or in French "Réseau interconnecté local").

[0156] According to another particular and non-limiting embodiment, the device 4 includes a communication interface 43 which enables communication with other devices (such as other computers in the embedded system) via a communication channel 430. The communication interface 43 corresponds, for example, to a transmitter configured to transmit and receive information and / or data via the communication channel 430. The communication interface 43 corresponds, for example, to a wired CAN (Controller Area Network) or CAN FD (Controller Area Network Flexible Data-Rate) type network. flexible data rate controllers”), FlexRay (standardized by ISO 17458) or Ethernet (standardized by ISO / IEC 802-3).

[0157] According to a particular and non-limiting embodiment, the device 4 can provide output signals to one or more external devices, such as a display screen 440, touch or not, one or more speakers 450 and / or other peripherals 460 via the output interfaces 44, 45, 46 respectively. According to a variant, one or more of the external devices is integrated into the device 4.

[0158] Of course, the present invention is not limited to the embodiments described above but extends to a method for determining the depth of a pixel in an image acquired by a vision system, and / or for measuring the distance between an object and a vehicle equipped with a vision system, the depth and / or distance being predicted and / or measured via a depth prediction model learned according to the learning method described above, which would include secondary steps without departing from the scope of the present invention. The same would apply to a device configured for implementing such a method.

[0159] The present invention also relates to a vehicle, for example an automobile or more generally an autonomous land-powered vehicle, comprising the device 4 of [Fig.4].

Claims

1. Demands Method for learning a depth prediction model implemented by a convolutional neural network associated with a stereoscopic vision system embedded in a vehicle (10), the stereoscopic vision system comprising a first camera (11) and a second camera (12) arranged so as to each acquire an image of a three-dimensional scene from a different point of view, said method being implemented by at least one processor, and being characterized in that it comprises the following steps: - reception (31) of a first image and a second image acquired by respectively the first camera (11) and the second camera (12) at the same time instant of acquisition; - generation (32) of a first depth map comprising depths associated with a set of pixels from the first image and of a second depth map comprising depths associated with a set of pixels from the second image, the depths being predicted with the depth prediction model from the first and second images; - determination (33) of three-dimensional positions of points from a first set of points from the first depth map, each point of the first set of points corresponding to a pixel of the first depth map, and determination of three-dimensional positions of points from a second set of points from the second depth map, each point of the second set of points corresponding to a pixel of the second depth map; - generation (34) of a third image and a third depth map from the first image, the first depth map and extrinsic parameters of the stereoscopic vision system, the third depth map comprising depths associated with a set of pixels of the third image, and generation of a fourth image and a fourth depth map from the second image, the second depth map and extrinsic parameters of the stereoscopic vision system, the fourth depth map comprising depths associated with a set of pixels of the fourth image; - determination (35) of a visibility mask as the union of a first visibility mask and a second visibility mask,

2. each pixel of the third image resulting from at least one pixel of the first image belonging to the first visibility mask and each pixel of the fourth image resulting from at least one pixel of the second image belonging to the second visibility mask; - assigning (36) a null value to the pixels of the third and fourth images not included in the visibility mask; - determination (37) of three-dimensional positions of points from a third set of points from the third depth map, each point of the third set of points corresponding to a pixel of the third depth map, and determination of three-dimensional positions of points from a fourth set of points from the fourth depth map, each point of the fourth set of points corresponding to a pixel of the fourth depth map; - learning (38) the depth prediction model by minimizing a loss error, the loss error being determined from: • a first consistency error determined by comparing the three-dimensional positions of points from the first and fourth sets of points, • a second consistency error determined by comparing the three-dimensional positions of points from the second and third sets of points, and • a photometric error determined by comparing the first and fourth images and by comparing the second and third images. Method according to claim 1, wherein the first consistency error and the second consistency error are determined respectively by the following function: With : • Lc^p) corresponding to the first consistency error Lc^p) for a pixel P of the first depth map, respectively the second consistency error Lc~\p) for a pixel P of the second depth map, a pixel P being defined by coordinates in two dimensions; • X*(p) a component along a first axis of a tridi- position Dimensional component of a point in the first set of points corresponding to pixel P of the first depth map, denoted Xj(p), respectively of the second set of points corresponding to pixel P of the second depth map, denoted X2(p); • x'*(p) a component along a first axis of a three-dimensional position of a point in the fourth set of points corresponding to pixel P of the fourth depth map, denoted p), respectively of the third set of points corresponding to pixel P of the third depth map, denoted X^Çp); * yÀP) a component along a second axis of a three-dimensional position of a point in the first set of points corresponding to pixel P of the first depth map, denoted y(p), respectively of the second set of points corresponding to pixel P of the second depth map, denoted (p);* P) a component along a second axis of a three-dimensional position of a point in the fourth set of points corresponding to pixel P of the fourth depth map, denoted y'(p), respectively of the third set of points corresponding to pixel P of the third depth map, denoted p); • z*(p) a component along a third axis of a three-dimensional position of a point in the first set of points corresponding to pixel P of the first depth map, denoted zY(p), respectively of the second set of points corresponding to pixel P of the second depth map, denoted z2(p) ' ct • z'^p) a component along a third axis of a three-dimensional position of a point in the fourth set of points corresponding to pixel P of the fourth depth map, denoted z\(p), respectively of the third set of points corresponding to pixel P of the third depth map, denoted z'2(p);

3. A method according to claim 2, wherein the third set of pixels comprises only pixels from the third depth map included in said visibility mask and the fourth set of pixels comprises only pixels from the fourth depth map included in said visibility mask.

4. A method according to claim 2 or 3, wherein the loss error is determined by the following function: L = ^p(min(Lcl(p\ Lc2(p) )+ min(Lpl(p), Lp2(p))) With: • L the loss error, • Lp{p) the first photometric error for a pixel P defined by its coordinates in two dimensions, • LP2 the second photometric error for the pixel P, • Lc\ the first consistency error for the pixel P, and • Lc2 the second consistency error for the pixel P.

5. A method according to any one of claims 1 to 4, wherein the determination of a three-dimensional position of a point corresponding to a pixel is obtained by the following function: p3O=^(pF'D(p,)) With: • the three-dimensional position of a point from the first, second, third and respectively fourth set of points, • K' a direction prediction model associated with the first camera (11), respectively with the second camera (12), • 0 a projection function in the three-dimensional scene of a pixel Pt as a function of its coordinates and the depth [^pj] associated with it in the first, second, third and respectively fourth depth map.

6. A method according to claim 5, wherein the direction prediction model is implemented by a neural network.

7. A method according to any one of claims 5 to 6, wherein the third and fourth depth maps are generated by the following function: ^=^(^1^(^^(^))]) With: • Ps the coordinates of a pixel of the third depth map, respectively of the fourth depth map, • 71 a function for converting homogeneous coordinates to pixel coordinates by removing one dimension from a vector, • K a direction prediction model associated with the second camera (12), respectively with the first camera (11), • K' a direction prediction model associated with the first camera (11), respectively with the second camera (12), • T an extrinsic matrix of the stereoscopic vision system, and • 0 a projection function in the three-dimensional scene of a pixel Pt as a function of its coordinates in the first image, respectively second image, and of the depth L^p^ associated with it in the first depth map, respectively second depth map.

8. A computer program comprising instructions for carrying out the method according to any one of the preceding claims, when such instructions are executed by a processor.

9. Device (4) configured to learn a depth prediction model by a vision system embedded in a vehicle (10), said device (4) comprising a memory (41) associated with at least one processor (40) configured to implement the steps of the method according to any one of claims 1 to 7.

10. Vehicle (10) comprising the device (4) according to claim 9.