Projection of a light pattern for object detection in a vehicle-facing scene

Abstract

The invention relates to a method that comprises determining (501) a first object detection result. If the first object detection result satisfies each activation condition of a driving assistance function, an activation signal is transmitted (504) to a control device that obtains (506) a light intensity map corresponding to a light pattern and indicative of light intensity values for controlling light elements of a pixelated light source of a light module. The light intensity map is transmitted (507) for projecting (508) a first pixelated light beam according to the light pattern. After projection, a first image representative of the scene is obtained (509) and a depth map is determined (510). A second object detection result is determined (511) from the determined depth map.

Description

[0001] Description

[0002] Title: Projecting a light pattern for object detection in a scene facing a vehicle

[0003] The present invention relates to the field of image processing, in particular, to the detection of an object in a scene facing a vehicle. More specifically, the invention concerns a system and a method for detecting an object in a scene facing a vehicle, such as a motor vehicle, after projecting a light pattern onto the scene.

[0004] Detecting and estimating the distance of an object in a scene facing a vehicle, such as a car or a motorized two-wheeler, is essential to many driver assistance functions, also known as ADAS (Advanced Driver Assistance Systems). These functions assist the driver in piloting the vehicle, and in the case of autonomous vehicles, they can even completely control certain driving parameters without driver intervention.

[0005] To detect the position and estimate the distance of objects in the scene facing the vehicle, it is common practice to use a vehicle sensor such as Lidar. Such a sensor allows for a precise assessment of the position and distance of objects in the scene.

[0006] However, it is preferable, and even necessary in the case of vehicles with high levels of autonomy, to ensure redundancy in object detection and in determining the position and distance of objects, since this information is used by ADAS functions that control the vehicle. Such redundancy ensures robustness in estimating the position and distance of an object, and thus provides greater safety in the execution of the driver assistance function.

[0007] This is particularly the case in a lost cargo scenario, in which objects forming obstacles located in the middle of the road must be detected by an ADAS function in order to operate an emergency braking or an evasive maneuver, in an automated manner.

[0008] Since lidar sensors are expensive, it is preferable not to provide such redundancy by equipping the vehicle with a second lidar sensor. Furthermore, for safety reasons, it is better to ensure redundancy using different technologies, rather than by multiplying sensors of the same type.

[0009] It is therefore desirable to confirm the detection of objects by a main sensor, for example a lidar, in a scene facing the vehicle, particularly in a lost package scenario, preferably without adding a new sensor in the vehicle.

[0010] Furthermore, the solution sought must allow for accurate object detection, including in night scenes or in poor ambient lighting conditions.

[0011] The present invention improves the situation.

[0012] A first aspect of the invention relates to a method for detecting an object in a scene facing a vehicle, the method comprising, during a typical phase, the following steps:

[0013] - determination of a first result of object detection in the scene facing the vehicle, based on a set of data captured by a main sensor of the vehicle;

[0014] - if the first object detection result satisfies each activation condition among at least one activation condition of a vehicle driving assistance function, transmission of an activation signal to a vehicle control device capable of controlling at least one light module comprising a pixelated light source, the pixelated light source comprising a set of individually controllable light elements;

[0015] - upon receipt of the activation signal, obtaining, by the control device, at least one light intensity map, each light intensity map corresponding to a light pattern and indicating light intensity values ​​to control the light elements of the pixelated light source of at least one light module;

[0016] - transmission of at least one light intensity map to at least one light module, for projection of at least one first pixelated light beam according to the light pattern corresponding to at least one light intensity map;

[0017] - obtaining at least one first representative image of the scene facing the vehicle, into which the first pixelated light beam is projected according to the light pattern;

[0018] - determination of a depth map from at least one initial image;

[0019] - determination of a second object detection result from the determined depth map.

[0020] The invention enables a second object detection result, thus ensuring redundancy with the dataset acquired by the primary depth sensor. Furthermore, this redundancy is achieved using hardware already present in most vehicles equipped with a driver assistance system, namely a scene-facing camera and a light module that can be integrated into the vehicle's headlights. In addition, the invention provides that, when data redundancy is required, a light pattern corresponding to a light intensity map is projected onto the scene. This allows for the precise determination of a depth map from which the second object detection result is derived. A depth map facilitates object detection and also provides a direct distance estimate for each detected object.Moreover, a high degree of responsiveness is possible for controlling the projected light beam, since the light intensity map generation model is capable of directly outputting a command that can be used to control the pixelated light source of at least one light module. In some embodiments, the method may further include confirming or refuting at least part of the first object detection result based on the second object detection result.

[0021] Thus, the second object detection result is comparable to the first object detection result, which allows the first object detection result to be confirmed or refuted, thereby strengthening the reliability of the driver assistance function based on the first object detection result.

[0022] According to some embodiments, the depth map can be determined by a determination module capable of applying a depth map determination model to at least one first image to determine the depth map, the determination model having one of the following structures:

[0023] - a convolutional neural network;

[0024] - an artificial neural network of the auto-encoder or variational auto-encoder type;

[0025] - a self-aware or transformative model; or

[0026] - a network generating a system of antagonistic generative networks.

[0027] Such model structures are particularly well-suited for receiving at least one initial image as input and generating a depth map as output. Furthermore, such structures can be used for machine learning.

[0028] In addition, the process may further include a training phase of the determination model, the training phase including a modification of at least one parameter of the determination model as a function of a loss evaluated from a training depth map determined by the determination model from at least one first training image representative of a scene in which a light pattern is projected.

[0029] Thus, the depth map determination model is specifically trained to produce a depth map from a representative image of a scene with a light pattern projection. This improves the accuracy of the depth map determined during the current phase.

[0030] In addition, the training phase may also include obtaining a reference depth map, and the loss can be assessed by comparison between the training depth map and the reference depth map.

[0031] Thus, the depth map determination model can be trained by supervised learning, which allows for accurate depth map determination during the current phase.

[0032] According to a first embodiment, at least one light intensity map can be predetermined and is stored in a vehicle memory.

[0033] Thus, obtaining the light intensity map is facilitated and allows for great responsiveness in projecting the light pattern into the scene facing the vehicle, and therefore great responsiveness in obtaining the second object detection result.

[0034] According to a second embodiment, obtaining at least one light intensity map may include selecting at least one light intensity map from a set of predefined light intensity maps, based on contextual information.

[0035] Such a second embodiment makes it possible to adapt at least one light intensity map obtained to the evolution of the scene in front of the vehicle, while facilitating its acquisition and ensuring high responsiveness of the system.

[0036] According to a third embodiment, obtaining at least one light intensity map may include:

[0037] - obtaining a representative image of the scene facing the vehicle;

[0038] - the application of a light intensity map generation model to the obtained image, to generate at least one light intensity map. Adapting the projected light pattern to the scene facing the vehicle (based on the image of the scene before the light pattern was projected) allows for an adaptive improvement in the accuracy of depth map determination. To this end, the light intensity map generation model can be designed, for example, using machine learning, particularly supervised learning, specifically to improve depth map determination compared to a situation where the light pattern is not projected onto the scene facing the vehicle.

[0039] According to some embodiments, for at least one light intensity map obtained, the light pattern corresponding to the light intensity map may include dark areas and lit areas.

[0040] Such a light pattern, presenting a plurality of strongly contrasting areas, makes it easier to identify the distortion of the light pattern by the scene, and thus to accurately determine the depth map of the scene.

[0041] In addition, the dark areas and the lit areas have identical shapes and sizes and form a checkerboard pattern.

[0042] Such a light pattern allows for precise determination of a depth map of the scene, while being associated with simple control of the pixelated light source.

[0043] According to some embodiments, a first and a second light intensity map can be obtained by the control device upon receipt of the activation signal. The first light intensity map can be transmitted to at least one light module for projection of a first pixelated light beam according to a first light pattern corresponding to the first generated light intensity map, in a first wavelength range. The second light intensity map can be transmitted to at least one light module for projection of a second pixelated light beam according to a second light pattern corresponding to the second generated light intensity map, in a second wavelength range.Obtaining at least one first image may involve acquiring a first image representing the scene onto which the first light pattern is projected in the first wavelength range, and acquiring a second image representing the scene onto which the second light pattern is projected in the second wavelength range. The depth map can then be determined from the first and second images.

[0044] Thus, the depth map can be determined from two images acquired from a scene in which several light patterns are projected into several distinct wavelength ranges, simultaneously or alternately during the same period. This improves the accuracy of the depth map determination.

[0045] According to some embodiments, the driver assistance function may be capable of controlling a vehicle's trajectory to perform an avoidance maneuver and / or emergency braking based on at least one object detected in the scene.

[0046] Such a critical function requires a reliable object detection result. The redundancy enabled by the invention in the object detection result ensures such reliability, which is particularly advantageous in a so-called lost package scenario.

[0047] According to some embodiments, the first image processing result and the second image processing result can each indicate:

[0048] - the presence or absence of at least one object in the scene; and

[0049] - for each detected object, a position of the detected object in the scene.

[0050] Thus, it is possible to detect objects in the scene, and in particular to reliably determine whether these objects are in a dangerous position relative to the vehicle, requiring an evasive maneuver or emergency braking. In some embodiments, the pixelated light source of the light module may comprise submillimeter-sized electroluminescent semiconductor elements epitaxially mounted directly onto a common substrate.

[0051] Such a light module allows the projection of a pixelated light beam according to a high-resolution light pattern, which improves accuracy when determining the depth map.

[0052] A second aspect of the invention relates to a vehicle comprising:

[0053] - a primary depth sensor capable of acquiring a set of data;

[0054] - a processing module capable of determining a first object detection result based on the dataset;

[0055] - a driver assistance module capable of implementing at least one driver assistance function, said driver assistance function being associated with at least one activation condition; the driver assistance module also being capable, when each activation condition of the driver assistance function is met by the first object detection result, of transmitting an activation signal; the vehicle further comprising:

[0056] - at least one light module comprising a pixelated light source, the pixelated light source comprising a set of individually controllable light elements;

[0057] - a control device capable of, upon receiving the activation signal, obtaining at least one light intensity map, each light intensity map corresponding to a light pattern and indicating light intensity values ​​to control the light elements of the pixelated light source of at least one light module;

[0058] - at least one camera arranged to obtain at least one first representative image of a scene facing the vehicle, into which is projected the light pattern corresponding to at least one light intensity map;

[0059] - a depth map determination module capable of determining a depth map based on at least one first image obtained; - an object detection module capable of determining a second object detection result based on the determined depth map.

[0060] Other features and advantages of the invention will become apparent upon examination of the detailed description below, and the accompanying drawings in which:

[0061] [Fig. 1a] illustrates an object detection system in a scene facing a vehicle, according to embodiments of the invention;

[0062] [Fig. 1 b] illustrates a light module of an object detection system for a scene facing a vehicle, according to embodiments of the invention;

[0063] [Fig. 2a] illustrates a light pattern projected by a light module of an object detection system of a scene facing a vehicle, according to embodiments of the invention;

[0064] [Fig. 2b] illustrates an image acquired by a camera following the projection of a light pattern in a scene facing a vehicle, according to embodiments of the invention;

[0065] [Fig. 3a] illustrates the structure of a control device, according to embodiments of the invention;

[0066] [Fig. 3b] illustrates the structure of a depth map determination module according to embodiments of the invention;

[0067] [Fig. 4a] illustrates a pixelated light source of a light module of an object detection system in a scene facing a vehicle, according to one embodiment;

[0068] [Fig. 4b] illustrates a pixelated light source from a light module of an object detection system of a scene facing a vehicle, according to another embodiment;

[0069] [Fig. 5] is a diagram illustrating the steps of a common phase of a method for detecting an object in a scene facing a vehicle, according to embodiments of the invention; [Fig. 6] is a training system for a depth map determination model, according to embodiments of the invention;

[0070] [Fig. 7] is a diagram illustrating the steps of a training phase of a depth map determination model, according to embodiments of the invention.

[0071] The description focuses on the features that distinguish the system and process from those known in the state of the art.

[0072] Figure 1a illustrates an object detection system of a scene 10 facing a vehicle 100, according to embodiments of the invention.

[0073] Vehicle 100 includes a right front projector 105.1 and a left front projector 105.2. The right front projector 105.1 includes at least one first light module 101.1 right and the left front projector includes at least one light module 101.2.

[0074] According to the invention, at least one of the light modules 101.1 and 101.2 is capable of projecting at least one light beam forming a light pattern in the scene 10. According to the invention, the light pattern can be projected in a first wavelength range, which may be a range of visible wavelengths or a range of wavelengths not including any visible wavelengths. The two light modules 101.1 and 101.2 can be capable of projecting light beams forming the same light pattern, by superimposing their respective beams, or of projecting two distinct light patterns, for example, a first light pattern and a second light pattern, either synchronously (simultaneously) or during separate time intervals.

[0075] A straight light beam 120.1 is projected by the straight light module 101.1. The straight light beam 120.1 may comprise several light beams in several distinct wavelength ranges, including the first wavelength range, as described below. Similarly, a left light beam 120.2 is projected by the left light module 101.2. The left light beam 120.2 may comprise several light beams in several distinct wavelength ranges, including the first wavelength range, as described below.

[0076] No restrictions are attached to the first wavelength range according to the invention, which may be a visible range, or an infrared or ultraviolet range, for example. By way of example, the first wavelength range may be within the near-infrared (NIR) range, the short-wave infrared (SWIR) range, the medium-wave infrared (MWIR) range, or the long-wave infrared (LWIR) range. Note that the MWIR and LWIR ranges are also called the thermal range.

[0077] In certain embodiments, and as explained below, the light module 101.1 and / or 101.2 may also be capable of projecting at least a second light beam in at least a second wavelength range, in addition to the first light beam in the first wavelength range, to project the same pattern as in the first wavelength range, to project another light pattern obtained according to the invention, or to perform a lighting or signaling function. For example, the light module 101.1 and / or 101.2 may project, alternately at a given frequency, or simultaneously, the first light beam in the first wavelength range and the second light beam in the second wavelength range (and optionally at least a third light beam in a third wavelength range).

[0078] Thus, the light module 101.1 and / or 101.2 can be capable of projecting during the same time interval (simultaneously or by alternating light beams at a frequency corresponding to a period smaller than the time interval):

[0079] - a first light beam according to a predefined light pattern or generated as described below, in the first wavelength range; and - a second light beam in the second wavelength range, according to a predefined light pattern or generated as described below, or performing a lighting or signaling function when the second wavelength range is the visible range.

[0080] Note that when the first range of wavelengths does not include any visible wavelengths, it is avoided to disturb the driver during the driving of the vehicle when detecting an object in scene 10 according to the invention, in the case where the driving of the vehicle is at least partially manual or under the responsibility of the driver.

[0081] According to the invention, the first light beam projected by the light module 101.1 and / or 101.2 (and optionally the second light beam when it is also projected) is pixelated, which allows the realization of the light pattern with a resolution depending on the number of pixels allowed by a pixelated light source of the light module 101.1 and / or 101.2.

[0082] Figure 1b illustrates the structure of a light module 101 with a pixelated light source 111, for example matrix-based, according to embodiments of the invention. The light module 101 can be the front right module 101.1 and / or the front left module 101.2 described previously.

[0083] The 101 light module includes:

[0084] - a control unit 110 of the pixelated light source 111;

[0085] - the pixelated light source 111;

[0086] - a projection optic for the light coming from the pixelated light source

[0087] 111 to produce at least the first pixelated light beam projected in front of the vehicle towards scene 10. No restrictions are attached to the projection optics, which may include any set of optical elements.

[0088] No restrictions are placed on the number of pixels of the pixelated light source 111. Preferably, the pixelated light source 111 is a high-definition light source, that is, one capable of projecting at least one initial light beam comprising more than one hundred pixels, preferably more than 1000 pixels. The pixelated light source 111 may also be capable of projecting at least one initial light beam comprising more than 10,000 pixels according to embodiments of the invention.

[0089] Furthermore, there are no restrictions attached to the technology associated with the pixelated light source 111, which can be:

[0090] - according to a first example, a matrix of individually controllable light elements, of which at least a first set of light elements is capable of emitting in the first wavelength range for the projection of the light pattern. As detailed below, according to embodiments of the invention, the same matrix of light elements may comprise a first set of light elements capable of emitting in the first wavelength range (for the projection of the light pattern) and a second set of light elements capable of emitting in a second wavelength range (for the projection of the same light pattern, another light pattern or a lighting or signaling beam);

[0091] - according to a second example, a first light source capable of emitting in the first wavelength range and a micromirror array, also called DMD for Digital Micromirror Devices, individually activatable to reflect the light from the first light source towards the projection optics 112. In addition, the pixelated light source 111 may include a second light source capable of emitting in the second wavelength range, as described later. In this case, the micromirror array is alternately controlled, at a given frequency, to produce a first light beam (for the projection of the light pattern) when the first light source is activated, and to produce a second light beam when the second light source is activated (for the projection of the same light pattern, another light pattern, or a lighting or signaling beam);

[0092] - According to a third example, a first laser light source capable of emitting in the first wavelength range (for projecting the light pattern), and a controllable mirror for scanning a predetermined set of positions. Such technology is called laser scanning. The control unit 110 is capable of synchronously controlling the movement of the mirror and the activation of the first laser light source. Furthermore, the pixelated light source 111 may include a second laser light source capable of emitting in the second wavelength range (for projecting the same light pattern, another light pattern, or a lighting or signaling beam), as described later.

[0093] Note that the pixelated light source 111 according to each of the three examples above, can also emit light in at least a third wavelength range, in addition to the first and second wavelength ranges.

[0094] Thus, more generally, the pixelated light source 111 is capable of emitting at least a first light beam according to a light pattern in at least one wavelength range, including at least the first wavelength range.

[0095] The three examples listed above have the advantage of allowing the realization of a light pattern by the projection of at least a first pixelated beam with a high resolution, from a light intensity map obtained and transmitted by a control device 103 described below.

[0096] In the first example described above, the light elements can be electroluminescent elements, individually controlled by a voltage applied to the terminals of each light element by the control unit 110. Each electroluminescent light element can be mounted on its own substrate. Alternatively, the electroluminescent light elements can be on the same substrate, in which case the pixelated light source is said to be monolithic.

[0097] A so-called "monolithic" light source can exhibit a particularly high density of light-emitting elements, making it especially attractive for a wide range of applications. A monolithic source consists of multiple submillimeter-sized electroluminescent semiconductor elements epitaxially bonded directly to a common substrate, typically silicon. Unlike conventional LED arrays, where each individual light-emitting element is an individually manufactured electronic component mounted on a substrate such as a printed circuit board (PCB), a monolithic source is considered a single electronic component. During its production, multiple arrays of electroluminescent semiconductor junctions are generated on a common substrate, forming a matrix.This production technique allows for the creation of closely spaced electroluminescent areas, each acting as a basic light element. The gaps between these light elements can be submillimeter in size. One advantage of this production technique is the high pixel density that can be achieved on a single substrate.

[0098] The first example has the advantage, when the pixelated light source comprises a first set of light elements and a second set of light elements, of projecting two light beams simultaneously in the first and second wavelength ranges. This is because the two sets of light elements can be controlled separately by the control unit 110.

[0099] In the second example described above, the control unit 110 controls the micromirrors of the array to produce the first light beam containing the light pattern in the first wavelength range from a first received light intensity map, when the first light source is active. The resolution of the light pattern then depends on the number of micromirrors in the array.Optionally, when the pixelated light source 111 includes at least one second light source and when the second light source is active, the control unit 110 controls the micro-mirrors of the matrix to realize the second light beam in the second wavelength range, according to the first received light intensity map (to project the same light pattern as in the first wavelength range) or according to a second received light intensity map (predefined or generated according to the invention and corresponding to a second light pattern).

[0100] In the third example described above, the control unit 110 controls the mirror and the first laser light source to scan the predetermined set of positions and produce the first light beam containing the light pattern in the first wavelength range, based on the first received light intensity map. The resolution of the light pattern then depends on the number of positions scanned by the mirror.Optionally, when the pixelated light source 111 includes at least one second laser light source, the control unit 110 controls the mirror and the second laser light source to realize the light beam in the second wavelength range, according to the first received light intensity map (to project the same light pattern as in the first wavelength range) or according to a second received light intensity map (predefined or generated according to the invention and corresponding to a second light pattern).

[0101] Referring again to Figure 1a, the vehicle 100 further comprises at least one camera 102 including a sensor capable of acquiring a first image, or a series of first images, of the scene 10 in the first wavelength range. When the first wavelength range is infrared, the at least one camera 102 comprises a camera including at least one infrared sensor, for example, a thermal camera. When the first wavelength range is visible, the at least one camera 102 comprises a color camera, for example, an RGB (Red Green Blue) camera.

[0102] According to advantageous embodiments, the camera 102 may comprise several sensors, including a first sensor capable of acquiring a first image, or a series of first images, in the first wavelength range, and a second sensor capable of acquiring a second image, or a series of second images, in the second wavelength range. When the pixelated light source is also capable of emitting in at least a third wavelength range, the camera 102 may further comprise at least one other sensor dedicated to at least a third wavelength range.

[0103] Alternatively, the vehicle 100 comprises several cameras 102: a first camera 102 capable of acquiring a first image, or a series of first images, in the first wavelength range, and a second camera 102 capable of acquiring a second image, or a series of second images, in the second wavelength range. When the pixelated light source 111 is also capable of emitting in at least a third wavelength range, the vehicle 100 may further comprise at least one other camera 102 dedicated to at least a third wavelength range.

[0104] Furthermore, in some of the embodiments of the invention, at least one camera 102 is capable of acquiring an image, or a series of images, in a predetermined wavelength range, which may be identical to the first wavelength range, identical to the second wavelength range (or identical to the third wavelength range), or distinct from the first and second wavelength ranges.

[0105] According to the invention, when several patterns (the same or different patterns) are projected into several different wavelength ranges, at least one camera 102 is capable of acquiring several images of the same scene (simultaneously), one image corresponding to each wavelength range into which a light pattern is projected.

[0106] When the same light module is capable of projecting light patterns (the same or different light patterns) in several wavelength ranges with the same matrix light source, and when the matrix light source is according to the second or third example, the alternation frequency between the first beam and the second beam is preferably greater than 100 Hz, for example equal to 700 Hz, so as to correspond to a period much shorter than the exposure time of at least one camera 102, which allows simultaneous capture of several images in the several wavelength ranges.

[0107] The vehicle 100 further comprises a depth map determination module 104.1 according to the invention, configured to determine a depth map (or depth image) of the scene 10 from at least one first image acquired by at least one camera 102 (and optionally from a second image), by applying a depth map determination model 106.1. As described below, the determination model 106.1 can further take as input at least one light intensity map obtained by the control device 103 (in particular when it is generated by the control device 103 according to the third embodiment).

[0108] As described below, the determination model 106.1 can be obtained by machine learning during a training phase of the determination model 106.1.

[0109] According to the invention, the projection of at least one light pattern in the first wavelength range in the scene 10, by controlling the light module 101 by at least one first light intensity map makes it possible to determine a depth map from at least one first image representative of the scene in which the at least one light pattern is projected and acquired in the first wavelength range.

[0110] According to the invention, each light intensity map (including at least the first light intensity map):

[0111] - is predetermined in a first embodiment. In this case, the light pattern projected in each wavelength range (including at least the first wavelength range) is identical regardless of the situation;

[0112] - is selected, in a second embodiment, from among several predefined light intensity maps, based on contextual information; or

[0113] - is generated, in a third embodiment, by a generation model 108 described below, based on a representative image of the scene before projection of a light pattern. The projection of at least one light pattern eliminates the need for a two-camera system capable of acquiring images in the same wavelength range, according to the principle of stereovision, to measure the disparity of each object or pixel of the scene 10 and to deduce depth information from the image.

[0114] Indeed, according to a known stereovision method, two cameras, whose relative positions are predefined and known, can each acquire an image of the same scene in the same wavelength range. The difference between pixels corresponding to the same object is called disparity, and it allows, geometrically, and from the known separation between the two cameras, the object's distance to be determined. It is thus possible to generate a depth map for a pair of images captured by the two cameras; the depth map indicates the distance (a depth) of each pixel in the captured scene.Projecting a light pattern onto the scene eliminates the need for a two-camera system; the distortions of the light pattern by the scene allow access to the depth information of each pixel of an image acquired by a single camera, as will be better understood in light of the description of figures 2a and 2b below.

[0115] The vehicle 100 further includes a control device 103 capable of obtaining and transmitting at least one light intensity map, including the first light intensity map, to at least one light module 101. As previously stated, obtaining at least one light intensity map may include:

[0116] - according to the first embodiment, obtaining at least one light intensity map stored in a vehicle memory, when at least one light intensity map is predetermined;

[0117] - according to the second embodiment, obtaining contextual information, for example from the image in the predetermined wavelength range from at least one camera 102, and selecting at least one light intensity map from a set of predefined light intensity maps, based on the contextual information;

[0118] - according to a third embodiment, obtaining an image in the predetermined wavelength range from at least one camera 102, and applying to said image the generation model 108 of light intensity map, to obtain at least one light intensity map.

[0119] Each light intensity map indicates a light intensity for each light element (or for a set of light elements when the source includes several sets emitting in different wavelength ranges) of the matrix source 111 of the light module 101. Upon receiving at least one light intensity map, the control unit 110 can thus control the matrix source 111 so as to project a light beam according to a light pattern corresponding to the light intensity map, at least in the first wavelength range.

[0120] Furthermore, in certain embodiments, the control unit 110 can control the matrix source 111 so as to project:

[0121] - a first beam of light following the same light pattern defined by the light intensity map in the first wavelength range, and

[0122] - a second light beam according to the same light pattern defined by the light intensity map, according to another light pattern defined by a second light intensity map, or according to a lighting or signaling instruction, in the second wavelength range.

[0123] As previously stated, the predetermined wavelength range can be the first wavelength range, the second wavelength range, or another wavelength range distinct from the first and second ranges. When the predetermined wavelength range is the other wavelength range, at least one camera 102 includes a sensor or camera capable of acquiring images in the other wavelength range.

[0124] According to the third embodiment, the generation model 108 is capable of generating a first light intensity map corresponding to a first light pattern and a second light intensity map corresponding to a second light pattern, from an image received as input in the predetermined wavelength range. The control device 103 can transmit the first light intensity map to the control unit 110, for projection of a first pixelated light beam according to the first light pattern in the first wavelength range, and can also transmit the second light intensity map to the control unit 110 (or to the control unit 110 of another light module 101) for projection of a second pixelated beam according to the second light pattern in the second wavelength range.According to other embodiments, the generation model 108 is capable of generating at least a third light intensity map corresponding to at least a third light pattern, for projection into at least a third wavelength range.

[0125] Alternatively, still in the third embodiment, the control device 103 includes a first generation model 108 capable of generating the first light intensity map from the image received as input in the predetermined wavelength range, and a second generation model 108 capable of generating the second light intensity map from the image received as input in the predetermined wavelength range (and optionally at least a third generation model 108 to generate the third light intensity map).

[0126] According to the first and second embodiments, the control device 103 does not include a generation model 108.

[0127] Thus, each light intensity map is interpreted by a control unit 110 to control the light intensity of each light element of a matrix source 111, or for a set of light elements of the matrix source 111 when the matrix source comprises several sets emitting in different wavelength ranges. For example, the light intensity map includes a light intensity value for each light element of a set (when the light intensity map and the set of light elements of the matrix source 111 have the same resolution) or to control the light intensity of a subset of light elements of the set of light elements (when the matrix source 111 has a higher resolution than the light intensity map).

[0128] The vehicle 100 according to the invention further comprises an object detection module 104.2 capable of detecting one or more objects in a scene, from at least the depth map representative of the scene determined by the determination module 104.1, and, optionally, also from the first image representative of the scene facing the vehicle captured by at least one camera 102 in the first predetermined wavelength range (and optionally from the second image in the second wavelength range).

[0129] The object detection module 104.2 can detect one or more objects in the scene represented by the depth map from the determination module 104.1 (and optionally by the first image captured by at least one camera in the first wavelength range, and again optionally by the second image captured by at least one camera 102 in the second wavelength range), optionally identify the category of each detected object from among several predefined categories, and determine the position of each detected object (or a portion of the image in which the object is located). Each detected object can also be associated with a detection score representing the certainty of the detection.

[0130] The object detection module 104.2 according to the invention can further determine a distance estimate for each detected object, based on the object's position in the depth map. The object's position can correspond to a set of pixels in the depth map, for example, a rectangle of pixels, and the object's distance estimate can be obtained by applying a predetermined function (for example, an average) to the respective depth values ​​of the pixels in the set indicating the object's position. Thus, the object detection module 104.2 can determine a second object detection result, indicating the presence or absence of one or more objects in the scene, and providing, for each detected object, a position in the scene facing the vehicle and optionally a distance estimate.

[0131] To this end, the object detection module 104.2 can implement an object detection model 106.2 capable of determining the second object detection result, from the depth map determined by the determination module 104.1, and optionally from at least one first image representative of the projection of the light pattern in the scene in the first wavelength domain (and optionally from a second image representative of the projection of the same light pattern or a second light pattern, in the second wavelength domain).

[0132] The 106.2 object detection model can be based on a known object detection model, for example a YOLO type model, for “You Only Look Once”.

[0133] According to embodiments of the invention, the object detection model 106.2 is trained in a preliminary phase by machine learning, the machine learning enabling the training of the parameters defining the object detection model 106.2, the model having a predefined structure.

[0134] No restrictions are attached to the machine learning applied to the object detection model 106.2 to optimize its parameters. The machine learning can, for example, be of the supervised type as described later with reference to Figures 6 and 7, in an embodiment in which the detection model 106.2 is trained jointly with the determination model 106.1.

[0135] Alternatively, the object detection model 106.2 is not derived from machine learning, but includes a set of rules capable of detecting one or more objects and determining their respective positions and distance estimates, by applying the set of rules at least to the depth map from the determination model 106.1, and optionally to at least one first image.

[0136] The vehicle 100 according to the invention further comprises at least one main sensor 107, which is, for example, a lidar, capable of capturing a set of data representative of the scene 10 facing the vehicle. The data set is submitted, in a known manner, to a processing module 104.3 capable of detecting, based on the data set, one or more objects in the scene 10, and of determining a distance value for each detected object. The processing module 104.3 is thus capable of producing a first object detection result as output, indicating the presence or absence of at least one detected object in the scene 10 and, for each detected object, a distance value between the detected object and the vehicle 100.

[0137] The system also includes a driver assistance module 109, also called ADAS module 109 (for “Advanced Driver-Assistance Systems”), capable of implementing at least one driver assistance function. At least one driver assistance function, known as ADAS, is based on the first object detection result from the processing module 104.3. For example, an automated vehicle trajectory control function 100 can control the longitudinal and / or lateral speed of vehicle 100 to adapt its trajectory based on the first object detection result output from processing module 104.3. In particular, if an object is detected in the vehicle's lane, the vehicle's trajectory can be adjusted to brake suddenly before the object or to avoid it, which is particularly useful in the lost package scenario described earlier.Furthermore, as described previously, it is necessary to ensure data redundancy for such an ADAS function, and therefore to confirm or refute all or part of the first object detection result, so as to confirm or refute the presence or absence of at least one object detected and indicated in the first processing result, and optionally to confirm the distance to such an object. The invention thus proposes to confirm or refute the first object detection result regarding the presence or absence of objects in the scene obtained from the lidar data 107, by the second object detection result from the object detection module 104.2.

[0138] Figure 2a shows an example of a light beam 120 according to a light pattern corresponding to a light intensity map obtained by the control device 103, in the first wavelength range, projected by a light module 101 as previously described.

[0139] In particular, Figure 2a represents a beam of light projected onto a screen equipped with an orthonormal coordinate system and positioned 25 meters from the projector. Therefore, Figure 2a does not represent the projection of the light pattern onto a real scene, which will be described with reference to Figure 2b.

[0140] According to embodiments of the invention, the control device 103 is capable of obtaining at least one light intensity map corresponding to a discontinuous light pattern, composed for example of dark areas and lit areas, the deformations induced by the scene 10 in the light pattern, in particular at the discontinuities which are the boundaries between dark areas and lit areas, allowing the determination of the depth map by the determination module 104.1.

[0141] Advantageously, according to the third embodiment, at least one light intensity map is generated from the image acquired by at least one camera 102 in the predetermined wavelength range, describing the scene facing the vehicle (before projection of light pattern): it is thus possible to improve the performance associated with the determination of the depth map by the determination model 106.1, as described below, in an adaptive manner as a function of the evolution of the scene 10 facing the vehicle 100.

[0142] In the example in Figure 2a, the light pattern corresponding to the light intensity map obtained by the control device 103 is a checkerboard pattern. The checkerboard pattern exhibits a regular alternation of dark areas 201 and illuminated areas 202, with a strong contrast between these areas.

[0143] Each zone 201 or 202 corresponds to a set of at least one pixel, and preferably to a plurality of pixels, for example several tens or hundreds of pixels.

[0144] Thus, a dark area is created by deactivating the pixels in the area, while a lit area is obtained by activating at least some of the pixels in the lit area. The activation or deactivation of pixels is implemented by the control unit 110, which is capable of controlling the pixelated light source 111. In the first example of a pixelated light source described previously, the control unit 110 creates the light pattern by applying a voltage to the light elements corresponding to the pixels of the lit areas 202, and applies no voltage to the light elements corresponding to the pixels of the dark areas 201, according to the light intensity map received from the control device 103.

[0145] A high-definition pixelated light source 111 thus allows the projection of a light pattern with a large number of dark and lit areas, which then allows high accuracy in determining the depth map, and therefore in determining the second object detection result.

[0146] The checkerboard pattern shown in Figure 2a is for illustrative purposes only. It comprises 9 columns and 8 rows of zones, each zone containing a plurality of pixels. There are no restrictions on the shape of the zones, nor on the distribution of light and dark areas.

[0147] Thus, according to the second and third embodiments of the invention, if the scene facing the vehicle changes, the image received by the control device 103 (submitted as input to the generation model 108 in the third embodiment) changes, which can induce a variation in the light intensity map, either generated as output by the generation model 108 in the third embodiment, or selected from several predefined maps in the second embodiment. Such an adaptation of the light pattern improves the performance of the depth map determination for the new scene facing the vehicle, and therefore improves the accuracy of the second object detection result.

[0148] Figure 2b presents a first image 210 acquired by the camera 102 following the projection of a light beam according to a light pattern in the scene 10 facing the vehicle 100, according to embodiments of the invention.

[0149] The scene shown in figure 2b is intentionally simplified, with few objects, in order to simplify the understanding of the invention.

[0150] The scene in which the light pattern is projected includes another vehicle, as well as a vertical wall located behind the other vehicle. The scene could thus correspond to the interior of a parking lot, a situation given for illustrative purposes only. The invention can advantageously be implemented in an outdoor driving scene, particularly during a night scene.

[0151] As previously explained, the light pattern projected into the scene can be formed in the first wavelength range, and the first image 210 captured by the camera 102 is thus an image in the first wavelength range.

[0152] Thus, at least one 102 camera can acquire:

[0153] - in embodiments, a single image 210 (the first image) of the scene in the first wavelength range during the projection of a light beam in the first wavelength range according to a light pattern corresponding to the light intensity map; or

[0154] - in one variant, the first image 210 described above, as well as a second image 210 of the scene in the second wavelength range, during the projection of a second light beam in the second wavelength range, according to the same light pattern, or according to a different light pattern (when the control device 103 obtains and transmits a first light intensity map and a second light intensity map). In the variant above in which first and second images are acquired, the first and second images are acquired simultaneously (i.e., the exposure time periods of the camera 102 for each of its sensors, or of the cameras 102, overlap or are identical).

[0155] Other variants are possible according to the invention, including the acquisition by at least one camera 102 of at least a third image in the third wavelength range, when a third light beam projects the light pattern or a third light pattern, in the third wavelength range.

[0156] Thus, at least one camera 102 can acquire as many images as there are wavelength ranges in which at least one light pattern is projected. In the examples described above, one or two wavelength ranges are used for projecting light patterns, but the invention also applies to strictly more than two wavelength ranges in which at least one light pattern is projected.

[0157] In what follows, for the sake of simplification, it is assumed that the first image 210 is in the first wavelength range and follows the projection of the light pattern in the first wavelength range by the two light modules 101.1 and 101.2, from a single light intensity map received from the control device 103.

[0158] It can be seen in figure 2b:

[0159] - that the dark and lit areas 211 projected onto a substantially horizontal plane such as the ground have an elongated shape and are thus lengthened;

[0160] - that the dark and lit areas 212 projected onto a substantially vertical and close plane, such as the rear of the other vehicle, retain a square format and have a first given size;

[0161] - that the dark and lit areas 213 projected onto a substantially vertical plane and further away than the areas 212, such as the wall behind the other vehicle, also retain a square format but have a second size greater than the first size; - that the dark and lit areas 214 projected onto irregular objects, with curves, such as the top of the other vehicle, are strongly distorted.

[0162] Thus, the projection of a light pattern into a scene provides, through analysis of the deformation of the projected pattern and after image processing, information on the disparity of the scene which allows us to estimate depth information of the scene and which therefore allows us to construct a depth map.

[0163] Thus, a light pattern comprising a regular repetition of contrasting sub-patterns, such as a repetition of dark and light square or rectangular areas to form a checkerboard, is particularly suited to estimating scene depth information, and therefore allows the accurate determination of a depth map by the determination model 106.1.

[0164] In the third embodiment, the 108 generation model can adapt the characteristics of such a regular repetition (size and shape of dark and light areas, for example) according to the image in the predetermined wavelength range received as input. Alternatively, also in the third embodiment, the 108 generation model is capable of generating a light intensity map corresponding to a light pattern without such a repetition of contrasting sub-patterns, according to the image in the predetermined wavelength range received as input.

[0165] The depth map is determined by the determination model 106 implemented by the image processing module 104, from the first image acquired by at least one camera 102 in the first wavelength domain, or from the first and second images acquired by at least one camera 102 in the first and second wavelength domains.

[0166] Figure 3a shows a structure of a control device 103 according to embodiments of the invention.

[0167] The control device 103 includes an input interface 303 for receiving an activation signal from the ADAS module 109 when the ADAS module 109 receives a first object detection result from the processing module 104.3 indicating that at least one object has been detected in front of the vehicle (for example, an object in the lane of vehicle 100). The activation signal transmitted from the ADAS module 109 to the control device 103 is thus intended to trigger the determination of a second object detection result to confirm or refute the first object detection result.

[0168] In the second and third embodiments, the input interface 303 is further capable of receiving the image captured by at least one camera 102, in the predetermined wavelength range, the image being representative of the scene 10 facing the vehicle (before projection of light pattern in the scene).

[0169] The control device 103 further includes a computing unit 301, such as a processor, configured to obtain one light intensity map, or two light intensity maps, upon receipt of the activation signal on the input interface 303.

[0170] The processor 301 is configured to communicate unidirectionally or bidirectionally, via one or more buses or via a direct wired connection, with a memory 302 such as Random Access Memory (RAM), Read Only Memory (ROM), or any other type of memory (Flash, EEPROM, etc.). Alternatively, the memory 302 may contain several of the aforementioned types. For example, in the second and third embodiments, the memory can temporarily store the image received on the input interface 303.

[0171] In the first embodiment, memory 302 stores the predetermined light intensity map:

[0172] - for projecting a luminous pattern onto the scene in one or more wavelength ranges; or

[0173] - the several predetermined light intensity maps for projecting respective light patterns in respective wavelength ranges. In the second embodiment 302, the memory 302 stores the predefined light intensity maps, from which one or more light intensity maps can be selected for projecting one or more light patterns into the scene, in one or more wavelength ranges.

[0174] In the third embodiment, the memory 302 can further permanently store an algorithm executing the generation model 108 capable of receiving as input an image in the predetermined wavelength range, and providing as output at least one light intensity map (for example a first light intensity map and a second light intensity map).

[0175] According to the third embodiment, the 108 generation model can be trained in a preliminary phase using machine learning, with machine learning enabling the training of the parameters defining the 108 generation model, the 108 generation model having a predefined structure. The 108 generation model can, for example, have one of the following structures:

[0176] - a convolutional neural network, such as an ll-Net type network for example;

[0177] - an artificial neural network of the auto-encoder or variational auto-encoder type, also called VAE in English;

[0178] - a self-aware or transformative model; or

[0179] - any other model structure capable of receiving as input an image representing a scene in the predetermined wavelength range, and of generating a light intensity map (or several light intensity maps).

[0180] No restrictions are attached to the machine learning applied to the generation model 108 to optimize its parameters. The machine learning can, for example, be supervised, as described later with reference to Figures 6 and 7, in embodiments where the generation model 108 is trained jointly with the determination model 106.1. Alternatively, the generation model 108 is not derived from machine learning, but comprises a set of rules capable of determining a light intensity map (or several light intensity maps) by applying the rule set to an image in the predetermined wavelength range, representing the scene facing the vehicle.

[0181] The control device 103 further includes an output interface 304 capable of transmitting to at least one light module 101, the light intensity map obtained, or the light intensity maps obtained, according to the first, second or third embodiment.

[0182] The control device 103 may further include another output interface 305 suitable for transmitting the generated or selected light intensity map, or the generated or selected light intensity maps, to the determination module 104.1, in the second and third embodiments.

[0183] Figure 3b shows a structure of the determination module 104.1 according to embodiments of the invention.

[0184] The determination module 104.1 includes an input interface 313 for receiving the first image, or the first and second images (and optionally the third image), captured by at least one camera 102 and representative of the projection of the light pattern, or light patterns, in the scene 10 facing the vehicle 100, from the light intensity map obtained by the control device 103, or from the light intensity maps obtained by the control device 103.

[0185] The determination module 104.1 further includes a computing unit 311, such as a processor, configured to determine a depth map of the scene 10, from at least the first image received on the input interface 313.

[0186] The processor 311 can, for example, be a graphics processing unit (GPU). The processor 311 is configured to communicate unidirectionally or bidirectionally, via one or more buses or via a direct wired connection, with a memory 312 such as Random Access Memory (RAM), Read Only Memory (ROM), or any other type of memory (Flash, EEPROM, etc.). Alternatively, the memory 312 may contain several of the aforementioned types. The memory can, for example, temporarily store the first image, or the first and second images, received on the input interface 313.

[0187] Memory 312 can also permanently store an algorithm running the determination model 106.1 capable of receiving as input a first image in the first wavelength domain, or first and second images in the first and second wavelength domains, and providing as output a scene depth map 10.

[0188] Memory 312 can, for example, store instructions enabling the execution of the determination model 106.1, which can be defined by a set of parameters optimized to generate a depth map of a scene from at least one first image representative of the projection of at least one light pattern in the scene.

[0189] According to the second and third embodiments, the determination model 106.1 can be capable of determining the depth map from at least one first representative image of the projection of at least one light pattern in the scene, but also from the light intensity map (or maps) corresponding to the projected pattern (or patterns). To this end, when the control device 103 transmits the light intensity map (or maps) to at least one light module 101, the control device 103 also transmits the light intensity map (or maps) to the determination module 104.1.

[0190] According to embodiments of the invention, the determination model 106.1 is trained in a preliminary phase by machine learning, the machine learning enabling the training of the parameters defining the determination model 106.1, the determination model 106.1 having a predefined structure. The determination model 106.1 may, for example, have one of the following structures:

[0191] - a convolutional neural network, such as an ll-Net type network for example;

[0192] - an artificial neural network of the auto-encoder or variational auto-encoder type, also called VAE in English;

[0193] - a self-aware or transformative model; or

[0194] - any other model structure capable of receiving as input a first image, or first and second images, representing the projection of at least one light pattern in the scene, at least in a first range of wavelengths, and optionally the light intensity map (or light intensity maps), and of determining a depth map of the scene.

[0195] There are no restrictions on the machine learning applied to the 106.1 determination model to optimize its parameters. The machine learning can, for example, be supervised, as described later with reference to Figures 6 and 7.

[0196] Alternatively, the determination model 106.1 is not derived from machine learning, but includes a set of rules capable of determining depth information by applying the set of rules to at least a first image representing the projection of at least one light pattern in a scene, in at least the first wavelength range, and optionally from the light intensity map (or maps) corresponding to the projected light pattern (or patterns).

[0197] The determination module 104.1 can apply preprocessing to each first received image (or first and second received images) before applying the depth map determination model 106.1. The preprocessing can consist of cropping the first received image to retain only the portion of the received image corresponding to the projection of the light pattern in the scene 10. Other preprocessing steps can be applied according to the invention. The determination module 104.1 further includes an output interface 314 capable of transmitting the determined depth map to the object detection module 104.2 described above, for application of the object detection model 106.2, determination of the second object detection result, and transmission of the second object detection result to the ADAS module 109.

[0198] The determination module 104.1 may further include another input interface 315 suitable for receiving the light intensity card, or light intensity cards, obtained by the control device 103, as described previously, in the second and third embodiments.

[0199] Figure 4a illustrates a pixelated light source 111 according to the first example above, capable of forming two light beams in two distinct wavelength ranges.

[0200] The pixelated light source 111 according to the first example includes:

[0201] - a first set of 401 light elements, individually controllable, capable of emitting light in the first range of wavelengths;

[0202] - a second set of 402 light elements, individually controllable, capable of emitting in the second wavelength range.

[0203] In the example in Figure 4a, the first wavelength range is an infrared range and the second wavelength range is a visible range: for this purpose, the second set of light elements 402 includes subsets of blue, red and green light elements, which makes it possible to project a colored light beam into scene 10 facing the vehicle:

[0204] - to project the same light pattern as the first set of light elements according to the light intensity map;

[0205] - to project, when the first and second light intensity cards are received, according to a light pattern corresponding to the second light intensity card; or

[0206] - to perform a lighting or signaling function, as previously described. Alternatively, each element of the second assembly 402 may be capable of emitting white light.

[0207] Thus, the light elements of the first set 401 can be controlled to form a first light beam according to the light pattern corresponding to the light intensity map received (or to the first light intensity map received when several light intensity maps are received from the control device 103), with a resolution depending on the number of light elements of the first set 401.

[0208] Simultaneously, the light elements of the second set 402 can be controlled to form a second light beam according to the light pattern corresponding to the light intensity map, according to another light pattern corresponding to the second light intensity map, or performing a lighting or signaling function according to a lighting or signaling instruction received from the control device 103.

[0209] According to embodiments not shown in the figures, at least a third set of light elements can be controlled to form a third light beam according to the light pattern, or according to the third light pattern, corresponding to the light intensity map or the third light intensity map received from the control device 103, in the third wavelength range.

[0210] Figure 4b illustrates a pixelated light source 111 according to the second example, comprising two light sources 411.1 and 411.2 and a DMD micromirror array.

[0211] The first light source 411.1 is capable of emitting light rays in the first wavelength range. The second light source 411.2 is capable of emitting light rays in the second wavelength range. According to undescribed variants, at least a third light source is capable of emitting light rays in the third wavelength range. The light sources 411.1 and 411.2 can be arranged on the same support 412, as shown in Figure 4b, or on two separate supports.

[0212] Only one micromirror 410 is shown in Figure 4b to facilitate understanding of the figure. However, in practice, the matrix comprises a large number of micromirrors, in particular more than 100, or even more than 1000 or more than 10,000 micromirrors individually controllable by the control unit 110 described previously.

[0213] Each micromirror 410 can be controlled to switch between at least one first position 420.1 and a second position 420.2. In the first position 420.1, the micromirror is arranged to reflect a light beam from a source, for example a light beam 413 emitted by the first light source 411.1, towards the projection system 112 so as to contribute to the formation of a light beam outside the vehicle 100. Thus, by controlling the positions of the micromirrors of the matrix by the control unit 110, the light module projects a first light beam according to the light pattern corresponding to the received light intensity map (or the first received light intensity map), in the first wavelength range, when the first light source 411.1 is activated.Similarly, by controlling the positions of the matrix micro-mirrors by the control unit 110, the light module projects a second light beam according to the same pattern corresponding to the first received light intensity map, according to another pattern corresponding to the second light intensity map, or to perform a lighting or signaling function (when the second wavelength range is the visible range).

[0214] The control unit 110 can thus control the same array of micromirrors, alternately at a given frequency, to project the first and second light beams alternately. The switching frequency between the two light beams can be greater than 700 Hz, so that the flicker of the second beam is not perceptible to the human eye (when the second wavelength range is the visible range) and therefore does not disturb the driver and other road users, nor is it perceptible to the camera 102 or cameras 102, having an exposure time longer than the period associated with the switching frequency.

[0215] Figure 5 is a diagram illustrating the steps of a common phase of a method for detecting an object in a scene facing a vehicle, according to embodiments of the invention.

[0216] At a stage 500, lidar 107 obtains the scene 10 dataset facing vehicle 100, as previously described.

[0217] At step 501, the processing module 104.3 determines the first object detection result from the dataset captured by the lidar 107. As described previously, the first object detection result indicates the presence or absence of at least one object detected in the scene 10 and, for each detected object, a position in the scene and a distance value between the detected object and the vehicle 100.

[0218] The first object detection result is transmitted by the processing module 104.3 to the ADAS module 109 at a step 502.

[0219] At step 503, the ADAS module determines whether an object is detected in the scene 10 based on the first object detection result, and can further determine whether the detected object is located on the road or in the vehicle's lane 100. More generally, at least one activation condition is predefined for an ADAS function capable of implementing an evasive maneuver or emergency braking: step 503 then consists of verifying whether each activation condition is met, based on the first object detection result. The detection of at least one object in the scene can thus be a first activation condition, and the object's location on the road or in the vehicle's lane can be a second activation condition. A third activation condition can be based on the distance value indicated in the first object detection result.For example, during an emergency braking maneuver, the driver assistance function can be activated if the distance value is sufficient given the vehicle's current speed. If at least one activation condition is not met, the ADAS function is not triggered and the process returns to step 500.

[0220] Conversely, if each activation condition is met, the process proceeds to a 504 step.

[0221] At step 504, the ADAS module 109 transmits the activation signal to the control device 103, to trigger the obtaining of a second object detection result.

[0222] Upon receiving the activation signal, the control device 103 can obtain, at a step 505, implemented in the second and third embodiments, an image in the predetermined wavelength range, representative of the scene 10 from at least one camera 102 (before projection of a light pattern).

[0223] At step 506, the control device 103 obtains at least one light intensity map. According to some embodiments, the control device 103 obtains a first light intensity map and a second light intensity map.

[0224] As described previously, step 506 of obtaining at least one light intensity map may include:

[0225] - according to the first embodiment, obtaining at least one light intensity map stored in a vehicle memory, when at least one light intensity map is predetermined;

[0226] - according to the second embodiment, the selection of at least one light intensity map from a set of predefined light intensity maps, based on contextual information including the image obtained in step 505 in the predetermined wavelength range;

[0227] - according to the third embodiment, the application to the image obtained in step 505 in the predetermined wavelength range of the generation model 108 of the light intensity map, to obtain at least one light intensity map. At a step 507, the control device 103 transmits the at least one light intensity map obtained to the at least one light module 101. In addition, the at least one light intensity map obtained can be transmitted to the determination module 104.1.

[0228] At step 508, the control unit 110 controls the matrix source 111 according to at least one received light intensity map, to project:

[0229] - a first light beam according to the light pattern corresponding to the light intensity map in the first wavelength range (when only one light intensity map is received); or

[0230] - a first light beam according to the light pattern corresponding to the light intensity map in the first wavelength range and a second light beam according to the light pattern corresponding to the light intensity map in the second wavelength range (when only one light intensity map is received), and optionally at least a third light beam according to the light pattern corresponding to the light intensity map in at least the third wavelength range; or

[0231] - a first light beam according to the light pattern corresponding to the first light intensity map in the first wavelength range and a second light beam according to the other light pattern corresponding to the second light intensity map in the second wavelength range (when at least two light intensity maps are received), and optionally at least a third light beam according to the third light pattern corresponding to the third light intensity map in at least the third wavelength range; or

[0232] - a first light beam according to the light pattern corresponding to the light intensity map in the first wavelength range and a second light beam according to a lighting or signaling instruction also received from the control device 103.

[0233] Following step 508, at least one camera 102 obtains, at a step 509, at least one first image representing the projection of the light pattern in the scene 10, in the first wavelength range. Optionally, at least one camera also obtains a second image representing the projection of the same light pattern (corresponding to the light intensity map) in the scene in the second wavelength range, or representing the other light pattern (corresponding to the second light intensity map) in the scene in the second wavelength range (and optionally a third image representing the projection of the same light pattern or the third light pattern in at least the third wavelength range).

[0234] At a step 510, the determination module 104.1 determines a depth map, for example by applying the determination model 106.1, from at least the first image in the first wavelength range obtained at step 509, and optionally from the second image in the second wavelength range obtained at step 509, and / or from at least one light intensity map obtained at step 506 and transmitted by the control device 103.

[0235] At step 511, the object detection module 104.2 determines, for example by applying the object detection model 106.2 to the determined depth map, and optionally to at least one first image obtained in step 509, a second object detection result from the depth map. The second object detection result indicates the presence or absence of one or more objects in the scene, and provides, for each detected object, a position in the scene facing the vehicle and a distance estimate.

[0236] At step 512, the second object detection result is passed to the ADAS 109 module.

[0237] At step 513, the ADAS 109 module compares the first object detection result received at step 502 with the second object detection result received at step 512 to confirm or refute all or part of the first object detection result. For example, the first object detection result can be confirmed if:

[0238] - the same number of objects are detected in the first object detection result and in the second object detection result; and

[0239] - the positions of the objects detected in the first object detection result correspond in the second object detection result; and

[0240] - Optionally, for each object detected and located, the distance value in the first object detection result and the distance estimate in the second object detection result are identical or close (e.g., their difference is less than a given threshold).

[0241] Otherwise, if at least one object detected in the first object detection result is not detected in the second object detection result, or if the positions of the respective objects detected in the first and second object detection results do not match, the first object detection result may be invalidated, at least partially. Optionally, if the difference between the estimated distance for a detected object and the actual distance value for that same object exceeds a given threshold, the first object detection result may also be invalidated, at least partially.

[0242] At step 314, the ADAS 109 module implements the ADAS emergency avoidance or braking function, depending on whether the first object detection result is confirmed or refuted by the second object detection result. For example, if the first object detection result is fully confirmed, at least one ADAS activation condition is met, allowing an emergency avoidance or braking maneuver to be performed. If there is a difference between the estimated distance to an object detected on the road and the actual distance to that same object, the distance value can be corrected (for example, using a weighted sum or an average of the estimated and actual distances), and the emergency avoidance or braking maneuver is adjusted based on the corrected distance value.Other adaptations of the ADAS function to the confirmation or refutation, partial or total, of the first object detection result by the second object detection result may be provided according to the invention.

[0243] If the second object detection result completely refutes the first object detection result, a new dataset can be acquired by the lidar 107 for determination of a new object detection result by the processing module 104.3.

[0244] Figure 6 is a 600 drive system of a 106.1 depth map determination model, according to embodiments of the invention.

[0245] Such a drive system 600 is external to the vehicle 100 and is capable of implementing the drive phase of the process according to the invention, which is described later with reference to Figure 7. The drive phase is prior to the current phase described with reference to Figure 5, which takes place during a driving situation of the vehicle 100. The drive phase can in particular be part of the design and manufacturing process of the depth map determination module 104.1 (or even of the generation model 108 of the control device 103 in the third embodiment, when the generation model 108 and the determination module 104.1 are driven jointly), before its integration into the vehicle 100.

[0246] The 600 drive system may differ depending on the embodiment.

[0247] The 600 drive system is first described in the first and second embodiments.

[0248] The 600 training system includes a 601 training database. The 601 training database is capable of storing training data associations, each training data association comprising:

[0249] - at least one first training image representative of a projection of a predefined (predetermined or selected) light pattern in a scene, in the first wavelength range. In the case where, in current phase, several light beams with at least one predefined light pattern are projected in several wavelength ranges, the at least one first training image is a set of several training images representative of projections of several light patterns in a scene in the first wavelength range, or a set of several training images representative of projections of a light pattern in several wavelength ranges;

[0250] - reference data including a reference depth map, for this same scene, indicating the “ground truth”.

[0251] The associations stored in the 601 training database can be, for example:

[0252] - obtained by accumulating initial training images and associated depth reference maps, in various real-world driving situations, for example for different scenes (city driving, motorway driving, rural driving) and in various real-world weather conditions; or

[0253] - obtained by simulating initial training images and associated reference depth maps, in various simulated driving situations, for example for various simulated scenes (in town, on highway, in the countryside) and in various simulated climatic conditions.

[0254] Preferably, the associations stored in the training database are varied, that is to say, they were obtained in scenes varying according to several criteria: scene composition, scene brightness level, weather conditions, etc.

[0255] In addition, the 601 training database includes more than one hundred training data associations, and preferably several thousand or even tens of thousands of training data associations.

[0256] The 600 training system further includes the 106.1 depth map determination model to be trained. The 106.1 depth map determination model is structurally capable of generating a depth map from at least one representative image of a scene in which at least one predefined light pattern (according to the first and second embodiments) is projected in at least one wavelength range.

[0257] For this purpose, the 106.1 depth map determination model can have one of the following structures:

[0258] - a convolutional neural network, such as an ll-Net type network for example;

[0259] - an artificial neural network of the auto-encoder or variational auto-encoder type, also called VAE in English;

[0260] - a generating network of a system of generative adversarial networks, also called GAN, for “Generative Adversarial Networks” in English;

[0261] - a self-aware or transformative model; or

[0262] - any other generative model structure capable of receiving as input at least one first image representing the projection of at least one predetermined light pattern in a scene in at least one first wavelength range, and of producing as output at least one depth map.

[0263] The 600 training system also includes a 604 loss assessment module, which is suitable for:

[0264] - to evaluate a loss by comparison between the ground truth reference data (reference depth maps) from training database 601 and the outputs of determination module 106.1 (training depth maps);

[0265] - to modify one or more parameters of the generation model 106.1 according to the evaluated loss, according to a predefined training strategy.

[0266] No restrictions are attached to the loss function used, nor to its application to the comparison between the ground truth reference data and the outputs of the determination module 106.1.

[0267] Such a 600 drive system is thus capable of implementing supervised learning of the depth map determination model 106.1. In the third embodiment, during the current phase, at least one light pattern is generated according to the scene 10 facing the vehicle by the generation model 108. Thus, in the third embodiment, the 600 drive system further includes the light intensity map generation model 108, which corresponds to the generation model 108 implemented in the control device 103 during the current phase.

[0268] In the third embodiment, the training images stored in the training database represent scenes with vehicles, but without the projection of at least one predefined lighting pattern, as in the first and second embodiments. The training images are then submitted to the generation model 108, and not to the determination model 106.1. Upon receiving a training image, the generation model 108 is capable of generating at least one training light intensity map.

[0269] In the third embodiment, the training system 600 further includes a synthesis module 602, capable of producing at least one synthetic image representative of the scene in the training image, illuminated by a light pattern corresponding to at least one light intensity map (called the training light intensity map during the training phase) produced as output by the generation model 108, in the first wavelength range. Thus, the synthesis module 602 makes it possible to simulate the capture by a camera of the scene in the training image, onto which a light pattern corresponding to the training light intensity map output by the generation model 108, in the first wavelength range, would be projected. To this end, the synthesis module 602 receives the following input:

[0270] - the training image; and

[0271] - at least one training light intensity map generated by the generation model 108. According to embodiments, the synthesis module 602 can generate a first synthetic image representative of the scene of the training image, illuminated by a light pattern corresponding to a training light intensity map produced at the output of the generation model 108, in the first wavelength domain, and a second synthetic image representative of the same scene illuminated by a light pattern (the same light pattern or a different light pattern) corresponding to a training light intensity map (the same training light intensity map as for the first synthetic image, or a second different training light intensity map) produced at the output of the generation model 108, in the second wavelength domain.

[0272] Thus, in the third embodiment, at least one synthetic image generated by the synthesis module 602 is transmitted to the determination model 106.1, which is capable of determining the depth map as previously described with reference to the first and second embodiments. In addition, the determination model can also receive at least one training light intensity map generated by the generation model 108.

[0273] In the third embodiment, the generation model 108 can be trained in a separate training phase from the training phase of the determination model 106.1. In this case, the generation model 108 used in the training system 600 is derived from such a prior learning phase and is therefore already optimized during the training phase of the determination model 106.1. Alternatively, the generation model 108 and the determination model 106.1 are trained jointly: in this case, the loss evaluation module 604 is capable of modifying at least one parameter of the generation model 108 and at least one parameter of the determination model 106.1, depending on the evaluated loss.

[0274] According to a variant of the 600 training system described with reference to Figure 6: - the reference data of the training database are not reference depth maps, but reference object detection results;

[0275] - the training system 600 further includes the object detection model 106.2 arranged between the determination model 106.1 and the loss evaluation module 604, the object detection model being already optimized following prior training, or being trained jointly with the determination model 106.1;

[0276] - the training depth map determined by the determination model 106.1 is transmitted to the object detection model 106.2 which determines, on the basis of the training depth map, and optionally on the basis of at least one synthetic image, a training object detection result;

[0277] - the training object detection result is transmitted from the object detection model 106.2 to the loss evaluation module 604;

[0278] - the loss evaluation module 604 evaluates a loss by comparing the training object detection result with the reference object detection result;

[0279] - the loss evaluation module 604 updates at least one parameter of the depth map determination model 106.1, and optionally at least one parameter of the generation model 108 and / or the object detection model 106.2, when the generation model 108 and / or the object detection model 106.2 are trained jointly with the determination model 106.1.

[0280] Thus, the 600 training system, according to one of the first to third embodiments, allows the implementation of supervised learning of the determination model 106.1, and optionally of the generation model 108 (in the third embodiment) and / or of the object detection model 106.2.

[0281] Alternatively, the determination model 106.1 can be derived from a machine learning method other than supervised learning. For example, the determination model 106.1 can be optimized by reinforcement learning, with a reward determined based on a training depth map defined by the determination model 106.1, or on the performance of the object detection module 106.2 receiving the determined training depth map.

[0282] Thus, in general, the determination model 106.1 can be trained by machine learning in order to improve the performance associated with the determination of the depth map.

[0283] Figure 7 is a diagram illustrating the steps of a training phase of a depth map determination model 106.1, in a method of detecting an object in a scene facing a vehicle, according to embodiments of the invention.

[0284] As previously explained, the training phase of the 106.1 depth map determination model can be implemented in the 600 training system described with reference to Figure 6.

[0285] Figure 7 illustrates both the training phase in the first, second and third embodiments.

[0286] The training phase for the first and second embodiments is described first.

[0287] At a stage 700, the training database 601 obtains a training data association as previously described, comprising at least one initial training image of a vehicle-facing scene into which at least one predetermined light pattern is projected, and a reference depth map representative of ground truth.

[0288] In the first and second embodiments, the process then proceeds directly to a step 703.

[0289] In step 703, the obtained training image is fed into the determination model 106.1, which generates a depth map based on at least one initial training image. In step 704, the loss evaluation model assesses the loss by comparing the depth map determined in step 703 with the reference depth map obtained using at least one initial reference image in step 700.

[0290] At a step 705, the loss evaluation module 604 can further determine whether a predefined convergence criterion is met or not, based in particular on the loss evaluated at step 704, and optionally on losses evaluated during previous iterations of steps 700 to 704.

[0291] If the convergence criterion is not met, the loss evaluation module 604 modifies at least one parameter of the determination model 106.1 at a step 706, based on the evaluated loss, according to the predefined training strategy. Such parameter optimization training strategies during supervised learning are well known and are not described further here.

[0292] Following step 706, or following step 707 described below in the third embodiment, the process training phase returns to step 700 to repeat steps 700 to 705 on the basis of a new association of training data from training database 601.

[0293] At step 708, when the loss evaluation module 604 determines that the convergence criterion is met during step 705, the training phase is completed and the determination model 106.1 can be implemented in the control device 103 for implementation of step 509 of the current phase of the process according to the invention.

[0294] In the third embodiment:

[0295] - during step 700, the association of training data obtained includes a reference depth map, and a training image representative of the scene, without projection of a light pattern (instead of at least a first training image representative of the projection of at least one predetermined light pattern as in the first and second embodiments); - following step 700, the process proceeds to a step 701 of generating at least one training light intensity map, by the generation model 108, as a function of the training image obtained in step 700;

[0296] - following step 701, the process includes the generation of at least one first synthetic image by the synthesis module 602, based on at least one training light intensity map generated in step 701 and the training image obtained in step 700;

[0297] - the application of the determination model 106.1 to at least one first synthetic image during step 703.

[0298] Still in the third embodiment, at least one training light intensity card can be passed from the generation model 108 to the determination model 106.1 during step 701, when the determination module 106.1 also takes the training light intensity card as input.

[0299] In addition, in the third embodiment, the loss evaluation module 604 can modify at least one parameter of the generation model 108, at a step 707, as a function of the evaluated loss, according to the predefined training strategy, when the determination model 106.1 and the generation model 108 are jointly trained.

[0300] The present invention is not limited to the embodiments described above by way of example; it extends to other variants.

Claims

Demands 1. A method for detecting an object in a scene facing a vehicle (100), the method comprising, during a typical phase, the following steps: - determination (501) of a first result of object detection in the scene facing the vehicle, based on a set of data captured (500) by a main sensor (107) of the vehicle; - if the first object detection result satisfies each activation condition among at least one activation condition of a vehicle driving assistance function, transmission (504) of an activation signal to a vehicle control device (103) capable of controlling at least one light module (101.1; 101.2; 101) comprising a pixelated light source (111), the pixelated light source comprising a set of individually controllable light elements; - upon receipt of the activation signal, obtaining (506), by the control device, at least one light intensity map, each light intensity map corresponding to a light pattern and indicating light intensity values ​​to control the light elements of the pixelated light source of at least one light module; - transmission (507) of at least one light intensity card to at least one light module, for projection (508) of at least one first pixelated light beam according to the light pattern corresponding to at least one light intensity card; - obtaining (509) at least one first representative image of the scene facing the vehicle, into which the first pixelated light beam is projected according to the light pattern; - determination (510) of a depth map from at least one first image; - determination (511) of a second object detection result from the determined depth map.

2. A method according to claim 1, further comprising a confirmation or refutation (513) of at least a part of the first object detection result as a function of the second object detection result.

3. A method according to claim 1 or 2, wherein the depth map is determined by a determination module (104.1) capable of applying a depth map determination model (106.1) to at least a first image to determine (510) the depth map, the determination model having one of the following structures: - a convolutional neural network; - an artificial neural network of the auto-encoder or variational auto-encoder type; - a self-aware or transformative model; or - a network generating a system of antagonistic generative networks.

4. Method according to claim 3, further comprising a training phase of the determination model, the training phase comprising a modification (706) of at least one parameter of the determination model (106.1) as a function of an evaluated loss (704) from a training depth map determined by the determination model from at least one first training image representative of a scene in which a light pattern is projected.

5. Method according to claim 4, wherein the training phase further comprises obtaining (700) a reference depth map, and wherein the loss is evaluated (704) by comparison between the training depth map and the reference depth map.

6. A method according to any one of the preceding claims, wherein at least one light intensity map is predetermined and is stored in a vehicle memory.

7. A method according to any one of claims 1 to 5, wherein obtaining (506) at least one light intensity map includes selecting at least one light intensity map from a set of predefined light intensity maps, based on contextual information.

8. A method according to any one of claims 1 to 5, wherein obtaining at least one light intensity map comprises: - obtaining (505) a representative image of the scene facing the vehicle; - an application (506) of a light intensity map generation model to the obtained image, to generate said at least one light intensity map.

9. A method according to any one of claims 1 to 8, wherein, for at least one light intensity map obtained, the light pattern corresponding to the light intensity map includes dark areas (201) and lit areas (202).

10. Method according to claim 9, wherein the dark areas (201) and the lit areas (202) have identical shapes and sizes and form a checkerboard pattern.

11. A method according to any one of the preceding claims, wherein a first light intensity map and a second light intensity map are obtained (506) by the control device (103), upon reception of the signal activation; in which the first light intensity map is transmitted (507) to at least one light module (101.1; 101.2; 101), for projection (508) of a first light beam pixelated according to a first light pattern corresponding to the first generated light intensity map, in a first wavelength range; wherein the second light intensity map is transmitted to at least one light module, for projection of a second light beam pixelated according to a second light pattern corresponding to the second generated light intensity map, in a second wavelength range; wherein the obtaining (509) of at least a first image includes obtaining a first image representative of the scene into which the first light pattern is projected in the first wavelength range, and obtaining a second image representative of the scene into which the second light pattern is projected in the second wavelength range; wherein the depth map is determined (510) from the first and second images.

12. A method according to any one of the preceding claims, wherein the driving assistance function is capable of controlling a vehicle trajectory (100) to perform an avoidance maneuver and / or emergency braking based on at least one object detected in the scene.

13. A method according to any one of the preceding claims, wherein the first image processing result and the second image processing result each indicate: - the presence or absence of at least one object in the scene; and - for each detected object, a position of the detected object in the scene.

14. A method according to any one of the preceding claims, wherein the pixelated light source (111) of the light module (101) comprises electroluminescent semiconductor elements of submillimeter dimensions, epitaxially mounted directly on a common substrate.

15. Vehicle (100) comprising: - a main sensor^ 07) capable of acquiring a set of data; - a processing module (104.3) capable of determining a first object detection result based on the dataset; - a driver assistance module (109) capable of implementing at least one driver assistance function, said driver assistance function being associated with at least one activation condition; the driver assistance module also being capable, when each activation condition of the driver assistance function is met by the first object detection result, of transmitting an activation signal; the vehicle further comprising: - at least one light module (101.1; 101.2; 101) comprising a pixelated light source (111), the pixelated light source comprising a set of individually controllable light elements; - a control device (103) capable of, upon receiving the activation signal, obtaining at least one light intensity map, each light intensity map corresponding to a light pattern and indicating light intensity values ​​to control the light elements of the pixelated light source of at least one light module; - at least one camera (102) arranged to obtain at least one first image representative of a scene (10) facing the vehicle, in which is projected the light pattern corresponding to at least one light intensity map; - a depth map determination module (104.1) capable of determining a depth map based on at least one first image obtained; - an object detection module (104.2) capable of determining a second object detection result based on the determined depth map.

Images