Improving semantic perception by controlling a vehicle's lighting

Abstract

The invention relates to a method for controlling a vehicle's lighting module. Upon obtaining (400; 401) a first image representing a scene facing the vehicle, a contour detection function (402) is applied to the first image to obtain a contour image. A light intensity map (404) is determined from the contour image. This light intensity map indicates a first light intensity control value for light elements capable of projecting light onto the contours of the scene elements represented by the contour image, and a second light intensity control value for light elements capable of projecting light outside the contours. The determined light intensity map (405) is transmitted to at least one lighting module for projecting a pixelated beam of light onto the scene facing the vehicle. FIG. 4

Description

Title of the invention: Improvement of semantic perception by controlling the lighting of a vehicle

[0001] The present invention relates to the field of controlling a motor vehicle's lighting module. More specifically, the invention relates to a method and device for controlling a motor vehicle's lighting module, to improve a semantic perception function, for example object detection or semantic segmentation, applied to an image captured in the motor vehicle.

[0002] Most motor vehicles are now equipped with a driver assistance module, also called AD AS, for “Advanced Driver-Assistance Systems”, capable of implementing at least one driver assistance function, allowing the driver to be assisted in driving or to control, in an automated manner, without contribution from the driver, certain driving parameters of the vehicle.

[0003] Such functions are based on data captured by sensors on the vehicle, such as a lidar, a radar, one or more cameras, etc.

[0004] Several driver assistance functions implemented by ADAS modules use images captured by a camera capable of obtaining representative images of a scene facing the vehicle. However, some ADAS functions require that these images be processed before being used. Such processing may include, in a known manner: - a semantic segmentation function designed to segment each image into several pixel regions, each region being labeled with a class from a set of predefined classes. In automotive applications, the following classes might be used: car, pedestrian, sign, road, etc. A semantic segmentation score can be associated with the semantic segmentation of the image, or with each segment determined within the image, the score representing the degree of certainty associated with the semantic segmentation; and / or - an object detection function designed to detect one or more objects in the scene represented by each image, to identify the category of each detected object from among several predefined categories, and to determine the position of each detected object (or a part of the image in which the object is located). Each detected object is further associated with a detection score representing the certainty associated with the object detection; - an instance segmentation function aimed at segmenting each image into several pixel regions for each object in the scene, as with semantic segmentation, but without labeling each region with a class from a set of predefined classes. An instance segmentation score can be associated with the instance segmentation of the image, or with each segment determined within the image, the score representing the degree of certainty associated with the instance segmentation; - a panoptic segmentation function, which implements both semantic segmentation and instance segmentation, thus aiming to segment each image into several pixel regions for the different objects in the scene, each object being labeled with a class from a set of predefined classes. A panoptic segmentation score can be associated with the panoptic segmentation of the image, or with each segment determined within the image, the score representing the degree of certainty associated with the panoptic segmentation.

[0005] However, such semantic perception functions are less effective when ambient light is low, or even very low, as is the case when the camera captures images representative of a night scene. In such situations, the performance of semantic perception functions is much lower than in daytime driving situations, which can prevent the implementation of certain driver assistance functions (or reduce their accuracy), or even lead to safety problems in the vehicle, particularly at high levels of autonomy.

[0006] There is therefore a need to improve the accuracy associated with at least one semantic perception function applied to an image of a scene facing a motor vehicle, when the ambient light is low.

[0007] The present invention improves the situation.

[0008] A first aspect of the invention relates to a method for controlling at least one lighting module in a vehicle, the method comprising the following steps: - obtaining a first image representative of a scene facing the vehicle; - applying a contour detection function to the first image, to obtain a contour image comprising contours of elements of the scene represented in the first image; - determination of a light intensity map, from the contour image, the light intensity map indicating light intensity control values ​​for controlling light elements of a pixelated light source from at least one vehicle lighting module, the light intensity map indicating at least a first light intensity control value for light elements capable of projecting light onto the contours of the scene elements represented by the contour image, and a second light intensity control value for light elements capable of projecting light outside the contours of the scene elements represented by the contour image, at least a first light intensity control value being different from the second light intensity control value; - transmission of the determined light intensity map to at least one lighting module, for projection of a pixelated lighting beam into the scene facing the vehicle.

[0009] Thus, the invention enables the projection of a contrast-enhancing beam (corresponding to the difference between the first and second light intensity control values) of the contours of scene elements facing the vehicle. This improves the ability of semantic perception functions, which are implemented using images of the scene, to obtain accurate semantic perception results. Indeed, in night scenes, semantic perception functions are less able to exploit texture information from the scene and must primarily process contour information. The invention thus enhances the ability of semantic perception functions to leverage scene contour information to obtain at least one semantic perception result.

[0010] Moreover, a high degree of responsiveness is allowed for the control of lighting, since the light intensity map is determined from a contour image of a first representative image of the scene, which does not involve any complex image processing.

[0011] According to some embodiments, the process may further comprise the following steps: - obtaining a second image representative of the scene facing the vehicle, following the projection of the pixelated lighting beam; - application of at least one semantic perception function to the second image obtained, to obtain at least one semantic perception result. Thus, the vehicle can implement at least one efficient semantic perception function, enabling the achievement of an accurate semantic perception result.

[0012] In addition, at least one semantic perception result is transmitted to a vehicle driving assistance module, capable of implementing at least one driving assistance function based on at least one semantic perception result.

[0013] Improving at least one semantic perception result improves the accuracy of the driver assistance function, thereby improving comfort and / or safety in the vehicle.

[0014] According to embodiments, at least a first light intensity control value may be lower than the second light intensity control value.

[0015] Thus, the outlines of the elements of the scene appear darker, which facilitates the implementation of at least one semantic perception function.

[0016] In addition, at least one first light intensity control value may include a zero light intensity value.

[0017] Thus, the contrast of the contours is maximized, the contours appear in black, which improves the performance of semantic perception functions in the vehicle in night driving situations.

[0018] According to some embodiments, the edge detection function may include the application of a Canny filter.

[0019] Such a contour detection function allows for rapid and precise acquisition of the contour image.

[0020] According to embodiments, the method may further include, following the obtaining of the contour image, obtaining a negative of the contour image, and the light intensity map may be determined as a function of the negative obtained.

[0021] In the case where the contour image initially has white contours, it is thus made possible to obtain a negative with black contours, and to directly deduce the light intensity map.

[0022] According to embodiments, the method may further include, following the obtaining of the contour image, an increase in the thickness of the contours in the contour image.

[0023] Thus, the visibility of contours is improved in the scene facing the vehicle, thereby improving the performance of semantic perception functions in the vehicle.

[0024] According to embodiments, the method may further include, after obtaining the contour image, cropping the contour image so that the scene represented in the contour image corresponds to a lighting area in which the light elements of the lighting module are able to project light.

[0025] Thus, the light intensity map can be deduced directly from the contour image (or the negative of the contour image), which simplifies its determination and the speed associated with lighting control.

[0026] According to embodiments, the pixelated light source of the lighting module may comprise electroluminescent semiconductor elements of submillimeter dimensions, epitaxially mounted directly on a common substrate.

[0027] Such a lighting module allows the projection of a high-resolution pixelated lighting beam, which improves the determination of a precise light intensity map and therefore a precise delimitation of contours whose contrast is reinforced in the scene.

[0028] A second aspect of the invention relates to a computer program comprising instructions for implementing the method according to the first aspect of the invention, when these instructions are executed by a processor.

[0029] A third aspect of the invention relates to a control device for at least one vehicle lighting module, comprising an interface for obtaining a first representative image of a scene facing the vehicle and a processor configured to: - Apply an edge detection function to the first image, to obtain an edge image including the edges of elements from the scene represented in the first image: - determine a light intensity map, from the contour image, the light intensity map indicating light intensity control values ​​to control light elements of a pixelated light source from at least one vehicle lighting module, the light intensity map indicating at least a first light intensity control value for light elements capable of projecting light onto the contours of the scene elements represented by the contour image, and a second light intensity control value for light elements capable of projecting light outside the contours of the scene elements represented by the contour image, the at least first light intensity control value being different from the second light intensity control value; - transmit the determined light intensity map to at least one lighting module, for projection of a pixelated lighting beam into the scene facing the vehicle.

[0030] A fourth aspect of the invention relates to a vehicle comprising: - a camera arranged to obtain a first image representative of a scene facing the vehicle; - at least one lighting module comprising a pixelated light source comprising a plurality of individually controllable light elements; - a control device according to the third aspect of the invention; - at least one image processing module, capable of applying at least one semantic perception function to the second image, to obtain at least one semantic perception result.

[0031] According to embodiments, the vehicle may further include a driver assistance module configured to implement at least one driver assistance function based on at least one semantic perception result.

[0032] Other features and advantages of the invention will become apparent from an examination of the detailed description below, and the accompanying drawings in which:

[0033] [Fig.1] illustrates a vehicle according to embodiments of the invention;

[0034] [Fig.2] illustrates a vehicle lighting module according to embodiments of the invention;

[0035] [Fig.3a] illustrates a first representative image of a scene facing a vehicle, according to embodiments of the invention;

[0036] [Fig.3b] illustrates a representative contour image of a scene facing a vehicle, according to embodiments of the invention;

[0037] [Fig.3c] illustrates a second image after application of a perception function semantics, according to embodiments of the invention;

[0038] [Fig.4] illustrates the steps of a method for controlling at least one lighting module in a vehicle, according to embodiments of the invention;

[0039] [Fig. 5] illustrates a structure of a control device of at least one module vehicle lighting, according to embodiments of the invention.

[0040] The description focuses on the characteristics that distinguish the module and the method from those known in the state of the art.

[0041] Fig. 1 illustrates a vehicle 100 according to embodiments of the invention.

[0042] The vehicle 100 according to the invention comprises at least one lighting module 120 including a pixelated light source, for example matrix, capable of forming a pixelated beam of light projected outwards from the vehicle so as to illuminate a scene facing the vehicle, particularly at night or when ambient light is low.

[0043] The lighting module 120 is integrated into a lighting device 110 such as a front headlight of the vehicle.

[0044] In some embodiments, the lighting device 110 may include at least one other lighting module, in addition to the lighting module 120, to perform a different or complementary lighting function to the lighting function implemented by the lighting module 120. However, the lighting module 120 may alternatively be the only module of the lighting device 110 performing a lighting function. In some embodiments, the lighting device 110 may further include a signaling module capable of implementing at least one vehicle signaling function.

[0045] In the example shown, the lighting device 110 in which the light module 120 is integrated can be the right front headlight or the left front headlight. Alternatively, a first light module 120 is arranged in the right front headlight and a second light module 120 is arranged in the left front headlight of the vehicle 100.

[0046] The vehicle 100 according to the invention further comprises a control device 130 capable of sending lighting commands to the light module 120 according to the images acquired by a camera 140.

[0047] According to the invention, the lighting control is a light intensity map indicating a light intensity for each light element of the matrix source of the lighting module 120, which is described below with reference to [Fig.2],

[0048] According to the invention, to generate the light intensity map, the control device 130 is capable of: - apply a contour detection function 131 to a first image acquired by the camera 140, to obtain a contour image comprising only contour information from the first image, the first image being representative of the scene facing the vehicle; - determine the light intensity map from the contour image.

[0049] No restrictions are attached to the edge detection function, which includes any function, or software module, capable of extracting the edges from a first image to obtain the edge image containing only edge information and no texture information, unlike the first image (the edge detection function can thus be considered a filter). The edge image can be a black and white or grayscale image and can include first pixels having a first light intensity value and forming the edges of the first image, and second pixels with a second light intensity value comprising all the pixels of the first image outside the edges.

[0050] For example, the first light intensity value may be zero and the second light intensity value may have a non-zero value, so that the contours appear dark in the contour image.

[0051] For example, the edge detection function may be a well-known function such as a Canny filter, or a Canny detector, capable of producing an edge image with the edges in white and the rest of the filtered image in black.

[0052] Advantageously, the light intensity map is determined from the contour image so as to enhance the contours of objects in the scene facing the vehicle, as explained below.

[0053] The control of the lighting module 120 is thus said to be adaptive, in that it is determined from the external scene facing the vehicle, represented by first images acquired by the camera 140.

[0054] Thus, the vehicle 100 according to the invention further comprises the camera 140, or a plurality of cameras 140, capable of acquiring an image, or a series of images, of the scene in which the lighting module 120 is capable of projecting the pixelated light beam. Thus, when the lighting module 120 is mounted in the front headlight of the vehicle 100, the lighting module 120 projects the pixelated light beam onto a scene facing the vehicle: in this case, the camera 140 is arranged in the vehicle 100 so as to acquire an image, or a series of images, representative of the scene facing the vehicle. For example, as illustrated in [Fig. 1], the camera 140 can be arranged in a central position at the top of the windshield of the vehicle 100.

[0055] More generally, the camera 140 is capable of acquiring the images described below (including the first and second images).

[0056] The vehicle 100 includes at least one image processing module 150, capable of applying at least one semantic perception function to at least one second image captured by the camera 140, for transmission of at least one semantic perception result to a driver assistance module 160, referred to as the AD AS 160 module in the following. The AD AS 160 module is capable of implementing at least one vehicle driver assistance function.

[0057] According to the invention, the AD AS 160 module uses at least one semantic perception result determined by at least one image processing module 150 for the implementation of at least one driver assistance function. No restrictions are attached to the at least one driver assistance function, which may be a pedestrian avoidance function, an emergency braking assistance function, an adaptive cruise control function, etc.

[0058] According to the invention, the image processing module 150 is a semantic perception module capable of implementing at least one semantic perception function. Thus, the image processing module 150 can implement one or more of the following semantic perception functions: - an object detection function capable of detecting one or more objects in an image or series of images, assigning a category to each detected object from a predefined set of categories, determining the position of each detected object in the image, and determining a detection score for each detected object; or - a semantic segmentation function capable of segmenting each captured image into several pixel regions, each region being labeled with a class from a set of predefined classes (car, pedestrian, sign, road, etc.). The semantic segmentation module can determine a segmentation score associated with the image segmentation, or with each segment determined within the image; or - an instance segmentation function capable of segmenting each image captured by the camera into several pixel regions for each instance of the scene, as with semantic segmentation, but without labeling each region with a class among a set of predefined classes. An instance segmentation score can be associated with the instance segmentation of the image, or with each segment determined within the image, the score being representative of the degree of certainty associated with the instance segmentation; or - a panoptic segmentation function capable of implementing both semantic and instance segmentation, thus able to segment each image captured by the camera into several pixel regions for the different objects in the scene, each object being labeled with a class from a set of predefined classes. A panoptic segmentation score can be associated with the panoptic segmentation of the image, or with each segment determined within the image, the score being representative of the degree of certainty associated with the panoptic segmentation; or - another function capable of performing a different semantic perception task.

[0059] The image processing module 150 can implement an object detection function by executing an object detection algorithm, such as an existing object detection algorithm not covered by the present invention. For example, a well-known YOLO (You Only Look Once) type algorithm can be implemented by the object detection module 150. Alternatively, the object detection algorithm implemented by the object detection module 150 is a machine learning-derived object detection model, not described herein.

[0060] Alternatively or in addition, the image processing module 150 may implement a semantic segmentation function by executing a semantic segmentation algorithm, such as a known semantic segmentation algorithm not covered by the present invention. Alternatively, the semantic segmentation algorithm implemented by the semantic segmentation module 150 is a machine learning-derived segmentation model, not described herein.

[0061] The control device 130 can be a centralized vehicle control device, called “Body Controller” in English, capable of implementing a plurality of vehicle functions, including the function of controlling the lighting module 120. Alternatively, the control device 130 is dedicated to determining the light intensity map to control the lighting module 120.

[0062] Figure 2 illustrates the structure of a lighting module 120 with a light source pixelated light 200, according to embodiments of the invention.

[0063] The lighting module 120 comprises: - a control unit 201, also called a pilot unit or “driver” in English, of the pixelated light source 200; - the pixelated light source 200; - a projection optic 202 for the light from the pixelated light source 200 to project a pixelated beam of light outwards from the vehicle, into the illuminated area facing the vehicle 100, corresponding at least partially to the scene in the field of view of the camera 140 described above. No restrictions are attached to the projection optic, which may comprise any set of optical elements.

[0064] The pixelated light source 200 comprises a plurality of light elements 210 that can be individually controlled by the control unit 201. The control unit 201 can thus individually control the light elements 210, and can thus control a pattern projected in the pixelated light beam by activating some light elements 210 and deactivating other light elements 210, and by controlling the light intensity of each light element 210 according to the received light intensity map.

[0065] Note that the light intensity map can directly include the light intensity control values ​​for all the light elements 210 of the pixelated light source 200, in which case the light intensity map is itself an image having the same resolution (the same number of pixels) as the number of light elements 210 of the pixelated light source 200. In this case, each pixel, called a control pixel, of the light intensity map indicates the light intensity control value for a corresponding light element 210 (having the same coordinates) of the pixelated light source 200. More generally, the light intensity map includes information enabling the control unit 201 of the lighting module 120 to determine the light intensities of the light elements 210 of the pixelated light source 200.Thus, the light intensity map has a format that is predefined based on the resolution of the pixelated light source 200.

[0066] There is no restriction on the number of light elements 210 in the pixelated light source 200. Preferably, the pixelated light source 200 is a high-definition light source, that is, one capable of projecting a beam of light comprising more than one hundred pixels, preferably more than 1000 pixels. The pixelated light source 200 can project a beam of light comprising more than 10,000 pixels according to embodiments of the invention.

[0067] The luminous elements 210 can be electroluminescent.

[0068] For example, each light element 210 can be a light-emitting diode of the LED type, for “Light Emitting Diode”, the plurality of LEDs forming a matrix network of LEDs.

[0069] According to some embodiments, the pixelated light source 200 can be monolithic. A so-called "monolithic" source can have a particularly high density of light elements 210, making it especially attractive for a wide range of applications. A monolithic source involves a plurality of submillimeter-sized electroluminescent semiconductor elements 210, epitaxially bonded directly onto a common substrate, the substrate generally being made of silicon.Unlike conventional LED arrays, where each individual light-emitting element is a uniquely produced electronic component mounted on a substrate such as a printed circuit board (PCB), a monolithic LED source is considered a single electronic component. During its production, multiple areas of light-emitting semiconductor junctions are generated on a common substrate, forming an array. This manufacturing technique allows for the creation of closely spaced areas of light-emitting elements, each acting as a single light-emitting element. The gaps between these elements can be submillimeter in size. One advantage of this manufacturing technique is the high pixel density that can be achieved on a single substrate.

[0070] The individual light elements can be individually controlled by the control unit 201, which is capable of receiving control signals from the control device 201 in the form of light intensity maps. The light intensity map is then interpreted by the control unit 201 to control the light intensity of each light element 210. For example, the light intensity map may include a light intensity control value for each light element 210 (when the light intensity map and the pixelated light source have the same resolution) or to control the light intensity of a set of light elements 210 (when the pixelated light source 200 has a higher resolution than the light intensity map).

[0071] Individual control of each light element 210 may include control of the power supply provided to the individual light element 210 by pulse-width modulation, or PWM. The control unit 201 may, for example, be an ASIC (Application-Specific Integrated Circuit), which has the advantage of being very compact and can be integrated into the monolithic light source.

[0072] Alternatively, the pixelated light source 200 comprises a light source and a micromirror array, each micromirror thus forming a light element 210 by reflection of the light rays from the light source. The array of micro-mirrors 210 is also called DMD for Digital Micromirror Devices, and the micro-mirrors 210 can be individually activated to reflect the light from the light source towards the projection optics 202, thus forming a pixelated beam of light, the pattern of which can be controlled by controlling the micro-mirrors 210.

[0073] [Fig.3a] illustrates an example of a first image acquired by the camera 140, representative of a scene facing the vehicle, according to embodiments of the invention.

[0074] The first image is used by the control device 130 to adaptively determine the light intensity map to be transmitted to the lighting module 120.

[0075] The first image 300 represents a scene facing the vehicle, which may in particular be a scene in a night driving situation.

[0076] In the example shown, the scene includes a first element 301, such as a first car travelling in the same direction as vehicle 100, a second element 302, such as a second car travelling in the same direction as vehicle 100, but closer to vehicle 100, and a third element 303, such as a pedestrian located on the left side of a road 304 also visible in the scene.

[0077] Such a scene is given by way of illustration and no restrictions are attached to the composition of the scene, in particular to the number of elements that compose the scene, nor to the respective categories or classes associated with these elements. The term “element” means any object or instance composing the scene, each delimited by a set of contours.

[0078] The outlines of the elements are sharp in [Fig. 3a], for illustrative purposes. However, it will be understood that the outline of the scene elements may be less clearly defined, particularly in night driving situations. Furthermore, in practice, the first image 300 comprises grayscale or color pixels, and not sharp outlines formed by black lines as shown, for simplification, in [Fig. 3a]. The first image is preferably a color image, or RGB image, for “Red Green Blue,” which can be acquired by most of the cameras 104 with which current vehicles are equipped. Thus, the first image is defined by the color and intensity of each of its pixels, and comprises a set of textures and outlines forming the scene in front of the vehicle, as captured by the camera 104. The textures formed by the pixels of the first image 300 are not shown in [Fig. 3a] for ease of understanding.

[0079] It should be noted that, during daytime driving situations, the semantic perception function implemented by at least one image processing module 150 primarily uses texture information to detect and qualify scene elements and obtain a semantic perception result. However, during In night scenes, texture information is less rich, as the contrasts in a nighttime image are lower than in a daytime image. Therefore, the invention proposes to enhance the legibility of the outlines of scene elements facing the vehicle in nighttime driving situations, in order to improve the performance of at least one semantic perception function.

[0080] To this end, the invention allows the lighting to be controlled by the lighting module 120, by enhancing the contrast between the pixels forming the contours of the scene elements and the pixels outside the contours, in order to improve at least one semantic perception function, in particular object detection or object segmentation, applied to a second image acquired after lighting control. Such a performance improvement may, in particular, be an improvement in the detection scores associated with object detection in the image 300, or an improvement in the segmentation score or scores, and / or a reduction in errors made during the detection, categorization, and / or localization of objects or instances in the scene.

[0081] Fig. 3b presents a contour image 310 obtained by applying the contour detection function 131 to the first image 300, by the control device 103.

[0082] The contour image 310 identifies the contours 305 of the scene elements and does not include any texture information. Preferably, the contour image 310 comprises only two sets of pixels: - a first set comprising the first pixels forming the contours 305 of the first image (in black on the [Fig.3b]); - a second set including the second set of pixels including all the pixels of the first image outside the outlines (in white on [Fig.3b]).

[0083] The contour image 310 can be a black and white image with: - the first pixels in black and the second pixels in white as in [Fig.3b]; or - the second pixels in black and the first pixels in white (i.e. the negative of the previous option), as obtained at the output of a Canny detector.

[0084] Alternatively, the contour image is a greyscale image, with the first set associated with at least one first light intensity value (for example a first greyscale level), and the second set associated with a second light intensity value (for example a second greyscale level) distinct from the first greyscale level.

[0085] According to the invention, the control device 103 is capable of determining a light intensity map from the contour image 310, in order to enhance the contrast between the contours 305 determined in the scene and the rest of the scene.

[0086] Before determining the light intensity map, the control device 130 can apply one or more treatments to the contour image to: - obtain a negative of the contour image (if the first pixels of the contours are white, or of a lower gray level than the second set of pixels in the rest of the scene), such that the first pixels have a lower light intensity value than the second set of pixels; and / or - increase the thickness of the outlines by transferring pixels from the second set to the first set; and / or - crop the contour image 310 so that it corresponds to the scene into which the light elements 210 of the lighting module 120 are projected.

[0087] According to some embodiments, the control device 103 can adapt the contour image 310 (optionally processed) to the predefined format, to obtain the light intensity map: - if the contour image 310 and the light intensity map have the same resolution, the light intensity value of each pixel of the contour image 310 is assigned to a corresponding control pixel of the light intensity map as a light intensity control value; - if the contour image 310 has a higher resolution than the light intensity map, a light intensity control value calculated from the light intensity values ​​of a set of pixels in the contour image 310 is assigned to a corresponding control pixel in the light intensity map, for each control pixel in the light intensity map; - if the contour image 310 has a resolution lower than the light intensity map, the light intensity value of each pixel of the contour image 310 is assigned to a set of corresponding control pixels of the light intensity map as a light intensity control value.

[0088] Thus, according to these embodiments, the control pixels are also divided into first control pixels associated with at least one first control value of light intensity and second control pixels associated with a second control value of light intensity.

[0089] Alternatively, the control pixels corresponding to the first pixels of the contour image 310 may exhibit a light intensity gradient (with at least one intermediate light intensity control value between the first light intensity control value and the second light intensity control value) between central control pixels (which have the first light intensity control value) and peripheral control pixels, which together form contours.

[0090] Thus, when the lighting module 120 projects a beam of light corresponding to the determined light intensity map, the contours of the elements of the scene are enhanced in a second image acquired by the camera 140 after projection of the lighting beam.

[0091] According to some embodiments, at least one light intensity control value is lower than the second light intensity control value, so that the edges are darker in the scene. For example, at least one light intensity control value may include a zero light intensity control value, so that no light rays are emitted towards the edges of the scene elements.

[0092] Alternatively, at least one light intensity control value is greater than the second light intensity control value, so that the contours are brighter in the scene. For example, the second light intensity control value could be zero.

[0093] Fig. 3c illustrates a second image 320 acquired by the camera 140, after projection into the scene facing the vehicle, of a beam of light corresponding to the light intensity map determined according to embodiments of the invention.

[0094] The scene is unchanged from the scene shown in [Fig.3a] in that the determination of the light intensity map, the projection of the corresponding lighting beam and the acquisition of the second image can be carried out in a very short time, in particular less than one tenth of a second.

[0095] In the second image, the contrast of the contours of elements 301 to 304 of the scene is enhanced. Figure 3c shows, in particular, the second image 310 after application of a semantic perception function by the image processing module 150, for example after application of the object detection function.

[0096] The second image 320 being captured by the camera 140, it is preferably a color image, also including texture information, which is not shown in [Fig. 3c], for simplification. In the second image 320, the contours are thus illuminated with at least a first level of light intensity (or several levels of light intensity in the example described above with a gradient of light intensity control values ​​between the central and peripheral pixels of the light intensity map), which can preferably be zero in order to enhance the contrast, and the rest of the scene, including the texture information, is illuminated according to a second level of light intensity, which is non-zero and higher than the first level of light intensity.The first level of light intensity results from the projection of light by the light elements 210 of the matrix source 200 to which the first light intensity control value is applied and the second level of light intensity results. of the projection of light by the luminous elements 210 of the matrix source 200 to which the second light intensity control value is applied.

[0097] In the example shown in [Fig.3c], the first element 301 is identified as a first object in a first part 311 of the second image 320, the second element 302 is identified as a second object 302 in a second part 312 of the second image 320 and the third element 301 is identified as a third object in a third part 313 of the second image 320.

[0098] As described previously, the image processing module 150, when executing the object detection function, can further assign a category to each identified object, for example, the category “car” to the first and second objects 301 and 302 and the category “pedestrian” to the third object 303. Parts 311 to 313 allow the position of the objects to be determined in the second image 320.

[0099] The strengthening of the contours of the elements of the scene by the projection of the lighting beam according to the determined light intensity map, allows the improvement of the performance of at least one semantic perception function.

[0100] No restriction is attached to the range of wavelengths in which the pixelated light beam is projected and in which the camera 150 is capable of obtaining images. For example, the range of wavelengths can be a range of visible wavelengths.

[0101] Alternatively, the wavelength range may not include any visible wavelengths. For example, it may be an infrared range, such as a near-infrared (NIR) range, a short-wave infrared (SWIR) range, a medium-wave infrared (MWIR) range, or a long-wave infrared (LWIR) range. Note that the MWIR and LWIR ranges are also called the thermal range.

[0102] Fig. 4 illustrates the steps of a method for controlling a lighting module in a vehicle, according to embodiments of the invention.

[0103] The method according to the invention can be implemented by the components of the vehicle 100 described above with reference to [Fig.1].

[0104] The method is implemented when the lighting module 120 is activated, therefore in driving situations where the brightness is low, in particular below a predefined brightness threshold, for example at night.

[0105] At a step 400, the camera 140 captures a first representative image of a scene facing the vehicle 100. Alternatively, the camera 140 captures several first images and obtains a first stabilized image. The first image can be the first image 300 described previously with reference to [Fig. 3a].

[0106] Optionally, the first image thus captured is transmitted to the image processing module 150 at a step 407, which applies at least one semantic perception function, and thus obtains at least one first semantic perception result at the end of step 407.

[0107] The at least first semantic perception result thus obtained is transmitted by the image processing module 150 to the AD AS module 160 at a step 408, for implementation of at least one AD AS function from the at least first semantic perception result.

[0108] It should be noted that during a first iteration, the lighting of the lighting module is not yet controlled according to the invention, and thus, the image processing module 150 processes the image with a first level of performance, which can be represented by a first semantic perception score associated with the first semantic perception result. The first level of performance is not determined during the current phase but is mentioned in this description to explain the technical effect associated with implementing the lighting control according to the invention, as described below.

[0109] At a step 401 following step 400, the first image captured by the camera 140 is received by the control device 130.

[0110] At a step 402, the device 130 applies the edge detection function to the first image received at step 401, and thus obtains an edge image, which may be the edge image 310 described previously with reference to [Fig.3b].

[0111] At a step 403, the control device 130 can apply at least one processing to the contour image 310, to obtain a contour image processed as described above, the at least one processing comprising one or more of the following: - obtaining a negative of the contour image 310; and / or - increasing the thickness of the contours in the contour image 310 by transferring pixels from the second set to the first set in the contour image; and / or - cropping the contour image 310 to match the scene into which the light elements 210 of the lighting module 120 project.

[0112] At a step 404, the control device 130 determines a light intensity map, from the contour image 310, optionally processed at step 403, as described previously.

[0113] At a step 405, the control device 130 transmits the light intensity map determined in step 404 to at least one lighting module 120.

[0114] Upon receiving the light intensity map, the previously described control unit 201 drives the pixelated light source 200 according to the light intensity map, to project a beam of light in front of the vehicle at a stage 406, depending on the light intensity control values ​​of the control pixels of the light intensity map.

[0115] Following the projection of the lighting beam according to the light intensity map determined by the control device 130, step 400 is repeated to capture a second image representative of the scene facing the vehicle 100, the second image being able to correspond to the second image 320 described previously with reference to [Fig.3c] (without the rectangles of parts 311 to 313, which are determined subsequently).

[0116] The second image representing the scene facing the vehicle 100 is transmitted from the camera 140 to the processing module 150, and the processing module 150 applies at least one semantic perception function to the second image, at a second iteration of step 407, to obtain at least one second semantic perception result

[0117] As shown in [Fig.3c], a second semantic perception result obtained from the second image 310, can include the respective positions of parts 311 to 313 and the identification of a category for each of parts 311 to 313, when the second semantic perception result is derived from an object detection function.

[0118] At least one second semantic perception result is transmitted by the processing module 150 to the AD AS module at a step 408 for implementation of at least one AD AS function based on at least one second result.

[0119] The second iteration following the capture of the second image follows step 406, which involves projecting a beam of light corresponding to the light intensity map determined by the control device 130 according to the invention. During this second iteration, the image processing module 150 processes the image with a second level of performance that is superior to the first level of performance, since the improved contrast of the scene's edges facilitates the implementation of one or more semantic perception functions. Like the first level of performance, the second level of performance is not determined during the current phase but is mentioned in this description to explain the improvement in image processing performance through the control of the lighting module 120, according to the invention.

[0120] Note that steps 407 and 408 may not be implemented during the first iteration, in which case only at least one second semantic perception result is transmitted to the AD AS 160 module during step 408. Thus, only semantic perception results obtained with a high level of performance are transmitted to the AD AS 160 module.

[0121] According to some embodiments, steps 401 to 406 are also applied to the second image captured after the second iteration of step 400 (following the acquisition of the second image). Thus, the pixelated lighting beam is controlled again, allowing the evolution of the scene facing the vehicle to be tracked. In these embodiments, a third image is captured at a third iteration of step 400, and steps 407 and 408 are applied to it on the one hand (to update the semantic perception) and steps 401 to 406 are applied to it on the other hand (to continue adapting the lighting to the evolution of the scene facing the vehicle). The steps of the current phase are thus repeated continuously, for successive iterations, during the vehicle's driving situation.

[0122] According to alternative embodiments, the lighting beam is adapted only once based on the first captured image, and the at least one semantic perception function is applied in step 407 only to the second image. Such embodiments can be provided, in particular, when the method is triggered by an event. No restrictions are attached to the event, which can be an event associated with a predetermined condition. The event can, for example, be that data captured by a sensor on the vehicle, separate from the camera 140, satisfies a predefined activation condition. For example, upon detection of an object on the road by another sensor on the vehicle, such as a lidar, the current phase of the method according to the invention can be triggered in order to verify the detection of the same object by the processing of step 505 applied to the second image.This allows for redundancy in the data captured by another sensor on the vehicle.

[0123] Improving the performance of at least one semantic perception function, for example the semantic segmentation or object detection function, thus allows for better reliability of the semantic perception results transmitted to the AD AS 160 module, and thus improves the accuracy and reliability of at least one ADAS function implemented by the AD AS module as a function of the images processed by the image processing module 150.

[0124] Figure 5 illustrates the structure of a control device 130 according to embodiments of the invention.

[0125] The control device 130 includes a processor 501 configured to communicate unidirectionally or bidirectionally, via one or more buses or via a direct wired connection, with a memory 502 such as a Random Access Memory (RAM), a Read Only Memory (ROM), or any other type of memory (Flash, EEPROM, etc.). Alternatively, the memory 502 comprises several memories of the aforementioned types.

[0126] The memory 502 is capable of storing, permanently or temporarily, at least some of the data used and / or resulting from the implementation of the process steps according to the invention illustrated with reference to [Fig.5]. In particular, the memory 502 can store a software module corresponding to the contour detection function 131, and can temporarily store the image received at each step 401 of the process according to the invention.

[0127] The processor 501 is capable of executing instructions, stored in memory 502, for the implementation of steps 402, 403, and 404 described above. Alternatively, the processor 501 can be replaced by a microcontroller designed and configured to perform steps 402, 403, and 404 described above.

[0128] The control device 130 includes a first interface 503 capable of receiving, in the step 401 described above, a first image captured by the camera 140 (and other images during subsequent iterations of the step 401).

[0129] The control device 130 includes a second interface 504 capable of transmitting the light intensity map determined, following the implementation of steps 402 to 404, to the lighting module 120 during step 405 described previously.

[0130] The present invention is not limited to the embodiments described above by way of example; it extends to other variants.

Claims

Demands

1. Method for controlling at least one lighting module (120) in a vehicle (100), the method comprising the following steps: - obtaining (400; 401) a first image (300) representative of a scene facing the vehicle; - applying (402) an edge detection function to the first image, to obtain a contour image (310) comprising contours of elements of the scene represented in the first image;- determination (404) of a light intensity map, from the contour image, the light intensity map indicating light intensity control values ​​to control light elements (210) of a pixelated light source (200) of at least one vehicle lighting module, the light intensity map indicating at least a first light intensity control value for light elements capable of projecting light onto the contours of the scene elements represented by the contour image, and a second light intensity control value for light elements capable of projecting light outside the contours of the scene elements represented by the contour image, the at least first light intensity control value being different from the second light intensity control value;- transmission (405) of the determined light intensity map to at least one lighting module, for projection of a pixelated lighting beam into the scene facing the vehicle.;

2. A method according to claim 1, further comprising the following steps: - obtaining (400) a second image representing the scene facing the vehicle, following the projection of the pixelated lighting beam; and - applying (407) at least one semantic perception function to the second image obtained, to obtain at least one semantic perception result.

3. A method according to claim 2, wherein at least one semantic perception result is transmitted (506) to a driver assistance module (160) of the vehicle, capable of implementing at least one driving assistance function based on at least one semantic perception result.

4. A method according to any one of claims 1 to 3, wherein at least one first light intensity control value is less than the second light intensity control value.

5. A method according to claim 4, wherein at least a first light intensity control value includes a zero light intensity control value.

6. A method according to any one of the preceding claims, wherein the contour detection function includes the application of a Canny filter.

7. A method according to any one of the preceding claims, further comprising, following the obtaining (402) of the contour image (310), obtaining (403) a negative of the contour image, in which the light intensity map is determined (404) as a function of the negative obtained.

8. A method according to any one of the preceding claims, further comprising, following the obtaining (402) of the contour image (310), an increase (403) in the thickness of the contours in the contour image.

9. A method according to any one of the preceding claims, further comprising, following the obtaining (402) of the contour image (310), a cropping (403) of the contour image so that the scene represented in the contour image corresponds to a lighting area in which the light elements of the lighting module are able to project light.

10. A method according to any one of the preceding claims, wherein the pixelated light source (200) of the lighting module (120) comprises electroluminescent semiconductor elements (210) of submillimeter dimensions, epitaxially mounted directly on a common substrate.

11. A computer program comprising instructions for carrying out the method according to any one of the preceding claims, when such instructions are executed by a processor.

12. A control device for at least one vehicle lighting module, comprising an interface for obtaining a first representative image of a scene facing the vehicle and a processor configured to: - apply an edge detection function to the first image, to obtain a contour image including the contours of elements of the scene represented in the first image; - determine a light intensity map from the contour image, the light intensity map indicating light intensity control values ​​to control light elements of a pixelated light source from at least one vehicle lighting module, the light intensity map indicating at least a first light intensity control value for light elements capable of projecting light onto the contours of the scene elements represented by the contour image, and a second light intensity control value for light elements capable of projecting light outside the contours of the scene elements represented by the contour image.at least one first light intensity control value being different from the second light intensity control value; - transmit the determined light intensity map to at least one lighting module, for projection of a pixelated lighting beam into the scene facing the vehicle.

13. Vehicle (100) comprising: - a camera (140) arranged to obtain a first representative image of a scene facing the vehicle; - at least one lighting module (120) comprising a pixelated light source (200) comprising a plurality of individually controllable light elements (210); - a control device according to claim 12; - at least one image processing module, capable of applying at least one semantic perception function to the second image, to obtain at least one semantic perception result.

14. Vehicle according to claim 13, further comprising a driver assistance module (160), wherein the driver assistance module is configured to implement at least one driver assistance function based on at least one semantic perception outcome.

Images