Visualization system for motor vehicles, associated method and computer program

Abstract

Visualization system for motor vehicle, method and associated computer program. The invention relates to a visualization system and method for motor vehicles. The method comprises steps for controlling the brightness level of at least one display surface, including: - control (40) of basic illumination on said display surface with a basic brightness level, and - control (36, 42) of increasing the brightness level on the display area of ​​obtained images, from the basic brightness level to at least one enhanced brightness level, when one of the following two conditions is met: a) detection (30) of a direction of the driver's gaze oriented towards said display area of ​​obtained images, or b) detection (32, 38) of at least one obstacle corresponding to a danger level greater than a first predetermined danger threshold, the detected direction of the driver's gaze not being oriented towards said display area.Figure for the abbreviation: Figure 2.

Description

Title of the invention: Visualization system for motor vehicles, associated method and computer program

[0001] The present invention relates to a visualization system for a motor vehicle, of the type comprising at least one image display surface obtained by at least one sensor positioned towards the outside of the vehicle, said display surface being illuminated by at least one associated lighting unit.

[0002] The invention also relates to an associated visualization method and an associated computer program.

[0003] The invention lies in the field of visualization systems for vehicles equipped with sensors or wireless communication means enabling the receipt of sensor data from a remote system, in particular image sensors, enabling the acquisition of images of the vehicle's external environment.

[0004] Indeed, in traditional motor vehicles, knowledge of the vehicle's external environment that is not located in the driver's direct field of vision through the windshield is provided by mirrors located at various points, for example, centrally, on the right, and on the left, facing the rear of the vehicle, thus allowing the driver to partially view the external rear and lateral spaces of the vehicle. This is, of course, necessary to ensure the avoidance of obstacles and hazards, to preserve the safety of the vehicle's occupants, other road users, pedestrians, etc.

[0005] In modern vehicles, several sensors, for example cameras, are positioned in various locations, and images captured in real time can be displayed on display surfaces, for example located in the vehicle's passenger compartment. This makes it possible to considerably increase the field of vision, and also to apply various processing techniques, such as image processing to better detect the presence of hazards and obstacles, thereby increasing safety, for example by anticipating the presence of hazards.

[0006] However, the various means put in place for this purpose, and in particular the extended display areas, also significantly increase the consumption of electrical energy, which is not desirable.

[0007] There is a need to limit the consumption of electrical energy in motor vehicles, while ensuring as much as possible protection against possible collision hazards in order to preserve road safety.

[0008] To this end, the invention proposes, according to one aspect, a visualization system for a motor vehicle operated by a driver present in the motor vehicle, comprising a display surface including an image display area for images obtained by at least one image sensor configured to capture images from outside the vehicle, said display surface being illuminated by at least one associated lighting unit, an obstacle detection module for the images obtained, configured to detect the presence of obstacle(s) by image processing and to calculate an associated danger level, a module for detecting the direction of the driver's gaze, and an electronic device comprising a processing unit. The processing unit implements a control module for said at least one lighting unit configured to:

[0009] - control basic lighting on said display surface with a level of basic brightness;

[0010] - to command an increase in the brightness level on said display area of images obtained, from the basic brightness level to at least one enhanced brightness level, when one of the following two conditions is met:

[0011] a) the driver's gaze direction detection module detects a direction oriented towards said image display area obtained, or

[0012] b) the obstacle detection module detects at least one obstacle corresponding to a calculated danger level greater than a first predetermined danger threshold, and the direction of the driver's gaze detected is not oriented towards said display area.

[0013] Advantageously, the proposed system reconciles energy-saving and safety objectives by increasing the lighting to achieve a higher level of brightness only when necessary for the driver. Furthermore, advantageously, increasing the brightness level upon detection of a hazard helps to attract the driver's attention, as the increased brightness acts as a visual stimulus.

[0014] The visualization system according to the invention may also have one or more of the following characteristics, taken independently or according to all technically feasible combinations.

[0015] The increase in the basic brightness level is carried out in an interval of first duration between one millisecond and twenty milliseconds.

[0016] The control module is further configured to, when the danger level falls below said first predetermined danger threshold and the direction of gaze detected by the driver's gaze direction detection module is not directed towards said image display area obtained, gradually decrease the brightness level towards the basic brightness level, during a time interval of second duration.

[0017] The system is configured to implement an alert generation module when the detected danger level is greater than a second predetermined danger threshold, greater than the first predetermined danger threshold and the gaze direction detected by the driver gaze direction detection module is not oriented towards said image display area obtained.

[0018] The alert generation module controls the emission of stimuli configured to alert the driver, according to the calculated danger level.

[0019] In the case where condition b) is met, the control module is configured to control an enhanced brightness level selected according to the calculated danger level, the brightness level increasing with the calculated danger level.

[0020] In the case where condition b) is met, the control module is configured to control lighting at maximum brightness level.

[0021] According to another aspect, the invention relates to a visualization method for a motor vehicle, operated by a driver present in the motor vehicle, for viewing images obtained by at least one image sensor configured to capture images from outside the vehicle, and displayed on an area of ​​a display surface, said display surface being illuminated by at least one associated lighting unit, implemented by an on-board programmable electronic device, configured to continuously receive a direction of gaze from the vehicle driver, to detect the presence of obstacle(s) by image processing, and to calculate an associated danger level. The method comprises steps for controlling the brightness level of said at least one display surface, comprising: - basic lighting control on said display surface with a basic brightness level, and - command to increase the brightness level on said image display area, from the basic brightness level to at least one enhanced brightness level, when one of the following two conditions is met: - a) detection of a direction of the driver's gaze oriented towards said image display area, or

[0022] b) detection of at least one obstacle corresponding to a level of danger greater than a first predetermined danger threshold, the direction of the driver's gaze detected not being oriented towards said display area.

[0023] According to a particular feature, the method includes, in the event of a calculated danger level exceeding the first danger threshold, a display on said display area, superimposed on the image display, of an indication relating to the obstacle or obstacles detected.

[0024] The advantages of the visualization method for motor vehicles are analogous to the advantages of the display system mentioned above.

[0025] According to another aspect, the invention relates to a computer program comprising program code instructions which, when executed by a programmable electronic device, implement a visualization method for a motor vehicle as briefly described above.

[0026] Other features and advantages of the invention will become apparent from the description given below, by way of example and not limitation, with reference to the accompanying figures, among which:

[0027] [Fig-1] [Fig.1] schematically represents a visualization system for motor vehicle according to an embodiment;

[0028] [Fig.2] [Fig.2] is a synoptic diagram of the main steps of a visualization process vehicle adaptation according to a specific embodiment.

[0029] Fig. 1 schematically illustrates a system 2 for displaying images representative of the exterior of a motor vehicle (not shown).

[0030] The visualization system 2 comprises a plurality of sensors 4, for example cameras, configured for image capture at a given frequency, for example positioned at various locations, oriented towards the outside of the vehicle, so as to capture images of the external environment of the vehicle.

[0031] For example, such sensors are on-board sensors, positioned outside the right, left, rear of the vehicle, allowing to obtain views of the outside of the vehicle analogous to the views of conventional rearview mirrors.

[0032] The images thus captured, in the form of digital images, are displayed, at a given display frequency, on display areas 6 of associated display surfaces, configured to display the successive captured images. Each display surface is illuminated by at least one associated lighting unit 8, dedicated to illuminating the display surface.

[0033] In particular, selective lighting by display zones of the display surface is provided.

[0034] According to one variant or in addition, sensors configured to provide image sequences from outside the vehicle are image sensors external to the vehicle, for example cameras positioned at the roadside or on other vehicles, and the image sequences are obtained by means of a wireless communication link between a wireless communication interface of the vehicle in question, and a remote communication system, configured to provide image sequences captured by the sensors external to the vehicle.

[0035] A display area 6 of a display surface, configured to display images obtained by sensors 4, forms an electronic mirror (in English “mirror”).

[0036] The extended display area is, for example, a crystal display surface liquids or LCD (for "Liquid Crystal Display"), or a surface integrated into a trim element of the vehicle, and backlit by controllable lighting units, for example LED matrices.

[0037] A display area is a portion of a display surface, the lighting of which is controllable.

[0038] In some embodiments, the display area occupies the entire display surface.

[0039] The control of the lighting units 8 allows, in particular, the control of a brightness level, varying, for example, between 0 and 100% of a maximum brightness level, with electrical energy consumption being correlated to this brightness level. In other words, the higher the brightness level, the more electrical energy the corresponding lighting units consume. Thus, controlling the lighting at a given brightness level allows for the control of electrical energy consumption.

[0040] Subsequently, the description is dedicated to a display area 6 associated with an image sensor 4, illuminated by controllable lighting units, it being understood that the method described, in its various variants, applies in an analogous way for each display area in the presence of multiple display areas.

[0041] Alternatively, a display surface can be configured to display images from several sources, i.e. from several sensors.

[0042] The system 2 further comprises a module 10 for detecting the direction of the driver's gaze, integrating, for example, an image sensor judiciously positioned in the vehicle's passenger compartment and a processing unit for detecting the driver's gaze direction, for example, by means of a 3D vector in a given spatial reference frame. Such driver gaze direction detection modules are known, for example, in the field of driver monitoring systems (DMS).

[0043] Consider the case of a vehicle piloted by a driver who is present in the passenger compartment and who operates the vehicle controls.

[0044] Of course, other people may be present in the vehicle; these are passengers.

[0045] The system 2 further comprises a programmable electronic device 12, for example an on-board computer, comprising a computing processor 14 and an electronic memory unit 16, connected by a communication bus 15.

[0046] The programmable electronic device 12 may include other elements, in particular communication interfaces with on-board equipment, which are not shown here.

[0047] The processor 14 of the programmable electronic device 12 is configured to

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056] implement a module 20 for obstacle detection in images obtained by a sensor 4, configured to detect the presence of obstacle(s) by applied image processing and to calculate an associated danger level. Here, the term obstacle generally refers to a stationary or moving object detected in the processed images that may be in the vehicle's path. The danger level indicates the risk of a potential collision, along with an associated time value. Thus, the term "obstacle" refers, for example, to fixed obstacles, such as road features (e.g., guardrails, parked vehicles), or moving obstacles, such as other vehicles, bicycles, or pedestrians. The list provided here is, of course, not exhaustive. It is clear that avoiding any collision between a moving vehicle and such obstacles is critical. The level of danger is calculated based on the estimated speeds and trajectories of the detected moving obstacles, as well as the vehicle in question. All methods known to a person skilled in the art of obstacle detection by processing successive images and calculating an associated danger level are applicable here. In one embodiment, the levels of danger are, for example, quantified on a scale from 0 to 4 as follows: - Danger level 0: no obstacle detected, - Danger level 1: "moderate" danger: potential obstacle to a horizon temporal greater than or equal to a temporal threshold Ts, for example Ts=5 seconds; - Danger level 2: "medium" danger: potential obstacle with a time horizon lower than the time threshold Ts, but higher than a time threshold of imminence Tsi, for example Tsi = 2 seconds; - Danger level 3: “high” danger: : potential obstacle with a time horizon lower than the time threshold of imminence Tsi. In addition, the processor 14 of the programmable electronic device 12 is configured to run a module 22 for determining the direction of gaze relative to a given display area 6, the module 22 cooperating with the module 10 for detecting a direction of gaze of the vehicle driver. Indeed, depending on the spatial position and dimensions of a given display area and the direction of the driver's gaze, module 22 is configured to determine whether the driver's gaze is directed towards the display area in question or not. Modules 20 and 22 are implemented continuously, as a background task. The processor 14 is further configured to execute a command module 24 of the brightness level of the display units associated with the display area under consideration.

[0057] Module 24 is configured to control the illumination of the display surface at a basic brightness level, and to increase the brightness level for the display area in question to a boosted brightness level, when one of the following two conditions is met:

[0058] a) the driver's gaze direction detection module detects a direction oriented towards the display area in question, or

[0059] b) the obstacle detection module detects an obstacle corresponding to a danger level higher than a first predetermined danger threshold, and the direction of the driver's gaze is not directed towards the display area considered.

[0060] For example, the first danger threshold is equal to 1, so condition b) is implemented as soon as a danger is detected.

[0061] In one embodiment, the basic brightness level is, for example, between 0% and 30% of the maximum brightness.

[0062] In one embodiment, the enhanced brightness level is for example between 60% and 100% of the maximum brightness, preferably between 60% and 80M% of the maximum brightness.

[0063] In one embodiment, several enhanced brightness levels are defined, in stages, according to the level of danger.

[0064] Module 24 is further configured to, when the danger level falls below the first predetermined danger threshold and the driver's gaze is no longer directed towards the display area in question, gradually decrease the brightness level from the enhanced brightness level to the basic brightness level. Advantageously, this gradual decrease is preferable so as not to disturb the driver.

[0065] In addition, the processor is configured to implement an alert generation module 26 by activating one or more alert devices capable of emitting stimuli configured to alert the driver: for example, audible sound, seat vibration, etc. Module 26 is implemented in conjunction with module 24, which controls the brightness level of the lighting units associated with the display area in question, in the event of detection of a hazard with a hazard level greater than or equal to a second hazard threshold, referred to as the high hazard threshold, for example, equal to 2, if the driver's gaze has not been directed towards the display area. Advantageously, this enhances safety.

[0066] In one embodiment, modules 20, 22, 24, 26 are implemented as software instructions forming a computer program, which, when executed by a computer, implements a visualization method for motor vehicles according to the invention.

[0067] The computer program comprising software instructions is further capable of being recorded on a non-transient, computer-readable medium. The computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. For example, the readable medium is an optical disc, a magneto-optical disc, a ROM, a RAM, any type of non-volatile memory (e.g., EPROM, EEPROM, FLASH, NVRAM), a magnetic card, or an optical card.

[0068] In an alternative not shown, modules 20, 22, 24, 26 are each implemented as programmable logic components, such as FPGAs (Field Programmable Gate Arrays), microprocessors, GPGPUs (General-Purpose Processing Units), or dedicated integrated circuits, such as ASICs (Application-Specific Integrated Circuits).

[0069] Fig. 2 is a synoptic diagram of the main steps of a visualization method for a motor vehicle, implemented in a visualization system 2 embedded in a motor vehicle as described with reference to Fig. 1.

[0070] The method is described herein in its application for the visualization of images of the exterior of the vehicle, obtained by one or more sensors 4 configured to capture images of the exterior of the vehicle, and displayed on an area of ​​an associated display surface, which is hereinafter referred to as the considered display area, referenced S.

[0071] For example, as explained above, the image sensor is, for example, a right or left side sensor, allowing visualization of external lateral areas of the vehicle.

[0072] Alternatively, the images are obtained by means of wireless communication, according to an implemented wireless communication protocol.

[0073] For example, the display surface considered, and in particular the display area, is illuminated with a basic brightness level at initialization, for example when starting the vehicle and switching on the display system.

[0074] The method comprises steps 30 for detecting the direction of the driver's gaze and 32 for detecting one or more potential obstacle(s) and calculating an associated hazard level Nd. These detection steps 30 and 32 are performed in the background at a given frequency.

[0075] Step 30 of detecting the direction of gaze is followed by a test 34 to determine if the direction of the driver's gaze is directed towards the display area S.

[0076] For example, it is determined whether a 3D vector characterizing the direction of the driver's gaze points towards a point in the display area S.

[0077] In case of a positive response to test 34, a first condition (condition a) above) The increase in brightness level to a reinforced brightness level is validated.

[0078] Step 34 is followed by a step 36 of increasing the brightness level. For example, the enhanced brightness level is between 60% and 80% of the maximum brightness, for example 70% of the maximum brightness.

[0079] Preferably, the increase in brightness is almost immediate, and occurs during an interval of first duration of between 1 millisecond and 20 milliseconds, preferably between 1 millisecond and 5 milliseconds.

[0080] However, if the brightness level of the display area considered S was already an enhanced brightness level (for example, when the driver looked towards the display area S at a previous time instant), step 40 consists of commanding a maintenance of the enhanced brightness level.

[0081] Approximately in parallel with the implementation of test 34, step 32 of obstacle detection and hazard level calculation is systematically followed by a comparison 38 of the hazard level Nd to a first predetermined hazard threshold Sd.

[0082] For example, considering the graduated scale of hazard levels between 0 and 3 described above, as soon as the hazard level Nd is strictly greater than 0, the answer to the comparison step is "yes". In other words, the first hazard threshold is equal to 1.

[0083] When the danger level has not reached the first danger threshold Sd (response "no" to comparison 38), and the driver's gaze is not directed towards the surface S, the brightness level is maintained at the basic level (step 40).

[0084] If, however, the brightness level of the display area considered S was an enhanced brightness level (for example, when the driver looked towards the display area S at a previous time instant), step 40 consists of commanding a gradual decrease in lighting for a gradual transition from the enhanced brightness level to the basic brightness level.

[0085] For example, the gradual decrease is carried out during a second time interval, greater than the first duration and between 1 second and 5 seconds, for example on the order of 1.5 seconds.

[0086] If the response to test 34 is negative, i.e. the driver is not looking towards the display area S, but the danger level has reached or exceeded the first danger threshold Sd, then a second condition (condition b)) of increasing the brightness level is validated.

[0087] Steps 34, 38 are then followed by a step 42 of increasing the brightness level by controlling the lighting units of the display area S, to move from the basic brightness level to an enhanced brightness level, with the objective of attracting the driver's attention, and in particular attracting the driver's gaze towards this surface.

[0088] In one embodiment, following the detection of a danger, regardless of the level of danger, the enhanced brightness level is the maximum brightness level.

[0089] In one embodiment, several levels of enhanced brightness are selectable, depending on the calculated danger level, with the brightness level increasing with the calculated danger level.

[0090] Preferably, the increase in brightness level between the base brightness level and an enhanced brightness level is almost instantaneous, the duration of the increase in brightness level being preferably shorter than the duration of the gradual decrease in brightness level. For example, the duration of the increase is between 1 millisecond and 20 milliseconds, preferably between 1 millisecond and 5 milliseconds.

[0091] In addition to displaying the images obtained for display in the display area S, preferably the method also includes a display 44 of a visual indication related to the detected obstacle, for example an obstacle framed, preferably by a colored or flashing frame.

[0092] The process also includes a step 46 of updating the calculated hazard level.

[0093] Indeed, the calculated level of danger can evolve, depending on possible changes in trajectory and speed of the objects involved (obstacle(s), vehicle).

[0094] Of course, the update of the hazard level is carried out following the iteration of step 32 of hazard level calculation previously described.

[0095] In a later step 48 it is determined whether the updated hazard level Nd is greater than or equal to a second hazard threshold or high hazard threshold Sdh, which is greater than the first hazard threshold.

[0096] For example, the high danger threshold is equal to 2.

[0097] If the danger level remains below the second danger threshold Sdh, the enhanced lighting (step 42) is maintained.

[0098] If the danger level is greater than or equal to the second danger threshold Sdh and the driver's gaze is not directed towards the display area S (step 34' analogous to step 34), the method includes an alert emission 50 by activation of one or more stimuli by vehicle devices, for example an audible alert, for example chosen according to the danger level.

[0099] However, if the driver's gaze is turned towards the display area S, which displays in particular the image and the detected obstacle(s), the activation of the warning devices is stopped (step 52).

[0100] The steps of the method are repeated so that the monitoring of the driver's gaze direction and the calculation of the danger level are applied almost continuously, as well as the brightness level controls according to the validation of conditions a) and b).

[0101] Advantageously, energy savings are achieved because the area of ​​the display surface considered is illuminated at a conditionally enhanced lighting level, which implies that over large time periods of driving in situations without particular danger, the lighting is maintained with a basic brightness level, thus with low electrical energy consumption.

[0102] Advantageously, the enhanced brightness level is chosen according to the calculated danger level.

[0103] Advantageously, lighting with an enhanced brightness level makes it possible to attract the driver's attention in the event of detection of a danger level exceeding the predetermined danger threshold, thereby enhancing safety.

Claims

Claims

1. Visualization system for a motor vehicle operated by a driver present in the motor vehicle, comprising - a display surface comprising a display area for images obtained by at least one image sensor configured to capture images from outside the vehicle, said display surface being illuminated by at least one associated lighting unit, - a module (20) for detecting obstacle(s) in said images obtained, configured to detect the presence of obstacle(s) by image processing and to calculate an associated danger level, - a module (10) for detecting a direction of gaze of the driver of the motor vehicle, and an electronic device comprising a calculation processor (14),characterized in that said calculation processor (14) implements a control module (24) of said at least one lighting unit configured to: - control basic lighting on said display surface with a basic brightness level; - control an increase in the brightness level on said display area of ​​obtained images, from the basic brightness level to at least one enhanced brightness level, when one of the following two conditions is validated: a) the module for detecting a direction of gaze of the driver detects a direction oriented towards said display area of ​​obtained images, or b) the module for detecting obstacle(s) detects at least one obstacle corresponding to a calculated danger level greater than a first predetermined danger threshold, and the detected direction of gaze of the driver is not oriented towards said display area.,

2. The system of claim 1, wherein the increase in the base brightness level is performed in an interval of first duration between one millisecond and twenty milliseconds.

3. System according to one of claims 1 or 2, in which the control module (24) is further configured to, when the danger level falls below said first predetermined danger threshold and the direction of gaze detected by the detection module (10) of a direction of gaze of the driver is not oriented towards said zone image display values ​​obtained, gradually decrease the brightness level towards the basic brightness level, during a time interval of second duration.

4. System according to one of claims 1 to 3, configured to implement a module (26) for generating an alert when the detected danger level is greater than a second predetermined danger threshold, greater than the first predetermined danger threshold and the direction of gaze detected by the module (10) for detecting a direction of gaze of the driver is not oriented towards said obtained image display area.

5. System according to claim 4, in which the alert generation module (26) controls an emission of stimuli configured to alert the driver, depending on the calculated danger level.

6. System according to one of claims 1 to 5, in which, in the case where condition b) is validated, the control module (24) is configured to control a reinforced brightness level selected according to the calculated danger level, the brightness level increasing with the calculated danger level.

7. System according to one of claims 1 to 5, in which, in the case where condition b) is validated, the control module is configured to control lighting at maximum brightness level.

8. Visualization method for a motor vehicle operated by a driver present in the motor vehicle, for visualizing images obtained by at least one image sensor configured to capture images from outside the vehicle, and displayed on an area of ​​a display surface, said display surface being illuminated by at least one associated lighting unit, implemented by an on-board programmable electronic device, configured to continuously receive a direction of gaze of the vehicle driver, to detect the presence of obstacle(s) by image processing and to calculate an associated danger level, the method being characterized in that it comprises steps of controlling a brightness level of said at least one display surface, comprising: - basic lighting control on said display surface with a basic brightness level,and - control for increasing the brightness level on said display area of ​​obtained images, the brightness level, basic at least one enhanced brightness level, when one of the following two conditions is validated: - a) detection of a direction of gaze of the driver oriented towards said display area of ​​obtained images, or b) detection of at least one obstacle corresponding to a danger level higher than a first predetermined danger threshold, the direction of gaze of the driver detected not being oriented towards said display area.

9. Method according to claim 8, comprising, in the event of a calculated danger level greater than the first danger threshold, a display on said display area, superimposed on the display of images, of an indication relating to the obstacle or obstacles detected.

10. A computer program comprising executable code instructions which, when executed by a programmable electronic device, implement a motor vehicle visualization method according to claims 8 or 9.

Images