Display process

Abstract

The invention relates to a method for displaying images comprising the following steps: - capturing (20) images by an imager, - detecting (22) whether an obstacle is present by a telemetry device and, in the event of the presence of said at least one obstacle, receiving (24) distance data between the detected obstacle and the vehicle, - determining (26) whether a quality of visibility outside the vehicle is lower than a defined threshold, - determining (28) whether predefined conditions representative of a danger for the vehicle are met, said predefined conditions comprising at least the received distance data, - if the predefined conditions representative of a danger are met and if the quality of visibility is lower than the defined threshold, displaying (48) the captured images on an area of ​​the display screen and superimposing on these captured images, in real time, a synthetic image to signal the detected obstacle.Figure to be published with the abstract: Figure 3.

Description

Title of the invention: Display method Technical field of the invention

[0001] The present invention relates to the field of methods for processing images and displaying images captured by cameras arranged on a motor vehicle. State of the prior art

[0002] It is known to display images containing augmented reality elements in order to assist the driver in driving a vehicle. Augmented reality is the superposition of reality and elements calculated by a computer system in real time. However, these augmented reality elements increase the cognitive load of the driver and can have a counterproductive effect. Presentation of the invention

[0003] A first aim of the present invention is to provide the driver with driving assistance by displaying elements in augmented reality but without increasing his cognitive load.

[0004] A second aim of the present invention is to identify situations likely to represent a danger for the vehicle from data measured by measuring devices, and to display augmented reality elements only when a situation likely to represent a danger has been identified.

[0005] A third aim of the present invention is to reduce the energy consumption of the display system.

[0006] A fourth object of the present invention is to increase driving safety. Summary of the invention

[0007] The present invention relates to a method for displaying images on a display screen of a motor vehicle, said method being implemented by a display system comprising at least one first imager capable of capturing images outside the vehicle, at least one obstacle detection telemetry device, a processing unit connected to said at least one first imager and to said at least one telemetry device, and a display screen connected to the processing unit, the display method comprising the following steps: - capturing images by said first imager, - detecting whether at least one obstacle is present by said telemetry device and, if said at least one obstacle is present, receiving distance data between said at least one detected obstacle and the vehicle, - determine whether the quality of visibility outside the vehicle is below a defined threshold, - determining whether predefined conditions representative of a danger for the vehicle are met, the danger coming from said at least one detected obstacle, said predefined conditions comprising at least the received distance data, - if the predefined conditions representative of a danger are met and if the visibility quality is lower than the defined threshold, displaying the captured images on an area of ​​the display screen and superimposing on these captured images and around said at least one detected obstacle, in real time, at least one synthetic image to signal said at least one detected obstacle.

[0008] The display method according to the present invention only displays an augmented reality element when there is a potential danger of collision. This method makes it possible to reduce the cognitive load on the driver and to reduce the energy consumption of the display system. In addition, the driver is more receptive to a danger signaling element, when he has not become accustomed to constantly viewing augmented reality elements. This display method is therefore more user-friendly.

[0009] The features set forth in the following paragraphs may optionally be implemented. They may be implemented independently of one another or in combination with one another:

[0010] - if the predefined conditions representing a danger are not met or if the visibility quality is equal to or greater than the defined threshold, the method comprises a step of displaying on said area of ​​the display screen the images captured by the first imager. - if the predefined conditions representative of a danger are not met and if the visibility quality is lower than said defined threshold, the method comprises a step of displaying on said area of ​​the display screen the images captured by said first imager. - the display system includes a second imager installed inside the vehicle and capable of capturing images representing the driver of the vehicle, and which also includes the following steps: - capture of images by said second imager, - determination of the vehicle driver's gaze position from the images captured by the second imager, - at least partial reduction of the brightness of said area of ​​the display screen, when the determined gaze position is not directed towards said area of ​​the display screen and when the predefined conditions representing a danger are not met. -the method comprises a step of generating said at least one synthetic image comprising at least one signaling element capable of signaling the presence of the obstacle, the signaling element being one of an arrow, an icon, a geometric figure, an overprint of said obstacle and an outline of said obstacle. - The method further comprises a step of identifying said obstacle from the distance data, and in which the generation step comprises the generation of a signaling element representative of the identified obstacle. - The step of determining the fulfillment of the predefined conditions includes the following steps: - calculation of the distance between said obstacle and the vehicle from the distance data, - comparison of the calculated distance to a first defined distance value. - the processing unit is capable of receiving at least one piece of data representative of the temperature outside the vehicle and at least one piece of data representative of the humidity outside the vehicle, and in which the step of determining whether the predefined conditions are met comprises the following steps: - calculation of the distance between said obstacle and the vehicle from the distance data, - reception of at least one temperature data item and / or at least one humidity data item, - comparison of said temperature data with at least one threshold temperature, and / or - comparison of said humidity data with at least one threshold humidity, - if said temperature data is lower than said threshold temperature and / or if said humidity data is lower than said threshold humidity, comparison of said calculated distance with a second defined distance value. -The step of determining the fulfillment of the predefined conditions includes the following steps: - calculation of an amplitude and / or an orientation of the speed of said obstacle, - -comparison of the calculated amplitude to a defined amplitude and / or comparison of the calculated orientation to a defined orientation range. - The step of determining the quality of visibility outside the vehicle is implemented from the images captured by the first imager and / or the distance data received. - The method further comprises a step of comparing the brightness level of the images captured by the first imager with a defined threshold brightness. - The method comprises a step of determining the position of said obstacle from the distance data, and generating synthetic images in which the signaling element is moved simultaneously with the movement of the obstacle. - The at least one telemetry device comprises one of a radar, a lidar, an infrared light device. Brief description of the figures

[0011] [Fig-1] is a perspective view of a vehicle interior showing a portion of the display system implementing the display method according to the invention;

[0012] [Fig.2] is a schematic view of the visualization system implementing the display method according to the invention;

[0013] [Fig.3] is a flowchart representing the steps of the display method according to the invention;

[0014] [Fig.4] is a flowchart representing a first example of a method of determination of the realization of predefined conditions representative of a danger;

[0015] [Fig.5] is a flowchart representing a second example of a method for determining the realization of predefined conditions representative of a danger;

[0016] [Fig.6] is a flowchart representing a third example of a method for determining the realization of predefined conditions representative of a danger;

[0017] [Fig.7] is a schematic view of a first image captured by the first imager;

[0018] [Fig.8] is a schematic view of the first image and of a computer-generated image superimposed on this first image;

[0019] [Fig.9] is a schematic view of a second image captured by the first imager and a computer-generated image superimposed on this second image; and

[0020] [Fig. 10] is a flowchart representing steps of the display method implemented when the display system comprises a second imager. Detailed description of the invention

[0021] With reference to Figures 1 and 2, the display system 2 implementing the display method according to the invention comprises a first imager 4 capable of capturing images outside the vehicle, a telemetry device 6 capable of detecting the presence of an obstacle, a processing unit 8 connected to the first imager and to the telemetry device 6, and a display screen 10 connected to the processing unit 8.

[0022] The first imager 4 may be a camera. The first imager is, for example, arranged in the exterior rearview mirror, for example, on the driver's side. It has a first field of vision extending to the side of the vehicle and towards the rear of the vehicle.

[0023] The telemetry device 6 comprises, for example, a radar. This radar has a second field of vision comprising at least a portion of the first field of vision. It is arranged at the rear of the vehicle. It is capable of detecting the presence of an obstacle 7 located around the vehicle and in particular on the side and at the rear of the vehicle. It is designed to receive distance data representative of the distance between the telemetry device and the obstacle 7. In this patent application, the term "obstacle" means any object, person or animal, mobile or fixed, located in the second field of vision which could possibly hinder the vehicle. An obstacle can for example be a pedestrian, a truck, a car, a bicycle, a motorcycle, a boy scout, a tractor, a flock of sheep, an animal, etc. A bicycle-type obstacle is shown in the image illustrated in [Fig.9]. Alternatively, the telemetry device 6 can also be arranged in the exterior rearview mirror 15.

[0024] Alternatively, the telemetry device is a lidar or an infrared light device, in particular a ToF (“time of flight” sensor in English).

[0025] Preferably, the display system 2 comprises several telemetry devices 6.

[0026] The processing unit 8 comprises a memory 14, in particular a flash memory, and a central processing unit 16 such as a processor or a microprocessor. The central processing unit 16 may be a programmable device which uses software, a specific integrated circuit (application specific integrated circuit (ASIC)) or a part of the electronic control unit (electronic control unit (ECU)). The memory 14 stores an executable image processing code, hereinafter called an “image processing application” and an executable code making it possible to implement the display method described below.

[0027] The memory 14 also stores a defined threshold S, a defined threshold brightness Ls, a first defined distance value Dl, a threshold temperature Ts, a threshold humidity Hs, a second defined distance value D2, a defined amplitude As and a defined orientation range Ps.

[0028] In the embodiment shown, the display screen 10 is fixed to the A-pillar. Alternatively, it can be fixed against the front door of the vehicle, or in the door, or on the central console. According to a first embodiment, the captured images and possibly the computer-generated images are displayed on the entire screen. According to a second embodiment, the display screen can be divided into several zones. Each zone can display different images. One of the zones displays the captured images and possibly the computer-generated images.

[0029] The connection links between the first imager 4 and the processing unit 8, the telemetry device 6 and the processing unit 8, the display screen 10 and the processing unit 8 may be wired or wireless.

[0030] In a particular embodiment, the display system 2 further comprises a second imager 12 connected to the processing unit. The second imager 12 is capable of capturing images representing the driver of the vehicle and transmitting them to the processing unit 8.

[0031] With reference to [Fig. 3], the display method according to the invention begins with a step 20 during which the first imager 4 captures images 16 outside the vehicle, and in particular, on the side and rear of the vehicle. The first imager 4 transmits the captured images 16 to the processing unit 8. An example of a captured image 16 is shown in [Fig. 7].

[0032] Then, during a step 22, the telemetry device 6 detects whether an obstacle 7 is present in the second field of vision.

[0033] When the telemetry device 6 detects an obstacle 7, it receives, during a step 24, data representative of the distance between the telemetry device 6 and the obstacle 7, hereinafter called distance data. It transmits this distance data to the processing unit.

[0034] During a step 26, the processing unit determines whether a quality of visibility outside the vehicle is lower than a defined threshold S. This step can be carried out by different methods implemented in isolation or combined with each other. According to a first method, step 26 of determining the quality of visibility outside the vehicle is implemented from the captured images 16, and the received distance data.

[0035] The image processing application searches for the presence of an obstacle 7 by processing the captured images 16, for example by a contour detection method. And, the processing unit 8 checks in parallel whether the telemetry device 6 has detected an obstacle 7. If the telemetry device 6 has detected an obstacle 7 and the image processing application does not detect any obstacle then the processing unit 8 determines that the quality of visibility outside the vehicle is lower than the defined threshold S. According to a second method, the image processing application determines the brightness level of at least a portion of the images captured by the first imager and compares this brightness level to a defined threshold brightness Ls. If the brightness level is lower than the defined threshold brightness Ls then the processing unit 8 determines that the quality of visibility outside the vehicle is lower than the defined threshold S. This method makes it possible to detect whether the vehicle is in a low visibility environment.

[0036] According to a third method, a brightness sensor of the vehicle determines the brightness outside the vehicle, and the processing unit determines whether the brightness is below a given threshold.

[0037] If the processing unit determines that the quality of visibility outside the vehicle is equal to or greater than the defined threshold S, or if the processing unit determines that the predefined conditions representative of a danger are not met, at least one area of ​​the display screen displays the images captured during a step 27. In particular, at least one area of ​​the display screen displays only the captured images. The processing unit 8 does not superimpose a synthetic image 18 on the captured images 16.

[0038] If the processing unit determines that the quality of visibility outside the vehicle is lower than the defined threshold S, the method continues with a step 28. During this step 28, the processing unit 8 determines whether the detected obstacle 7 meets predefined conditions representative of a danger for the vehicle. This step can be carried out by different methods implemented in isolation or combined with each other.

[0039] According to a first method illustrated in [Fig.4], the determination step 28 comprises a sub-step 30 of calculating the distance between the detected obstacle 7 and the telemetry device 6 from the distance data, and a sub-step 32 of comparing the calculated distance with a first defined distance value D1. The first defined distance value D1 has been previously stored in the memory 14. The first defined distance value D1 is equal to or greater than a predefined safety distance to be respected. If the calculated distance is less than the first defined distance value Dl, the processing unit determines that the detected obstacle 7 fulfills or achieves the predefined conditions representative of a danger for the vehicle. Thus, when step 28 is carried out according to the first method, the predefined conditions include the fact that the distance calculated during sub-step 30 is less than the first defined distance value DL According to a second method illustrated in [Fig.5], the processing unit 8 calculates the distance between the detected obstacle 7 and the telemetry device 6 from the distance data during sub-step 30. Then, the processing unit 8 receives, during a sub-step 34, at least one data item T representative of the temperature outside the vehicle and / or at least one data item H representative of the humidity outside the vehicle. These data are for example measured by a temperature sensor and a humidity probe mounted on the vehicle and connected to the processing unit. Then, during a sub-step 36, the processing unit 8 compares the temperature data T with a threshold temperature Ts, and / or compares the humidity data H with a threshold humidity Hs. If the temperature data T is lower than the threshold temperature Ts, and / or if the humidity data H is lower than the threshold humidity Hs, the processing unit 8 compares, during a sub-step 38, the calculated distance to a second defined distance value D2. This second defined distance value D2 corresponds to the safety distance in the event of rain or fog. If the temperature data T is lower than the threshold temperature Ts, and / or if the humidity data H is lower than the threshold humidity Hs and if the calculated distance is lower than the second defined distance value Dl, the processing unit determines that the detected obstacle 7 meets predefined conditions representative of a danger for the vehicle. If the temperature data T is not lower than the threshold temperature Ts and / or if the humidity data H is not lower than a threshold humidity Hs, the processing unit compares, during a step 32, the calculated distance with the first defined distance value Dl.

[0040] Thus, when step 28 is carried out according to the second method, the predefined conditions comprise a calculated distance less than the second defined distance value D2 and a temperature less than the threshold temperature Ts and / or a humidity less than the threshold humidity Hs.

[0041] According to a third method illustrated in [Fig.6], the processing unit 8 calculates during a sub-step 40, an amplitude and / or an orientation of the speed of the obstacle 7. This amplitude and / or this orientation can be calculated from several distance data measured during a predefined duration.

[0042] During a sub-step 42, the processing unit 8 compares the amplitude of the speed of the obstacle 7 to a defined amplitude As. In the same way, the processing unit 8 checks whether the orientation of the movement of the obstacle 7 is included in a defined orientation range Ps relative to the vehicle. If the calculated speed amplitude is greater than the defined amplitude As and / or if the calculated orientation is included in a defined orientation range Ps relative to the vehicle, the processing unit determines that the detected obstacle 7 fulfills predefined conditions representative of a danger for the vehicle. This sub-step 42 advantageously makes it possible to anticipate a change of lane, an overtaking or an acceleration of an obstacle of the car or motorcycle or bicycle type.

[0043] When step 28 is carried out according to the third method, the predefined conditions include the fact that the amplitude of the speed of the detected obstacle is greater than a defined amplitude As and / or that the orientation of the movement of the detected obstacle is included in a defined orientation range Ps.

[0044] Alternatively or additionally, the predefined conditions may also include the level of brightness outside the vehicle. When this level of brightness is lower than a brightness threshold, it is considered that the driver has a reaction time greater than the reaction time, in daytime driving, and that, consequently, the braking distance will be increased, the defined distance value is therefore also increased.

[0045] If the visibility is lower than the defined threshold S and if the predefined conditions representative of a danger are realized, the method continues with a step 44 during which the processing unit 8 identifies the obstacle using the distance data and possibly the images captured by the first imager 4. The identification of the obstacle consists of identifying whether the obstacle is a pedestrian, a truck, a car, a bicycle, a motorcycle, a scout, a tractor, a flock of sheep, an animal, etc.

[0046] Step 44 is optional. During a step 46, the processing unit 8 determines the position of the obstacle from the distance data and generates synthetic images 18.

[0047] The computer-generated images 18 represent at least one signaling element 19 capable of signaling the presence of the obstacle. The signaling element 19 is, for example, an arrow, an icon or a geometric figure such as a square or a circle.

[0048] The computer-generated images are generated in real time from the distance data received continuously so that on the computer-generated images the signaling element 19 is moved simultaneously with the movement of the obstacle located outside the vehicle.

[0049] When the method comprises a step 44, the signaling element 19 represents the identified obstacle. Thus, the signaling element 19 can be an icon representative of the identified obstacle or an overprint representative of the identified obstacle or an outline of the obstacle. Thus, an icon representing a bicycle has been generated on the synthetic image 18 which is superimposed on a captured image 16, in the example illustrated in [Fig. 9]. An overprint of a vehicle arranged at the location where the vehicle was identified has been generated on the synthetic image 18 which is superimposed on a captured image 16 in the example illustrated in [Fig. 8]. An overprint is an impression of two or more images on the same sensitive surface. This sensitive surface is here an area of ​​the display screen. When the method does not include a step 44 of identifying the obstacle, the processing unit 8 generates synthetic images comprising an icon or an arrow showing the location of the obstacle without representing its identification.

[0050] During a step 48, the processing unit 8 displays the images captured by the first imager 4 on the display screen 10 and superimposes in real time the synthetic images 18 on the captured images to signal the presence and the position of the obstacle 7.

[0051] If after step 28, the detected obstacle 7 does not meet the predefined conditions representative of a danger, the method continues with a step 50. During step 50, at least one area of ​​the display screen displays the captured images. In particular, at least one area of ​​the display screen displays only the captured images. The processing unit 8 does not superimpose a synthetic image 18 on the captured images 16.

[0052] In the embodiment in which the display system 2 comprises a second imager 12 capable of capturing images representing the driver of the vehicle, the display method may comprise, with reference to [Fig. 10], a step 52 of capturing images by said second imager 12. Then, during a step 54, the processing unit 8 determines the position of the gaze of the driver of the vehicle from the images captured by the second imager.

[0053] During a step 56, the brightness of the display screen 10 is reduced at least partially, when the determined position corresponds to a gaze of the driver which is not directed towards the display screen.

Claims

1.

2. Claims Method for displaying images on a display screen (10) of a motor vehicle, said method being implemented by a display system (2) comprising at least one first imager (4) capable of capturing images outside the vehicle, at least one telemetry device (6) for detecting obstacles, a processing unit (8) connected to said at least one first imager and to said at least one telemetry device, and a display screen (10) connected to the processing unit, the display method comprising the following steps: - capturing (20) images (16) by said first imager, - detecting (22) whether at least one obstacle (7) is present by said telemetry device and, in the event of the presence of said at least one obstacle (7), receiving (24) distance data between said at least one obstacle (7) detected and the vehicle, - determine (26) whether a quality of visibility outside the vehicle is lower than a defined threshold (S), - determining (28) whether predefined conditions representative of a danger for the vehicle are met; the danger coming from said at least one detected obstacle (7), said predefined conditions comprising at least the received distance data, - if the predefined conditions representative of a danger are met and if the visibility quality is lower than the defined threshold (S), displaying (48) on at least one area of ​​the display screen (10) the captured images (16) and superimposing (48) on these captured images and around said at least one detected obstacle (7), in real time, at least one synthetic image (18) to signal said at least one detected obstacle (7), and in which if the predefined conditions representative of a danger are not met and if the quality of visibility is lower than said defined threshold (S), the method comprises a step of displaying (50) on said area of ​​the display screen (10) the images captured (16) by said first imager. Display method according to claim 1, in which if the predefined conditions representative of a danger are not met or if the visibility quality is equal to or greater than the defined threshold (S), the method comprises a display step (27) on said area of ​​the display screen (10) of the images captured (16) by the first imager.

3. Display method according to any one of claims 1 and 2, wherein the display system (2) comprises a second imager (12) installed inside the vehicle and capable of capturing images representing the driver of the vehicle, and which further comprises the following steps: - capture (52) of images by said second imager (12), - determination (54) of the position of the gaze of the driver of the vehicle from the images captured by the second imager (12), - at least partial reduction (56) of a brightness of said area of ​​the display screen (10), when the determined position of the gaze is not directed towards said area of ​​the display screen (10) and when the predefined conditions representative of a danger are not met.

4. Display method according to any one of claims 1 to 3, which comprises a step of generating (46) said at least one synthetic image (18) comprising at least one signaling element capable of signaling the presence of the obstacle, the signaling element being one of an arrow, an icon, a geometric figure, an overprint of said obstacle and an outline of said obstacle.

5. Display method according to claim 4, which further comprises a step of identifying (44) said obstacle from the distance data, and in which the generation step (46) comprises the generation of a signaling element (19) representative of the identified obstacle.

6. Display method according to any one of claims 1 to 5, in which the step of determining (28) whether the predefined conditions are met comprises the following steps: - calculating (30) the distance between said obstacle (7) and the vehicle from the distance data, - comparing (32) the calculated distance with a first defined distance value (Dl).

7. Display method according to any one of claims 1 to 6, in which the processing unit (8) is capable of receiving at least one piece of data (T) representative of the temperature outside the vehicle and at least one piece of data (H) representative of the humidity outside the vehicle, and wherein the step of determining (28) whether the predefined conditions are met comprises the following steps: - calculating (30) the distance between said obstacle (7) and the vehicle from the distance data, - receiving (34) at least one temperature data item and / or at least one humidity data item, - comparing (36) said temperature data item with at least one threshold temperature (Ts), and / or - comparing (36) said humidity data item with at least one threshold humidity (Hs), - if said temperature data item is lower than said threshold temperature (Ts) and / or if said humidity data item is lower than said threshold humidity (Hs), comparing (38) said calculated distance with a second defined distance value (D2).

8. Display method according to any one of claims 1 to 7, in which the step of determining (28) the fulfillment of the predefined conditions comprises the following steps: - calculation (40) of an amplitude and / or an orientation of the speed of said obstacle, - - comparison (42) of the calculated amplitude with a defined amplitude (As) and / or comparison of the calculated orientation with a defined orientation range (Ps).

9. A display method according to any one of claims 1 to 8, wherein the step of determining (26) the quality of visibility outside the vehicle is implemented from the images captured by the first imager and / or the received distance data.

10. A display method according to any one of claims 1 to 9, wherein the at least one telemetry device (6) comprises one of a radar, a lidar, an infrared light device.

Images