Display method

Abstract

The invention relates to an image display method comprising the steps of: capturing (20) an image via an imager; detecting (22), via the telemetry device, whether at least one obstacle is present and, if the at least one obstacle is present, receiving (24) distance data between the detected obstacle and the vehicle; determining (26) whether a visibility quality outside the vehicle is below a defined threshold; determining (28) whether a predetermined condition indicative of the presence of a hazard in the vehicle is satisfied, the predetermined condition comprising at least the received distance data; if a predetermined condition indicative of a hazard is satisfied and if the visibility quality is below a defined threshold, the captured images are displayed (48) on the area of the display screen and at least one composite image is superimposed on these captured images in real time in order to send a signal regarding the detected obstacle.

Description

Technical Field

[0001] The invention relates to the field of image processing and display methods for images captured by a camera arranged on board a motor vehicle. Background Art

[0002] It is known to display images with augmented reality features to assist the driver in driving a vehicle. Augmented reality is the superposition of reality with features calculated in real time by a computer system. However, these augmented reality features increase the cognitive load on the driver and may have a counterproductive effect. SUMMARY OF THE INVENTION

[0004] A first object of the present invention is to provide driving assistance to the driver without increasing the cognitive load by displaying augmented reality features.

[0005] A second object of the invention is to identify situations that may represent a danger to the vehicle based on the data measured by the measuring device and to display an augmented reality feature only when a situation that may represent a danger has been identified.

[0006] A third object of the present invention is to reduce the energy consumption of a display system.

[0007] A fourth object of the present invention is to improve driving safety. Summary of the invention

[0008] The object of the invention is a method for displaying an image on a display screen of a motor vehicle, said method being implemented by a display system comprising at least one first imager suitable for capturing images of the exterior of the vehicle, at least one telemetry device for detecting obstacles, a processing unit connected to said at least one first imager and said at least one telemetry device, and a display screen connected to the processing unit, said display method comprising the following steps:

[0009] - capturing an image via said first imager,

[0010] - detecting via the telemetry device whether at least one obstacle is present and, if the at least one obstacle is present, receiving distance data between the at least one detected obstacle and the vehicle,

[0011] - determining whether the quality of visibility outside the vehicle is below a defined threshold,

[0012] - determining whether a predetermined condition indicating a danger to the vehicle resulting from said at least one detected obstacle is satisfied, said predetermined condition comprising at least the received distance data,

[0013] - If predetermined conditions indicating a danger are met and if the visibility quality is below a defined threshold, the captured images are displayed on an area of ​​the display screen and at least one synthetic image is superimposed in real time on these captured images and around the at least one detected obstacle in order to send a signal about the at least one detected obstacle.

[0014] The display method according to the invention only displays the augmented reality feature when there is a potential collision hazard. This method reduces the cognitive load on the driver and reduces the energy consumption of the display system. In addition, when drivers are not yet accustomed to constantly viewing the augmented reality feature, they are more likely to accept the hazard warning feature. Therefore, this display method is more user-friendly.

[0015] The features disclosed in the following paragraphs may be optionally implemented. These features may be implemented independently of each other or in combination with each other:

[0016] If the predetermined condition indicating a danger is not met, or if the visibility quality is equal to or greater than a defined threshold, the method comprises a step of displaying the image captured by the first imager on said area of ​​the display screen.

[0017] If the predetermined condition indicative of a hazard is not met and if the visibility quality is less than said defined threshold, the method comprises a step of displaying the image captured by said first imager on said area of ​​the display screen.

[0018] - the display system further comprises a second imager mounted inside the vehicle and adapted to capture an image representing a driver of the vehicle, and the display method further comprises the steps of:

[0019] - capturing an image via said second imager,

[0020] - determining the gaze position of the vehicle driver from the image captured via the second imager,

[0021] - at least partially reducing the brightness of said area of ​​the display screen when the determined gaze position is not directed towards said area of ​​the display screen and when a predetermined condition indicative of a danger is not met.

[0022] - The method comprises the step of generating said at least one composite image comprising at least one signalling feature suitable for signalling the presence of an obstacle, the signalling feature being one of an arrow, an icon, a geometric figure, an overlay of said obstacle and an outline of said obstacle.

[0023] - The method further comprises the step of identifying said obstacle based on the distance data, and wherein the generating step comprises generating a transmission signal characteristic representative of the identified obstacle.

[0024] - The step of determining whether a predetermined condition has been met comprises the following steps:

[0025] - calculating the distance between the obstacle and the vehicle based on the distance data,

[0026] - Comparing the calculated distance with a first defined distance value.

[0027] - the processing unit is adapted to receive at least one data representative of the temperature outside the vehicle and at least one data representative of the humidity outside the vehicle, and wherein the step of determining whether a predetermined condition has been met comprises the following steps:

[0028] - calculating the distance between the obstacle and the vehicle based on the distance data,

[0029] - receiving at least one temperature data and / or at least one humidity data,

[0030] - comparing said temperature data to at least one threshold temperature, and / or

[0031] - comparing said humidity data to at least one threshold humidity,

[0032] - if the temperature data is below the threshold temperature and / or if the humidity data is below the threshold humidity, comparing the calculated distance with a second defined distance value.

[0033] - The step of determining whether a predetermined condition has been met comprises the following steps:

[0034] - calculating the magnitude and / or orientation of the speed of the obstacle, - comparing the calculated magnitude with a defined magnitude and / or comparing the calculated orientation with a defined orientation range.

[0035] - A step of determining a visibility quality outside the vehicle is performed based on the image captured by the first imager and / or the received distance data.

[0036] - The method further comprises the step of comparing the brightness level of the image captured by the first imager with a defined threshold brightness.

[0037] - The method comprises the steps of determining the position of said obstacle from the distance data and generating a composite image, wherein the transmission signal feature moves simultaneously with the movement of the obstacle.

[0038] - At least one remote sensing device comprises a radar, a lidar or an infrared light device. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] [ Figure 1] is a perspective view of the interior of a vehicle showing a portion of a display system implementing a display method according to the present invention;

[0040] [ Figure 2 ] is a schematic diagram of a display system for implementing a display method according to the present invention;

[0041] [ Figure 3 ] is a flow chart showing the steps of the display method according to the present invention;

[0042] [ Figure 4 ] is a flow chart illustrating a first example of a method for determining that a predetermined condition indicating a danger has been satisfied;

[0043] [ Figure 5 ] is a flow chart illustrating a second example of a method for determining that a predetermined condition indicating a danger has been satisfied;

[0044] [ Figure 6 ] is a flow chart illustrating a third example of a method for determining that a predetermined condition indicating a danger has been satisfied;

[0045] [ Figure 7 ] is a schematic diagram of a first image captured by a first imager;

[0046] [ Figure 8 ] is a schematic diagram of a first image and a composite image superimposed on the first image;

[0047] [ Fig. 9 ] is a schematic diagram of a second image captured by a first imager and a composite image superimposed on the second image; and

[0048] [ Fig.10 ] is a flowchart showing the steps of a display method implemented when the display system includes a second imager. DETAILED DESCRIPTION

[0049] refer to Figure 1 and Figure 2 The display system 2 implementing the display method according to the present invention includes a first imager 4 suitable for capturing images outside the vehicle, a telemetry device 6 suitable for detecting the presence of obstacles, a processing unit 8 connected to the first imager and the telemetry device 6, and a display screen 10 connected to the processing unit 8.

[0050] The first imager 4 may be a camera. The first imager is for example arranged in an outside rear view mirror, for example on the driver's side. The first imager has a first field of view extending along the side of the vehicle and towards the rear of the vehicle.

[0051] The remote sensing device 6 comprises, for example, a radar. The radar has a second field of view that includes at least a portion of the first field of view. The radar is located at the rear of the vehicle. The radar is capable of detecting the presence of obstacles 7 located around the vehicle, in particular at the sides and rear of the vehicle. The radar is configured to receive distance data representing the distance between the remote sensing device and the obstacle 7.

[0052] In this patent application, the term "obstacle" refers to any object, person or animal located in the second field of view that may potentially hinder the movement or stationary of the vehicle. Obstacles may be, for example, pedestrians, trucks, cars, bicycles, motorcycles, reconnaissance aircraft, tractors, flocks of sheep, animals, etc. Fig. 9 A bicycle type obstacle is shown in FIG.

[0053] Alternatively, the telemetry device 6 can also be installed in the exterior rearview mirror 15 .

[0054] Alternatively, the telemetry device may be a lidar or infrared light device, in particular a time of flight (ToF) sensor.

[0055] Preferably, the display system 2 comprises several telemetry devices 6 .

[0056] The processing unit 8 comprises a memory 14, in particular a flash memory, and a central processing unit 16, such as a processor or a microprocessor. The central processing unit 16 may be a programmable device using software, a part of an application specific integrated circuit (ASIC) or an electronic control unit (ECU).

[0057] The memory 14 stores executable image processing codes (hereinafter referred to as “image processing application program”) and executable codes for implementing the display method disclosed below.

[0058] The memory 14 also stores a defined threshold value S, a defined threshold brightness Ls, a defined first distance value D1, a threshold temperature Ts, a threshold humidity Hs, a defined second distance value D2, a defined size As, and a defined orientation range Ps.

[0059] In the embodiment shown, the display screen 10 is mounted on the A-pillar. Alternatively, the display screen may be mounted on the front door of the vehicle, or in the door, or on the center console.

[0060] In a first embodiment, the captured image is displayed on the entire screen, and if necessary, the composite image is displayed. In a second embodiment, the display screen can be divided into several areas. Each area can display a different image. One of these areas displays the captured image and, if applicable, displays the composite image.

[0061] The connection links between the first imager 4 and the processing unit 8, the telemetry device 6 and the processing unit 8, and the display screen 10 and the processing unit 8 may be wired or wireless.

[0062] In a particular embodiment, the display system 2 further comprises a second imager 12 connected to the processing unit 8. The second imager 12 is designed to capture an image of the vehicle driver and transmit the image to the processing unit 8.

[0063] refer to Figure 3 The display method according to the invention starts at step 20, in which the first imager 4 captures an image 16 of the vehicle exterior, in particular the side and rear of the vehicle. The first imager 4 transmits the captured image 16 to the processing unit 8. Figure 7 An example of a captured image 16 is shown.

[0064] Then, in step 22, the telemetry device 6 detects whether there is an obstacle 7 in the second field of view.

[0065] When the telemetry device 6 detects an obstacle 7, the telemetry device receives data representing the distance between the telemetry device 6 and the obstacle 7, hereinafter referred to as distance data, in step 24. The telemetry device transmits the distance data to the processing unit.

[0066] In step 26 , the processing unit determines whether the visibility quality outside the vehicle is below a defined threshold S. This step can be performed by various methods implemented alone or in combination.

[0067] According to a first method, a step 26 of determining the visibility quality outside the vehicle is performed based on the captured image 16 and the received distance data.

[0068] The image processing application searches for the presence of an obstacle 7 by processing the captured image 16, for example using a contour detection method. At the same time, the processing unit 8 checks whether the telemetry device 6 has detected the obstacle 7. If the telemetry device 6 has detected the obstacle 7 and the image processing application has not detected the obstacle, the processing unit 8 determines that the visibility quality outside the vehicle is below a defined threshold S.

[0069] In a second method, the image processing application determines the brightness level of at least some of the images captured by the first imager and compares the brightness level with a defined threshold brightness Ls. If the brightness level is below the defined threshold brightness Ls, the processing unit 8 determines that the visibility quality outside the vehicle is below a defined threshold S. This method detects whether the vehicle is in a low visibility environment.

[0070] In a third method, a vehicle brightness sensor determines brightness outside the vehicle, and a processing unit determines whether the brightness is below a given threshold.

[0071] If the processing unit determines that the visibility quality outside the vehicle is equal to or greater than the defined threshold S, or if the processing unit determines that the predetermined condition indicating a danger is not met, at least one area of ​​the display screen displays the image captured during step 27. In particular, at least one area of ​​the display screen displays only the captured image. The processing unit 8 does not superimpose the synthetic image 18 on the captured image 16.

[0072] If the processing unit determines that the quality of visibility outside the vehicle is below a defined threshold S, the method continues to step 28. During this step 28, the processing unit 8 determines whether the detected obstacle 7 satisfies a predetermined condition indicating a danger to the vehicle. This stage can be carried out by different methods implemented alone or in combination.

[0073] according to Figure 4 In the first method shown, the determination step 28 comprises: a sub-step 30 of calculating the distance between the detected obstacle 7 and the telemetry device 6 based on the distance data; and a sub-step 32 of comparing the calculated distance with a first defined distance value D1. The first defined distance value D1 has been stored in the memory 14. The first defined distance value D1 is equal to or greater than a predetermined safety distance to be followed.

[0074] If the calculated distance is less than the first defined distance value D1, the processing unit determines that the detected obstacle 7 meets or fulfills a predetermined condition indicating that there is a danger to the vehicle.

[0075] Thus, when step 28 is carried out according to the first method, the predetermined condition consists in the fact that the distance calculated during sub-step 30 is smaller than the first defined distance value D1 .

[0076] according to Figure 5 In the second method shown, the processing unit 8 calculates the distance between the detected obstacle 7 and the telemetry device 6 based on the distance data in sub-step 30 .

[0077] Then, in sub-step 34, processing unit 8 receives at least one data T representative of the temperature outside the vehicle and / or at least one data H representative of the humidity outside the vehicle. These data are measured, for example, by temperature and humidity sensors mounted on the vehicle and connected to the processing unit.

[0078] Then, in sub-step 36 , the processing unit 8 compares the temperature data T with a threshold temperature Ts, and / or compares the humidity data H with a threshold humidity Hs.

[0079] If the temperature data T is below the threshold temperature Ts, and / or if the humidity data H is below the threshold humidity Hs, the processing unit 8 compares the calculated distance with a second defined distance value D2 in sub-step 38. The second distance value D2 corresponds to a safe distance in rainy or foggy conditions.

[0080] If the temperature data T is below the threshold temperature Ts, and / or if the humidity data H is below the threshold humidity Hs, and if the calculated distance is below the second defined distance value D1, the processing unit determines that the detected obstacle 7 meets the predetermined condition indicating a danger to the vehicle.

[0081] If the temperature data T is not lower than the threshold temperature Ts and / or the humidity data H is not lower than the threshold humidity Hs, then in step 32 the processing unit compares the calculated distance with the first defined distance value D1.

[0082] Thus, when step 28 is performed according to the second method, the predetermined conditions include the calculated distance being lower than the second defined distance value D2 and the temperature being lower than the threshold temperature Ts and / or the humidity being lower than the threshold humidity Hs.

[0083] according to Figure 6 In the third method shown in , in sub-step 40, the processing unit 8 calculates the magnitude and / or orientation of the speed of the obstacle 7. The magnitude and / or orientation can be calculated based on a number of distance data measured within a predetermined time period.

[0084] In sub-step 42, the processing unit 8 compares the magnitude of the speed of the obstacle 7 with the defined magnitude As. In the same way, the processing unit 8 checks whether the moving orientation of the obstacle 7 is within the defined orientation range Ps relative to the vehicle. If the calculated magnitude of the speed is greater than the defined magnitude As and / or if the calculated orientation is within the defined orientation range Ps relative to the vehicle, the processing unit determines that the detected obstacle 7 meets a predetermined condition indicating that the vehicle is in danger. Advantageously, this sub-step 42 can be used to predict lane changes, overtaking or acceleration of obstacles (such as cars, motorcycles or bicycles).

[0085] When step 28 is performed according to the third method, the predetermined conditions include that the speed magnitude of the detected obstacle is greater than the defined magnitude As and / or the movement orientation of the detected obstacle is within the defined orientation range Ps.

[0086] Alternatively or additionally, the predetermined condition may also include a brightness level outside the vehicle. When the brightness level is below a brightness threshold, the driver's reaction time is considered to be greater than that of daytime driving, and thus the braking distance increases, thereby also increasing the defined distance value.

[0087] If the visibility is below a defined threshold S and if a predetermined condition indicating a danger is met, the method continues to step 44, where the processing unit 8 identifies an obstacle using the distance data and possibly the image captured by the first imager 4. Obstacle identification includes identifying whether the obstacle is a pedestrian, a truck, a car, a bicycle, a motorcycle, a reconnaissance aircraft, a tractor, a flock of sheep, an animal, etc.

[0088] Step 44 is optional.

[0089] In step 46 , the processing unit 8 determines the position of the obstacle based on the distance data and generates a composite image 18 .

[0090] The composite image 18 depicts at least one signaling feature 19 designed to indicate the presence of an obstacle. The signaling feature 19 is, for example, an arrow, an icon or a geometric figure such as a square or a circle.

[0091] A synthetic image is generated in real time based on the continuously received distance data, so that in the synthetic image, the transmission signal feature 19 moves simultaneously with the movement of the obstacle located outside the vehicle.

[0092] When the method comprises step 44, the signal feature 19 represents the identified obstacle. In this way, the signal feature 19 may be an icon representing the identified obstacle or an overlay representing the identified obstacle or the outline of the obstacle. Fig. 9 In the example shown, an icon representing a bicycle has been generated on the composite image 18, which is superimposed on the captured image 16. Figure 8 In the example shown, a vehicle overlay positioned where the vehicle is identified is generated on the composite image 18, which is superimposed on the captured image 16. An overlay is two or more images printed on the same photosensitive surface. In this case, the photosensitive surface is an area of ​​a display screen.

[0093] When the method does not comprise the step 44 of identifying an obstacle, the processing unit 8 generates a composite image comprising an icon or arrow showing the position of the obstacle without depicting the identification of the obstacle.

[0094] In step 48, the processing unit 8 displays the image captured by the first imager 4 on the display screen 10 and superimposes the synthetic image 18 on the captured image in real time to signal the presence and position of the obstacle 7.

[0095] After step 28, if the detected obstacle 7 does not meet the predetermined condition indicating a danger, the method continues to step 50. In step 50, at least one area of ​​the display screen displays the captured image. In particular, at least one area of ​​the display screen displays only the captured image. The processing unit 8 does not superimpose the composite image 18 on the captured image 16.

[0096] In embodiments where the display system 2 further includes a second imager 12 capable of capturing an image depicting the vehicle driver, reference Fig.10 , the display method may include a step 52 of capturing an image via the second imager 12 .

[0097] Then, in step 54 , the processing unit 8 determines the gaze position of the vehicle driver from the image captured by the second imager.

[0098] In step 56 , the brightness of display screen 10 is at least partially reduced when the determined position corresponds to the driver not looking at the display screen.

Claims

1. A method for displaying an image on a display screen (10) of a motor vehicle, the method being implemented by a display system (2), the display system comprising at least one first imager (4) adapted to capture an image of the exterior of the vehicle, at least one telemetry device (6) for detecting obstacles, a processing unit (8) connected to the at least one first imager and the at least one telemetry device, and a display screen (10) connected to the processing unit, the display method comprising the following steps: - capturing (20) an image (16) via said first imager, - detecting (22) via the telemetry device whether at least one obstacle (7) is present and, if the at least one obstacle (7) is present, receiving (24) distance data between the at least one detected obstacle (7) and the vehicle, - determining (26) whether the visibility quality outside the vehicle is below a defined threshold (S), - determining (28) whether a predetermined condition indicating a danger to the vehicle is met, the danger arising from the at least one detected obstacle (7), the predetermined condition comprising at least the received distance data, - If the predetermined condition indicating danger is met and if the visibility quality is below the defined threshold (S), the captured images (16) are displayed (48) on the area of ​​the display screen (10) and at least one synthetic image (18) is superimposed (48) in real time on these captured images and around the at least one detected obstacle (7) in order to signal the at least one detected obstacle (7).

2. A display method according to claim 1, wherein if the predetermined condition indicating danger is not met, or if the visibility quality is equal to or greater than the defined threshold (S), the method comprises the step of displaying (27) the image captured (16) by the first imager on the area of ​​the display screen (10).

3. A display method according to claim 1 or 2, wherein if the predetermined condition indicating danger is not met and if the visibility quality is less than the defined threshold (S), the method comprises the step of displaying (50) the image captured (16) by the first imager on the area of ​​the display screen (10).

4. The display method according to any one of claims 1 to 3, wherein the display system (2) further comprises a second imager (12) mounted inside the vehicle and adapted to capture an image representing a driver of the vehicle, and the display method further comprises the following steps: - capturing an image (52) via said second imager (12), - determining (54) a gaze position of the vehicle driver based on the image captured via the second imager (12), - at least partially reducing (56) the brightness of said area of ​​said display screen (10) when the determined gaze position is not directed towards said area of ​​said display screen (10) and when said predetermined condition indicating a danger is not met.

5. A display method according to any one of claims 1 to 4, comprising the step of generating (46) at least one composite image (18), wherein the at least one composite image comprises at least one signalling feature suitable for sending a signal regarding the presence of the obstacle, the signalling feature being one of an arrow, an icon, a geometric figure, an overlay of the obstacle and an outline of the obstacle.

6. A display method according to claim 5, further comprising the step of identifying (44) the obstacle based on the distance data, and wherein the generating step (46) comprises generating a transmission signal feature (19) representing the identified obstacle.

7. A display method according to any one of claims 1 to 6, wherein the step of determining (28) whether a predetermined condition has been met comprises the following steps: - calculating (30) the distance between the obstacle (7) and the vehicle based on the distance data, - comparing (32) the calculated distance with a first defined distance value (D1).

8. A display method according to any one of claims 1 to 7, wherein the processing unit (8) is adapted to receive at least one data (T) representing the temperature outside the vehicle and at least one data (H) representing the humidity outside the vehicle, and wherein the step of determining (28) whether the predetermined condition has been met comprises the following steps: - calculating (30) the distance between the obstacle (7) and the vehicle based on the distance data, - receiving (34) at least one temperature data and / or at least one humidity data, - comparing (36) said temperature data with at least one threshold temperature (Ts), and / or - comparing (36) said humidity data with at least one threshold humidity (Hs), - if the temperature data is below the threshold temperature (Ts) and / or if the humidity data is below the threshold humidity (Hs), comparing (38) the calculated distance with a second defined distance value (D2).

9. A display method according to any one of claims 1 to 8, wherein the step of determining (28) whether a predetermined condition has been met comprises the following steps: - calculating (40) the magnitude and / or orientation of the velocity of the obstacle, - Comparing (42) the calculated size with a defined size (As) and / or comparing the calculated orientation with a defined orientation range (Ps).

10. Display method according to any one of claims 1 to 9, wherein the step of determining (26) the visibility quality of the exterior of the vehicle is performed based on the image captured by the first imager and / or the received distance data.

11. Display method according to any one of claims 1 to 10, wherein the at least one remote sensing device (6) comprises a radar, a lidar or an infrared light device.

Images