Enhanced display system and method for motor vehicles

Abstract

Enhanced display system and method for motor vehicles. This enhanced display system for motor vehicles comprises a head-up display surface, allowing viewing of the exterior of the vehicle, a first camera positioned towards the exterior of the vehicle, at least a second camera positioned towards the interior of the vehicle and associated with a unit for detecting the gaze direction of a vehicle occupant, and a programmable electronic device configured to implement modules for: - receiving (30) first images captured by said first camera and a gaze direction of said vehicle occupant, the gaze direction being obtained by implementing said gaze direction detection unit from second images captured by the second camera, - identifying (32) an object placed in the gaze direction detected in said first images, - obtaining (34) information relating to the identified object,and - display (36) on said display surface of information relating to the identified object. Figure for the abbreviation: Figure 1,

Description

Title of the invention: Enhanced display system and method for a motor vehicle

[0001] The present invention relates to an enhanced display system for a motor vehicle and a motor vehicle comprising such an enhanced display system.

[0002] It also relates to an associated enriched display method and an associated computer program.

[0003] The invention lies in the field of motor vehicles equipped with improved functionalities, in particular to increase safety and user satisfaction.

[0004] Motor vehicles are increasingly equipped with various sensors and electronic computing devices to add various functionalities, including functionalities to increase the comfort and safety of vehicle occupants, whether the driver or passengers.

[0005] The present invention falls within this technological context and aims to improve the experience of the occupants of a motor vehicle.

[0006] To this end, the invention relates to an enhanced display system for a motor vehicle, the display system comprising a head-up display surface, making it possible to view the exterior of the vehicle, the system comprising a first camera positioned towards the exterior of the vehicle, at least one second camera positioned towards the interior of the vehicle and associated with a unit for detecting a direction of gaze of an occupant of the vehicle, and a programmable electronic device configured to implement modules of:

[0007] -receiving first images captured by said first camera and a direction of gaze of said occupant of the vehicle, the direction of gaze being obtained by implementing said unit for detecting a direction of gaze from second images captured by the second camera,

[0008] -identification of an object placed in the direction of gaze detected in said first images,

[0009] -obtaining at least one piece of information relating to the identified object, and

[0010] -display on said display surface of information relating to the identified object.

[0011] Advantageously, the proposed enriched display system makes it possible to detect the direction of gaze of one or more occupants of the motor vehicle, and then to provide additional information relating to the objects viewed by the occupant or occupants of the vehicle whose direction of gaze has been detected.

[0012] According to other advantageous aspects of the invention, the enriched display comprises one or more of the following characteristics, taken individually or in all technically possible combinations.

[0013] The programmable electronic device comprises an enhancement module configured to calculate a brightness level in at least one of said first images, and when the brightness level is lower than a first threshold, apply a processing to increase the brightness level of said first images to obtain first images with a brightness level higher than said first threshold and display on said display surface said first images with a brightness level higher than said first threshold.

[0014] The programmable electronic device is further configured to calculate a brightness level in at least one of said first images, and when the brightness level is greater than a second threshold, apply a processing of reducing the brightness level of said first images to obtain first images with a brightness level lower than said second threshold and display on said display surface said first images with a brightness level lower than said second threshold.

[0015] The display surface is formed from a transparent glass on which a reflective film is added.

[0016] The second camera is oriented towards the driver of the vehicle.

[0017] The system further comprises a geolocation receiver, and the identification of a object placed in the detected gaze direction is carried out using a first geolocation position of the vehicle at the time of capturing said first images and a terrain database, the identification comprising a determination of a second geolocation position corresponding to the detected gaze direction, and an extraction from said terrain database of an identifier of an object placed at said second geolocation position.

[0018] The at least one piece of information relating to said identified object is obtained from an external database, based on said object identifier.

[0019] The invention also relates to an enriched display method implemented in an enriched display system as briefly described above, comprising steps of:

[0020] -receiving first images captured by said first camera and a direction of gaze of said occupant of the vehicle, the direction of gaze being obtained from second images captured by the second camera,

[0021] -identification of an object placed in the direction of gaze detected in said first images,

[0022] -obtaining information relating to the identified object, and

[0023] -display control on said display surface of information relating to the identified object.

[0024] The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement an enhanced display method as defined above.

[0025] The invention also relates to a motor vehicle comprising an enhanced display system as briefly described above.

[0026] The invention will appear more clearly on reading the description which follows, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0027] [Fig-1] [Fig.l] is a schematic representation of a display system enriched according to an embodiment;

[0028] [Fig.2] [Fig.2] is an example of an enriched display;

[0029] [Fig.3] [Fig.3] is a synopsis of the main stages of a display process enriched according to an embodiment.

[0030] The invention applies to any type of motor vehicle, and more generally to any type of vehicle comprising a passenger compartment adapted to accommodate people (driver and passengers), the space inside the passenger compartment being the interior space of the vehicle, and comprising at least one display surface allowing the exterior of the vehicle to be viewed.

[0031] The passenger compartment comprises, in a known manner, seats intended to accommodate the driver and passengers respectively.

[0032] [Fig.l] schematically illustrates a display system 2 called an enriched display system, according to one embodiment.

[0033] The display system 2 is adapted to be carried by a vehicle 4.

[0034] The display system 2 comprises a display surface 6 called “head-up” or HUD (for “Head-Up Display”), allowing you to view the exterior of the vehicle.

[0035] For example, the display surface 6 is formed from a transparent window, for example the windshield of the vehicle 4, conventionally intended for viewing the front exterior of the vehicle, on which a reflective film is added, thus making it possible to form a display surface 6 on which it is possible to display images.

[0036] The enhanced display system 2 further comprises a first camera 8 for capturing images, for example an optical camera, positioned on or in the vehicle and oriented so as to capture first images of the exterior of the vehicle. The first images are digital images, and they are captured continuously with a given shooting frequency, for example of the order of 50 images per second.

[0037] In one embodiment, the first camera 8 is oriented so as to capture, in its field of view, with a given shooting angle, the front exterior of the vehicle. In this case, the field of view of the first camera 8 corresponds to the field of vision of the driver of the vehicle.

[0038] In the following, it is considered that only one first camera 8 is present, but a similar operation is envisaged for a plurality of first cameras positioned towards the outside of the vehicle, so as to widen the field of view, first combined images of widened field of view then being obtained.

[0039] In addition, the enriched display system 2 comprises at least one second camera 10 for taking pictures, for example an optical camera, positioned inside the passenger compartment of the vehicle and oriented towards the interior of the passenger compartment of the vehicle.

[0040] In the example of [Fig.l], two such second cameras 10 are shown.

[0041] Each second camera 10 is associated with or comprises a unit 12 for detecting the direction of gaze of an occupant of the vehicle located in the shooting field of the second camera. Each second camera 10 associated with a unit 12 for detecting the direction of gaze forms a device for detecting the direction of gaze. Such devices for detecting the direction of gaze of the driver are known, for example in the field of driver supervision systems or DMS (for “Driver Monitoring Systems” in English).

[0042] The enhanced display system 2 further comprises a programmable electronic device 14, configured to control a display of data on the display surface 6 and connected respectively to the first camera 8 and to the or each second camera 10 via internal communication buses 16.

[0043] The programmable electronic device 14 is for example an on-board electronic calculation unit ECU.

[0044] The programmable electronic device 14 comprises at least one calculation processor 18 and an electronic memory 20.

[0045] The enhanced display system 2 comprises or is configured to communicate with an on-board geolocation receiver 22, for example a GPS receiver. In a known manner, a geolocation receiver is capable of providing a geolocation position of the vehicle, in a fixed terrestrial reference frame, for example of providing coordinates of the vehicle in this terrestrial reference frame. Such a geolocation receiver is for example used to display, on an additional display screen (not shown) a symbol representing the vehicle on a map, the map being obtained from a terrain database 24. The terrain database 24 can be stored locally or remotely and accessible via a wireless communication protocol.

[0046] Furthermore, the enhanced display system 2 comprises or is configured to communicate with a communication interface 26. The communication interface 26 implements a wireless communication protocol, allowing communication with remote data servers, in particular with one or more external databases 28.

[0047] The programmable electronic device 14 comprises a reception module 30 which is configured to receive first images captured by the first camera 8 and a direction of gaze of an occupant of the vehicle, the direction of gaze being obtained by implementing the unit for detecting a direction of gaze from second images captured by the second camera 10. Preferably, the second images are captured substantially at the same time as the first images.

[0048] For example, the direction of gaze is expressed by a 3D vector in a spatial reference system associated with the vehicle.

[0049] In the case where several second cameras 10 and associated gaze direction detection units 12 are present, the module 30 is configured to receive a gaze direction per second camera 10.

[0050] The programmable electronic device 14 further comprises a module 32 for identifying an object placed in the first images as a function of the detected direction of gaze, a module 34 for obtaining at least one item of information relating to the identified object and a module 36 for displaying on the display surface 6 the information relating to the identified object.

[0051] In addition, the programmable electronic device 14 optionally comprises a module 38 for improving the brightness of the first images, and for displaying the first improved images on the display surface 6, thus making it possible to improve the visual comfort of the occupants, as well as the visibility of the exterior of the vehicle, in the shooting field of the first camera 8.

[0052] In particular, the module 38 is configured to calculate a brightness level in at least one of the first images, and when the brightness level is lower than a first threshold SI, apply a processing to increase the brightness level of said first images to obtain first images with a brightness level higher than the first threshold and control a display on the display surface 6 of the first images with a brightness level higher than the first threshold.

[0053] In addition, the module 38 is further configured to, and when the calculated brightness level is greater than a second threshold S2, apply a processing of reducing the brightness level of the first images to obtain first images with a brightness level lower than the second threshold S2 and control display on the display surface 6 of the first images with a brightness level lower than the second threshold.

[0054] The processor 18 is configured to implement the modules 30, 32, 34, 36 and 38 when the programmable electronic device 14 is powered up.

[0055] In one embodiment, the modules 30, 32, 34, 36 and 38 are produced in the form of software instructions forming a computer program, which, when executed by a computer, implements an enhanced display method according to the invention.

[0056] The computer program comprising software instructions is further capable of being recorded on a non-transitory computer-readable medium. The computer-readable medium is, for example, a medium capable of storing the electronic instructions and of being coupled to a bus of a computer system. For example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example EPROM, EEPROM, FLASH, NVRAM), a magnetic card or an optical card.

[0057] In a variant not shown, the modules 30, 32, 34, 36 and 38 are each produced in the form of programmable logic components, such as FPGAs (Field Programmable Gate Arrays), microprocessors, GPGPU components (General-purpose processing on graphics processing), or even dedicated integrated circuits, such as ASICs (Application Specific Integrated Circuits)•

[0058] [Fig.2] schematically illustrates an example of a display surface 6 comprising an enriched display.

[0059] In this schematic example, the occupant of the vehicle concerned is the driver C, who is seated in the passenger compartment, behind the steering wheel. The second camera 10 is also shown schematically, which, in collaboration with the direction of gaze detection unit, makes it possible to detect the direction of gaze of the driver C.

[0060] The shooting field of the camera 10 is represented by the axes A1, A2.

[0061] A driver's field of vision is located between axes A3 and A4. The direction of the The driver's gaze is represented by the vector V.

[0062] In the first image 40 which is visible on the display surface 6, the vector V intercepts an object 42.

[0063] The object 42 is for example a building, and more specifically a monument, the external environment being a city in the illustrated example.

[0064] In other examples, it may be a remarkable element of the landscape, for example a portion of relief (hill, mountain), a distant building, etc.

[0065] Information 'XXX' relating to the identified object 42 is displayed inside a small superimposed window 44, which is displayed superimposed on the first image 40. For example, the window 44 is a pop-up window, displayed temporarily.

[0066] For example, the information 'XXX' is textual information, and represents a name identifying the identified object 42, for example the name of the monument or element re- notable part of the landscape.

[0067] Alternatively or in addition, safety information is also displayed, for example information relating to potential hazards, work in progress, etc.

[0068] For example, the window 44 is displayed in relation to the identified object 42, and therefore its position on the display surface 6 moves, when the vehicle is moving, so as to remain in the same relative position with respect to the object 42, and is no longer displayed when the object 42 leaves the shooting field of the first camera.

[0069] Thus, advantageously, information relating to the objects is displayed on the display surface 6, thus forming a display enriched with additional information.

[0070] [Fig. 3] is a block diagram of the main steps of an enriched display method, implemented by an enriched display system as described above, according to one embodiment.

[0071] The method comprises steps 50 of capturing first images by the first camera and 52 of capturing second images by the or each second camera, and a step 54 of detecting a direction of gaze of an occupant of the vehicle from each second image.

[0072] The method comprises receiving 56 the first images and 3D vectors indicating each direction of gaze of an occupant of the vehicle.

[0073] The method also comprises a step 58 of identifying, for each direction of gaze detected, represented by a vector V, an object placed in the first image in the direction V.

[0074] In one embodiment, step 58 comprises a detection 60 of a first geolocation position PI of the vehicle at the time of capturing the first image considered, a determination 62 of a second geolocation position P2 corresponding to the detected direction of gaze. The first geolocation position PI and the second geolocation position P2 are both represented in a fixed terrestrial reference frame.

[0075] For example, the step 62 of determining a second geolocation position P2 implements a third spatial position P3 of intersection of the direction of gaze and the first image considered, the third spatial position being expressed for example in a spatial reference system associated with the vehicle, and a calculation of the second geolocation position P2 as a function of the third spatial position P3 and the first geolocation position PI of the vehicle.

[0076] The identification step 58 further comprises an extraction 64 from a terrain database of an object identifier located at the second geolocation position P2.

[0077] The method further comprises a step 66 of obtaining information relating to the object identified by the object identifier. For example, step 66 implements an external database, to extract textual information relating to the identified object, for example the name of a monument etc.

[0078] Finally, the method comprises a step 68 of controlling the display of information relating to the identified object, for example in the form of text in a “pop-up” type window.

[0079] As indicated, the display relating to the same object is for example maintained as long as the object is identified in a first image.

[0080] Optionally, the method also comprises steps relating to improving the brightness of the first images, thus allowing an improved display making it possible to improve driving safety.

[0081] The method then comprises a calculation of a brightness level of one or more successive first images.

[0082] Any conventional method of calculating brightness level is applicable.

[0083] The method then comprises a comparison of the calculated brightness level with a first threshold SL

[0084] If the brightness level is lower than the first threshold SI, the method comprises a step of processing the first images to increase the brightness level of said first images and obtain first images with a brightness level higher than the first threshold SL.

[0085] Then, the method comprises a display on the display surface 6 of the first images of brightness level higher than the first threshold.

[0086] The method also comprises, if the calculated brightness level is not lower than the first threshold, a comparison of the calculated brightness level with a second threshold S2.

[0087] The second threshold S2 is greater than or equal to, preferably greater than, the first threshold SL

[0088] If the brightness level is greater than the second threshold S2, the method comprises a step of processing the first images to reduce the brightness level of said first images and obtain first images with a brightness level lower than the second threshold S2.

[0089] The first threshold SI and second threshold S2 are chosen according to external conditions, for example day / night, weather conditions (clouds, rain, sun etc.), presence or absence of external lighting.

[0090] As an optional addition, other image processing operations making it possible to improve the quality of the first captured images in real time are applied, the first improved images being displayed on the display surface, which makes it possible to increase the vi sibility from the outside by the driver and therefore improve safety.

[0091] Advantageously, the proposed enriched display makes it possible to improve the experience of vehicle occupants, providing additional information and improving safety.

Claims

Claims

1. Enhanced display system for a motor vehicle, the display system comprising a head-up display surface (6), making it possible to view the exterior of the vehicle, the system comprising a first camera (8) positioned towards the exterior of the vehicle, at least one second camera (10) positioned towards the interior of the vehicle and associated with a unit (12) for detecting a direction of gaze of an occupant of the vehicle, and a programmable electronic device (14) characterized in that it is configured to implement modules for: -receiving (30) first images captured by said first camera (8) and a direction of gaze of said occupant of the vehicle, the direction of gaze being obtained by implementing said unit (12) for detecting a direction of gaze from second images captured by the second camera (10), -identifying (32) an object placed in the direction of gaze detected in said first images,-obtaining (34) at least one piece of information relating to the identified object, and -displaying (36) on said display surface (6) information relating to the identified object.,

2. Display system according to claim 1, wherein the programmable electronic device comprises an enhancement module (38) configured to calculate a brightness level in at least one of said first images, and when the brightness level is lower than a first threshold, apply a processing of increasing the brightness level of said first images to obtain first images of brightness level higher than said first threshold and display on said display surface said first images of brightness level higher than said first threshold.

3. Display system according to one of claims 1 or 2, wherein the programmable electronic device is further configured to calculate a brightness level in at least one of said first images, and when the brightness level is greater than a second threshold, apply a processing of decreasing the brightness level of said first images to obtain first images of brightness level lower than said second threshold and display on said display surface said first images of brightness level lower than said second threshold.

4. A display system according to any one of claims 1 to 3, wherein said display surface is formed of a transparent glass pane to which a reflective film is added.

5. A display system according to any one of claims 1 to 4, wherein said second camera is oriented towards the driver of the vehicle.

6. A display system according to any one of claims 1 to 5, further comprising a geolocation receiver (22), and wherein the identification of an object placed in the detected direction of gaze is performed using a first geolocation position of the vehicle at the time of capturing said first images and a terrain database (24), the identification comprising a determination of a second geolocation position corresponding to the detected direction of gaze, and an extraction from said terrain database (24) of an identifier of an object placed at said second geolocation position.

7. System according to claim 6, wherein the at least one information relating to said identified object is obtained from an external database (28), based on said object identifier.

8. A motor vehicle comprising an enhanced display system according to any one of claims 1 to 7.

9. An enriched display method implemented in an enriched display system according to claims 1 to 7, comprising steps of: -receiving (56) first images captured by said first camera and a direction of gaze of said occupant of the vehicle, the direction of gaze being obtained from second images captured by the second camera, -identifying (58) an object placed in the direction of gaze detected in said first images, -obtaining (66) at least one item of information relating to the identified object, and -controlling display (68) on said display surface of information relating to the identified object.

10. A computer program comprising software instructions which, when executed by a programmable electronic device, implement an enhanced display method for a motor vehicle according to claim 9.

Images