Motor vehicle comprising a navigation system displaying warning information in augmented reality

Abstract

The invention relates to a motor vehicle (VHL) comprising a navigation system (CNS), a camera (CMF) coupled to the navigation system (CMF), which is capable of capturing a road scene in front of the vehicle (VHL), and a navigation screen (ECN) coupled to the navigation system (CNS); wherein the navigation system (SNC) further comprises an augmented reality display module (MRA) capable of displaying at least one item of warning information on the screen (ECN) when the vehicle (VHL) is geolocated as approaching a pedestrian collision risk zone, in the form of a virtual dynamic graphical representation overlaid on real-time images captured by the camera (CMF); and wherein the augmented reality display module (MRA) is capable of modifying the appearance of the virtual graphical representation according to time ranges determined in relation to the periods during which the risk zone is frequented.

Description

DESCRIPTION Title of the invention: Motor vehicle comprising a navigation system displaying alert information in augmented reality The present invention claims priority from French application 2312907 filed on 23.11.2023, the content of which (text, drawings and claims) is incorporated herein by reference.

[0001] The present invention relates generally to navigation systems for motor vehicles and relates more particularly to a motor vehicle comprising a navigation system displaying alert information in augmented reality and in particular alert information concerning the proximity of an area at high risk of collision with pedestrians likely to enter the roadway upstream of the vehicle, such as an area comprising a school establishment.

[0002] An on-board navigation system in a vehicle is generally integrated into, or coupled with, a so-called vehicle infotainment system which manages exchanges between the vehicle and the outside world: remote servers, portable electronic terminals, etc.

[0003] An in-vehicle infotainment system, also known as an IVI, refers to a system that combines entertainment and information provision for drivers and passengers.

[0004] Such an infotainment system typically comprises a central unit associated with a digital display screen equipped with a touch-sensitive control surface, also called a multimedia screen, allowing the user, and in particular the driver, to navigate through a menu tree giving him access to numerous vehicle features, including connected navigation via a vehicle navigation system.

[0005] The navigation system uses digital map data and real-time vehicle location data provided by a geolocation device commonly referred to as a “GPS” receiver, an acronym for “Global Positioning System”.

[0006] The navigation system benefits from one of the features of the infotainment system: connectivity to the exterior of the vehicle and in particular to one or more remote servers, capable of providing real-time data relating to car navigation.

[0007] This data complements that already present, pre-recorded in a cartographic database of the on-board navigation system.

[0008] Warning information relating to control zones, work zones, etc. is displayed in the form of icons by the navigation system in a so-called standard navigation mode and traffic information is displayed in a so-called connected navigation mode, for example by superimposing on a route, colored segments (“orange” or “red”) depending on the traffic density. This information is transmitted to the vehicle by remote servers for a subscription.

[0009] Some vehicles have a navigation system enhanced with an option called "augmented reality." This option, implemented by an augmented reality display module, allows virtual information to be superimposed on a visualization of the real road scene in front of the vehicle, filmed by a front camera of the vehicle.

[0010] We speak of augmented reality when graphic objects representing virtual elements are displayed superimposed on the driver's road scene seen directly by the driver or through real images of the road scene captured by a camera and displayed on a screen present in the cockpit of the vehicle, in particular a screen designated as a multimedia screen or even a navigation screen.

[0011] This type of camera, also called a multifunction camera, is used in particular by an ADAS (Advanced Driver Assistance System) driving assistance system to assess safety distances between vehicles, for sign recognition, etc.

[0012] One advantage of augmented reality is that it can superimpose floating virtual arrows, a road marking or a heading / compass, or any other guidance information indicating the route to follow and / or pointing in the direction of the road to follow on the actual road scene displayed on a vehicle screen, in particular a vehicle navigation screen.

[0013] To date, there are no virtual elements that are realistic and sufficiently impactful to alert the vehicle driver of a danger ahead of the vehicle's route, without polluting the display of other information displayed in real time on the navigation screen.

[0014] A risky situation requiring the display of an alert in augmented reality could be, for example, the presence of an educational establishment (school) on a vehicle route.

[0015] The objective is therefore to enrich on-board navigation systems with a striking and dynamic alert display functionality in augmented reality linked to a risk zone. collision with pedestrians and in particular an area crossed by the vehicle's route containing a school.

[0016] To this end, the invention has as its first subject a motor vehicle comprising a navigation system, a camera coupled to the navigation system, capable of filming a road scene in front of the vehicle, a navigation screen coupled to the navigation system; said navigation system further comprising an augmented reality display module capable of displaying at least one alert information on the screen when the vehicle is geolocated approaching a zone at risk of collision with pedestrians, in the form of a virtual dynamic graphic representation superimposed on real images captured by the camera; said augmented reality display module being capable of modifying the appearance of the virtual graphic representation according to time slots determined in relation to periods of frequentation of said risk zone.

[0017] According to one feature, the virtual reality display module is capable of automatically controlling the stopping of the display of the virtual signaling gantry when the navigation system has geolocated the vehicle outside the risk zone.

[0018] According to one feature, the navigation system includes a navigation menu allowing the selection of an option for displaying in augmented reality alert information relating to the proximity of said risk zone, from the navigation screen.

[0019] According to another characteristic, the augmented reality display module is capable of controlling the display of a virtual signaling gantry, superimposed on the real images of the road scene, when the navigation system geolocates the vehicle near the risk zone; said augmented reality display module augmented being able to assign a “red” color to the virtual signaling gantry when the vehicle is geolocated near the risk zone in determined time slots of attendance of the risk zone, and able to assign an “orange” color to the virtual signaling gantry outside these determined time slots.

[0020] According to one characteristic, the virtual signaling gantry comprises at least one virtual road warning sign relating to the type of risk zone, accompanied or not by at least one virtual road traffic light in flashing or fixed lighting mode.

[0021] According to one characteristic, the risk zone contains at least one educational establishment.

[0022] According to one feature, the navigation system cooperates with a cartographic database listing at least said educational establishment and the attendance time slots of said establishment; said time slots being used by the augmented reality display module to adapt the color of the virtual signaling gantry according to said time slots.

[0023] According to one characteristic, the motor vehicle further comprises an infotainment system capable of providing the augmented reality display module with information on determined time slots of attendance in relation to the risk zone; said information being capable of being updated via waves.

[0024] The second subject of the present invention is a method for managing the display of a graphic representation of alert information in a vehicle as described above, said method, implemented by the augmented reality display module when the augmented reality display option of a alert information relating to the proximity of said risk zone is selected, consisting of displaying on the navigation screen a virtual signaling gantry, superimposing real images of the road scene captured by the camera when the vehicle is geolocated near the risk zone, and assigning a “red” color to the virtual signaling gantry when the vehicle is geolocated near the risk zone in determined time slots of frequentation of the risk zone and assigning an “orange” color outside these determined time slots.

[0025] Finally, the present invention has as a third object, a computer program product comprising instructions which, when the program is executed by a computer, lead the latter to implement the method as described above.

[0026] The present invention also has the advantage of providing precision on the level of danger by taking into account the time slots for which high attendance in a risk zone is expected.

[0027] Other advantages and characteristics of the present invention may emerge more clearly from the description which follows, given solely by way of non-limiting example and made with reference to the drawings in which:

[0028] [Fig. 1] illustrates a schematic view of a vehicle according to the invention;

[0029] [Fig. 2] illustrates a first aspect of an augmented reality graphic representation of alert information displayed on a navigation screen of the vehicle according to the invention;

[0030] [Fig. 3] illustrates a second aspect of the augmented reality graphic representation of the alert information displayed on the vehicle navigation screen according to the invention; and

[0031] [Fig. 4] illustrates a flowchart of the main steps of a method according to the invention implemented by the augmented reality display module of the vehicle according to the invention.

[0032] The present invention is described below using the example of a situation in which a vehicle route crosses a risk zone in which there is an educational establishment which will hereinafter be referred to as a “school”, regardless of the type of educational establishment.

[0033] A school is generally physically indicated by danger and speed limit signs, near the school, several dozen meters before and after the school and by ground markings of the type "Slow Down School" in the immediate vicinity of the school and / or in front of the school entrances.

[0034] Although the speed limit in the zone and the recommendations to slow down must be respected regardless of the time and day of the week, during or outside school holidays, such a zone is only really an area in which driver vigilance is required during specific time slots surrounding the hours of entry and exit from classes (the most critical times in terms of concentration of pedestrians in the zone: children and accompanying persons and therefore a population at high risk of collision with a vehicle).

[0035] The main idea of ​​the present invention is therefore to be able to alert the driver upstream of the “school” zone in the augmented reality navigation scene by adding information on the level of vigilance expected of the driver according to determined time slots. The term “dynamic” introduced previously, translates this ability to adapt the alert information, according to busy periods in the “school” zone.

[0036] Figure 1 illustrates a schematic view of a VHL motor vehicle according to the invention.

[0037] The VHL vehicle is equipped with at least one front camera of the CMF multifunction camera type, arranged at the top of the PRB windshield of the VHL vehicle and being substantially centered relative to the PRB windshield.

[0038] As already indicated in the preamble of the present invention, the CMF camera can be the same camera as that which is used by an ADAS system of the VHL vehicle in particular to evaluate the distances between vehicles for the control of braking, speed, or even for recognizing traffic signs.

[0039] The VHL vehicle further comprises an on-board SNC navigation system capable of being connected to a remote server, in particular a server providing navigation data, using communication functionalities of the VHL vehicle provided for example by a SID infotainment system via a wireless communication link supporting 3G, 4G or 5G communication protocols, not shown.

[0040] The SNC on-board navigation system (in connected mode or not) is capable of locating points of interest or POIs (Point Of Interest) such as the exact locations of schools pre-recorded and listed in a CAR cartographic database, with the school attendance time slots, and searchable from a navigation engine of the SNC navigation system, not shown, on a vehicle route. The SNC navigation system also makes it possible to locate the VHL vehicle in relation to this route thanks in particular to GPS geolocation of the VHL vehicle via a GPS receiver of the SNC navigation system, not shown.

[0041] The SNC navigation system is coupled to, or includes an augmented reality module MRA, capable of displaying dynamic alert information in the form of graphic representations of determined virtual elements and relating to the “schools” POI which will be superimposed on the real images IMG captured in real time by the CMF front camera when the VHL vehicle is geolocated approaching the school.

[0042] Access to the augmented reality display mode for alert information, for example titled "school proximity alert", is presented, for example, as an option that can be selected from a menu of the SNC navigation system displayed on a central screen of the VHL vehicle, generally an ECN multimedia screen (or navigation screen) with a touch-sensitive HMI (Human Machine Interface), coupled to the SNC navigation system.

[0043] Thanks to the knowledge of the GPS position of all the schools and the VHL vehicle, by the SNC navigation system, the MRA augmented reality module, in cooperation with the SNC navigation system, determines when to display alert information called "school proximity alert" in the scene filmed by the CMF front camera of the VHL vehicle, when approaching the physical location of the school.

[0044] In a preferred embodiment, as illustrated in Figures 2 and 3, the graphic element is advantageously chosen to represent a virtual PSV signaling gantry, delimited by broken lines in the figures, supporting one or more PAS traffic signs and one or more virtual FSL traffic lights.

[0045] The MRA augmented reality display module controls the display of the PSV virtual signaling gantry from the moment the VHL vehicle is geolocated by the SNC navigation system at a determined distance upstream of the school's geolocation (50 meters for example in an urban area) and disappears automatically when the VHL vehicle is geolocated at a determined distance after the HVL vehicle has passed in front of the school's geolocation (50 meters for example in an urban area).

[0046] The MRA augmented reality display module uses an “orange” or “red” color code to modify the visual appearance of the PSV virtual signaling gantry depending on the time at which the VHL vehicle is geolocated near a school in order to specify the level of attendance and therefore danger in the area.

[0047] Thus, the virtual signaling gantry is colored "red" (figure 2) during busy times, i.e. at times when children enter or leave school, typically between 8:00 a.m. and 9:30 a.m., between 11:00 a.m. and 1:30 p.m. and between 4:00 p.m. and 5:00 p.m. That is to say, if the VHL vehicle uses a traffic lane near a school during these times, the PSV virtual signaling gantry will be displayed with a "red" color and the graphic representations of the FLS virtual lights can virtually reproduce light signals in flashing mode or in "flash" mode in "red" color, for example.

[0048] The PSV virtual signaling gantry is “orange” (figure 3) during times of lowest traffic near schools, i.e. outside the time slots indicated above.

[0049] According to the embodiment described with reference to figures 2 and 3, the augmented reality display module MRA also controls the display on the virtual PSV gantry of a virtual pictogram representing a PAS traffic sign, centered on the horizontal bar of the virtual PSV gantry, allowing the user to identify that it is a “school proximity alert” type alert. Graphical representations of FLS flashing or steady traffic lights can also be displayed on the virtual gantry on either side of the PSV traffic sign. Other virtual warning signs can be displayed on the PSV virtual traffic gantry to reinforce the signaling of a school.

[0050] The SNC navigation system cooperates with a CAR map database listing schools and school attendance time slots; time slots which are used by the MRA augmented reality display module to adapt the appearance (the “red” or “orange” color) of the PSV virtual signaling gantry according to the time slots.

[0051] The SNC navigation system can cooperate with an SID infotainment system capable of providing, to the MRA augmented reality display module, information on determined time slots of attendance in relation to the risk zone such as schools; said information being capable of being updated via waves, in particular via a wireless communication network using the 3G, 4G or 5G protocol, in connection with a remote server, not shown.

[0052] The present invention provides a visualization and a fluid user experience of the SNC navigation system thanks to augmented reality, without the need to add a “popup” type message which would pollute the scene filmed by the CMF camera.

[0053] The CMF camera can be used either permanently to scan the road scene or temporarily when approaching the area where the school is physically located.

[0054] A method for managing the display of the PSV virtual signaling gantry, using a two-color color code, respectively “red” and “orange”, implemented by the module augmented reality display MRA, is described below with reference to the flowchart in figure 4.

[0055] The method consists, when an option for displaying in augmented reality alert information relating to the proximity of a risk zone (school) is selected 100, from a menu of the SNC navigation system, in displaying 300 on the ECN navigation screen a virtual signaling gantry PSV, superimposed on the real images IMG of the road scene captured by the CMF camera when the VHL vehicle is geolocated 200 (yes) near the risk zone.

[0056] The process consists of assigning 400 a “red” color to the PSV virtual signaling gantry when the VHL vehicle is geolocated (yes) near the risk zone in specific time slots for frequenting the risk zone and assigning 400 an “orange” color (no) outside these specific time slots.

[0057] The method implements a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to implement the steps of the method described above.

Claims

CLAIMS 1. Motor vehicle (VHL) comprising a navigation system (SNC), a camera (CMF) coupled to the navigation system (CMF), capable of filming a road scene in front of the vehicle (VHL), a navigation screen (ECN) coupled to the navigation system (SNC); said navigation system (SNC) further comprising an augmented reality display module (MRA) capable of displaying at least one alert information on the screen (ECN) when the vehicle (VHL) is geolocated approaching a zone at risk of collision with pedestrians, in the form of a virtual dynamic graphic representation (PSV) superimposed on real images (IMG) captured by the camera (CMF); said augmented reality display module (MRA) being capable of modifying the appearance of the virtual graphic representation according to time slots determined in relation to periods of frequentation of said risk zone.

2. Vehicle (VHL) according to the preceding claim, in which the virtual reality display module (MRA) is capable of automatically controlling the stopping of the display of the graphic representation (PSV) when the navigation system (SNC) has geolocated the vehicle (VHL) outside the risk zone.

3. Vehicle (VHL) according to one of the preceding claims, in which the navigation system (SNC) comprises a navigation menu making it possible to select an option for displaying in augmented reality alert information relating to the proximity of said risk zone, from the navigation screen (ECN).

4. Vehicle (VHL) according to one of the preceding claims, in which the augmented reality display module (MRA) is capable to control the display of a virtual signaling gantry (PSV), superimposed on the real images (IMG) of the road scene, when the navigation system (SNC) geolocates the vehicle (VHL) near the risk zone; said augmented reality display module (MRA) being able to assign a “red” color to the virtual signaling gantry (PSV) when the vehicle (VHL) is geolocated near the risk zone in determined time slots of frequentation of the risk zone, and able to assign an “orange” color to the virtual signaling gantry (PSV) outside these determined time slots.

5. Vehicle (VHL) according to the preceding claim, in which the virtual signaling gantry (PSV) comprises at least one virtual road warning sign (PAS) relating to the type of risk zone, accompanied or not by at least one virtual road signal light (FLS) in flashing or fixed lighting mode.

6. Vehicle (VHL) according to the preceding claim, in which the risk zone contains at least one educational establishment.

7. Vehicle (VHL) according to the preceding claim, in which the navigation system (SNC) cooperates with a cartographic database (CAR) listing at least said educational establishment and the time slots of attendance of said establishment; said time slots being used by the augmented reality display module (MRA) to adapt the color of the virtual signaling gantry (PSV) according to said time slots.

8. Motor vehicle (VHL) according to one of the preceding claims, further comprising an infotainment system (SID) capable of providing the augmented reality display module (MRA) information on specific time slots of attendance in relation to the risk zone; said information being able to be updated via the airwaves.

9. Method for managing the display of a graphic representation of alert information in a vehicle (VHL) according to the preceding claims, said method implemented by the augmented reality display module (MRA) when the option of displaying in augmented reality alert information relating to the proximity of said risk zone is selected (100), consisting of displaying (300) on the navigation screen (ECN) a virtual signaling gantry (PSV), superimposing the real images (IMG) of the road scene captured by the camera (CMF) when the vehicle (VHL) is geolocated (200) near the risk zone, and assigning (400) a “red” color to the virtual signaling gantry (PSV) when the vehicle (VHL) is geolocated near the risk zone in determined time slots of frequentation of the risk zone and assigning an “orange” color outside these determined time slots.

10. Computer program product comprising instructions which, when the program is executed by a computer, cause the latter to implement the method according to claim 9.

Images