Display method in an augmented reality system of a motor vehicle

Abstract

A method for displaying information in a motor vehicle augmented reality system, said augmented reality system having a predetermined field of view, and said method comprises the steps of: determining (601) the relative position of a virtual object to the augmented reality system; if the determined position is such that the object is within the field of view, displaying (605) by the augmented reality system an image representing the virtual object in its relative position; otherwise, displaying (607) by the augmented reality system a pictogram indicating the direction in which the virtual object is located. A motor vehicle augmented reality system and a motor vehicle comprising the system are also described. Figure to be published with the abstract: Fig 6

Description

Title of the invention: Display method in an augmented reality system of a motor vehicle technical field

[0001] The present invention relates to a display method in an augmented reality system of a motor vehicle, a computer program product, an augmented reality system of a motor vehicle and a motor vehicle comprising such a system. State of the art

[0002] In order to enable a driver of a motor vehicle to be informed without his eyes leaving the road, more and more motor vehicle models include an augmented reality display system or augmented reality system (the two expressions will be used interchangeably in this text to refer to the same system).

[0003] These systems include a display and a calculator associated with a memory.

[0004] The display can be a head-up display (HUD), a technology originating from the aeronautics industry. Through various mechanisms, it projects information at windshield height. Another technology used involves the driver wearing special glasses.

[0005] In all cases, information is projected as an overlay onto the real-world view. The distinguishing feature of augmented reality, compared to a simple overlay display, is that the information is positioned to visually overlap with the relevant external real-world element. For example, assuming the displayed information is directional information in a route guidance application, the direction arrow will visually position itself on the lane the vehicle should take. Another example is given by US patent application US2021 / 0088351, which describes how an augmented reality system allows a driver to pick up a waiting passenger.

[0006] However, augmented reality systems as described are not suitable for signaling events or objects that are outside their field of vision.

[0007] There is therefore a real need for a method and system for augmented reality in a motor vehicle that resolves all or part of the aforementioned drawbacks. Description of the invention

[0008] To resolve one or more of the aforementioned drawbacks, according to a first embodiment, a display method in a reality system augmented by a motor vehicle, the augmented reality system having a predetermined field of view, and the process includes the steps of: • Determining the relative position of a virtual object to the augmented reality system; • if the determined position is such that the object is in the field of vision, the augmented reality system displays an image representing the virtual object in its relative position; • otherwise, the augmented reality system displays a pictogram indicating the direction in which the virtual object is located.

[0009] Thus, the user of the system can locate the position of the virtual object even when it is not in the field of vision of the augmented reality system.

[0010] Specific features or embodiments, usable alone or in combination, are: • the pictogram is displayed on the edge of the augmented reality system's field of vision corresponding to the relative position of the virtual object; • the pictogram is displayed in the center of the augmented reality system's field of vision and includes an arrow indicating the side of the virtual object's relative position; • The display of the pictogram is accompanied or replaced by the emission of a spatialized sound originating virtually from the relative position of the virtual object; and / or • The pictogram varies depending on the type of virtual object.

[0011] In a second embodiment, a computer program product includes program code instructions for implementing the above process when the program product is executed on a computer.

[0012] In a third embodiment, an augmented reality system for a motor vehicle includes a display having a predetermined field of view and a computer associated with a memory configured to implement the method described above.

[0013] In a fourth embodiment, a motor vehicle includes a system according to the third embodiment. Brief description of the figures

[0014] The invention will be better understood upon reading the following description, given solely by way of example, and with reference to the figures in the appendix in which: • [Fig.1] represents a side view of a vehicle comprising an augmented reality system according to one embodiment; • [Fig.2] represents a top view of a first driving situation illustrative; • [Fig.3] represents an augmented reality view of the situation in [Fig.2]; • [Fig. 4] represents a top view of a second driving situation illustrative; • [Fig.5] represents an augmented reality view of the situation in [Fig.4]; • [Fig. 6] represents a flowchart of a display method for the system of augmented reality of the [Fig. 1]; and • [Fig.7] represents a flowchart of the display process according to a mode of more detailed execution. Methods of implementation

[0015] The embodiments presented below refer to a motor vehicle, a car. However, those skilled in the art understand that they are also applicable to other types of vehicles such as vans, trucks, etc.

[0016] The terms "front", "rear", "top", "bottom", "transverse" are understood in relation to the vehicle.

[0017] With reference to [Fig. 1], a motor vehicle 101 includes an augmented reality system 103. The latter consists of a pair of glasses 105 connected to a computer 107. In this embodiment, the pair of glasses 105 is the display for the augmented reality system 103.

[0018] The computer 107 is also connected to a GNSS antenna 109 (“Geolocation and Navigation by a Satellite System”) whose function is to provide the geographical position, or geopositioning, and speed of the vehicle 101 as well as the time at any time.

[0019] In addition, the computer 107 is connected to a device 111 for tracking the position and orientation of the pair of glasses 105. Thus, the computer 107 knows at all times the direction of the gaze of the driver wearing the pair of glasses 105 and can therefore determine his field of vision and the virtual objects to be displayed.

[0020] The calculator 107 is associated with a memory (not shown) containing software adapted to implement the process described below in relation to [Fig.2] to 7.

[0021] In order to give a concrete example of application of the embodiment, it is assumed that the vehicle 101 is in competition with a virtual vehicle 201.

[0022] Fig. 2 represents a top view of a situation in which vehicle 101 arrives at the entrance to a right-hand bend of lane 203.

[0023] Fig. 3 represents this situation as seen from the driver wearing the augmented reality glasses 105.

[0024] In order to alert the driver, the augmented reality system 103 displays over the view of the lane in the bend arrows 205 alerting about the bend as well as a braking zone 207.

[0025] Moreover, in this situation, the virtual vehicle 201 is in the field of vision of the glasses 105, field of vision represented in [Fig.2] by the dotted lines 209.

[0026] Thus, the vehicle 201 appears as such, in the form of an image, in the augmented reality vision presented to the driver.

[0027] Fig. 4 and Fig. 5 represent the same situation as before, but the virtual vehicle 201 is located outside the field of vision 209.

[0028] Consequently, the virtual vehicle 201 cannot appear as such in the field of vision 209 under penalty of misleading the driver by not being represented in its true position.

[0029] Also, in [Fig.5], the virtual vehicle 201 is replaced by a pictogram in the form of an arrow 501 oriented so as to indicate to the driver the direction in which the virtual vehicle 201 is located.

[0030] The display of [Fig.3] and [Fig.5] is carried out according to the following process, implemented by system 103, [Fig.6].

[0031] In a step 601, the relative position of the virtual car 201 is determined in relation to the vehicle 101 and more particularly the augmented reality system 103.

[0032] In step 603, this relative position is compared with respect to the field of vision 209.

[0033] If the determined position is such that the vehicle 201 is within the field of In this vision, the virtual object is displayed, step 605, by the augmented reality system 103 as an image representing the virtual object in its relative position. In this case, the image represents a vehicle seen from a three-quarter rear view.

[0034] Otherwise, system 103 displays, at step 607, a pictogram indicating the direction in which the virtual object is located. In our example, the pictogram corresponds to arrow 501.

[0035] A more detailed embodiment of the process is now described in relation to [Fig.7].

[0036] The process is to display the virtual objects in near real time, the steps are repeated for each time step, step 701.

[0037] Furthermore, since the process is applicable when there are multiple virtual objects, the following steps are repeated, step 703, for each virtual object in the list of virtual objects to be represented. It should be noted that this list can itself evolve over time.

[0038] In step 705, the system determines the distance between the virtual object and the augmented reality system. If this distance is greater than a predetermined distance, the virtual object is not considered and the process moves on to the next object in the list.

[0039] If this distance is less than the predetermined distance, the system determines, in step 707, the spatial area visible through the glasses, i.e., the system's field of vision. For this purpose, the system uses in particular the device 111 for determining the position and orientation of the glasses.

[0040] In step 709, the relative position of the object is compared to the field of vision thus calculated.

[0041] If the object is in the field of vision, it is displayed, step 711, at the location calculated by its relative position, and then the process moves on to the next virtual object.

[0042] If the object is not in the field of vision, a projection of its position in the field of vision is calculated, step 713, in order to determine which side of the field of vision the virtual object is on.

[0043] Then at step 715, the system displays the visual element indicating the direction of the position of the virtual object. The process then proceeds to the next virtual object.

[0044] Figure 1 illustrates a system according to certain embodiments. The breakdown presented is for pedagogical purposes to highlight the different functions. However, it is understood that each block can be implemented using different means or combinations thereof, such as hardware components, software, one or more computers, and / or electronic circuits. Each component may include at least one computer or a control unit. At least one memory may be included in each component. The memory may include computer program instructions or software code.

[0045] The computers can be implemented by any type of data processing device, such as a central processing unit, a signal processing unit, a specific application integrated circuit, a programmable gate network, etc. The computers can be implemented in the form of a single controller, or a plurality of controllers or computers.

[0046] The different modules are connected to each other by data links adapted to the environment. These can be wired or wireless.

[0047] For the software, the implementation may comprise modules or units distributed in the form of procedures, functions, etc. The memories may be any type of storage circuit. They may be part of the processor circuit, or separate from it and connected via electrical data links. These may be non-volatile memories, hard drives, RAM, flash memory, etc.

[0048] The software product can be downloaded from a communication network and / or stored on a computer-readable medium. It can be directly executable by a processor or be in the form of a high-level language requiring one or more intermediate operations to be executable.

[0049] Thus, the program instructions stored in memory and processed by the computers can be any type of program code, for example, a compiled or interpreted program written in a suitable programming language.

[0050] The computer program instructions stored in memory are such that, when executed by the computer, the latter carries out one or more of the steps of the processes described above.

[0051] The invention has been illustrated and described in detail in the drawings and the preceding description. This description is to be considered illustrative and given by way of example and not as limiting the invention to this single description. Numerous embodiments are possible.

[0052] In a first variant, the pictogram in the form of an arrow can be replaced or supplemented by positioning the pictogram on the side where the virtual object is located.

[0053] In a second variant, the display of the pictogram is accompanied or replaced by the emission of a spatialized sound also giving the user an indication of the direction of the virtual object.

[0054] In a third variant, the pictogram varies according to the type of virtual object.

[0055] In a fourth variant, the virtual object may be composed of a group of virtual objects, or even a group of groups of virtual objects. Within a group, the virtual objects are treated with respect to their relative positions. In particular, it is possible to hide virtual objects that are concealed by other virtual objects in the group.

Claims

Demands

1. A method for displaying in an augmented reality system of a motor vehicle, said augmented reality system having a predetermined field of view, and said method comprising the steps of: • determining (601) the position relative to the augmented reality system of a virtual object; • if the determined position is such that the object is in the field of view, displaying (605) by the augmented reality system an image representing the virtual object in its relative position; • otherwise, displaying (607) by the augmented reality system a pictogram indicating the direction in which the virtual object is located.

2. A method according to claim 1, wherein the pictogram is displayed on the edge of the field of vision of the augmented reality system corresponding to the relative position of the virtual object.

3. A method according to claim 2, wherein the pictogram is displayed in the center of the field of view of the augmented reality system and includes an arrow indicating the side of the relative position of the virtual object.

4. Method according to claim 1, 2 or 3, wherein the display of the pictogram is accompanied by the emission of a spatialized sound originating virtually from the relative position of the virtual object.

5. A method according to claim 1, 2 or 3, wherein the display of the pictogram is replaced by the emission of a spatialized sound originating virtually from the relative position of the virtual object.

6. A method according to any one of the preceding claims, wherein the pictogram varies according to the type of virtual object.

7. Computer program product characterized in that it includes program code instructions for implementing the method according to any one of claims 1 to 6 when the program product is executed on a computer.

8. A motor vehicle augmented reality system (103) comprising a display (105) having a predetermined field of view and a computer (107) associated with a memory

9. configured to implement the method according to any one of claims 1 to 6. Motor vehicle (101) comprising a system according to claim 8.

Images