Method for rendering virtual elements

Abstract

The invention relates to a method for presenting a virtual element (10) in a display area (12) of at least one display device of a vehicle. The method is described in which a virtual element located outside the display area (12) of a display device (14) is perceived by the driver at a certain distance or direction estimate by signaling the driver of the vehicle if the determined three-dimensional coordinates of at least one virtual element (10) are located outside the display area (12) of the display device (14).

Description

Technical Field

[0001] The present invention relates to a method for presenting virtual elements in a display area of ​​at least one display device of a vehicle, wherein a three-dimensional space is presented in the display area of ​​the display device, wherein three-dimensional coordinates for positioning at least one virtual element are determined based on at least one data source, wherein if the three-dimensional coordinates of the at least one virtual element in the presented three-dimensional space are located within the display area of ​​the display device, the virtual element is transformed as a two-dimensional image into the three-dimensional space presented in the display area of ​​the display device, and wherein the at least one virtual element is presented in the display area of ​​the display device in a perspective-accurate manner relative to the three-dimensional space in the field of view of the driver. Background Art

[0002] With the continuous advancement of virtual and augmented reality technologies and applications, these technologies and applications are also entering the automotive industry. Augmented reality (AR) enriches the real world with virtual elements that are accurately registered or positioned in three-dimensional space and allow real-time interaction. Head-up displays (HUDs) provide a corresponding technical solution for enriching the cockpit with perspective-accurate virtual augmentation.

[0003] In the case of a head-up display, a virtual display is created using an imaging unit integrated into the instrument panel, such as a TFT display. This image is deflected toward the windshield via multiple mirrors, where light is reflected into the driver's eyes. The driver perceives this image as a virtual display in their field of vision. In the design of such systems, the area into which the light is reflected is spatially limited to achieve higher brightness with less light scattering. This area is referred to as the "EyeBox" because the driver's field of vision must be within it in order to perceive the virtual image.

[0004] Due to the relatively narrow field of view, it may not be possible to present virtual elements perspectively correctly, since the virtual objects must be positioned outside the “EyeBox.” Different efforts have been made to display objects that are not within the driver's field of view.

[0005] DE 10 2017 211 244 B3, for example, discloses a method for enriching a vehicle driver's field of view with additional information. For this purpose, it is provided that if a vehicle actively flashes its lights to indicate an intention to change lanes, the vehicle is typically displayed in a flashing manner in the driver's field of view.

[0006] The disadvantage of the above-mentioned prior art is that the driver does not feel the actual distance to the important objects outside the field of vision. Summary of the Invention

[0007] The object of the present invention is therefore to specify a method in which virtual elements located outside the display area of ​​a display device can be perceived by the driver with a certain distance or direction estimate.

[0008] The present invention achieves this problem by signaling the vehicle driver if the determined three-dimensional coordinates of the at least one virtual element are outside the display area of ​​the display device. This notifies the vehicle driver that an object important to the driver is outside the driver's field of view. The signaling can be visual, but can also be acoustic, for example.

[0009] The display area of ​​the display device can be a display in the center area or in the cockpit area, for example, in the vehicle's instrument cluster. Alternatively or additionally, the display area of ​​the display device can be designed as a head-up display. A head-up display is understood to be a display area in which the driver can maintain their head position or gaze direction because information is projected into their field of vision, for example, onto the vehicle's windshield.

[0010] The presented three-dimensional space can preferably describe an area outside the vehicle in the direction of travel. In the case of a head-up display, the display area or three-dimensional space of the display device can be equated with the field of view through the windshield. This three-dimensional space is thus the space in front of the vehicle within the field of view of the driver or front passenger. The corresponding three-dimensional coordinates calculated for the virtual elements can be, for example, Cartesian coordinates.

[0011] The data sources required for calculating these coordinates can be of different types. Preferably, these data sources are vehicle data or navigation data, which are recorded by a positioning sensor device of the vehicle, such as a GPS, a rotation speed sensor or a camera.

[0012] When calculating the three-dimensional coordinates of a virtual element, it is determined whether the element can be presented perspectively correctly within the driver's field of view. If, on the other hand, this is not the case because the three-dimensional coordinates lie outside the field of view, the driver is informed that the virtual element is not within the display area of ​​the display device or within his field of view.

[0013] Virtual elements can be various graphic elements and / or buttons that are displayed within the display area of ​​the display device. Virtual elements can be linked to vehicle data and / or depend on the vehicle's driving characteristics. This data can, for example, come from existing driver assistance systems. However, it is also conceivable that the displayed virtual elements are not relevant to the vehicle's driving characteristics.

[0014] Further preferred embodiments of the invention result from the remaining features mentioned in the dependent claims.

[0015] In a first embodiment of the present invention, the three-dimensional coordinates of the virtual element are determined based on a distance measurement to the vehicle ahead. This allows the use of data from so-called Adaptive Cruise Control (ACC). With this function, the user can choose between, for example, five levels, each resulting in a different distance to the vehicle ahead.

[0016] It is conceivable that markings are projected into the driver's field of view as virtual elements for contact simulation, which simplify the estimation of spatial distances. If such markings are no longer within the driver's field of view, for example because the distance is too great for a perspective-correct representation of such markings within the display area of ​​the display device, the method according to the present invention indicates this to the driver.

[0017] Alternatively or additionally, another embodiment of the method according to the present invention may provide for determining the three-dimensional coordinates of the virtual element based on the time interval to the vehicle ahead. Using distance control, the driver of the vehicle can select, for example, between five levels, each resulting in a different distance to the vehicle ahead. The term "distance" is used to define each level by a specific time interval to the vehicle ahead, such as 1 second. Based on the actual speed and the distance, a distance to be maintained is calculated and displayed as a contact-simulating line within the display area of ​​the display device. Other representations are also conceivable.

[0018] According to this definition, the line moves perspectively backward or forward (upward or downward in the 2D field of view), so that it may move out of the field of view. Ideally, all selectable levels should be visible in the field of view. However, depending on the speed, individual lines may move out of the field of view, so that the currently active time interval is no longer displayed. Therefore, the method according to the present invention notifies the driver that one of these lines or markings is outside the display area.

[0019] In one advantageous embodiment of the method according to the present invention, provision is made for displaying, in an edge region of the display area of ​​the display device, the three-dimensional coordinates of the at least one virtual element located outside the display area of ​​the display device. This signalizes to the driver that the virtual element cannot be displayed within the display area because it lies outside the display area. However, if this signaling were performed only within the edge region, the driver would be unnecessarily distracted. Consequently, other displayed virtual elements are not superimposed, and the display area of ​​the display device as a whole does not clutter the driver's perception.

[0020] In order to make the method according to the present invention more intuitive, another design scheme stipulates that if the three-dimensional coordinates of the at least one virtual element are so far away in the background perspectively that the two-dimensional image of the virtual element is located above the display area of ​​the display device, then the upper edge area of ​​the display area of ​​the display device presents: the three-dimensional coordinates of the at least one virtual element are located outside the display area of ​​the display device.

[0021] Elements located at a greater distance behind the display area of ​​the display device are shifted upwards in the 2D field of view. This means that the driver is informed by the signaling in the upper edge region that an element has moved into the background or has moved so far to the rear that it can no longer be displayed in the display area of ​​the display device.

[0022] Correspondingly, in another design scheme of the method according to the present invention, it is stipulated that: if the three-dimensional coordinates of the at least one virtual element are so far away in the foreground perspectively that the two-dimensional image of the virtual element is located below the display area of ​​the display device, then the lower edge area of ​​the display area of ​​the display device presents: the three-dimensional coordinates of the at least one virtual element are located outside the display area of ​​the display device.

[0023] This means that the driver is informed by the signaling in the lower edge region that an element has moved into the foreground or has moved so far forward that it can no longer be represented within the display area of ​​the display device.

[0024] Advantageously, another embodiment of the method according to the present invention provides for the signaling to be performed by presenting a virtual signaling element within the display area of ​​the display device. A virtual signaling element that fades into the driver's field of view can simply signal that the actual virtual element is not within the display area of ​​the display device. This virtual signaling element can have a variety of different designs. Importantly, it is suitable for indicating to the driver that other important virtual elements are not presented within the display area of ​​the display device.

[0025] For example, it is conceivable that the virtual signaling element has a graphically simple design so as not to unnecessarily distract the driver. The virtual signaling element can be, for example, a dot or a line. However, it is also conceivable to present text, for example, in the form of the name of a corresponding, unpresented object.

[0026] In another embodiment of the method according to the present invention, the virtual signaling element has the width of an unrepresented virtual element in a perspective-accurate representation for the virtual element. This allows the vehicle driver to roughly estimate how far away the unrepresented virtual element is. Therefore, the perspective-accurate width allows the driver to see how much narrower the virtual signaling element is compared to the nearest presented virtual element, which would be, for example, the distance line. Because the perspective-accurate width, which relates to the actual coordinates of the virtual element, allows the driver to visually expand the perspective and visually integrate the virtual signaling element with the perspective accuracy.

[0027] Correspondingly, another embodiment of the method according to the present invention provides that the virtual signaling element corresponds to a virtual element that is not presented, and that the virtual signaling element has the size of the virtual element in a perspectively accurate presentation. The perspectively accurate size of the virtual signaling element relative to actual three-dimensional coordinates that are not within the display area of ​​the display device makes it easier for the driver to visually expand their perspective and visually integrate the virtual signaling element in a perspectively accurate manner. The driver can thus visually expand his or her field of view.

[0028] Alternatively or additionally, provision can be made for the virtual signaling element to be presented as an icon. The icon can have any size and can be presented accordingly either in the edge region of the display area of ​​the display device or, in the case of relatively small icons, also in the driver's direct field of vision.

[0029] Unless otherwise indicated in individual cases, the different embodiments of the invention mentioned in this application can be advantageously combined with one another. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be described below in the form of an exemplary embodiment with reference to the accompanying drawings.

[0031] Figure 1 A schematic diagram showing a display area of ​​a display device when performing an embodiment of the method according to the present invention;

[0032] Figure 2 A further diagram showing a display area of ​​a display device during the execution of a further exemplary embodiment of the method according to the invention; and

[0033] Figure 3 A schematic diagram of a vehicle is shown having a device for carrying out the method according to the present invention. DETAILED DESCRIPTION

[0034] Figure 1The representation of virtual elements 10 in a display area 12 of a display device 14 is schematically shown. A three-dimensional space 16 is shown in display area 12. Virtual elements 10 are represented in display area 12 by two-dimensional images 18, which are, if possible, perspective-correct. To this end, three-dimensional coordinates for positioning at least one of virtual elements 10 are determined based on at least one data source. These three-dimensional coordinates should indicate where exactly two-dimensional image 18 must be positioned in order to create the most realistic possible impression of two-dimensional image 18 contacting a portion of the surrounding environment that is perceived in a simulated manner, i.e., as being perceived.

[0035] When determining these three-dimensional coordinates, it is possible that, based on the data source, the virtual elements 10 to be displayed must be located outside the display area 12 of the display device 14. For this purpose, it is provided that the driver of the vehicle can be informed of this by a signal if any virtual elements 10 that are important to him or her cannot be displayed in the display area 12. These virtual elements 10 can, for example, be activated functions of a driver assistance system.

[0036] In this embodiment, a virtual signaling element 20 informs the driver that the virtual element 10 associated with the selected function cannot be displayed in the display area 12 of the display device 14 . In the present embodiment, the three-dimensional coordinates of the virtual element 10 are determined based on a distance measurement to the vehicle 22 traveling ahead. This allows the use of data from so-called Adaptive Cruise Control (ACC). With this function, the user can, for example, select between multiple display levels, each indicating a different distance to the vehicle 22 traveling ahead.

[0037] In the present case, the time interval or headway adjustment to the vehicle 22 traveling ahead is determined. According to the term "headway," each level is defined by a specific time interval, such as 1 second, to the vehicle 22 traveling ahead. Based on the actual speed and the headway, the distance to be maintained is calculated, which can be displayed as a contact-simulating line as a virtual element 10 in the display area 12 of the display device 14. Other representations are also conceivable.

[0038] exist Figure 1 It can be seen that these time intervals are projected onto roadway 24 as lines in a manner that is noticeable to the driver. In this exemplary embodiment, three selected lines are displayed to the driver, which depend on the time interval to vehicle 22 traveling ahead. It can be seen that the line closest to the driver in terms of perspective, in the form of the lowest virtual element 10, is not arranged in display area 12 of display device 14. In the method shown, it is recognized that the three-dimensional coordinates of this virtual element 10 are not located in display area 12 of display device 14.

[0039] This is communicated to the driver using a virtual signaling element 20. Virtual signaling element 20 is presented at the lower edge of display area 12 of display device 14 and corresponds to a virtual element 10 whose three-dimensional coordinates lie outside of display area 12. This means that virtual signaling element 20 is not displayed perspectively in display area 12, but rather with the size and characteristics that would otherwise be required for accurate positioning of virtual signaling element 20. In this way, the driver of the vehicle can infer that the virtual element 10 that was originally intended to be presented—in this case, a line that should be displayed at a specific time interval from vehicle 22 traveling ahead—is much closer than it would otherwise be if presented via display device 14.

[0040] That is, by presenting virtual signaling element 20 in the lower edge region of display area 12 of display device 14 , the driver is informed that an element has moved into the foreground or has moved so far forward that it can no longer be presented in display area 12 of display device 14 .

[0041] Similar to Figure 1 , Figure 2 The example shows that these time intervals are projected as lines onto roadway 24 in a manner that is noticeable to the driver. In this exemplary embodiment, three selected lines are displayed to the driver, which depend on the time interval to vehicle 22 traveling ahead. It can be seen that the line, in the form of the uppermost virtual element 10, which is perspectively farthest from the driver, is not arranged in display area 12 of display device 14. In the method shown, it is recognized that the three-dimensional coordinates of this virtual element 10 are not located in display area 12 of display device 14.

[0042] This is communicated to the driver using a virtual signaling element 20. Virtual signaling element 20 is presented at the upper edge of display area 12 of display device 14 and corresponds to a virtual element 10 whose three-dimensional coordinates are located outside of display area 12. This means that virtual signaling element 20 is not displayed perspectively in display area 12, but rather is displayed with the size and characteristics that would have been intended for accurate positioning of virtual signaling element 20. Consequently, the virtual signaling element is presented smaller than it would have been if it had been accurately positioned. In this way, the driver of the vehicle can infer that the virtual element 10 that was originally intended to be presented—in this case, a line that is to be displayed at a specific time interval from vehicle 22 traveling ahead—is much further away than it would have been if it had been presented via display device 14.

[0043] Figure 3Another embodiment of the method or a vehicle having a corresponding device for carrying out the method is shown as an example. A virtual element 10 in the form of navigation instructions is presented in the display area 12 of the display device 14. In this case, the navigation instructions indicating a turn are not presented completely perspectively in the display area 12 of the display device 14. This is also indicated to the vehicle driver in the form of a virtual signaling element 20. In the example shown, the left turn is to be driven, which is not completely perspectively represented by the presented virtual element 10.

[0044] By presenting a virtual signaling element 20 in the left edge region of the display area 12 of the display device 14, the driver of the vehicle is shown that the turn is not yet complete if the driver merely follows the presented virtual element 10. By presenting the virtual signaling element 20 at a size such that the virtual element 10 lies outside the display area 12 of the display device 14 with its three-dimensional coordinates, the driver can estimate how far the turn will continue beyond the presented instructions.

[0045] Reference Signs List

[0046] 10 Virtual Elements

[0047] 12 display area

[0048] 14 Display device

[0049] 16 Three-dimensional space

[0050] 18 2D images

[0051] 20 Virtual Signaling Elements

[0052] 22 Vehicle ahead

[0053] 24 lanes

Claims

1. A method for presenting a virtual element (10) in a display area (12) of at least one display device of a vehicle, wherein a three-dimensional space (16) is presented in the display area (12) of the display device (14), wherein three-dimensional coordinates for positioning the at least one virtual element (10) are determined based on at least one data source, wherein if the three-dimensional coordinates of the at least one virtual element (10) in the presented three-dimensional space (16) are located within the display area of ​​the display device, the virtual element is transformed as a two-dimensional image (18) into the three-dimensional space (16) presented in the display area (12) of the display device (14), and wherein the at least one virtual element (10) is presented perspectively correctly in the display area (12) of the display device (14) in the field of vision of the driver, characterized in that If the determined three-dimensional coordinates of the at least one virtual element (10) are outside the display area (12) of the display device (14), signaling this to the driver of the vehicle, The signaling is achieved by presenting a virtual signaling element (20) in a display area (12) of the display device (14).

2. The method according to claim 1, characterized in that The three-dimensional coordinates of the virtual element (10) are determined based on a distance measurement from a vehicle traveling ahead.

3. The method according to claim 1 or 2, characterized in that The three-dimensional coordinates of the virtual element (10) are determined based on a time interval from a vehicle (22) traveling ahead.

4. The method according to claim 1 or 2, characterized in that In the edge area of ​​the display area (12) of the display device (14), it is presented that the three-dimensional coordinates of the at least one virtual element (10) are located outside the display area (12) of the display device (14).

5. The method according to claim 1 or 2, characterized in that If the three-dimensional coordinates of the at least one virtual element (10) are so far away in the background perspectively that the two-dimensional image (18) of the virtual element (10) is located above the display area (12) of the display device (14), then the upper edge area of ​​the display area (12) of the display device (14) shows that the three-dimensional coordinates of the at least one virtual element (10) are located outside the display area (12) of the display device (14).

6. The method according to claim 1 or 2, characterized in that If the three-dimensional coordinates of the at least one virtual element (10) are so far away in the foreground perspectively that the two-dimensional image (18) of the virtual element (10) is located below the display area (12) of the display device (14), then the lower edge area of ​​the display area (12) of the display device (14) shows that the three-dimensional coordinates of the at least one virtual element (10) are outside the display area (12) of the display device (14).

7. The method according to claim 1 or 2, characterized in that The virtual signaling element (20) has the width of a non-rendered virtual element (10) in a perspectively correct representation of the virtual element (10).

8. The method according to claim 1 or 2, characterized in that The virtual signaling element (20) corresponds to the virtual element (10) that is not presented; and the virtual signaling element (20) has the size of the virtual element (10) in a perspectively accurate presentation form.

9. The method according to claim 1 or 2, characterized in that The virtual signalling element (20) is presented as an icon.

Images