User interface, means of transportation, and method for outputting information to a user of a means of transportation

Abstract

The invention relates to a user interface, to a means of transportation, and to a method for outputting information (1) to a user of a means of transportation. The method comprises the following steps: automatically determining a distance from an environmental object (11) travelling ahead or located ahead and, depending on the distance, automatically outputting the information (1) to the user via a head-up display with augmented reality, hereinafter "AR HUD", wherein the information (1) is positioned on the AR HUD depending on a pheno position of the environmental object (11), the method further comprising automatically determining that a portion of the environmental object (11) is not located within a visible region (5) of the AR HUD and, in response thereto, automatically intensifying a representation of the information (1).

Description

[0001] Description

[0002] User interface, means of transport and method for issuing a notification to a user of a means of transport

[0003] The present invention relates to a user interface, a means of transportation, and a method for displaying a message to a user of a means of transportation. In particular, the present invention relates to an improved and intuitively understandable highlighting of environmental objects by means of a front-view display with augmented reality.

[0004] Modern vehicles are equipped with a multitude of sensors, driver assistance systems, and related display devices. One example is front-view displays that enable augmented reality. In other words, information about an object in the environment can be displayed, which, from the user's / occupant's perspective, appears virtually "attached" to that object—a phenomenon known as contact analog. Put another way, the environment can be modified by displays using an AR-HUD (Augmented Reality Head-Up Display) through optical cues, such that artificial cues or artifacts are displayed in such a way that they track the pheno-position of an object in the environment. For the user, this results in a display in which the cues appear to be positionally linked to the object in the environment.

[0005] A range of environmental sensors enables vehicles to determine distances to surrounding objects. Examples include ultrasound-based sensors, lidar, radar, optical cameras, 2D / 3D cameras, etc., which allow for the detection and measurement of the environment.

[0006] US 2020 / 0183157 A1 discloses a display device for a means of transportation which can display a sign virtually embedded in the road surface. For example, navigation instructions are displayed virtually embedded in the road surface.

[0007] US 2018 / 0261014 A1 discloses a method for displaying information in a means of transportation in which the size, display interval or color of the displayed object can be adjusted depending on the distance between a foreign vehicle and an ego vehicle.

[0008] Based on the aforementioned prior art, it is an object of the present invention to present distance-dependent information to an occupant of a means of transport in an intuitively understandable manner. This object is achieved according to the invention by a method with the features of claim 1, a user interface with the features of claim 11, and a means of transport with the features of claim 12. The dependent claims describe preferred embodiments of the invention.

[0009] This procedure is used to issue a warning to a user of a means of transport. The means of transport can be, for example, a car, van, truck, aircraft, and / or watercraft. The user can be, for example, the driver of the means of transport, a passenger / co-pilot, or another occupant. The warning can be displayed as a visual indicator.

[0010] In a first step, the distance to a moving or stationary object in the environment (depending on its state of motion) is determined. This can be done using sensors. In other words, the vehicle's environmental sensors determine the distance to the object. Based on this distance, a notification is then displayed to the user via a front-view display using augmented reality.

[0011] The front view display is hereinafter referred to as "AR-HUD". It consists of a light projection which is transferred from the projector to the windshield by means of mirrors in such a way that a display is created in the area of ​​the windshield through which the vehicle occupant, usually the driver, looks at the road.

[0012] Due to the vehicle's design, the area in which the front view display can be shown is spatially limited, i.e., it only covers a portion of the windshield, and is hereinafter referred to as the visible area of ​​the front view display. Other terms for the visible area known to those skilled in the art include field of view (FOV), viewing area, etc.

[0013] Preferably, this visible area is adjustable to the user; that is, depending on the size, a vertical orientation can be adjusted so that the field of view is not always located in the same position on the windshield. Horizontal swiveling is also known and possible. This visible area is usually not discernible, for example, by a fixed frame; that is, a display element of the front view display moving out of the visible area would simply disappear abruptly. Users are accustomed to a normal display where, if a display element moves at the visible edge, it simply moves out of the picture behind the screen boundary. No change in the display is required for this. However, the situation is different with a front view display according to the invention, where a response to this situation is required.

[0014] To accomplish this task, the cue is positioned on the AR-HUD based on the pheno-position of the surrounding object. The pheno-position of the surrounding object, as viewed on the AR-HUD, is defined as the position at which the object's appearance, relative to the user's / occupant's eye position, intersects the front view display or the windshield of the vehicle. In other words, the optical axis between the user's focus and the surrounding object lies at the object's pheno-position within the front view display. Put another way, the optical axis intersects the front view display at the object's pheno-position.

[0015] Furthermore, it is determined that a portion of the surrounding object is not within the visible area of ​​the AR-HUD. In other words, a section of the surrounding object is located in such a way that the optical axis between the user's focus and that portion of the object lies outside the visible area of ​​the front-view display. Therefore, direct or contact-like highlighting of this portion of the surrounding object cannot be achieved via the front-view display, as the portion has moved out of the front-view display in the Y and / or X directions. The surrounding object can be tracked using an environmental sensor on the vehicle, and the aforementioned fact can be determined based on geometric characteristics. Upon this, the display of the warning is intensified. The warning is still displayed via the AR-HUD.However, since the migrated part of the environment object can no longer be directly or contact-analogously highlighted by the indicator, the indicator is continuously displayed in an enhanced form. For example, the Y-position of the part of the environment object can be retained if the environment object has migrated out of the AR-HUD's area in the X-direction, and / or the X-coordinate of the part of the environment object can be retained for the indicator if the environment object has migrated out of the AR-HUD in the Y-direction. This does not preclude the indicator from initially being displayed in an area on the AR-HUD located at the position between the edge of the AR-HUD and the pheno-position of the environment object beyond which the environment object migrates out of the AR-HUD.In other words, the indicator virtually jumps to the opposite side of the pheno-position of the surrounding object, which remains within the range of the AR HUD. Other movements are also possible instead of the display jumping.

[0016] Intensifying the display of the cue can therefore be accompanied by a change in the cue's location and / or an adjustment of its light intensity and / or color. By virtually moving the surrounding object out of the AR-HUD, thus reducing the cue's connection to the surrounding object or the part of the object that has moved out, intensifying the cue's display can at least partially compensate for the visual connection to the surrounding object and the visual comprehensibility of the connection between the cue and the part of the surrounding object. In this way, it can be avoided that the user does not receive intuitive and quickly understandable support regarding the surrounding object when surrounding objects move out of the frame.

[0017] Preferably, the position of the surrounding object can be determined using sensor technology (e.g., optical and / or ultrasound-based and / or lidar and / or laser-based) in the vicinity of the vehicle, and from this, it can be inferred that a part of the surrounding object has moved out of view. For example, a lower part of the surrounding object (unlike an upper part) may no longer be within the visible area of ​​the AR HUD. In response, the display of the notification above the object's pheno-position can be intensified. In other words, the notification is assigned to the area of ​​the pheno-position that remains within the AR HUD. A notification previously displayed at this position may thus have a higher intensity, a larger screen area, greater dynamics, a changed color, or similar features.This is done to adjust the display and compensate for the previously diminishing relationship between the environment object and the hint. The same applies if an upper part of the environment object (as opposed to a lower part) is no longer within the visible area of ​​the AR-HUD. In response, the hint's display below the environment object's phenoscopic position can be enhanced. Similarly, it can be determined if a left part of the environment object (as opposed to a right part) is no longer within the user's visible area of ​​the AR-HUD. In response, the hint's display to the right of the environment object's phenoscopic position can be enhanced, or a hint can be displayed at that position for the first time.

[0018] Accordingly, it can be determined that a right-hand part of the environment object (as opposed to a left-hand part) is no longer within the visible area of ​​the AR-HUD from the user's perspective. In response, the display is supplemented with a message to the left of the environment object's pheno-position, or an existing message is amplified. This ensures the strongest possible visual relationship between the outgoing environment object and the message displayed by the AR-HUD.

[0019] The warning can be issued in response to a predefined distance being detected, particularly in the form of or as part of a driver takeover request, a predefined threshold, a takeover request, or similar. For example, a parameter of the warning can be varied essentially linearly or in steps depending on the distance determined by the sensors. For instance, the width, height, light intensity, color, or animation of the warning can be adjusted based on the detected distance.

[0020] Depending on the type of indicator regarding the surrounding object, different colors, animation styles, and heights / widths of the indicator can be selected. This does not preclude the semantics or symbolism from also being adapted to the current situation in order to dynamically reflect the changing relevance of the surrounding object or other circumstances. For example, the visual indicator can be supplemented or replaced with a further, and in particular more prominent, indicator when responding to the determination of a predefined distance. For example, an exclamation mark can be added to an existing indicator / marker when a predefined threshold is exceeded or fallen below. The second indicator can also be repositioned according to the invention if a part of the surrounding object threatens to move out of, or has already moved out of, the display area of ​​the AR-HUD.

[0021] The visual display of the message on the AR-HlID does not preclude the possibility that the message and / or further information may include an acoustic, static, and / or haptic component. In particular, the message or further information may be augmented by a component displayed on a conventional screen. Specifically, the additional display unit is not necessarily capable of producing contact-like displays.

[0022] The indication can be changed quantitatively and / or qualitatively depending on the distance, so that, for example, a color, an intensity, a numerical value inherent in the indication, a bar or its length and / or width visually encodes the distance in the indication. The same applies to any further indication provided, to be displayed in accordance with the invention.

[0023] Preferably, a bar-shaped indicator can illustrate the current distance to an object in the environment by linearly affecting a dimension of the bar. Preferably, the length of the bar depends linearly on the distance between the means of transport and the object in the environment and is adjusted essentially continuously.

[0024] In particular, it can be provided that the vehicle's sensors detect the color and / or color gradient of the surrounding object. Depending on the color and / or gradient, a color or gradient can be automatically selected and used to display the message. A complementary color can be used to ensure optimal visibility. In other words, a color contrasting with the actual color of the surrounding object can be used to visually differentiate the message from the object's appearance. Alternatively, an identical or nearly identical color can be chosen for the message. This results in a particularly homogeneous and, if desired, visually appealing appearance for the message.the contact-analogous augmented display according to the invention. According to a second aspect of the present invention, a user interface for outputting a message to a user of a means of transportation is proposed. The user interface can be designed as a driver assistance system. It can have a data input, an evaluation unit (for example, a microprocessor, a programmable processor, etc.), and a data output.

[0025] The evaluation unit is configured to determine the distance to a moving or stationary object in the environment via its data input. Depending on this distance, a notification can be displayed to the user via an AR-HUD using the data output. The positioning of the notification on the AR-HUD can be determined based on the pheno-position of a part of the object and, optionally, on the user's eye position. The user interface can utilize optical cameras and / or environmental sensors to measure the geometric relationships. These sensors can also be used for other driver assistance systems, thereby minimizing or even completely eliminating the hardware required for implementing the present invention.The user interface is configured via the data input to identify a portion of the environment object as being outside the visible area of ​​the AR-HUD and, by means of the data output, to enhance the display of the indication accordingly. In this way, the user interface according to the invention can implement the same features, feature combinations, and the resulting advantages of the method according to the invention in such a clear manner that, to avoid repetition, reference is made to the above explanations.

[0026] According to a third aspect of the present invention, a means of transport (car, truck, van, road vehicle, motorcycle, aircraft and / or watercraft) is proposed which has a user interface according to the second aspect of the invention, so that a method according to the first aspect of the invention can be carried out and the features, combinations of features and advantages also result accordingly with regard to the means of transport.

[0027] Brief description of the characters

[0028] Further details, advantages and features of the present invention will become apparent from the following description of exemplary embodiments with reference to the drawing. Figure 1 shows a schematic representation of an exemplary embodiment of a means of locomotion according to the invention, together with an exemplary embodiment of a user interface according to the invention.

[0029] Fig. 2 shows a schematic representation of a front-view display screen usable according to the invention for an embodiment of a user interface according to the invention, below which a static display is located;

[0030] Fig. 3 shows a schematic representation of an AR-HUD display view in a first driving situation;

[0031] Fig. 4 shows a schematic representation of an AR-HUD display view in a second driving situation;

[0032] Fig. 5 shows a schematic representation of an AR-HUD display view in a third driving situation, where the indicator has optically detached itself from the pheno-position of the surrounding object;

[0033] Fig. 6 shows a schematic representation of an AR-HUD display view in a fourth driving situation, where the indicator previously located below the pheno-position of the surrounding object now overlays the pheno-position of the surrounding object; and

[0034] Fig. 7 shows a flowchart illustrating the steps of an embodiment of a method according to the invention for issuing a notification to a user of a means of transportation.

[0035] Fig. 1 shows a passenger car as an embodiment of a means of transport 10 according to the invention, in which a user 2 is seated in a driver's seat. The user 2 is responsible for the longitudinal and lateral control of the means of transport 10 as the driver. In this capacity, the user is supported according to the invention by a projector 4 that can project optical cues via mirrors into the user 2's primary field of vision on the windshield 7. The visible area 5 designates the area through which the pheno-positions of cues can be virtually represented only within a region bounded by a lower line of sight 61 and an upper line of sight 62. The optical axis 6 of the user 2's focus lies within this area, allowing them to perceive surrounding objects (not shown) located within the visible area.

[0036] An electronic control unit (ECU) is connected to the electronic control unit (ECU) via a touchscreen 9, acting as an evaluation unit 12. For this purpose, the touchscreen is connected to the ECU via a data input 13. A data output 14 of the ECU is connected to the projector 4. In this way, the user 2 can adjust the visual display to their preferences and configure how many indicators should be assigned to a given object in the environment, depending on specific environmental conditions or driving situations, and how these indicators should be displayed visually, audibly, or haptically.

[0037] Fig. 2 shows a view of the front view display as presented to the user in Fig. 1, without surrounding objects, visual cues, or other elements. Instead, the visible area 5, positioned on the windshield 7, is defined by its upper edge 52 and its lower edge 51, which are not visible to the user. Below the visible area 5, as well as the static display located at a distance from the visible area 5, a further visible area 8 is discernible. This exemplary configuration of static and non-static front view displays allows for the display of status elements on the one hand, and display elements positioned in the front view display in a contact-analogous manner, i.e., with precise positioning relative to a surrounding object.

[0038] Fig. 3 shows a possible front view display (AR-HUD) for the user, as shown in Fig. 1, when an environmental object 11 is completely within the visible area 5 such that sufficient space is provided below the environmental object 11 for the contact-analog display of an optical indicator 1. The optical indicator 1 comprises a horizontal bar whose length and thickness are determined by the current distance of the environmental object 11 from the Ego vehicle (not shown). Above the bar, an exclamation mark is displayed as a second indicator, which signals when a predefined (critical) distance to the environmental object 11 has been breached. A collision warning is also displayed in the visible area 8 of the static area.

[0039] Fig. 4 shows the display depicted in Fig. 3 after the ambient object 11 has once again approached the ego-mobility device (not shown). The bar in the indicator 1 has increased in width and length, as has the exclamation mark. Fig. 5 shows a representation according to the invention, which is displayed after a portion or the entire ambient object 11 has moved out of the visible area 5. The ambient object 11 is a schematic representation of the vehicle actually present to the (not shown) user. Since this representation is located entirely above the upper edge 52 of the visible area 5, an indicator 1 is displayed directly at the upper edge 52, the Y-position of which corresponds to the Y-position of the ambient object 11. Furthermore, the Y-position of the optical indicator 1 is continuously adjusted to the actual Y-position of the ambient object 11.In the illustration, clue 1 comprises a corona with a horizontal top and a convex bottom. In other words, clue 1, which is shown as sharply outlined in Figures 3 and 4, has given way to a comparatively soft, yet brightly lit corona. Due to space constraints above the corona, the second clue is shown below it in the form of an exclamation mark.

[0040] Fig. 6 shows a view from the user's perspective in an ego-mobility device designed according to the invention, after the surrounding object 11 has sunk significantly below a lower edge 51 of the visible area 5, or the eye position of the user (not shown) has dropped so that the lower edge 51 is higher than the lower part of the surrounding object 11 or its pheno-position. To still be able to provide the user with a warning 1 about the critical distance, the optical warning 1 is superimposed on the pheno-position of the surrounding object 11. In particular, in response to the need to display the warning 1 superimposed on the surrounding object 11, the intensity and / or the area of ​​the warning and / or an animation of the warning 1 can be automatically modified to adapt the user's perception or the warning effect of the warning 1 to the potentially less favorable contrast situation.

[0041] Fig. 7 shows steps of an embodiment of a method according to the invention for issuing a notification to a user of a means of transportation. In step 100, a distance to a moving or stationary environmental object is determined. For this purpose, the ego-mobility device can evaluate environmental sensors and / or wirelessly received messages. Depending on the distance, a notification is issued to the user via an AR-HUD in step 200, whereby the positioning of the notification on the AR-HUD depends on a pheno-position of the environmental object. In addition, the user's eye position can be estimated and / or determined by sensors and / or determined from a setting of the AR-HUD. In step 300, it is then determined that a part of the environmental object is no longer within the visible area of ​​the AR-HUD from the user's perspective.In response, the display of the indicator is visually enhanced in step 400. For this purpose, the optical extent and / or light intensity of the indicator can be automatically adjusted / enhanced using technical means. Finally, in step 500, a bar is visually displayed below the pheno-position of the surrounding object, the length and width of which are linearly related to the sensor-measured distance from the surrounding object.

[0042] The result is a proposed operating mode that is intuitively understandable and meets the technical restrictions of an AR-HUD display device.

[0043] Reference symbol list

[0044] Notice

[0045] user

[0046] AR-HUD (Front View Display with Augmented Reality)

[0047] Projector visible area of ​​the front view display optical axis

[0048] Windscreen visible area of ​​the static front view display

[0049] Touchscreen

[0050] Ego means of transportation

[0051] Surrounding object

[0052] Evaluation unit

[0053] Data input

[0054] Data output lower edge upper edge lower viewing ray upper viewing ray -500 process steps

Claims

Patent claims 1. A method for issuing a notification (1) to a user (2) of a means of transport (10) comprising the steps of: automatically determining (100) a distance to a forward-moving or forward-lying environmental object (11) and, depending on the distance, automatically issuing (200) the notification (1) to the user (2) via a front-view display (3) with augmented reality, hereinafter referred to as "AR HUD (3)", wherein the positioning of the notification (1) on the AR-HUD (3) is dependent on a pheno-position of the environmental object (11), further comprising automatically determining (300) that a part of the environmental object (11) is not located within a visible area (5) of the AR-HUD (3) and, in response thereto, automatically amplifying (400) a representation of the notification (1).

2. The method according to claim 1 further comprising automatically determining that a lower part of the environment object (11) is not located within the visible area (5) of the AR-HUD (3) and, in response thereto, automatically amplifying a display of the hint (1) above the pheno-position of the environment object (11), or automatically determining that an upper part of the environment object (11) is not located within the visible area (5) of the AR-HUD (3) and, in response thereto, automatically amplifying a display of the hint below the pheno-position of the environment object (11), and / or automatically determining that a left part of the environment object (11) is not located within the visible area (5) of the AR-HUD (3) and, in response thereto, automatically amplifying a display of the hint to the right of the pheno-position of the environment object (11), or automatically determiningthat a right part of the surrounding object (11) is not located within the visible area (5) of the AR-HUD (3) and in the, Responding to this automatically amplifies the display of the hint to the left of the pheno-position of the environment object (11).

3. Method according to claim 1 or 2, wherein the indication (1) is issued in response to an automatic determination of a predefined distance, in particular in the form of or as part of a driver takeover request.

4. Method according to claim 3, wherein the reference (1) in response to the determination of the predefined distance is supplemented or replaced by a further reference.

5. Method according to claim 4, wherein the reference (1) and / or the further reference comprises an acoustic and / or static component.

6. Method according to any of the preceding claims, wherein the reference (1) encodes the distance quantitatively and / or qualitatively.

7. Method according to one of the foregoing claims further encompassing Display (500) a bar below and / or above and / or to the side of the pheno position.

8. Method according to any of the preceding claims, wherein the reference (1) encodes the distance by a length and / or thickness of a bar on the AR HUD (3).

9. Method according to any of the preceding claims, wherein the indication (1) is an optical indication.

10. Method according to one of the preceding claims further comprising automatic determination of a color and / or a color gradient of the environment object (11) and, depending on the color and / or gradient, automatically select a color for the indicator.

11. User interface for issuing a message (1) to a user (2) of a means of transport (10) comprising: a data input (13), an evaluation unit (12) and - 15 - a data output (14), wherein the evaluation unit (12) is configured to determine a distance to a preceding or preceding environmental object (11) by means of the data input (13) and, depending on the distance, to output the message (1) to the user (2) via a front view display (3) with augmented reality, hereinafter referred to as "AR HUD (3)" by means of the data output (14), wherein a positioning of the message (1) on the AR-HUD (3) takes place depending on a pheno-position of the environmental object (11), to determine by means of the data input (13) that a part of the environmental object (11) is not located within the visible area (5) of the AR-HUD (3) and, in response to this, to enhance a representation of the message (1) by means of the data output (14).

12. Means of transport comprising a user interface according to claim 11.

Images