Method for controlling a display device, control device, and motor vehicle

Abstract

The invention relates to a method for controlling a display device, a control device and a motor vehicle. The invention relates to a method for controlling a display device (1), which is designed to output a virtual marking (4) for an object (5) detected in the surroundings of a motor vehicle (2) for a driver of the motor vehicle (2), in which method the virtual marking (4) is output by means of the display device (1) in such a way that the object (5) is identified in a real driving scene, when the motor vehicle (2) is decelerated by the driver assistance function in order to avoid a collision with the object (5) and the difference between the deceleration of the motor vehicle (2) and a predetermined limit value for the deceleration falls below a predetermined threshold value, the virtual marker (4) is adapted, whereby the adapted marker (4) signals that the deceleration nearly has reached the limit value, the control of the motor vehicle (2) is transferred from the driver assistance function to the driver in the case of the limit value.

Description

Technical Field

[0001] The invention relates to a method for controlling a display device, a control device, and a motor vehicle. Background Art

[0002] US2016 / 0200249A1 discloses a vehicle head-up display device. The device includes a display control module that receives signals from multiple sources and determines one or more warnings based on the signals. The warnings may include at least lane departure warnings, blind angle warnings, forward / rearward distance display, front hazard warnings, and side hazard warnings. The display device includes multiple light sources that selectively project multiple light patterns onto multiple locations on the vehicle's windshield, such that the multiple light patterns overlap with objects outside the vehicle.

[0003] Furthermore, US2019 / 0077418A1 discloses a method for displaying the braking process of a preceding transport vehicle in a transport vehicle. A graphic object can be generated on a display surface, wherein each preceding transport vehicle is associated with a corresponding graphic object that displays the acceleration of the transport vehicle.

[0004] In addition, a vehicle approaching system is known from US2019 / 0299855A1, which includes a sensor, a vehicle computing device, and a head-up display of the own vehicle, so as to display graphic elements for approaching display to vehicle passengers (e.g., the driver of the own vehicle). The graphic elements for approaching display are displayed via a head-up display and projected onto a transparent surface of the own vehicle, such as a window, a windshield, a screen, or glass (or a sheet, i.e., Scheiben). The graphic elements for approaching display indicate that there is a vehicle approaching the own vehicle. Alternatively, the vehicle can travel in front of and / or behind the own vehicle in a row with the own vehicle. Summary of the Invention

[0005] The object of the present invention is to provide a solution which allows a particularly extensive notification of the driver of a motor vehicle regarding the support provided by a driver assistance function.

[0006] According to the invention, this object is achieved by the subject matter of the independent claims. Further possible embodiments of the invention are disclosed in the dependent claims, the description, and the drawings. The features, advantages, and possible embodiments stated within the description for one of the subject matter of an independent claim are at least analogously to the features, advantages, and possible embodiments of the corresponding subject matter of the other independent claims, as well as any possible combination of the subject matter of the independent claims, if applicable, in combination with one or more of the dependent claims.

[0007] The present invention relates to a method for controlling a display device configured to display virtual markers for objects detected in the vehicle's surroundings to the driver of a motor vehicle. The display device is particularly a heads-up display (HUD) or head-mounted display, particularly augmented reality glasses. The driver can thus continue to perceive the vehicle's real surroundings through the display device, with the markers appearing superimposed (or faded in, or "eingeblended") into the real surroundings for the driver. In this method, the virtual markers are output in a manner that identifies the objects in the real driving scene. If the display device is a HUD, it can particularly be a contact-simulating HUD. With a contact-simulating HUD, display elements (such as virtual markers) can be superimposed in the driver's current field of view so that they appear to be a permanent part of the environment. This allows, for example, navigation arrows to appear as if they are directly on the road. In the case of safety distance stripes, stripes can be superimposed on the windshield area for the driver, depending on speed, to indicate the distance they should maintain from vehicles ahead. Contact-simulated markings that overlap the vehicle's surroundings offer the advantage of directly directing attention to the marked objects, allowing the driver to assess the vehicle's driving situation particularly well with minimal visual input. Contact-simulated heads-up displays are also known as augmented reality heads-up displays. In contact-simulated heads-up displays, optical information is used to identify elements in the real vehicle's surroundings. Examples of applications include optical markings of lane boundaries, optical highlighting of pedestrians at the roadside, navigation instructions directly in the lane, or markings of vehicles that have been identified by a driver assistance function. Contact-simulated heads-up displays present indications of driver assistance functions (such as adaptive cruise control). Objects within the driver's field of view, particularly within the available display area of ​​the heads-up display, can be marked to indicate to the driver that the driver assistance function has identified them and is adjusting them accordingly. Objects marked with virtual markings are objects that have been identified by the vehicle's driver assistance function and that the vehicle has been adjusting accordingly. Currently, the vehicle is being slowed down by the driver assistance function to avoid a collision with the object.

[0008] The method further provides for adapting a virtual flag when the driver assistance function decelerates the vehicle to avoid a collision with an object and the difference between the vehicle's deceleration and a predefined deceleration limit falls below a predefined threshold. In this case, the adapted flag signals that the vehicle's deceleration has almost reached the limit at which control of the vehicle is transferred from the driver assistance function to the driver. Without this method, the driver would not know how close an active control system of the vehicle controlled by the driver assistance function is to a permissible functional control limit, nor at what point the driver would need to be particularly alert so that he or she can intervene in the vehicle control if necessary.

[0009] Limit values ​​can be predefined, for example, by legal regulations. In other words, the limit values ​​can be selected based on legal regulations to ensure that the vehicle complies with legal requirements. The limit values ​​describe the maximum permissible deceleration of the vehicle when a driver assistance function is controlling the vehicle longitudinally, or when the driver of the vehicle is assisted by the driver assistance function in controlling the vehicle longitudinally. If the predefined deceleration limit value must be exceeded to avoid a collision with an object, the driver must take over control of the vehicle. The driver can brake the vehicle at a deceleration greater than the limit value. To effectively direct the driver's attention to the identified object, a virtual marker for the object can be displayed using a display device, superimposed on the real driving scene. Thus, in an augmented reality head-up display, it can appear to the driver that the virtual marker and the object overlap, at least partially. In particular, it can be provided that the optical properties of the augmented reality head-up display or augmented reality head-mounted display are adapted for the driver using the virtual marker. Thus, the object appears to the driver to have the optical properties adapted by the virtual marker. This allows the driver's attention to be particularly effectively directed to the object, allowing the driver assistance function to adjust the vehicle to account for the object. The method allows a driver to be particularly well prepared, who may have to take over control of the motor vehicle in the near future when a deceleration of the motor vehicle triggered by a driver assistance function approaches a limit value for the deceleration.

[0010] In a possible refinement of the present invention, the color scheme and / or animation of an object can be adjusted by adapting a marking. Adapting the color scheme of an object can be understood as, for example, coloring the object orange for the driver using a virtual marking. Depending on the distance between the vehicle's current deceleration and a preset deceleration limit, the object can be colored different colors using the virtual marking. If the distance between the vehicle's current deceleration and the limit is particularly large and therefore exceeds a preset threshold, the object can be colored green, for example, using the virtual marking. If the vehicle's deceleration is between the threshold and the limit, or if the deceleration corresponds to the threshold, the object can be colored orange using the virtual marking. If the vehicle's deceleration is equal to or greater than the preset limit, the object can be colored red using the virtual marking. Adapting the animation of an object can be understood as dynamically changing the appearance of an object using a virtual marking. For example, by superimposing a virtual marking, an object can be alternately enlarged and then reduced again to draw the driver's attention to the object. By adapting the color or dynamically changing the appearance of an object with the aid of a virtual marker, the driver's attention can be drawn particularly effectively to the object and to the information represented by the virtual marker, which indicates that the driver may have to take over control of the motor vehicle in the near future as part of a driver assistance function.

[0011] In another possible embodiment of the present invention, the object is presented as an adaptation by means of a virtual marker in a flashing manner, wherein the flashing presentation is a dynamic adaptation of the appearance and, therefore, of the object's animation. The flashing frequency increases as the difference between the vehicle's deceleration and a preset limit value decreases. The dynamics of the flashing can thus indicate to the driver whether the likelihood of the driver having to take control of the vehicle in the near future is increasing or decreasing. Furthermore, the flashing frequency can indicate whether the driver will have to take control of the vehicle soon or not. The greater the increase in the flashing frequency, the earlier the driver will have to take control of the vehicle. This allows the driver to be particularly well prepared for taking over control from the driver assistance function.

[0012] In another possible embodiment of the present invention, in addition to the adaptation flag, an acoustic warning is triggered in the interior of the vehicle. The acoustic warning indicates that the deceleration is approaching a limit value relative to a predetermined threshold value, and therefore that the driver may need to take over control of the vehicle from a driver assistance function in the near future if the deceleration reaches the limit value. The acoustic warning can alert the driver not only visually via a virtual flag but also acoustically that control may be taken over from the driver assistance function in the near future. This minimizes the risk of the driver failing to perceive the information indicating that the vehicle's deceleration is approaching a limit value relative to the predetermined threshold value.

[0013] In this regard, it can be provided in particular that a voice output or warning tone is output as an acoustic prompt within the vehicle interior. The warning tone is particularly effective in gaining the driver's attention, as it alerts the driver. The voice output allows for particularly precise information to be conveyed to the driver. For example, the voice output could read: "The vehicle's deceleration is approaching a limit value. Prepare to take over control of the vehicle." This ensures that the driver is particularly well informed of the actions that may be required of him in the near future. Consequently, the driver can be particularly well prepared to take over control of the vehicle.

[0014] In another possible embodiment of the present invention, the outer contour of the marker corresponds to the outer shape of the object, allowing the marker to be output so that it completely covers the object. In other words, the marker can have at least substantially the same contour or outer shape as the object in the driver's field of view. This allows the virtual marker to be particularly well-adapted to the driver's visual appearance. For example, the color scheme of the object as perceived by the driver can be particularly well-adapted. In particular, this allows the virtual marker to only adapt the visual design of the object in the driver's field of view and otherwise minimally impact the driver's field of view. This provides the driver with a particularly good view of the actual driving scene in the vehicle's surroundings.

[0015] In another possible embodiment of the present invention, in addition to the adaptation marker, a textual and / or pictographic notification is triggered, which is superimposed on the actual driving scene. This means that the textual and / or pictographic notification is displayed by means of a display device, in particular, projected into the driver's field of view. This allows the driver to perceive the textual or pictographic notification particularly well without shifting their gaze from the driving scene surrounding the vehicle. A textual or pictographic notification, such as an exclamation mark or a warning triangle, visually provides the driver with a particularly high level of information about why the object has been identified by means of a virtual marker. This allows the driver to easily and with minimal effort confirm that they are expected to take over control of the vehicle in the near future. This allows the driver to be particularly well prepared for taking over control. For example, the textual notification could read: "Vehicle deceleration approaching limit value. Prepare to take over control of the vehicle."

[0016] In another possible embodiment of the present invention, in addition to the adaptation marker, ambient lighting in the vehicle interior and / or vibration of the vehicle seat and / or steering wheel is activated or adapted when the difference between the detected deceleration of the vehicle and a preset deceleration limit value falls below a preset threshold value. This means that, in addition to identifying the object in the driver's field of view using the virtual marker, the driver is alerted to the possible need to take over control of the vehicle in the near future by adapting the color and / or output dynamics of the ambient light emitted in the vehicle interior or by vibrating or changing the vibration of the vehicle seat and / or steering wheel. Adapting the ambient light or the vibration of the steering wheel and / or vehicle seat makes it possible to alert the driver particularly effectively, thereby allowing the driver to be particularly well prepared for the possible need to take over control of the vehicle.

[0017] The present invention further relates to a control device configured to perform a method as described in conjunction with the method for controlling a display device according to the present invention. In particular, the control device is configured to control and thereby trigger a display device so that a virtual marker for an object detected in the surroundings of the vehicle is output to the driver of the motor vehicle via the display device. The control device can thus trigger the display device to output the virtual marker in a manner that identifies the object in the real driving scenario. Furthermore, the control device can be triggered so that the virtual marker is adapted via the display device when the motor vehicle is decelerated by a driver assistance function to avoid a collision with an object and the difference between the motor vehicle's deceleration and a preset deceleration limit falls below a preset threshold. The control device can be configured to compare the deceleration of the motor vehicle, triggered by the driver assistance function, with a preset limit and thus compare the difference with a preset threshold. The control device can thus determine whether the virtual marker should be adapted. If the control device confirms that the virtual marker should be adapted, the control device controls the display device so that the virtual marker is adapted according to the control performed by the control device.

[0018] The present invention further relates to a motor vehicle having a control device as has already been described in conjunction with the control device according to the invention.

[0019] Further features of the invention can be gathered from the following description of the drawings and from the drawings. The features and feature combinations mentioned above in the description and the features and feature combinations shown individually in the following description of the drawings and / or in the drawings can be used not only in the respectively specified combination but also in other combinations or alone without departing from the scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In the attached figure: Figure 1The diagram shows the line of sight of a driver of a motor vehicle contacting a simulated head-up display, in which objects in the real driving scene are identified by virtual markers. DETAILED DESCRIPTION

[0021] exist Figure 1 2 shows a contact-simulating head-up display 1, also known as an augmented reality head-up display, as seen from the perspective of the driver of motor vehicle 2. Head-up display 1 is a display device in which the driver can maintain their gaze direction and, therefore, their head posture, while looking toward windshield 3 of motor vehicle 2. Corresponding information is projected into the driver's field of view, superimposed on windshield 3. Head-up display 1 is a display device configured to display virtual markers 4 for objects 5 detected in the surroundings of motor vehicle 2 to the driver of motor vehicle 2.

[0022] The motor vehicle 2 comprises a control device which is configured to operate a display device, in the present case a contact-simulated head-up display 1, in the method described below. In the method for controlling the display device, in the present case a contact-simulated head-up display 1, which is executed by the control device, a virtual marker 4 is output in such a way as to identify an object 5 in the real driving scene. Figure 1 As can be seen in FIG, an object 5 is marked in the driver's field of vision by means of a virtual marker 4. Currently, the outer contour of the marker 4 corresponds to the outer shape of the object 5, so that the virtual marker 4 can be output in such a way that it completely covers the object 5. Figure 1 As can be seen in the figure, virtual marker 4 overlaps object 5 in the driver's field of view. Thus, object 5 is covered by virtual marker 4. To ensure that the driver can still see object 5 (despite its overlap with marker 4), virtual marker 4 can be displayed transparently, at least in sections. In this case, object 5 is a vehicle, particularly another motor vehicle. Virtual marker 4 directs the driver of motor vehicle 2's attention to object 5, which is captured in the surroundings of motor vehicle 2.

[0023] Currently, motor vehicle 2 is configured to be controlled with the aid of a driver assistance function. This means that the driver of motor vehicle 2 can be assisted by the driver assistance function in controlling motor vehicle 2, or the driver assistance function is configured to perform longitudinal and / or lateral control of motor vehicle 2 at least partially automatically, in particular fully automatically.

[0024] If the driver assistance function decelerates motor vehicle 2 to avoid a collision with object 5 and the difference between the deceleration of motor vehicle 2 and a preset deceleration limit falls below a preset threshold, virtual marker 4 is adapted. This means that the adapted virtual marker 4 indicates to the driver of motor vehicle 2 that the difference between the current deceleration of motor vehicle 2 and the preset threshold has been determined and that the distance is therefore less than the preset threshold. In other words, the adapted virtual marker 4 indicates that the current deceleration of motor vehicle 2 is closer to the preset limit than the preset threshold. The limit specifies, in particular, the maximum deceleration at which motor vehicle 2 can be decelerated to its maximum extent when controlled by the driver assistance function. Decelerations of motor vehicle 2 exceeding the limit are currently permitted only by the driver of motor vehicle 2. Therefore, the adapted marker 4 signals to the driver that the current deceleration of motor vehicle 2 has almost reached the limit. Once the deceleration of motor vehicle 2 reaches the limit, control of motor vehicle 2 is transferred from the driver assistance function to the driver. If it is determined that the current deceleration of motor vehicle 2 (by means of which the driver assistance function is decelerating motor vehicle 2) is insufficient to prevent a collision between motor vehicle 2 and object 5, and that a deceleration of motor vehicle 2 equal to or greater than a limit value is required to avoid the collision, control of motor vehicle 2 is transferred to the driver so that the driver can decelerate motor vehicle 2 sufficiently to avoid the collision. Therefore, by correspondingly adapting the appearance of object 5 in contact-simulating head-up display 1 by means of virtual marking 4, the driver of motor vehicle 2 can be particularly well informed whether the driver assistance function, by means of which motor vehicle 2 is being controlled, is approaching a corresponding limit for controlling motor vehicle 2 by means of the driver assistance function, and therefore that the driver should take over control of motor vehicle 2 if the driver assistance function has actually reached its limit.

[0025] Currently, the color scheme of object 5 is adapted by adapting virtual marker 4 . Alternatively or additionally, the animation and, therefore, the dynamic changes in the appearance of object 5 can be adapted by adapting virtual marker 4 . Object 5 can be presented as an adaptation by means of virtual marker 4 in a flashing manner. In particular, the flashing frequency can be increased as the difference between the deceleration of motor vehicle 2 and a predefined limit value decreases. Adapting virtual marker 4 indicates a criticality, with the driver potentially having to take over control of motor vehicle 2 in the near future and, therefore, the driver assistance function approaching a system limit of the driver assistance function, particularly a system limit of the speed control. Criticality is indicated by the difference between the deceleration of motor vehicle 2 and a predefined limit value being less than a predefined threshold value. This criticality can be indicated by flashing an object, particularly a vehicle ahead. The frequency increases as the deceleration approaches the limit value. Alternatively or additionally, virtual marker 4 can be presented in a pulsating or vibrating manner, with the pulsating or vibrating increasing as the difference between the deceleration of motor vehicle 2 and the predefined limit value decreases. The smaller the difference between the deceleration of motor vehicle 2 and the limit value, the more the transparency of virtual marking 4 is reduced.

[0026] In order to alert the driver and to particularly effectively alert the driver that a driver assistance function has reached its system limit and that the driver should, if necessary, take over control of motor vehicle 2 , an acoustic warning can be output in the interior of motor vehicle 2 in addition to adapting virtual marker 4 . This acoustic warning can be a warning tone or a voice output. The voice output can, for example, read: "ACC is approaching its system limit." Alternatively or in addition to outputting the acoustic warning, the ambient lighting in the interior of motor vehicle 2 can be adapted, particularly by flashing or changing color, and / or the steering wheel of motor vehicle 2 can vibrate and / or the driver's seat of motor vehicle 2 can vibrate. The warning tone can be a beep.

[0027] In this method, when the difference between the deceleration of the motor vehicle 2 and a preset limit value has fallen below a preset threshold value and is therefore about to reach a critical deceleration at which the driver must take over control of the motor vehicle 2, the optical or visual presentation of an object 5 for which the motor vehicle 2 is adjusted by means of a driver assistance function is adapted in the contact-simulating head-up display 1 by means of a virtual marker 4.

[0028] In a contact-simulating head-up display 1, an object 5 can be marked, which is detected by sensors and cameras of motor vehicle 2 and targeted by a driver assistance function of motor vehicle 2. The corresponding object 5 can be identified using a virtual marker 4. A green marker 4 on an object 5 can indicate that the driver assistance function is targeting this object 5. Objects 5 can be marked with the virtual marker 4 only in specific situations and not permanently. A red marker 4 on an object 5 can indicate that the functional limit of the driver assistance function has been reached, and that the maximum deceleration permitted by the driver assistance function is not expected to be sufficient to avoid a collision between motor vehicle 2 and object 5, requiring the driver to intervene in the control of motor vehicle 2. Within the scope of the described method for controlling a display device, another display category is additionally provided, which informs the driver that the driver assistance function can still be actively controlled, but that the deceleration of motor vehicle 2 is relatively high, and therefore a driver takeover request may occur, requiring the driver to be particularly vigilant. If a defined high deceleration is applied in motor vehicle 2 by the driver assistance function, the marking of object 5 changes color upon triggering the function controller, in particular, displaying object 5 in orange. By adapting the marking of object 5 , similar to the color logic used in warning toolkits and human-machine interfaces, the driver is informed that a high deceleration is necessary and that a system limit for a driver assistance function may be reached, at which point the driver must take over control of vehicle 2 . The color of virtual marking 4 thus indicates the driver assistance function's ability to maintain a current time or adjust a predetermined time interval to the vehicle ahead. Influencing factors include, in addition to the distance, the actual speeds of vehicle 2 and the vehicle ahead, as well as the driver assistance function's ability to adjust vehicle 2 given a given relative speed between them. Thus, virtual marking 4 enables reliable display of the driver assistance function, rather than simply providing a "you're driving too close" warning. Virtual marking 4 dynamically and contextually communicates the system limit of the driver assistance function. To assist, virtual marking 4 can be supplemented with icons and / or text, which convey the system limit more clearly than color. When virtual marking 4 indicates an object 5 in orange, the following message can be displayed: "ACC system approaching system limit." When virtual marking 4 indicates an object 5 in red, the following message can be displayed: "ACC system limit exceeded."

[0029] As an alternative to designing the display device as a contact-simulating head-up display, the display device can also be a head-mounted display, in particular augmented reality glasses.

[0030] In summary, the present invention shows how a criticality display of an adjustable object can be used in an augmented reality heads-up display.

[0031] Reference Number List 1 Contact analog head-up display 2 Motor vehicles 3 Windshield 4 Virtual Tagging 5 Objects

Claims

1. A method for controlling a display device (1), which is configured to output a virtual marker (4) for an object (5) detected in the surroundings of the motor vehicle (2) to a driver of a motor vehicle (2), wherein the method comprises: - outputting the virtual marker (4) in such a way as to identify the object (5) in a real driving scenario, - When the motor vehicle (2) is decelerated by the driver assistance function to avoid a collision with the object (5) and the difference between the deceleration of the motor vehicle (2) and a preset limit value for the deceleration falls below a preset threshold value, the virtual marker (4) is adapted, whereby the adapted marker (4) signals that the deceleration has almost reached the limit value, at which control of the motor vehicle (2) is transferred from the driver assistance function to the driver.

2. The method according to claim 1, wherein the color scheme of the object (5) and / or the animation of the object (5) is adjusted by adapting the marking (4).

3. The method according to claim 1 or 2, wherein as an adaptation, the object (5) is presented in a flashing manner with the aid of the virtual marker (4), wherein the flashing frequency increases as the difference between the deceleration of the motor vehicle (2) and a preset limit value becomes smaller. 4 . The method according to claim 1 , wherein, in addition to the adaptation of the marking ( 4 ), the output of an acoustic warning in the interior of the motor vehicle ( 2 ) is triggered. The method according to claim 4 , wherein a speech output or a warning tone is output as the acoustic prompt.

6. The method according to any one of the preceding claims, wherein the outer contour of the marking (4) corresponds to the outer shape of the object (5), so that the marking (4) can be output in such a way that it completely covers the object (5).

7. The method according to any of the preceding claims, wherein in addition to the adaptation of the marking (4), the output of a prompt in the form of text and / or in the form of an icon is triggered, which prompt overlaps with the real driving situation.

8. A method according to any one of the preceding claims, wherein when the difference between the deceleration of the motor vehicle (2) and a preset limit value for the deceleration falls below a preset threshold value, in addition to adapting the mark (4), the ambient light in the interior space of the motor vehicle (2) and / or the vibration of the vehicle seat and / or the vibration of the steering wheel are activated or adapted. 9 . A control device configured to carry out the method according to claim 1 .

10. A motor vehicle (2) having a control device according to claim 9.

Images