Method for controlling a display device, controller, and motor vehicle

Abstract

The invention relates to a method for controlling a display device (1) which is designed to output a virtual marking (4) for an object (5) detected in the surroundings of a motor vehicle (2) to a driver of the motor vehicle (2), wherein the virtual marking (4) is output by means of the display device (1) so as to indicate the object (5) in a real driving scene, and the virtual marking (4) is adapted if the motor vehicle (2) is decelerated by a driver assistance function in order to prevent a collision with the object (5) and the difference between the deceleration of the motor vehicle (2) and a specified threshold for the deceleration falls below a specified threshold, whereby the adapted marking (4) signals that the deceleration has almost reached the threshold at which the control of the motor vehicle (2) is transferred from the driver assistance function to the driver.

Description

[0001] Description

[0002] Method for controlling a display device, control device and motor vehicle

[0003] The present invention relates to a method for controlling a display device, a control device and a motor vehicle.

[0004] A vehicle head-up display device is known from US 2016 / 0200249 A1. It comprises a display control module that receives signals from a plurality of sources and determines one or more warnings based on the signals. The warnings may include at least one of a lane departure warning, a blind spot warning, a forward / rear distance indicator, a forward hazard warning, and a side hazard warning. The display device comprises a plurality of light sources that selectively project a plurality of light patterns onto a plurality of locations on a windshield of a vehicle, such that the plurality of light patterns overlay objects outside the vehicle.

[0005] Furthermore, US 2019 / 0077418 A1 discloses a method for displaying braking processes of preceding transport vehicles in a transport vehicle. Graphic objects can be generated on a display surface, with each preceding transport vehicle being assigned a respective graphic object that indicates the acceleration of the transport vehicle.

[0006] Furthermore, US 2019 / 0299855 A1 discloses a vehicle approach system comprising sensors, a vehicle computing device, and a head-up display of an ego vehicle for displaying graphic elements for proximity indication to a vehicle occupant, for example, the driver of the ego vehicle. The graphic elements for proximity indication are displayed via a head-up display and projected onto a transparent surface of the ego vehicle, such as windows, the windshield, a screen, or panes. The graphic elements for proximity indication indicate the presence of a vehicle in the vicinity of the ego vehicle. Alternatively, the vehicle can travel in line with the ego vehicle in front of and / or behind the ego vehicle. The object of the present invention is to create a solution that makes it possible to provide a driver of the motor vehicle with particularly comprehensive information regarding support provided by a driver assistance function.

[0007] This object is achieved according to the invention by the subject matter of the independent claims. Further possible embodiments of the invention are disclosed in the subclaims, the description, and the figures. Features, advantages, and possible embodiments presented in the description for one of the subject matter of the independent claims are to be regarded at least analogously as features, advantages, and possible embodiments of the respective subject matter of the other independent claims, as well as any possible combination of the subject matter of the independent claims, optionally in conjunction with one or more of the subclaims.

[0008] The invention relates to a method for controlling a display device which is designed to output to a driver of a motor vehicle a virtual marker for an object detected in the surroundings of the motor vehicle. The display device is in particular a head-up display or a head-mounted display, in particular augmented reality glasses. The driver can thus continue to perceive the real surroundings of the motor vehicle through the display device, wherein the marker is superimposed on this real surroundings for the driver. The method provides for the virtual marker to be output as identifying the object in a real driving scene. If the display device is a head-up display, then it can in particular be a contact-analog head-up display.Using the contact-analog head-up display, display elements such as virtual markers can be superimposed onto the driver's current view in such a way that they feel as if they are an integral part of the environment. This makes a navigation arrow, for example, appear as if it were lying directly on the road. In the case of a safety distance bar, a speed-dependent bar could be displayed in a windshield area to indicate to the driver how much distance they should maintain from the vehicle in front. These contact-analog markers, which are superimposed on the vehicle's surroundings, offer the advantage of directing attention to the marked object, allowing the driver to assess the vehicle's driving situation particularly accurately with very few glances. The contact-analog head-up display is also known as an augmented reality head-up display.The contact-analog head-up display uses optical information to mark elements in the real vehicle environment. Examples of applications include optical marking of lane boundaries, optical highlighting of pedestrians at the side of the road, navigation instructions directly on the road, or marking of vehicles detected by the driver assistance function. The contact-analog head-up display shows information from driver assistance functions, such as an adaptive cruise control system. Objects within the driver's field of vision, particularly within a playable display area of ​​the head-up display, can be marked to indicate to the driver that the driver assistance function has detected them and is responding accordingly.The object marked by the virtual marker is an object that has been detected by a driver assistance function of the vehicle and toward which the vehicle is controlled by the driver assistance function. In this case, the vehicle is decelerated by the driver assistance function to avoid a collision with the object.

[0009] The method further provides for the virtual marking to be adjusted when the motor vehicle is decelerated by the driver assistance function to avoid a collision with the object and the difference between the deceleration of the motor vehicle and a predefined limit value for the deceleration falls below a predefined threshold. In this case, the adjusted marking signals that the deceleration of the motor vehicle has almost reached the limit value at which control of the motor vehicle is transferred from the driver assistance function to the driver. Without the method, the driver cannot determine how close an active control system of the motor vehicle controlled by the driver assistance function is to a permitted functional control limit and at what point in time the driver must be particularly alert in order to be able to intervene in the control of the motor vehicle if necessary.

[0010] The limit value can, for example, be specified by legal requirements. In other words, the limit value can be selected depending on legal requirements to ensure that the motor vehicle complies with the legal requirements. The limit value describes the maximum deceleration of the motor vehicle when the motor vehicle is being controlled longitudinally by the driver assistance function or when the driver of the motor vehicle is assisted in steering the vehicle longitudinally by the driver assistance function. If the specified limit value for the deceleration must be exceeded in order to avoid a collision between the motor vehicle and the object, the driver must take control of the motor vehicle. The driver can brake the motor vehicle with a deceleration that is greater than the limit value.In order to particularly effectively direct the driver's attention to the marked object, the virtual marking for the object can be output by means of the display device so as to overlay the real driving scene. It can therefore appear to the driver in the augmented reality head-up display as if the virtual marking and the object overlap at least in part. In particular, it can be provided that an optical property in the augmented reality head-up display or augmented reality head-mounted display is adapted for the driver by means of the virtual marking. It therefore looks to the driver as if the object has the optical property adapted by the virtual marking. This makes it possible to particularly effectively direct the driver's attention to the object with regard to which the motor vehicle is controlled by means of the driver assistance function.The method enables the driver to be particularly well prepared for the fact that he may have to take over control of the motor vehicle in the near future if the deceleration of the motor vehicle triggered by the driver assistance function approaches the limit value for the deceleration.

[0011] In a possible development of the invention, it is provided that by adjusting the marking, a coloring of the object and / or an animation of the object can be set. Adjusting the coloring of the object means that the object is colored orange for the driver, for example, using the virtual marking. Depending on the distance between the current deceleration of the motor vehicle and the limit value specified for the deceleration, the object can be colored in different colors using the virtual marking. If the distance between the current deceleration of the motor vehicle and the limit value is particularly large and thus greater than the specified threshold, the object can be colored green, for example, using the virtual marking.If the vehicle's deceleration lies between the threshold and the limit value, or if the deceleration corresponds to the threshold value, the object can be colored orange using the virtual marker. If the vehicle's deceleration is equal to or greater than the specified limit value, the object can be colored red using the virtual marker. Adjusting the object's animation means dynamically changing the object's appearance using the virtual marker. For example, the object can be alternately enlarged and reduced in size by overlaying it with the virtual marker to draw the driver's attention to the object.By adjusting the color or dynamically changing the appearance of the object using the virtual marker, the driver's attention can be particularly effectively directed to the object and to the information represented by the virtual marker, which indicates that the driver may have to take over control of the motor vehicle in the driver assistance function in the near future. In a further possible embodiment of the invention, it is provided that the object is displayed flashing using the virtual marker as an adaptation, whereby the flashing display represents a dynamic adjustment of the appearance and thus an adjustment of the animation of the object. The flashing frequency increases as the difference between the deceleration of the motor vehicle and the predefined limit decreases.The dynamics of the flashing can thus indicate to the driver that the probability that the driver will need to take control of the vehicle in the near future is increasing or decreasing. Furthermore, the frequency of the flashing can indicate whether the driver will need to take control of the vehicle very soon or more recently. The more the flashing frequency increases, the sooner the driver will need to take control of the vehicle. This allows the driver to prepare particularly well for taking control of the vehicle from the driver assistance function.

[0012] In a further possible embodiment of the invention, in addition to adjusting the marking, an acoustic signal is triggered in the interior of the motor vehicle. This acoustic signal indicates that the deceleration has approached the limit value closer than the predefined threshold, and thus the driver may have to take over control of the motor vehicle from the driver assistance function in the near future if the deceleration reaches the limit value. The acoustic signal makes it possible for the driver to be alerted not only visually via the virtual marking, but also acoustically, to the possible imminent takeover of control by the driver assistance function in the near future.The risk of the driver not perceiving the information indicating that the deceleration of the motor vehicle has already approached the limit value closer than the specified threshold value can thus be kept particularly low.

[0013] In this context, it can be provided in particular that a voice output or a warning tone is emitted in the interior of the motor vehicle as an acoustic warning. The warning tone is a particularly effective way of gaining the driver's attention, as it can put the driver on alert. The voice output can be used to give the driver the information very precisely. The voice output could, for example, be: "Deceleration of the motor vehicle is approaching the limit. Prepare to take over control of the motor vehicle." This can ensure that the driver is particularly well informed about what action may be required of them in the near future. This allows the driver to prepare particularly well for taking over control of the motor vehicle.

[0014] In a further possible embodiment of the invention, it is provided that an outer contour of the marking corresponds to an outline of the object, whereby the marking can be output so as to completely cover the object. In other words, the marking can have at least substantially the same contour or the same outlines as the object in the driver's field of vision. As a result, the visual appearance of the object for the driver can be particularly well adapted by means of the virtual marking. For example, the coloring of the object as perceived by the driver can be particularly well adapted. In particular, this makes it possible for the virtual marking to only be used to adapt the visual design of the object in the driver's field of vision, and moreover, the driver's field of vision is impaired very little.This gives the driver a particularly good view of the real driving scene in the vehicle’s surroundings.

[0015] In a further possible embodiment of the invention, in addition to adjusting the marking, the output of a message in text form and / or in the form of an icon is triggered, which is superimposed on the actual driving scene. This means that the message in text form and / or in the form of an icon is introduced, in particular projected, into the driver's field of vision by means of the display device. This allows the driver to perceive the message in text form or in the form of an icon particularly well without taking their eyes off the driving scene in the surroundings of the motor vehicle.The textual warning or the icon-like warning, for example, in the form of an exclamation mark or a warning triangle, allows the driver to be given a particularly comprehensive visual overview of why the object is marked with the virtual marker. This allows the driver to determine particularly easily and with minimal effort that they are expected to take control of the vehicle in the near future if necessary. This allows the driver to prepare particularly well for taking control. The textual warning could, for example, read: "Deceleration of the vehicle is approaching the limit. Prepare to take control of the vehicle."

[0016] In a further possible embodiment of the invention, it is provided that, in addition to adjusting the marking, an ambient light in the interior of the motor vehicle and / or a vibration of a vehicle seat and / or a vibration of a steering wheel is activated or adjusted when it is determined that the difference between the deceleration of the motor vehicle and the predetermined limit value for the deceleration falls below the predetermined threshold value.This means that, in addition to marking the object in their field of vision, the driver is alerted to the possible need to take control of the vehicle in the near future by means of the virtual marking. This is done by adjusting the color and / or output dynamics of the ambient lighting emitted in the interior of the vehicle, or by vibrating the vehicle seat or steering wheel, or by changing the vibration of the vehicle seat and / or steering wheel. By adjusting the ambient lighting or the vibration of the steering wheel and / or vehicle seat, the driver can be particularly well alerted, which in turn allows the driver to prepare particularly well for the possible need to take control of the vehicle.

[0017] The invention further relates to a control device configured to carry out a method as already described in connection with the method according to the invention for controlling a display device. In particular, the control device is configured to control the display device and thus trigger the display device to output the virtual marker for the object detected in the surroundings of the motor vehicle to the driver of the motor vehicle. The control device can thus trigger the display device to output the virtual marker identifying the object in the real driving scene.Furthermore, the control device can trigger the virtual marking to be adjusted by means of the display device if the motor vehicle is decelerated by the driver assistance function to avoid a collision with the object and the difference between the deceleration of the motor vehicle and the predefined limit value for the deceleration falls below the predefined threshold. The control device can be configured to compare the deceleration of the motor vehicle triggered by the driver assistance function with the predefined limit value and thus compare the difference with the predefined threshold value. Thus, the control device can determine whether the virtual marking needs to be adjusted.If the control device determines that the virtual marking needs to be adjusted, the control device controls the display device in such a way that the display device adjusts the virtual marking in accordance with the control by the control device.

[0018] The invention further relates to a motor vehicle with a control device as already described in connection with the control device according to the invention. Further features of the invention can be derived from the following description of the figures and from the drawing. The features and combinations of features mentioned above in the description, as well as the features and combinations of features shown below in the description of the figures and / or in the figures alone, can be used not only in the respective specified combination, but also in other combinations or on their own, without departing from the scope of the invention.

[0019] The drawing shows:

[0020] Fig. 1 shows a view of a driver of a motor vehicle onto a contact analogue head-up display in which an object in a real driving scene is marked with a virtual marker.

[0021] Fig. 1 shows a contact-analog head-up display 1, also referred to as an augmented reality head-up display, from the perspective of a driver of a motor vehicle 2. The head-up display 1 is a display device that allows the driver to maintain their line of sight and thus their head position when looking at a windshield 3 of the motor vehicle 2, with relevant information being projected into their field of vision, superimposed on the windshield 3. The head-up display 1 is a display device that is configured to output to the driver of the motor vehicle 2 a virtual marker 4 for an object 5 detected in the surroundings of the motor vehicle 2.

[0022] The motor vehicle 2 comprises a control device configured to control the display device, in this case the contact-analog head-up display 1, using a method as described below. In the method performed by the control device for controlling the display device, in this case the contact-analog head-up display 1, the virtual marker 4 will output the object 5 in a real driving scene in a characterizing manner. As can be seen in Fig. 1, the object 5 in the driver's field of vision is thus marked by the virtual marker 4.

[0023] In the present case, an outer contour of the marking 4 corresponds to an outline of the object 5, whereby the virtual marking 4 can be output so as to completely cover the object 5. As can be seen in Fig. 1, the virtual marking 4 overlays the object 5 in the driver's field of vision. The object 5 is thus covered by the virtual marking 4. To ensure that the driver can still see the object 5 despite the overlay with the marking 4, the virtual marking 4 can be output transparently, at least in some areas. In the present case, the object 5 is a vehicle, in particular another motor vehicle. By means of the virtual marking 4, the attention of the driver of the motor vehicle 2 is directed to the object 5 detected in the surroundings of the motor vehicle 2.

[0024] In the present case, the motor vehicle 2 is configured to be controlled by means of a driver assistance function. This means that the driver of the motor vehicle 2 can be assisted in controlling the motor vehicle 2 by means of the driver assistance function, or the driver assistance function is configured to perform the longitudinal and / or lateral control of the motor vehicle 2 at least partially, in particular fully, automatically.

[0025] If the motor vehicle 2 is decelerated by the driver assistance function to avoid a collision with the object 5 and a difference between the deceleration of the motor vehicle 2 and a predefined limit value for the deceleration falls below a predefined threshold value, the virtual marking 4 is adjusted. This means that, based on the adjusted virtual marking 4, the driver of the motor vehicle 2 can recognize that it has been determined that the difference and thus the distance between the current deceleration of the motor vehicle 2 is smaller by the predefined limit value than the predefined threshold value. In other words, the adjusted virtual marking 4 indicates that the current deceleration of the motor vehicle 2 has approached the predefined limit value more than the predefined threshold value.In particular, the limit value specifies a maximum deceleration by which motor vehicle 2 may be decelerated when controlled by the driver assistance function. Deceleration of motor vehicle 2 exceeding the limit value may only be undertaken by the driver of motor vehicle 2. The adjusted marking 4 thus signals to the driver that the current deceleration of motor vehicle 2 has almost reached the limit value. As soon as the deceleration of motor vehicle 2 has reached the limit value, control of motor vehicle 2 is transferred from the driver assistance function to the driver.

[0026] If it is thus determined that the current deceleration of the motor vehicle 2, by means of which the motor vehicle 2 is decelerated by the driver assistance function, is not sufficient to prevent the collision of the motor vehicle 2 with the object 5, and a deceleration of the motor vehicle 2 that is equal to or greater than the limit value is necessary to avoid the collision, then control of the motor vehicle 2 is transferred to the driver so that the driver can decelerate the motor vehicle 2 sufficiently to avoid the collision.By adapting the optics of the object 5 in the contact-analog head-up display 1 by means of the virtual marking 4, the driver of the motor vehicle 2 can thus be particularly well informed as to whether the driver assistance function, by means of which the motor vehicle 2 is controlled, is approaching the respective limits for controlling the motor vehicle 2 by means of the driver assistance function and thus the control of the motor vehicle 2 must be taken over by the driver, should the driver assistance function actually reach the limits.

[0027] In the present case, it is provided that the coloring of the object 5 is adjusted by adjusting the virtual marking 4. Alternatively or additionally, an animation and thus a dynamic change in the appearance of the object 5 can be adjusted by adjusting the virtual marking 4. As an adjustment, the object 5 can be represented flashing by means of the virtual marking 4. In particular, the frequency of the flashing can increase as the difference between the deceleration of the motor vehicle 2 and the predetermined limit value decreases. A criticality for the possibly required takeover of control of the motor vehicle 2 by the driver in the near future and thus the proximity of the driver assistance function to system limits of the driver assistance function, in particular to system limits of a cruise control, is indicated by adjusting the virtual marking 4.Criticality is given by the fact that the difference between the deceleration of the motor vehicle 2 and the predefined limit value is smaller than the predefined threshold value. Criticality can thus be indicated by the flashing display of the object, in particular the vehicle in front. The frequency increases as the deceleration approaches the limit value. The virtual marking 4 can alternatively or additionally be displayed shaking or vibrating, with the shaking or vibrating increasing the smaller the difference between the deceleration of the motor vehicle 2 and the predefined limit value becomes. The smaller the difference between the deceleration of the motor vehicle 2 and the limit value becomes, the more the transparency of the virtual marking 4 can be reduced.

[0028] In order to alert the driver and to draw their attention particularly clearly to the fact that the driver assistance function is reaching its system limits and that the driver may need to take over control of the motor vehicle 2, in addition to adjusting the virtual marking 4, an acoustic signal can be output in the interior of the motor vehicle 2. This acoustic signal can be a warning tone or a voice output. The voice output can be, for example, "ACC near system limit." Alternatively or in addition to outputting the acoustic signal, an ambient light in the interior of the motor vehicle 2 can be adjusted, in particular with regard to flashing or a color change, and / or the steering wheel of the motor vehicle 2 can vibrate and / or the driver's seat of the motor vehicle 2 can vibrate. The warning tone can be a beep.

[0029] In the method, an optical or visual representation of the object 5, to which the motor vehicle 2 is controlled by means of the driver assistance function, is adjusted in the contact-analog head-up display 1 by means of the virtual marking 4 when the difference between the deceleration of the motor vehicle 2 and the predetermined limit value has fallen below the predetermined threshold value and thus the critical deceleration is soon reached at which the driver has to take over control of the motor vehicle 2.

[0030] In the contact-analog head-up display 1, objects 5 that have been detected by sensors and cameras of the motor vehicle 2 and that are controlled by driver assistance functions of the motor vehicle 2 can be marked. The respective object 5 can be marked using the virtual marking 4. A green marking 4 of the object 5 can indicate that this object 5 is being controlled by the driver assistance function. In this case, the object 5 can only be marked situationally and not permanently using the virtual marking 4. A red marking 4 of the object 5 can indicate that a functional limit of the driver assistance function has been reached and a maximum deceleration permitted for the driver assistance function is probably not sufficient to avoid a collision between the motor vehicle 2 and the object 5, and thus the driver must intervene with regard to the control of the motor vehicle 2.Within the scope of the described method for controlling the display device, an additional display category is provided, which informs the driver that the driver assistance function can still actively regulate, but that the deceleration of motor vehicle 2 is relatively high and therefore a driver takeover request may occur, and the driver should therefore be particularly vigilant. In the event that a defined high deceleration is applied in motor vehicle 2 by the driver assistance function, the color of the marking of object 5 is changed upon a trigger from a function control unit; in particular, object 5 is displayed in orange.By adjusting the marking of object 5, analogous to a warning kit and a color logic used in the human-machine interface, the driver is signaled that a significant deceleration is necessary and that a system limit for the driver assistance function may occur, upon reaching which limit the driver must take over control of motor vehicle 2. A color of the virtual marking 4 thus characterizes a current ability of the driver assistance function to maintain or adjust a specified time gap to the vehicle in front. In addition to the distance, the actual speed of motor vehicle 2 and the vehicle in front, as well as the ability of the driver assistance function to control motor vehicle 2 at a given relative speed between motor vehicle 2 and the vehicle in front, are also taken into account.The virtual marker 4 thus enables a reliability indicator of the driver assistance function and not per se a "you're driving too close" warning. The virtual marker 4 communicates the system limits of the driver assistance function dynamically and in a situation-specific manner. To support this, an icon and / or text can be added to the virtual marker 4, which communicates the system limits more clearly than the color itself. When the object 5 is marked orange by the virtual marker 4, the following can also be displayed: "ACC system near system limit." When the object 5 is marked red by the virtual marker 4, the following can also be displayed: "ACC system limits exceeded."

[0031] As an alternative to the design of the display device as a contact-analog head-up display, the display device can be a head-mounted display, in particular augmented reality glasses.

[0032] Overall, the invention shows how a criticality display of control objects can be used in the augmented reality head-up display.

[0033] List of reference symbols contact analogue head-up display motor vehicle windshield virtual marking object

Claims

Patent claims 1. Method for controlling a display device (1) which is designed to output a virtual marking (4) for an object (5) detected in the surroundings of the motor vehicle (2) to a driver of a motor vehicle (2), in which method the virtual marking (4) is output by means of the display device (1) in a manner characterising the object (5) in a real driving scene, the virtual marking (4) is adapted when the motor vehicle (2) is decelerated by a driver assistance function in order to avoid a collision with the object (5) and a difference between the deceleration of the motor vehicle (2) and a predetermined limit value for the deceleration falls below a predetermined threshold value, whereby the adapted marking (4) signals that the deceleration has almost reached the limit value at which control of the motor vehicle (2) is transferred from the driver assistance function to the driver.

2. Method according to claim 1, wherein by adjusting the marking (4) a coloring of the object (5) and / or an animation of the object (5) is set.

3. Method according to claim 1 or 2, wherein as an adaptation the object (5) is represented flashing by means of the virtual marking (4), wherein a frequency of the flashing increases as the difference between the deceleration of the motor vehicle (2) and the predetermined limit value decreases.

4. Method according to one of the preceding claims, wherein in addition to the adjustment of the marking (4), an output of an acoustic signal in the interior of the motor vehicle (2) is triggered.

5. The method according to claim 4, wherein a voice output or a warning tone is output as an acoustic indication.

6. Method according to one of the preceding claims, wherein an outer contour of the marking (4) corresponds to an outline of the object (5), whereby the marking (4) can be output so as to completely cover the object (5).

7. Method according to one of the preceding claims, wherein in addition to the adjustment of the marking (4), an output of a note in text form and / or in the form of an icon is triggered, which overlays the real driving scene.

8. Method according to one of the preceding claims, wherein in addition to adjusting the marking (4), an ambient light in the interior of the motor vehicle (2) and / or a vibration of a vehicle seat and / or a vibration of a steering wheel is activated or adjusted when it is determined that the difference between the deceleration of the motor vehicle (2) and the predetermined limit value for the deceleration falls below the predetermined threshold value.

9. Control device which is configured to carry out a method according to one of the preceding claims.

10. Motor vehicle (2) with a control device according to claim 9.

Images