CONTROL DEVICE AND METHOD FOR CONTROLLING THE OPERATION OF A VIEWING DEVICE FOR A MOTOR VEHICLE

Abstract

A control device (1) is provided for controlling the operation of a field-of-view display device (2) for a motor vehicle (10), wherein the control device (1) is designed to obtain (S1) the current driving speed of the motor vehicle (10) traveling in the forward direction, and to control (S2) the field-of-view display device (2) to output as augmented reality a warning (20) regarding a traffic-relevant situation that occurs at an entry point in the forward direction in front of the motor vehicle (10), wherein the warning (20) is virtually applied to a road (30) in the forward direction.The control device (1) is designed to control the field of vision display device (2) (S3), to virtually apply the message (20) for the first time at a position on the road (30) located in the direction of travel between the entry point and the motor vehicle (10), and to virtually move the message (20) along the road (30) depending on the current driving speed until the motor vehicle (10) has reached the entry point.

Description

[0001] The present disclosure relates to a control device for controlling an operation of a visual field display device for a motor vehicle, a motor vehicle comprising the control device, and / or a method for controlling an operation of a visual field display device for a motor vehicle. Additionally or alternatively, a computer program is provided which comprises instructions which, when executed by a computer, cause the computer to at least partially carry out the method. Additionally or alternatively, a computer-readable medium is provided which comprises instructions which, when executed by a computer, cause the computer to at least partially carry out the method.

[0002] Augmented reality (AR) displays are already known in their first forms for motor vehicles.

[0003] Video AR is known, for example, in which a live image of the driving scene is displayed on a screen, e.g. a central information display (CID) and / or an instrument cluster, which is enriched with virtual content such as turn-by-turn instructions.

[0004] Head-up display systems (HUDs) are also beginning to feature AR content. However, due to the currently limited display area, this content is very limited.

[0005] In addition, AR glasses are also known that are currently not (yet) permitted for use while driving a motor vehicle.

[0006] In motor vehicles, AR displays can be used for navigation, for example, with the displays typically becoming more staggered with increasing proximity to the maneuver. For example, a map including the route is available on one screen, e.g., the CID. As a maneuver is approached, a so-called guiding display, i.e., a route guidance output, is displayed, e.g., in the instrument cluster or HUD. Finally, the AR guiding, i.e., the route guidance or route guidance information, is displayed as augmented reality.

[0007] In principle, AR displays can provide excellent support for drivers during driving, as mental effort and thus driver distraction are reduced when relevant information is directly visible in real-life situations. Currently, the three application areas of navigation, driver assistance, and warnings are approved for motor vehicles.

[0008] Regardless of whether an AR HUD, AR glasses, or another system is used, the problem arises that at a distance of approximately 50-70 m from a location where, for example, a driving maneuver is to be performed, the objects or situations to be augmented become too small to be accurately augmented. Human perception is the main limiting factor here, but technical complexity such as precision or smoothness are also critical. Furthermore, above this threshold, there are often occlusions (e.g., a curve before a turning situation), so augmentation doesn't yet make sense.

[0009] At typical driving speeds, the range from 50-70 m to 0 m is traversed very quickly, so there's often too little time to react. While augmentation can provide real support, the driver must be given the opportunity to mentally prepare for the upcoming maneuver or traffic situation.

[0010] Against the background of this prior art, the object of the present disclosure is to provide a device and / or a method which are each suitable for enriching the prior art.

[0011] The problem is solved by the features of the independent claims. The subordinate claims and the dependent claims each contain optional developments of the disclosure.

[0012] The object is then achieved by a control device for controlling an operation of a field of view display device for a motor vehicle.

[0013] The control device is configured to obtain (optionally continuously and / or repeatedly) a current driving speed of the motor vehicle traveling in the forward direction. In other words, the control device can be configured to obtain the current driving speed of the motor vehicle when it is traveling or moving in the forward direction.

[0014] The control device is configured to control the field of view display device to output an indication regarding a traffic-relevant situation occurring at an entry point in the forward direction of travel in front of the motor vehicle as an augmented reality display (optionally in the field of view of a driver of the motor vehicle in the forward direction of travel and / or through a windshield of the motor vehicle). The indication is virtually applied to a road ahead in the forward direction of travel (optionally on a road surface of the road).

[0015] The control device is designed to control the field of view display device, to virtually apply the notice for the first time at a position on the road that is located in the forward direction of travel between the entry point and the motor vehicle, and to virtually move the notice along the road as a function of the current driving speed (optionally obtained continuously and / or repeatedly) until the motor vehicle has reached the entry point.

[0016] Augmented reality (AR) refers to the visual representation of (additional) information, i.e., the supplementation of the field of vision of a driver and / or other vehicle occupants (or, in general, images or videos) with computer-generated additional information or virtual objects by means of overlay and / or superimposition. The field of vision may be limited to the area in the forward direction of travel in front of the vehicle, e.g., the field of vision when looking through the windshield of the vehicle.

[0017] The field of vision display device can be a head-up display or a head-up display system (HUD). With the head-up display, information, e.g., the warning, can be superimposed in the form of a virtual image onto a real image of the surroundings in front of the motor vehicle observed by the driver or another vehicle occupant. For example, when looking through the windshield of the motor vehicle, the virtual image can be integrated into the surroundings (in the forward direction of travel) of the motor vehicle, e.g., with correct perspective. For this purpose, the head-up display can, for example, have an imaging unit for generating light beams with the desired content to be displayed in the virtual image. The head-up display can further have a mirror optic for projecting the light beams in a predetermined shape and direction onto a partially transparent reflective panel, e.g., the windshield of the motor vehicle.The desired content can be projected into the eyes of, for example, the driver via reflections on the reflective disc, optionally in such a way that the virtual image is displayed to the driver behind the reflective disc at the desired distance, size and quality.

[0018] Alternatively, or in addition, the field of view display device can comprise or display augmented reality glasses (AR glasses for short). These glasses can be worn, for example, by the driver of the motor vehicle while driving.

[0019] The control device or control unit can be part of the driver assistance system or represent it. The control device can be, for example, an electronic control unit (ECU). The electronic control unit can be an intelligent processor-controlled unit that can communicate with other modules, for example, via a central gateway (CGW) and, if necessary, can form the vehicle's on-board network via fieldbuses such as the CAN bus, LIN bus, MOST bus, FlexRay, and / or via the automotive Ethernet, e.g., together with telematics control units and / or environmental sensors.

[0020] The control device described above offers a number of advantages. Among other things, it provides improved technology that increases the driver's preparation time for a traffic-relevant situation. This advantageously allows the information to be displayed for a longer period of time and thus remain in the driver's field of vision longer than would be the case with a conventional, world-class AR display. The information can be displayed at a time when a "true" augmentation—i.e., a world-class AR display—of the information at the point of occurrence of the traffic-relevant situation does not yet make sense in terms of human perception and technical aspects, such as the precision or smoothness of the augmentation.

[0021] Possible further developments of the control device described above are explained in detail below.

[0022] The control device can be configured to control the visual field display device to virtually move the sign along the road at a virtual speed less than or equal to the current driving speed. A virtual distance between the sign and the motor vehicle can be kept constant or steadily (or continuously) decreased depending on the current driving speed.

[0023] The control device may be configured to control the field of view display device to output the indication in a non-world-fixed manner.

[0024] The position at which the sign is first virtually applied to the road may depend on the point at which the traffic-relevant situation occurs, as well as on the current driving speed and / or the currently permitted speed limit. The higher the current driving speed and / or the currently permitted speed limit, the greater the virtual distance between the position and the point at which it occurs.

[0025] The control device may be configured to control the field of view display device to virtually move the indication along a recommended and / or planned trajectory of the motor vehicle along the road.

[0026] The control device can be configured to control the field of view display device so that the indication, e.g. as an animation, deviates laterally from the recommended and / or planned trajectory within a predetermined distance from the entry point to indicate a direction of a recommended driving maneuver.

[0027] The traffic-relevant situation can be a traffic regulation and / or a maximum permissible speed that applies from the point of entry.

[0028] The traffic-relevant situation can be a turning maneuver and / or a lane change that is to be carried out according to a recommended and / or planned trajectory at the entry point.

[0029] The traffic-relevant situation can be a charging and / or refueling process of the motor vehicle, which is to take place at a charging and / or refueling station at the entry point.

[0030] It is conceivable that the traffic-relevant situation can be any situation that can be assigned to a specific position in the world. The traffic-relevant situation can refer to a (or any) point of interest (POI) and / or take the point of interest into account, e.g., a restaurant, a landmark, etc.

[0031] The sign can be placed virtually on the road in a perspective-correct manner (optionally in the real world).

[0032] The control device can be designed to control the field of view display device so that the information changes within a predetermined distance from the entry point into a world-fixed representation and / or into a representation in which the information is virtually applied to the road in a perspectively correct and fixed manner.

[0033] The above description can be summarized in other words and in a possible more concrete embodiment of the disclosure as described below, whereby the following description is not to be interpreted as limiting the disclosure.

[0034] The indication regarding a traffic-relevant situation can also be referred to as a pseudo-AR preview. The pseudo-AR preview can be displayed and, for example, inserted in addition to a guiding display or an AR display. This pseudo-AR preview can already be world-based in some aspects, i.e., quasi-AR, but not yet attached to the object in question.

[0035] World-fixed AR aspects can be perspectively correct positioning and orientation depending on the real world, e.g., AR objects can be correctly positioned on the street.

[0036] Pseudo AR aspects can include the pseudo AR preview not being fixed in the world, but being able to move depending on the movement of the vehicle. The influence of the vehicle's movement on the representation in the virtual image does not have to be physically correct, but can be consciously controlled (enlarged or reduced). This allows virtual objects to move faster or slower relative to the vehicle than they would in real life. By reducing the influence of the vehicle's movement, it could be ensured, for example, that virtual objects remain in the display's field of view for longer.

[0037] The goal of this pseudo-AR preview may be to increase the driver's preparation time. This avoids surprising the driver with the AR display, with the goal of facilitating the driver's mental preparation and allowing enough time to initiate a maneuver. This can be useful and important, for example, for navigation and warnings.

[0038] The pseudo-AR preview can be displayed before a point in time at which real augmentation makes sense.

[0039] Depending on the permitted speed (road class), the pseudo-AR forecast cannot be initiated too early to avoid distraction. On country roads, a pseudo-AR forecast can be initiated from approximately 200 meters. In 30 km / h zones, a pseudo-AR forecast can only be initiated from 100 meters, or in certain situations, the pseudo-AR forecast can be omitted.

[0040] The pseudo-AR preview can follow the vehicle's own lane. It can move along the trajectory recommended next. In some situations, the pseudo-AR preview can deviate from this towards the end of the animation and, for example, subliminally indicate the direction of the recommended maneuver.

[0041] The pseudo-AR preview can partially anticipate the symbolism of the subsequent augmentation. Furthermore, the pseudo-AR preview can be discreet and not obstruct the driver's field of vision.

[0042] Furthermore, a motor vehicle is provided which comprises the control device described above.

[0043] The motor vehicle may include a field of view display device configured to display augmented reality. The field of view display device may, for example, comprise or be a head-up display.

[0044] The motor vehicle may be a passenger car, in particular an automobile, or a commercial vehicle, such as a truck.

[0045] The control device can be connected to an on-board network of the motor vehicle and / or a sensor device of the motor vehicle for signaling purposes in order to obtain the current driving speed.

[0046] The motor vehicle may include a navigation device that may be signal-connected to the control device. The navigation device may be configured to provide a recommended and / or planned trajectory or route of the motor vehicle. The navigation device may be configured to detect a traffic-relevant situation along the recommended and / or planned trajectory or route (e.g., in the forward direction of travel of the motor vehicle).

[0047] The motor vehicle may be automated. The motor vehicle may be configured to at least partially and / or at least temporarily assume longitudinal and / or lateral guidance during automated driving of the motor vehicle by means of the control device.

[0048] Automated driving can be implemented in such a way that the motor vehicle's movement is (largely) autonomous. Automated driving can be controlled at least partially and / or temporarily by the control device.

[0049] It is conceivable that the motor vehicle intervenes in the lateral guidance of the motor vehicle through a driver assistance system actively, e.g. by adjusting an actual steering wheel position, and optionally passively, e.g. by displaying a turn-off instruction.

[0050] What has been described above with reference to the control device also applies analogously to the motor vehicle and vice versa.

[0051] Furthermore, a method for controlling an operation of a head-end display device for a motor vehicle is provided.

[0052] The method comprises obtaining a current driving speed of the motor vehicle traveling in the forward direction.

[0053] The method comprises controlling the field of view display device to output, as augmented reality, an indication regarding a traffic-relevant situation occurring at an entry point in the forward direction of travel in front of the motor vehicle. The indication is virtually applied to a road ahead in the forward direction of travel.

[0054] The method comprises controlling the field of view display device to virtually apply the notice for the first time at a position on the road that is located in the forward direction of travel between the entry point and the motor vehicle, and to virtually move the notice along the road depending on the current driving speed until the motor vehicle has reached the entry point.

[0055] The (control) method may be a computer-implemented method, i.e. one, several or all steps of the method may be carried out at least partially by a computer or a data processing device, optionally the control device.

[0056] What has been described above with reference to the control device and the motor vehicle also applies analogously to the method and vice versa.

[0057] Furthermore, a computer program is provided, comprising instructions which, when the program is executed by a computer, cause the computer to at least partially carry out or implement the method described above.

[0058] A program code of the computer program may be in any code, in particular in a code suitable for motor vehicle controls.

[0059] What has been described above with reference to the control device, the motor vehicle and the method also applies analogously to the computer program and vice versa.

[0060] Furthermore, a computer-readable medium, in particular a computer-readable storage medium, is provided. The computer-readable medium comprises instructions which, when executed by a computer, cause the computer to at least partially execute or carry out the method described above.

[0061] This means that a computer-readable medium can be provided that contains a computer program as defined above. The computer-readable medium can be any digital data storage device, such as a USB flash drive, a hard disk, a CD-ROM, an SD card, or an SSD card (or SSD drive / SSD hard disk).

[0062] The computer program does not necessarily have to be stored on such a computer-readable storage medium in order to be made available to the motor vehicle, but can also be obtained via the Internet or otherwise externally.

[0063] What has been described above with reference to the method, the control device, the computer program and the motor vehicle also applies analogously to the computer-readable medium and vice versa.

[0064] Below is an optional embodiment with reference to Fig. 1 to described. Fig. 1 schematically shows a motor vehicle according to the disclosure with a control device for controlling an operation of a field of view display device of the motor vehicle, Fig. 2 schematically shows a flow diagram of a method according to the disclosure for controlling an operation of the field of view display device; and Fig. 3 to 5 schematically show exemplary views by means of the field of view display device.

[0065] The Fig. 1, the motor vehicle 10 shown only schematically comprises the control device 1.

[0066] The control device 1 is signal-connected to an imaging unit 2 of the field of vision display device. The field of vision display device can be, for example, a head-up display, wherein the imaging unit 2 is designed to generate light beams with a desired display content. These light beams are reflected, for example, via a mirror optics and a partially transparent reflective screen, e.g., the windshield 11 of the motor vehicle 10, in order to project the desired display content into the eyes of the driver (or another vehicle occupant). The mirror optics ensure that the desired display content is presented to the driver as a virtual image and thus as augmented reality behind the reflective screen at the desired distance, size, and quality. This virtual image can, for example, be an indication of a traffic-relevant situation that may, for example,world-wide or quasi-world-wide as an augmented reality in the driver's field of vision.

[0067] Alternatively, or in addition, other field of view display devices are also conceivable, e.g., glasses for displaying augmented reality, which are worn by the driver while driving the motor vehicle 10.

[0068] To control the operation of the field of view display device or the imaging unit 2, the control device 1 is designed to carry out the following with reference to Fig. 2 to carry out the procedures 100 described in detail.

[0069] In a first method step S1, a current driving speed of the motor vehicle 10 traveling in the forward direction is obtained.

[0070] In a second method step S2, the field of view display device is controlled to output an indication regarding a traffic-relevant situation that occurs at an entry point in the forward direction of travel in front of the motor vehicle 10 as an augmented reality, wherein the indication is applied virtually, optionally with a correct perspective in the real world, to the road on a road ahead in the forward direction of travel.

[0071] Various traffic-relevant situations are conceivable that can be indicated by the notice.

[0072] The traffic-relevant situation can be a traffic regulation and / or a speed limit that applies from the point of entry. The information can, for example, be a traffic sign corresponding to the traffic regulation or the speed limit.

[0073] Another possible traffic-relevant situation could be a turning maneuver and / or a lane change that is to be performed according to a recommended and / or planned trajectory at the entry point. The indication can be indicated, for example, by arrows or trajectory stripes, with the arrows or trajectory stripes pointing in the direction of the turn or the direction of the new lane.

[0074] Furthermore, it is conceivable, for example, that the traffic-relevant situation can be a motor vehicle-related situation, e.g., an upcoming charging and / or refueling process of the motor vehicle 10, which is to take place at a charging and / or refueling station at the entry point.

[0075] In a third method step S3, the field of view display device is controlled to virtually apply the notice for the first time at a position on the road that is located in the forward direction of travel between the entry point and the motor vehicle 10, and to virtually move the notice depending on the current driving speed along the road, e.g., on a recommended and / or planned trajectory of the motor vehicle 10 along the road, until the motor vehicle 10 has reached the entry point.

[0076] For this purpose, the sign can be moved virtually along the road at a virtual speed less than or equal to the current driving speed. Thus, the motor vehicle 10 can approach the sign or "push" the sign in front of it until the entry point is reached.

[0077] The position at which the sign is first virtually applied on the road may depend on the location where the traffic-relevant situation occurs and also on the current driving speed and / or a currently permitted maximum speed.

[0078] The indication may, for example, be an animation and may point in the direction of a recommended driving maneuver within a specified distance from the entry point or deviate laterally from the recommended and / or planned trajectory.

[0079] This can, for example, indicate a recommended turning maneuver and / or lane change.

[0080] Fig. 3 to 5 schematically show exemplary views using the field of view display device. While driving the motor vehicle 10, the driver can look through the windshield 11 in the forward direction as usual, with the field of view display device or the imaging unit 2 generating virtual information in the driver's eyes. The virtual information can accordingly be the indication 20.

[0081] In this case, the indication 20 is virtually applied to the road 30 ahead in the forward direction of travel and is virtually moved along the road depending on the current driving speed of the motor vehicle 10.

[0082] In the Fig. 3(a) and (b), Notice 20 is a caution sign indicating an upcoming traffic regulation, e.g., an upcoming traffic junction such as an intersection or a junction, where the driver must drive with particular caution and be ready to brake at any time. Notice 20 is therefore displayed before the point of entry into this traffic regulation so that the driver has sufficient time to adjust to this traffic-relevant situation.

[0083] The indication 20 moves depending on the current driving speed of the motor vehicle 10, whereby the virtual speed v H of note 20 less than or equal to the current driving speed v K of the motor vehicle 10, as indicated by the arrows in Fig. 3(b). Thus, the indicator 20 is not displayed in the driver's field of vision in a fixed, world-wide manner, but also virtually moves in the forward direction, so that the indicator 20 remains in the driver's field of vision longer than a typical fixed, world-wide representation of augmented reality.

[0084] The Fig. 4 and Fig. 5 show two further examples of Note 20. In Fig. 4, an arrow on road 30 indicates an imminent right turn according to a recommended and / or planned trajectory at the entry point. Fig. 5, the indicator 20 is a lightning bolt, which is intended to indicate an imminent charging process of, for example, an electrically powered motor vehicle 10. For this purpose, there may be a charging station located ahead along road 30, for example, which the driver should drive to. This indicator 20 allows the driver to prepare accordingly in advance. List of reference symbols 1 control device 2 imaging unit of the field of view display device 10 motor vehicle 11 Windshield 20 Note 30 Street 100 procedures S1-S3 process steps

Claims

[1] Control device (1) for controlling an operation of a field of view display device (2) for a motor vehicle (10), wherein the control device (1) is designed to: - to obtain (S1) a current driving speed of the motor vehicle (10) traveling in the forward direction; and - to control (S2) the field of vision display device (2) to output an indication (20) regarding a traffic-relevant situation occurring at an entry point in the forward direction of travel in front of the motor vehicle (10) as an augmented reality, wherein the indication (20) is virtually applied to a road (30) ahead in the forward direction of travel, characterized by that the control device (1) is designed to: - to control (S3) the field of view display device (2), to virtually apply the indication (20) for the first time at a position on the road (30) which is located in the forward direction of travel between the entry point and the motor vehicle (10), and to virtually move the indication (20) along the road (30) as a function of the current driving speed until the motor vehicle (10) has reached the entry point. [2] Control device (1) according to claim 1, characterized by that the control device (1) is designed to: - to control the field of view display device (2) to virtually move the indication (20) along the road (30) at a virtual speed less than or equal to the current driving speed. [3] Control device (1) according to claim 1 or 2, characterized bythat the position at which the indication (20) is first virtually applied to the road (30) depends on the point of occurrence of the traffic-relevant situation and also on the current driving speed and / or a currently permissible maximum speed. [4] Control device (1) according to one of the preceding claims, characterized by that the control device (1) is designed to: - to control the field of view display device (2) to virtually move the indication (20) along a recommended and / or planned trajectory of the motor vehicle (10) along the road (30). [5] Control device (1) according to claim 4, characterized by that the control device (1) is designed to: - to control the field of view display device (2) so that the indication (20) deviates laterally from the recommended and / or planned trajectory within a predetermined distance from the entry point for indicating a direction of a recommended driving maneuver. [6] Control device (1) according to one of the preceding claims, characterized by that the traffic-relevant situation - a traffic regulation and / or a maximum speed limit that applies from the point of entry, - a turning manoeuvre and / or a lane change that is to be carried out according to a recommended and / or planned trajectory at the point of entry, and / or - a charging and / or refueling process of the motor vehicle (10) which is to take place at a charging and / or refueling station at the entry point. [7] Control device (1) according to one of the preceding claims, characterized by that the sign is virtually placed on the road in the real world with the correct perspective. [8] Motor vehicle (10), characterized by that the motor vehicle (10) comprises the control device (1) according to one of the preceding claims. [9] Method (100) for controlling an operation of a visual field display device (2) for a motor vehicle (10), the method (100) comprising: - obtaining (S1) a current driving speed of the motor vehicle (10) traveling in the forward direction; and - controlling (S2) the field of view display device (2) to output an indication (20) regarding a traffic-relevant situation occurring at an entry point in the forward direction of travel in front of the motor vehicle (10) as an augmented reality, wherein the indication (20) is virtually applied to a road (30) ahead in the forward direction of travel, characterized by that the method (100) comprises: - controlling (S3) the field of view display device (2) to virtually apply the indication (20) for the first time at a position on the road (30) which is located in the forward direction of travel between the entry point and the motor vehicle (10), and to virtually move the indication (20) along the road (30) as a function of the current driving speed until the motor vehicle (10) has reached the entry point. [10] Computer program and / or computer-readable medium, characterized by that the computer program and / or the computer-readable medium comprises instructions which, when the program is executed by a computer, cause the computer to carry out the method (100) according to claim 9.

Images