METHOD FOR CALCULATING AN "AUGMENTED REALITY" OVERLAY FOR DISPLAYING A NAVIGATION ROUTE ON AN AR DISPLAY UNIT, DEVICE FOR EXECUTING THE METHOD, AS WELL AS MOTOR VEHICLE AND COMPUTER PROGRAM

Description

[0001] The proposal concerns the technical field of driver information systems, also known as infotainment systems. Such systems are primarily used in vehicles. However, the invention could also be used by pedestrians, cyclists, etc., with smart glasses. The proposal further includes a suitably designed device for carrying out the method, as well as a motor vehicle and a computer program.

[0002] A future vision in the automotive industry is the ability to project virtual elements onto the windshield of one's own vehicle, offering the driver several advantages. This utilizes so-called "Augmented Reality" (AR) technology. The corresponding German term "erweiterte Realität" (extended reality) is less common. In this technology, the real environment is enhanced with virtual elements. This has several advantages: Looking down at displays other than the windshield becomes unnecessary, as much relevant information is projected onto the windshield. Thus, the driver doesn't have to take their eyes off the road. The special feature of AR displays is the ability to precisely position the virtual elements within the real environment. The virtual element is displayed at the point where the driver's gaze is directed in the real world.These overlays allow the real-world environment to be "superimposed" on the user's viewpoint and supplemented with additional information, such as a navigation route. This reduces the cognitive effort required from the driver, as there is no need to interpret an abstract graphic; instead, an intuitive understanding based on normal perceptual habits can occur.

[0003] Currently, head-up displays (HUDs) are used as AR display units in vehicles. These have the advantage that the HUD image appears closer to the real world. These displays are essentially projection units that project an image onto the windshield. However, depending on the module's design, this image is located a few to 15 meters in front of the vehicle from the driver's perspective. This has the advantage that the displayed information is presented in such a way that the driver's eyes are relieved of the need for accommodation.

[0004] The "image" is composed as follows: It's less of a virtual display and more of a "keyhole" into the virtual world. The virtual environment is theoretically overlaid on the real world and contains the virtual objects that support and inform the driver. The limited display area of ​​the HUD means that only a portion of it is visible. In other words, you're looking through the HUD's display area at this section of the virtual world. Because this virtual environment complements the real world, it's also referred to as "mixed reality."

[0005] Currently, intensive work is also underway on technologies that will eventually enable autonomous driving. A first approach is not to completely relieve the driver of their duties, but rather to ensure that the driver can take control of the vehicle at any time. The driver also performs monitoring functions. Newer technologies in the field of driver information systems, such as head-up displays, make it possible to better inform the driver about what is happening around the vehicle.

[0006] Due to the current trend towards higher levels of autonomy, where many vehicles are still driven by a human operator, it can be assumed that corresponding additional information will be usable for manually driven vehicles in the medium term, and not only for highly automated systems in the long term. The solution described in more detail below can be used for both manually driven and automatically driven vehicles.

[0007] For driver-vehicle interaction, the question arises as to how this information can be presented in a way that creates genuine added value for the human driver and allows them to quickly and intuitively locate the information provided. The following solutions in this area are already known from the state of the art.

[0008] Most vehicles today are equipped with a navigation system to provide destination and route guidance for the driver. Furthermore, vehicles with a head-up display (HUD) are available on the market, where the HUD projects desired information onto the windshield, allowing the driver to view the projected information while looking ahead.

[0009] German patent application DE 10 2009 047 410 A1 proposes a method for dynamically determining a route using a navigation system. Several alternative routes are calculated. Up to a junction point, the routes are identical and displayed as a single route. From the junction point onward, the routes diverge. The navigation system recognizes which route has been selected based on the corresponding driving maneuver. When the vehicle turns, the selected route is chosen.

[0010] From US patent 2017 / 314945 A1, another method for generating routes is known, in which a main route from an origin to a destination is first generated. During the journey, decision points along the main route are identified, and alternative routes between each decision point and the destination are generated, which can then be used as the new main route from the decision point to the destination.

[0011] From DE 10 2013 016 241 A1, a method for the augmented display of at least one additional piece of information in at least one recorded digital image of an environment, in particular a vehicle environment, is known, wherein the recorded image is output as an output image forming a background on a display and the additional information is superimposed on the output image depending on the associated background area of ​​the output image.

[0012] A method for reducing driver distraction is known from DE 10 2017 211 821 A1. The method can include projecting one or more graphic elements onto a windshield in the driver's field of vision using a head-up display (HUD). The method can also include determining the content status of one or more projected graphic elements. Furthermore, the method can include detecting one or more objects in the vehicle's surroundings. The method can also include notifying the driver about one or more detected objects if the content status meets a distraction criterion. The method is also used in vehicle navigation.

[0013] Relatively obvious examples usually relate to the field of navigation. While classic navigation displays (with conventional LCD displays) generally show schematic representations (e.g., a right-angled arrow indicating that a right turn should be made at the next opportunity), AR displays offer significantly more effective possibilities. Since the displays can be presented as "part of the environment," extremely fast and intuitive interpretations are possible for the user. Nevertheless, the approaches known to date also exhibit various problems for which no solutions are currently known. This was recognized within the scope of the invention.

[0014] In the future, in-vehicle navigation will be increasingly supported by displays on a head-up display (augmented or with 2D maneuver indicators). To provide the user with consistent road and route guidance, the system augments a navigation path directly onto the road.

[0015] If route changes are necessary due to traffic disruptions, congestion, detours, or similar issues, current navigation devices calculate a new route and inform the user of the recalculation. US Patent 2010 / 0292886 A1 discloses how an AR projection using a head-up display generates information about the driving path of a road, showing lane information, directions, distances, and warnings. However, until now, there has been no information or visualization of what has changed on the route. Therefore, the new route is not visible to the user, and they are only informed about the imminent maneuver when the navigation device provides voice prompts. This problem was addressed in the invention.

[0016] Therefore, there is a need for further improvements in vehicle route guidance and the related feedback to the driver via the infotainment system.

[0017] The invention aims to better support the driver when routes change.

[0018] This problem is solved by a method for calculating an augmented reality overlay for displaying a navigation route on an AR display unit according to claim 1, a device for carrying out the method according to claim 7, a motor vehicle according to claim 10, and a computer program according to claim 11. The overlay serves the purpose of assisting the driver in the longitudinal control of the vehicle.

[0019] The dependent claims include advantageous further developments and improvements of the invention in accordance with the following description of these measures.

[0020] The method for calculating an AR overlay of a route calculated by a navigation system, which is displayed to a vehicle driver on an AR display unit, in particular a vehicle's head-up display, consists of recalculating the route from time to time. A navigation arrow is then displayed when the vehicle is on a set route, but no immediate driving maneuver is imminent.At a certain distance before a junction, a notification of a newly calculated route is displayed, giving the driver the option to display an overview of the newly calculated route, and when this option is exercised, the overview of the newly calculated route is displayed, and giving the driver another option to switch back and forth between the overview of the newly calculated route and an overview of a previously calculated route, and when this further option is exercised, either the overview of the newly calculated route or the overview of the previously calculated route is displayed, and the respective route is shown along its course to the destination.

[0021] At a second distance before the junction, which is shorter than the first distance, a navigation path augmented onto the road for the newly calculated route is displayed next to a navigation path augmented onto the road for the previously calculated route, and the navigation path for the route that the vehicle is not following is hidden as soon as the navigation system has recognized which route the vehicle is following by the vehicle's turn signal being activated.

[0022] To recalculate the route periodically, the system can consider specific criteria and then decide whether to recalculate. For example, if delays are detected on the current route or time improvements are identified on alternative routes, the route can be recalculated. This method has the advantage that the system recommends an alternative route and displays it directly in the user's field of vision. The driver experiences minimal distraction but gains the ability to quickly assess the new route. The user interface allows the driver to compare both routes side-by-side.

[0023] The user is shown an overview of the originally calculated route combined with the new route recommendation and can choose one of the two options (preferably the new recommendation) based on the different routes. If information regarding the estimated travel time for each route is also displayed (minutes to the destination), the user can save time by choosing the route with the shorter travel time.

[0024] Before the newly calculated route is displayed, an AR overlay is calculated to give the driver a hint about the newly calculated route. An advantageous measure is to provide operating instructions on how the driver can display the newly calculated route.

[0025] To provide an overview of the route, the AR overlay is calculated to display the newly calculated route and the previously calculated route in such a way that the respective route is shown in its course up to the destination point.

[0026] Another advantageous measure is to calculate the AR overlay for the route overview in a zoomable manner, allowing zooming to be performed using steering wheel controls. This is beneficial for the driver when they want to take a closer look at the route.

[0027] Furthermore, it is advantageous for the process if an assessment of the driving situation is carried out before the route overview is displayed, and the display is only carried out if the assessment shows that the driving situation does not require increased attention from the driver. This minimizes the potential for driver distraction.

[0028] It is also advantageous if the AR overlay for displaying the newly calculated route is designed to differ in color from the AR overlay for displaying the previously calculated route. This further improves the distinguishability of the routes.

[0029] It is advantageous to represent the route requiring more time in red or orange, while the more time-efficient route is represented in blue or green. This corresponds to the usual way of indicating that one option is more advantageous than the other.

[0030] Once the driver has chosen one of the two routes, the route the vehicle is not following is hidden. The navigation system uses "position tracking" to determine which route the vehicle is following. Position tracking is also known as GPS tracking. The navigation system analyzes the vehicle's current position. After the junction, the vehicle can only be on one of the two routes. The other route can then be hidden.

[0031] A device for carrying out the method according to the invention comprises an AR display unit, a processing unit, and a navigation system. The navigation system calculates a route and is designed to recalculate the route periodically to adapt to changing conditions, particularly traffic conditions. The processing unit performs the steps for calculating an AR overlay of a route calculated by the navigation system on the AR display unit. The processing unit is designed to calculate the AR overlay such that a navigation arrow is displayed when the vehicle is on a preset route but no immediate driving maneuver is imminent.a message indicating a newly calculated route is displayed at a certain distance before a junction, wherein the message includes a control option to display an overview of the newly calculated route, and when this control option is exercised, the overview of the newly calculated route is displayed, and wherein a further control option allows switching back and forth between the display of the route overview for the newly calculated route and the previously calculated route, and when this further control option is exercised, either the overview of the newly calculated route or the overview of the previously calculated route is displayed, and wherein the respective route is displayed in its course up to the destination point;and at a second distance, which is shorter than the first distance before the junction, a navigation path augmented onto the road for the newly calculated route is displayed next to a navigation path augmented onto the road for the previously calculated route, and the navigation path for the route that the vehicle is not following is hidden as soon as the navigation system has recognized, from the vehicle's turn signal activation, which route the vehicle is following.

[0032] The device is designed so that the calculations for AR overlays, as listed in the corresponding process steps of the inventive method, are performed using the appropriately programmed computing unit.

[0033] The AR display unit can advantageously be implemented as a head-up display.

[0034] Furthermore, the same advantages apply to the device for carrying out the method with the correspondingly programmed computing unit as mentioned in the claims with the corresponding method steps.

[0035] The device according to the invention can be used in a motor vehicle. In the vehicle, the invention is preferably implemented in such a way that the display unit is permanently installed in the vehicle, e.g. in the form of a head-up display.

[0036] For a computer program that is executed in the computing unit of the device in order to carry out the method according to the invention, the corresponding advantages apply as described for the method according to the invention.

[0037] Exemplary embodiments of the invention are shown in the drawings and are explained in more detail below with reference to the figures.

[0038] They show: Fig. 1 The principle of displaying information in the driver's field of vision while driving using a head-up display; Fig. 2 The typical cockpit of a vehicle; Fig. 3 A block diagram of the vehicle's infotainment system; Fig. 4 A representation of an AR display for the driver indicating that the route has been recalculated; Fig. 5 A representation of an AR display for the driver visualizing a route overview of the recalculated route with the option to switch to the display of the previously calculated route; Fig. 6 A representation of an AR display for the driver visualizing a route overview of the original route with the option to switch to the display of the recalculated route; Fig. 7 A representation of an AR display for the driver with a simplified navigation display for situations where no special driving maneuvers are imminent; Fig.Fig. 8 shows an AR overlay for the driver displaying the current navigation path in the event of an upcoming driving maneuver; Fig. 9 shows an AR overlay for the driver displaying the current and a newly calculated navigation path in the event of an upcoming turning maneuver; Fig. 10 shows an AR overlay for the driver displaying the newly calculated navigation route selected by the driver; and Fig. 11 shows a flowchart for a program to calculate AR overlays for displaying the different navigation routes.

[0039] The present description illustrates the principles of the inventive disclosure. It is therefore understood that those skilled in the art will be able to design various arrangements which, although not explicitly described here, embody principles of the inventive disclosure and which are also intended to be protected in their scope.

[0040] Fig. 1 This illustrates the basic functionality of a head-up display. The head-up display 20 is mounted in the vehicle 10 below / behind the instrument cluster in the dashboard area. Additional information is projected onto the windshield and displayed within the driver's field of vision. This additional information appears as if it were projected onto a projection surface 21 located 7–15 m in front of the vehicle 10. However, the real world remains visible through this projection surface 21. The displayed additional information essentially creates a virtual environment. Theoretically, this virtual environment is superimposed on the real world and contains virtual objects that support and inform the driver while driving. However, it is only projected onto a portion of the windshield, meaning the additional information cannot be arbitrarily positioned within the driver's field of vision.

[0041] Fig. 2 Figure 10 shows the cockpit of vehicle 10. A passenger car is depicted. However, any other vehicle could also be considered vehicle 10. Examples of other vehicles include: buses, commercial vehicles, especially trucks, agricultural machinery, construction machinery, rail vehicles, etc. The invention could generally be used in land vehicles, rail vehicles, watercraft, and aircraft.

[0042] In the cockpit, two display units of an infotainment system are highlighted with reference symbols. These are the head-up display 20 and a touchscreen 30, which is mounted in the center console. While driving, the center console is not in the driver's field of vision. Therefore, the additional information is not displayed on display unit 30 while driving.

[0043] The touchscreen 30 is used primarily for operating vehicle functions 10. For example, it can be used to control a radio, a navigation system, playback of stored music, and / or air conditioning, other electronic equipment, or other comfort functions or applications of the vehicle 10. This is often referred to collectively as an "infotainment system." In motor vehicles, especially passenger cars, an infotainment system refers to the integration of the car radio, navigation system, hands-free system, driver assistance systems, and other functions into a central control unit. The term "infotainment" is a portmanteau word, composed of the words "information" and "entertainment."The infotainment system is primarily operated via the touchscreen 30, which is easily visible and operable by both the driver and passenger of the vehicle 10. Below the touchscreen 30, mechanical controls, such as buttons, rotary knobs, or combinations thereof, like rotary push-button controls, may be arranged in an input unit 50. Typically, parts of the infotainment system can also be operated via the steering wheel. This unit is not shown separately but is considered part of the input unit 50.

[0044] Fig. 3 Figure 1 schematically shows a block diagram of the infotainment system 200, as well as examples of some subsystems or applications of the infotainment system. The operating device comprises the touch-sensitive display unit 30, a computing unit 40, an input unit 50, and a memory unit 60. The display unit 30 includes both a display area for showing variable graphical information and a user interface (touch-sensitive layer) arranged above the display area for entering commands by a user.

[0045] The display unit 30 is connected to the computer unit 40 via a data line 70. The data line can be designed according to the LVDS standard, corresponding to Low Voltage Differential Signaling. The display unit 30 receives control data from the computer unit 40 via data line 70 to control the display area of ​​the touchscreen 30. Control data for the entered commands from the touchscreen 30 to the computer unit 40 is also transmitted via data line 70. The input unit is designated by the reference number 50. It includes the aforementioned operating elements such as buttons, rotary knobs, sliders, or rotary push-buttons, which the operator uses to make inputs via the menu navigation. Input generally refers to selecting a menu option, changing a parameter, switching a function on and off, etc.

[0046] The storage unit 60 is connected to the computing unit 40 via a data line 80. Storage unit 60 contains a pictogram directory and / or symbol directory with pictograms and / or symbols for the possible display of additional information. The points / symbols that serve as the basis for calculating the grid display can also be stored here.

[0047] The remaining components of the infotainment system—camera 150, radio 140, navigation system 130, telephone 120, and instrument cluster 110—are connected to the infotainment system control unit via data bus 100. The high-speed version of the CAN bus according to ISO standard 11898-2 is a suitable option for data bus 100. Alternatively, a bus system based on Ethernet technology, such as BroadR-Reach, could be used. Bus systems that transmit data via fiber optic cables are also possible. Examples include the MOST bus (Media Oriented System Transport) and the D2B bus (Domestic Digital Bus). It should also be noted that camera 150 can be configured as a conventional video camera. In this case, it records 25 full frames per second, which corresponds to 50 half fields per second in interlaced recording mode.Alternatively, a special camera can be used that captures more images per second to increase the accuracy of object detection for faster-moving objects. Multiple cameras can be used for environmental monitoring. In addition, the previously mentioned radar or lidar systems could be used to supplement or expand environmental monitoring. For internal and external wireless communication, the vehicle is equipped with a communication module 160. This module is often referred to as an on-board unit. It can be designed for mobile communication, e.g., according to the LTE standard (Long Term Evolution). It can also be designed for WLAN communication (Wireless LAN), whether for communication with occupant devices in the vehicle or for vehicle-to-vehicle communication, etc.

[0048] The inventive method for calculating an AR overlay of additional information for a display on an AR display unit 20 is explained below using two exemplary embodiments.

[0049] For the other figures, the same reference numbers denote the same fields and symbols as in the description of the Figuren 1 bis 3 explained.

[0050] First, the process of visualizing route changes at a greater distance (outside the field of view) is considered. Fig. 4 The diagram shows that vehicle 10 is being guided along the selected route by navigation system 130. An example of minimalist route guidance is shown, where only the navigation arrow 2001 is displayed to indicate that the vehicle is traveling on the set route. From time to time, navigation system 130 recalculates the route. If another route proves to be more advantageous than the previous route based on one or more criteria, a notification 2002 is displayed to the driver indicating that the route has been recalculated.

[0051] Possible criteria by which the routes are assessed include: Distance, travel time, energy consumption, road quality, traffic density.

[0052] The list is not exhaustive and further criteria may be considered, individually or in combination with one or more of the other criteria.

[0053] If the junction of the new route is far enough away and not in sight, only message 2002 will initially appear, indicating that the route has been recalculated. A certain minimum distance, e.g., 5 km, can be programmed to determine whether the distance to the junction is sufficient. Along with message 2002 about the recalculated route, a user guide 2003 may also be displayed, explaining how to view further details about the recalculated route.

[0054] This allows the user to display a route overview of the newly calculated route. To do this, the driver selects the "Detailed View 2003" option using the multifunction steering wheel controls (MFL). The new route will then be displayed on the HUD 20. For example, the detailed view can be selected by pressing the up arrow button on the MFL.

[0055] The route overview for the new route is in Fig. 5 The route is shown and has the reference number 2005. The general outline of the newly calculated route is visible there. The starting point is indicated by the typical navigation arrow. The destination of the route is also marked with a flag. The route overview displays the route, for example, in blue. The surrounding roads are shown in a map-like manner in a different color, such as gray. Below route 2005, you will find the navigation arrow 2001 and the note 2004, indicating that the new route involves a distance of 52 km and an estimated travel time of 22 minutes. To the left and right of the route are two arrow buttons 2006, which serve as operating instructions. Pressing the right arrow button or the left arrow button allows you to switch between the route overview of the various alternative routes.

[0056] In the Fig. 6 The route overview for the original route is shown in 2008. The original route is displayed in a different color, preferably red. Below this are the navigation arrow (2001) and the message (2007) indicating that this is the original route, with the information: 44 km distance and 28 minutes travel time. The arrow buttons (2006) are also visible next to the route overview. In another selection option, both routes (2005 and 2008) are displayed simultaneously. This makes it easier for the driver to assess the difference between the two routes.

[0057] The remaining controls via the multifunction steering wheel (MFL) can be operated using the following buttons. The OK button (confirmation button) accepts the displayed route for the 130 navigation system. An MFL button (back or down arrow) exits the overview.

[0058] For displaying route overviews, the following rules are advantageous: The route with the longer travel time should be marked in orange or red to indicate that a more attractive route exists. The more attractive route should be displayed in the same color as the active route (e.g., blue).

[0059] The estimated travel time should be displayed for both routes. Alternatively, the additional travel time can be shown only for the route with the longer travel time.

[0060] The newly calculated route should indicate the type of road using a color code. For the driver, especially on longer journeys, it's helpful to know whether they can continue on a highway, are being diverted onto a country road, or if toll roads have been included.

[0061] The proposed route overview should be zoomable using the steering wheel controls, so that even short route changes can be viewed.

[0062] The route overview should only be displayed in undemanding driving situations (due to its size and surface area). Alternatively, limiting the route overview display to situations where the vehicle is stationary is conceivable. Additionally, displaying the route overview on display unit 30 in the center console is a possibility.

[0063] The following describes an exemplary implementation that enables a visualization of route changes located directly in front of the vehicle.

[0064] The initial situation is in Fig. 7 It is also a simplified navigation view, similar to the one shown in Fig. 4 The user recognizes from the navigation arrow 2001 that navigation is active, but no immediate driving maneuver is imminent. If a driving maneuver becomes necessary, the user is notified with sufficient advance notice. In this case, an augmented navigation path 2009 expands from the reduced navigation display. This is in Fig. 8 The reduced view and the navigation path both use the same graphic element (the navigation arrow 2001 or a point), which is augmented onto the road at a certain distance. Vehicle 10 then drives over the displayed navigation arrows.

[0065] When a new route recommendation is given, a new route is displayed alongside the originally calculated route. The routes are color-coded: The original route with a longer travel time to the destination is shown in red (or orange), while the new recommendation is displayed in the usual navigation color (e.g., blue). Additionally, the delay duration is displayed for both route recommendations. This is in the Fig. 9 The original route, shown in red, is labeled with the reference number 2009. The new route, labeled with the reference number 2010, is shown in blue. Additionally, the following information can be found in... Fig. 9 Another turn instruction from 2011 indicates that a maneuver is pending at a distance of 50 m. The driver must decide which route to follow. Besides the option of displaying both routes side by side, an alternative implementation allows the newly calculated route recommendation (blue) to be displayed in such a way that it pushes the original route from 2009 (red) to the left.

[0066] The new route recommendation can be accepted automatically by the user following it. As previously described, this can be detected via position tracking when the vehicle reaches the junction. The new route is then automatically adopted, and the original route is hidden.

[0067] In an alternative embodiment, the route selection can be intelligently recognized and adopted as user intention via the turn signal.

[0068] In another alternative implementation, the user actively selects one of the two routes. This can be done via a menu, e.g., using the multifunction steering wheel [click left or right and press the confirmation / OK button], or via voice command.

[0069] Fig. 10 This shows a moment when the driver has already selected the new route, for example, by activating the right turn signal. The original route has already been hidden. The navigation path to be followed is indicated by navigation arrows (2010). At the junction, a turn instruction (2011) is permanently superimposed onto the road surface.

[0070] Based on Fig. 11 A computer program, 400, for calculating the AR overlays for the various routes will be explained. Program 400 is executed in processing unit 40. The program start is designated by reference number 402. In program step 404, the route is recalculated periodically. This recalculation is performed cyclically to take into account changing traffic conditions such as traffic jams, accidents, breakdowns, and roadworks.

[0071] In program step 406, a query is performed to determine whether a junction is located ahead on one of the found routes at a certain distance, e.g., at least 5 km. If not, the program proceeds to step 430. Otherwise, in step 408, an AR overlay is displayed with the notification 2002 regarding the route recalculation, as shown in Fig. 4 represented, calculated.

[0072] When the driver requests a detailed view of the newly calculated route, the driving situation is assessed in program step 410. Here, a special driving maneuver may be announced via car-to-car communication, such as a cooperative braking maneuver intended to prevent a rear-end collision. This would be important, for example, in intelligent platooning. Another possibility is that a driver assistance system has detected a specific driving situation, such as insufficient distance to the vehicle ahead or excessive speed. Other examples are conceivable.

[0073] If such a special driving situation is not detected, the AR display of the 2005 route overview for the newly calculated route is calculated in step 412. The display is in Fig. 5 shown.

[0074] Using the left and right arrow keys, the driver can switch between the offered routes. Therefore, if the driver selects the route overview for the original route, the calculation of the AR overlay for the route overview of the original route takes place in step 414, as shown in Fig. 6 shown.

[0075] The program then checks in step 416 whether there is a junction a short distance ahead where vehicle 10 should turn onto one of the offered routes. The short distance could be 2 km or less. Other values ​​are possible. If this is not the case, an AR overlay for a minimalist route display is calculated in step 415. Such a minimalist route display is in Fig. 7 shown.

[0076] The program then branches back to query 416. If it detects that the waypoint where the vehicle must turn is a short distance ahead, step 418 calculates the AR overlay for displaying the current navigation path. This overlay is displayed in Fig. 8 It is displayed. It serves to indicate that a driving maneuver is imminent. This alerts the driver.

[0077] As the vehicle progresses further, the display changes. In step 420, the AR overlay is calculated to display both the original 2009 route and the newly calculated 2010 route simultaneously. This AR overlay is in Fig. 9 shown. The turn arrow 2011 and the information regarding the travel times of the two routes are also part of the calculated AR overlay.

[0078] The subsequent query 422 determines which route the driver chooses. This can be recognized by the activation of the turn signal. The other previously mentioned possibility involves the aforementioned position tracking. However, in this case, the route choice would be recognized somewhat later because both routes are identical up to the turning point.

[0079] If query 422 detects that the turn signal has been activated, the program branches to step 424, in which a new AR overlay is calculated. The original route 2009 is then hidden.

[0080] Step 428 follows, in which another AR overlay is calculated. This displays the new 2010 route as shown in... Fig. 10 The turning arrow 2011 is augmented onto the road in such a way that it appears permanently fixed at the turning point.

[0081] If neither the activation of the turn signal nor the selection of the new route 2010 is detected in step 422, another AR overlay is calculated in step 426. In this overlay, the newly calculated route 2010 is hidden, and the original route 2009 is retained.

[0082] Then, in step 430, the route guidance switches to minimalist mode. In step 430, the corresponding AR overlay is calculated. This step is also executed if the new route 2010 was selected. The minimalist view will be similar to that in Fig. 7 depicted.

[0083] Finally, in step 432, the program checks whether the destination of the navigation route has been reached. If not, the program returns to the beginning and the route is recalculated in step 404.

[0084] The program thus forms a loop through steps 404 to 432, which is executed repeatedly until query 432 detects that the planned navigation destination has been reached. If this is the case, the program terminates in program step 434. The program can be terminated at any time if the driver intervenes and disengages the comfort function.

[0085] All examples mentioned herein, as well as conditional formulations, are to be understood without limitation to such specifically cited examples. For instance, it is recognized by those skilled in the art that the block diagram shown here represents a conceptual view of an exemplary circuit arrangement. Similarly, it is understood that a flowchart, state transition diagram, pseudocode, and the like are different ways of representing processes that are essentially stored in computer-readable media and can thus be executed by a computer or processor. The object mentioned in the patent claims can expressly also be a person.

[0086] It should be understood that the proposed method and associated apparatus can be implemented in various forms of hardware, software, firmware, specialized processors, or a combination thereof. Specialized processors can include application-specific integrated circuits (ASICs), reduced instruction set computers (RISCs), and / or field-programmable gate arrays (FPGAs). Preferably, the proposed method and apparatus are implemented as a combination of hardware and software. The software is preferably installed as an application program on a program storage device. Typically, this is a machine based on a computer platform that includes hardware such as one or more central processing units (CPUs), random access memory (RAM), and one or more input / output (I / O) interfaces. An operating system is also typically installed on the computer platform.The various processes and functions described here may be part of the application program or a part that is executed via the operating system.

[0087] The disclosure is not limited to the embodiments described here. There is scope for various adaptations and modifications that a person skilled in the art would consider based on their expertise and in relation to the disclosure itself.

[0088] The invention is explained in more detail in the exemplary embodiments using the example of its use in vehicles. The possible use in airplanes and helicopters, for example during landing maneuvers or search operations, etc., is also mentioned.

[0089] However, it should be noted that the application is not limited to this. The invention can be used whenever AR overlays can enhance the field of vision of a driver, operator, or even simply a person wearing smart glasses.

[0090] Even with remotely controlled devices like robots, where remote control is via a monitor displaying a camera image, AR overlays can simplify operation. Therefore, this is another potential application. Reference symbol list

[0091] 10 Vehicle 20 Head-Up Display (HUD) 21 Virtual Projection Surface 30 Touch-Sensitive Display Unit 40 Processing Unit 50 Input Unit 60 Storage Unit 70 Data Line to Display Unit 80 Data Line to Storage Unit 90 Data Line to Input Unit 100 Data Bus 110 Instrument Cluster 120 Telephone 130 Navigation System 140 Radio 150 Camera 160 Communication Module 200 Infotainment System 400 Computer Program 402 Various 445 Program Steps 2001 Navigation Arrow 2002 Notification of New Route Calculation 2003 Operating Instruction for Detailed View 2004 Notification of New Route 2005 Route Overview of New Route 2006 Operating Instruction: Switch to View of Other Route 2007 Notification of Original Route 2008 Route Overview of Original Route 2009 Navigation Path for Original Route 2010 Navigation Path for New Route 2011 Turning Instructions

Claims

1. Method for calculating an AR overlay for a route (2009, 2010) calculated by a navigation system (130), which overlay is displayed to a driver of a vehicle on an AR display unit (20), in particular a head-up display of a vehicle (10), wherein the route (2009, 2010) is calculated anew from time to time and characterized in that - a navigation arrow (2001) is displayed when the vehicle is on a set route but no immediate driving maneuver is coming up; - at a first distance before a branch-off point, a notification (2002) about a newly calculated route is overlaid, wherein the driver is given an operating option (2003) to overlay an overview of the newly calculated route (2005), and if this operating option (2003) is selected, the overview of the newly calculated route is displayed, and wherein the driver is given a further operating option (2006) to switch back and forth between the overview of the newly calculated route (2005) and an overview of a previously calculated route (2008), and if this further operating option (2006) is selected, either the overview of the newly calculated route or the overview of the previously calculated route is displayed, and wherein the particular route is shown in its course to the destination; and - at a second distance before the branch-off point shorter than the first distance, a navigation path, which is augmented onto the road, for the newly calculated route (2010) is displayed next to a navigation path, which is augmented onto the road, of the previously calculated route (2009) and the navigation path for the route that the vehicle (10) does not follow is hidden as soon as the navigation system has recognized which route the vehicle (10) is following by the vehicle's turn signal being activated.

2. Method according to claim 1, wherein before the newly calculated route (2010) is overlaid, an AR overlay is calculated such that the driver is given a notification (2002) about the newly calculated route (2010), with an operating notification (2003) about how the driver can have the route overview (2005) of the newly calculated route (2010) overlaid.

3. Method according to either of the preceding claims, wherein the AR overlay for the route overview (2005, 2008) is calculated so to be zoomable, wherein a zoom operation can be carried out using a steering wheel control (50).

4. Method according to any of the preceding claims, wherein before the route overview (2005, 2008) is overlaid, an assessment of the driving situation (410) is performed and the overlaying is only performed if the assessment of the driving situation (410) shows that the driving situation does not require increased attention from the driver.

5. Method according to any of the preceding claims, wherein the AR overlay for displaying the newly calculated route (2010) is calculated such that it differs in color from the AR overlay for displaying the previously calculated route (2008).

6. Method according to claim 5, wherein the route requiring more time is displayed in a red or orange color, whereas the more time-favorable route is displayed in a blue or green color.

7. Device for performing the method according to any of the preceding claims 1 to 6, comprising an AR display unit (20), a navigation system (130) which calculates a route (2009, 2010), and a computing unit (40) for calculating an AR overlay for a route (2009, 2010) calculated by the navigation system (130) on the AR display unit (20), wherein - the navigation system (130) is designed to calculate the route (2009, 2010) anew from time to time, characterized in that - the computing unit (40) calculates the AR overlay such that - a navigation arrow (2001) is displayed when the vehicle is on a set route but no immediate driving maneuver is coming up; - at a first distance before a branch-off point, a notification about a newly calculated route (2010) is overlaid, wherein the notification about the newly calculated route contains an operating option (2003) to overlay an overview of the newly calculated route, and if this operating option (2003) is selected, the overview of the newly calculated route is displayed, and wherein a further operating option (2006) allows switching back and forth between the display of the route overview (2005, 2008) for the newly calculated route (2010) and of the previously calculated route (2009), and if this further operating option (2006) is selected, either the overview of the newly calculated route or the overview of the previously calculated route is displayed, and wherein the particular route is shown in its course to the destination; and - at a second distance before the branch-off point shorter than the first distance, a navigation path, which is augmented onto the road, for the newly calculated route (2010) is displayed next to a navigation path, which is augmented onto the road, of the previously calculated route (2009) and the navigation path for the route that the vehicle (10) does not follow is hidden as soon as the navigation system has recognized which route the vehicle (10) is following by the vehicle's turn signal being activated.

8. Device according to claim 7, wherein the computing unit (40) is further designed to perform the calculations of AR overlays according to any of claims 2 to 6.

9. Device according to claim 7 or 8, wherein the display unit (20) is a head-up display.

10. Motor vehicle, characterized in that the motor vehicle (10) has a device according to any of claims 7 to 9.

11. Computer program, characterized in that the computer program is designed to perform, when executed in a computing unit (40), the steps of the method for calculating an AR overlay for displaying a navigation route (2009, 2010) on an AR display unit (20) according to any of claims 1 to 6.