Display method for navigation information, head-up display device, vehicle, and storage medium

Abstract

The present invention discloses a method for displaying navigation information, a head-up display device, a vehicle, and a medium. The method includes: when detecting a display condition of a target AR navigation element, obtaining first position description information of the target AR navigation element in the real world; determining second position description information of the target AR navigation element in a head-up display coordinate system on a virtual image plane according to the first position description information of the target AR navigation element in the real world; and determining a standard layout position of the target AR navigation element in a target projection image formed by a head-up display device in the vehicle according to the second position description information. By adopting the above technical solution, it is possible to integrate the AR navigation element displayed on the vehicle imaging element with the real scene, thereby solving the problem that the existing AR navigation elements cannot accurately fit the actual road conditions, and providing a better head-up display visual experience for the driver during the navigation process.

Description

Technical Field

[0001] The present invention relates to the technical field of navigation, and in particular, to a method for displaying navigation information, a head-up display device, a vehicle, and a storage medium. Background Art

[0002] A head-up display (HUD) refers to a device that displays vehicle driving information on the front windshield in front of the driver, enabling the driver to view important driving information without lowering their head or turning their head.

[0003] In the prior art, the head-up display of navigation information can be achieved by directly projecting the navigation prompt information (such as turning left or right) of the vehicle navigation onto the front windshield of the vehicle through the HUD device configured on the vehicle.

[0004] In the process of implementing the present invention, the inventors found that the prior art has the following problems: when the existing head-up display method is implemented, the fitting effect between the navigation information displayed on the front windshield and the real moving environment is poor. The driver's direct feeling is that in some cases, the navigation information displayed on the front windshield is misaligned with the real scene, which will interfere with the driver's driving decision to a certain extent and affect the user experience. Summary of the Invention

[0005] Embodiments of the present invention provide a method for displaying navigation information, a head-up display device, a vehicle, and a storage medium to achieve precise fitting of the AR (Augmented Reality) navigation elements displayed on the front windshield with the actual road elements.

[0006] According to one aspect of the present invention, there is provided a method for displaying navigation information, including:

[0007] When detecting the display condition of a target augmented reality (AR) navigation element, obtaining first position description information of the target AR navigation element in the real world;

[0008] According to the first position description information of the target AR navigation element in the real world, determining second position description information of the target AR navigation element in the head-up display coordinate system of the virtual image plane;

[0009] According to the second position description information, determining the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle, and projecting the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0010] Optionally, detecting the display condition of the target AR navigation element includes:

[0011] Obtaining the navigation data of the vehicle in real time;

[0012] Filter the next expected vehicle action that matches the vehicle in the navigation data, and obtain in real time the distance value that the vehicle needs to move to execute the next expected vehicle action;

[0013] When the distance value reaches the first pre - prompt threshold, determine the first layout position preset for the pre - prompt element in the target projection image, and project the target projection image displayed by the head - up display device onto the imaging element of the vehicle;

[0014] When the distance value reaches the second pre - prompt threshold, determine the second layout position preset for the leading - prompt animation of the preset duration in the target projection image, and project the target projection image displayed by the head - up display device onto the imaging element of the vehicle;

[0015] When it is determined that the leading - prompt animation has finished playing, determine the display condition for detecting the target AR navigation element.

[0016] The advantage of such a setting is that: by projecting the pre - prompt element and the leading - prompt animation onto the imaging element of the vehicle, it is possible to give a pre - prompt to the driver in a period of time before the vehicle action changes, thus facilitating the driver to prepare for the vehicle action change.

[0017] Optionally, obtain the first position description information of the target AR navigation element in the real world, including:

[0018] In the pre - established database, obtain the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis;

[0019] According to the navigation data of the vehicle, obtain the current distance value that the vehicle needs to move to execute the next expected vehicle action;

[0020] Take the combination of the length and width values, the height value from the reference plane, the left offset, and the current distance value as the first position description information.

[0021] The advantage of such a setting is that: by using the navigation data of the vehicle to obtain the position description information of the target AR element in the real world, it is possible to accurately describe the position description parameters that the AR navigation element should have when the AR navigation element is virtually placed in the real world before displaying the AR navigation element on the windshield. Furthermore, accurate data preparation is made for the accurate calculation of the second position information of the AR navigation element in the virtual image plane. This enables the driver to feel a better fitting effect between the AR navigation element and the real world, and thus can provide a better head - up display visual experience for vehicle navigation for the driver.

[0022] Optionally, in a pre-established database, obtain the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis, including:

[0023] Obtain the target environmental attribute of the current moving environment of the vehicle according to the navigation data of the vehicle;

[0024] According to the mapping relationship between the environmental attribute and the position parameter in the database, query and obtain the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis under the target environmental attribute.

[0025] The advantage of this setting is that by obtaining the environmental attribute of the current driving road of the vehicle and querying the relevant information of the first position description information according to the mapping relationship between the environmental attribute and the position parameter in the database, the environmental attribute can be considered when setting the projection position of the target AR projection element, so that the target AR navigation element displayed on the front windshield has a higher matching degree with the real world environment.

[0026] Optionally, determine the second position description information of the target AR navigation element in the head-up display coordinate system of the virtual image plane according to the first position description information of the target AR navigation element in the real world, including:

[0027] In a pre-established database, obtain at least one optical performance parameter corresponding to the virtual image plane;

[0028] According to each optical performance parameter and the first position description information, determine the corner coordinates of the target AR navigation element in the head-up display coordinate system as the second position description information.

[0029] The advantage of this setting is that by using each optical performance parameter corresponding to the virtual image plane and the first position description information to obtain the second position description information, the specific imaging position of the target AR navigation element in the virtual image plane can be accurately determined in combination with the actual optical performance parameters of the HUD, further ensuring the precise fit between the AR navigation element and the real world.

[0030] Optionally, the optical performance parameters include:

[0031] The horizontal field of view angle of the virtual image plane, the vertical field of view angle of the virtual image plane, the left view angle of the virtual image plane, the lower view angle of the virtual image plane, the horizontal resolution of the virtual image plane, the vertical resolution of the virtual image plane, the virtual image distance, and a preset human eye-to-reference plane distance value.

[0032] The advantage of such a setting is that by presetting the optical performance parameters, the projection position and size of the target AR navigation element in the virtual image plane can be calculated more accurately, and at the same time, the calculation efficiency can be effectively improved.

[0033] Optionally, projecting the target projection image displayed by the head-up display device onto the imaging element of the vehicle includes:

[0034] Determine multiple refresh time points according to the refresh rate of the target projection image, and determine the real-time distance value that the vehicle needs to move to execute the next expected vehicle action at each refresh time point according to the real-time vehicle moving speed;

[0035] At each refresh time point, determine the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0036] The advantage of such a setting is that by combining the use of the real-time vehicle speed, at each refresh time point of the target AR navigation element, the target AR navigation element is displayed on the imaging element of the vehicle according to the corrected layout position, and the near-big-and-far-small display effect can be achieved on the premise of closely fitting the real scene only through one calculation of the standard layout position.

[0037] Optionally, at each refresh time point, determining the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle, and projecting the target projection image displayed by the head-up display device onto the imaging element of the vehicle includes:

[0038] Obtain the current real-time distance value matching the current refresh time point;

[0039] According to the current real-time distance value, calculate the current magnification ratio of the target AR navigation element, and calculate the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point according to the standard layout position and the current magnification ratio of the target AR navigation element;

[0040] At the current refresh time point, project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0041] The advantages of such a setting are as follows: By calculating the current magnification ratio of the target AR navigation element based on the current real-time distance value that matches the current refresh time point, and then calculating the current corrected layout position, and displaying the target AR navigation element on the imaging element of the vehicle, it is possible to appropriately correct the display size of the target AR navigation element in the virtual image plane according to the real-time distance value, and thus adaptively adjust the target AR navigation element displayed on the imaging plane to a suitable size, enabling the driver to feel the near-big-and-far-small display effect of the target AR navigation element just like a real object, and enhancing the driver's AR navigation usage experience.

[0042] Optionally, the target AR navigation element is a left-turn prompt element or a right-turn prompt element;

[0043] Calculating the current magnification ratio of the target AR navigation element according to the current real-time distance value, and calculating the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point according to the standard projection parameters and the current magnification ratio of the target AR navigation element, including:

[0044] Calculating the current magnification ratio of the target AR navigation element and the current lateral movement distance according to the current real-time distance value;

[0045] Calculating the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point according to the standard layout position, the current magnification ratio of the target AR navigation element, and the current lateral movement distance.

[0046] The advantages of such a setting are as follows: By using the current real-time distance value of the vehicle to determine the lateral movement distance of the target AR navigation element at different real-time distance values, the target AR navigation element can be gradually translated to the left or right as the vehicle approaches the left-turn position or the right-turn position while ensuring the near-big-and-far-small display effect, so as to further improve the user experience.

[0047] Optionally, at each refresh time point, after determining the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the real-time distance value and projecting the target projection image displayed by the head-up display device onto the imaging element of the vehicle, it further includes:

[0048] When it is determined that the real-time distance value reaches the preset end display distance threshold, generating an end display animation that matches the target AR navigation element;

[0049] Determine the preset third layout position of the end display animation in the target projection image, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0050] The advantage of such a setting is that it can prompt the driver that the current AR navigation is temporarily ended by projecting the end display animation on the imaging element of the vehicle, which can improve the driver's experience of using AR navigation to a certain extent.

[0051] According to another aspect of the present invention, there is provided a head-up display device, comprising:

[0052] A first position description information acquisition module, configured to acquire first position description information of a target AR navigation element in the real world when detecting a display condition of the target augmented reality (AR) navigation element;

[0053] A second position description information determination module, configured to determine second position description information of the target AR navigation element in a head-up display coordinate system in a virtual image plane according to the first position description information of the target AR navigation element in the real world;

[0054] A navigation information projection module, configured to determine a standard layout position of the target AR navigation element in a target projection image formed by a head-up display device in a vehicle according to the second position description information, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0055] According to another aspect of the present invention, there is provided a vehicle, the vehicle comprising:

[0056] At least one processor; and

[0057] A memory communicatively connected to the at least one processor; wherein,

[0058] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the navigation information display method according to any embodiment of the present invention.

[0059] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the navigation information display method according to any embodiment of the present invention when executed.

[0060] The technical solution of the embodiment of the present invention determines the second position description information of the target AR navigation element in the head-up display coordinate system of the virtual image plane by obtaining the first position description information of the target AR navigation element in the real world, and determines the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle. By displaying the target AR navigation element on the imaging element of the vehicle, it is possible to obtain the AR navigation element that the user feels has the first position description information in the real world according to the actual navigation route and the movement of the vehicle, and obtain the AR navigation element that can bring an immersive navigation element and an enhanced display effect of real scene fusion projected onto the imaging element of the vehicle, so that the AR navigation prompt can accurately fit the real world and improve the user experience.

[0061] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0063] Figure 1a is a flowchart of a method for displaying navigation information according to an embodiment of the present invention;

[0064] Figure 1b is a schematic diagram of the head-up display effect of vehicle navigation information according to an embodiment of the present invention;

[0065] Figure 1c is a schematic diagram of the spatial imaging of an AR navigation element according to an embodiment of the present invention;

[0066] Figure 2a is a flowchart of another method for displaying navigation information according to an embodiment of the present invention;

[0067] Figure 2b is a schematic diagram of a pre-prompt element according to an embodiment of the present invention;

[0068] Figure 2c is a schematic diagram of a leading prompt animation according to an embodiment of the present invention;

[0069] Figure 2d is a schematic diagram of an AR navigation element according to an embodiment of the present invention;

[0070] Figure 3a is a flowchart of another method for displaying navigation information provided according to an embodiment of the present invention;

[0071] Figure 3b is a schematic diagram of the horizontal field of view of a virtual image plane and the longitudinal viewing angle of the virtual image plane provided according to an embodiment of the present invention;

[0072] Figure 3c is a schematic diagram of the downward viewing angle of a virtual image plane provided according to an embodiment of the present invention;

[0073] Figure 3d is a schematic diagram of the virtual image distance provided according to an embodiment of the present invention;

[0074] Figure 3e is a schematic diagram of the projection of AR navigation elements in a virtual image plane provided according to an embodiment of the present invention;

[0075] Figure 3f is a hardware structure diagram provided according to an embodiment of the present invention;

[0076] Figure 4 is a schematic diagram of the structure of a head-up display device provided according to an embodiment of the present invention;

[0077] Figure 5 is a schematic diagram of the structure of a vehicle terminal for implementing the method for displaying navigation information according to an embodiment of the present invention. Detailed implementation manners

[0078] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0079] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0080] The navigation information display method provided by the embodiments of the present application can be applied to a head-up display device, which can be applied to any variety of transportation means (not shown). For example, the transportation means may include, but is not limited to, land transportation means such as vehicles, air transportation means such as aircraft (or called flying vehicles), or water or underwater transportation means, etc.

[0081] For the sake of convenience of description, only the vehicle-mounted application scenario will be taken as an example below to introduce each exemplary embodiment of the technical solution of the present application in detail.

[0082] Figure 1a It is a flowchart of a navigation information display method provided by an embodiment of the present invention. This embodiment is applicable to the situation where the position information of the AR navigation element in the head-up display coordinate system is determined to realize the display of the AR navigation element on the imaging element of the vehicle. This method can be executed by a head-up display device, and the head-up display device can be implemented in the form of hardware and / or software.

[0083] Correspondingly, as Figure 1a shown, the method includes:

[0084] S110. When the display condition of the target AR navigation element is detected, obtain the first position description information of the target AR navigation element in the real world.

[0085] AR navigation is a navigation technology that uses augmented reality technology and combines the real-time moving environment, and uses elements such as virtual models, arrows, and dotted lines to guide the direction for users. Among them, the elements such as virtual models, arrows, and dotted lines used to guide the direction are AR navigation elements.

[0086] In this embodiment, taking a vehicle as an example, the moving environment is the specific road scene.

[0087] Taking a common vehicle as an example, the driver can use the in-vehicle navigation function in the car computer, and then display the navigation information on the in-vehicle screen. However, during actual driving, if the driver looks down at the in-vehicle screen while driving, to a certain extent, it will affect driving safety. When performing AR navigation in the form of a head-up display, the AR navigation elements can be matched with the road environment to indicate the road to the driver without disturbing the driver's normal driving state.

[0088] When implementing the head-up display of navigation elements, the main technical difficulty lies in: during driving, the vehicle sensors move with the vehicle, and as the displacement changes, the coordinates of the vehicle sensors and the HUD device also change. What the driver directly feels is that the image that fits the real scene is misaligned with the real scene. The technical effect sought in this case is to accurately fit the AR navigation elements projected on the imaging element of the vehicle with the actual road elements, so that the driver can view the AR navigation information that can be accurately fitted with the actual road conditions through the imaging element of the vehicle, and the user experience is good.

[0089] For the convenience of understanding the technical effects that the present invention can achieve, Figure 1b Fig. shows a schematic diagram of the head-up display effect of an optional vehicle navigation information. As Figure 1b shown, the steering wheel, the controller (i.e., the vehicle controller), and the HUD device are all conventional devices in the vehicle. The vehicle controller can be used for navigation and send AR navigation element projection information to the HUD device according to the navigation content. The HUD device can be used to project the AR navigation elements onto the windshield. The windshield is an optional imaging element of the vehicle. The image formed by the reflection of the windshield is usually called a virtual image. The user can see the virtual image through the windshield and can also see the real road conditions in front of the road. In other words, the user can see the virtual image superimposed on the real environment. Among them, the buildings and the road are shown in a simple form. The three evenly arranged black triangles in the figure are an optional AR navigation element for indicating a right turn of the vehicle.

[0090] In the example as Figure 1b shown, the technical effect that the present invention can achieve is to display the AR navigation elements on the imaging element in front of the driver according to the real-time navigation information, and, in the driver's perspective, the AR navigation elements can be matched with the road. In a more specific example, when the vehicle is about to turn right, the right-turn indication arrow is displayed on the driver's front windshield, and in the driver's perspective, the right-turn indication arrow is tiled or perpendicular to the target road that the vehicle is about to enter, so as to achieve the purpose of indicating the driving route.

[0091] In an embodiment of the present invention, in order to improve the navigation experience, an optional form of displaying AR navigation elements is proposed: before the target AR navigation element is about to be displayed, a leading prompt animation is played first to provide the driver with a prompt for vehicle movement transformation, and after the leading prompt animation ends, the target AR navigation element is projected onto the imaging element.

[0092] Among them, the specific element form of the target AR navigation element can be determined according to the next expected vehicle movement (i.e., the next vehicle movement). For example, when the vehicle is about to turn right, the target AR navigation element can be multiple arrows arranged evenly and pointing to the right.

[0093] Furthermore, an optional display condition for the target AR navigation element can be that the leading prompt animation has finished playing.

[0094] Among them, the first position description information of the target AR navigation element in the real world can include the length and width values of the target AR navigation element in the real world, the height value from the reference plane, the left offset of the center point of the target AR navigation element relative to the driving axis, and the current distance value of the vehicle from the distance required to perform the next expected vehicle movement. It can be further understood that the first position description information can represent the expected display position of the target AR navigation element in the real world.

[0095] Optionally, the above first position description information can be obtained according to the current navigation information of the vehicle and in combination with a pre-generated database.

[0096] S120. According to the first position description information of the target AR navigation element in the real world, determine the second position description information of the target AR navigation element in the head-up display coordinate system in the virtual image plane.

[0097] Optionally, according to the first position description information of the AR navigation element in the real world, in combination with at least one optical performance parameter corresponding to the virtual image plane formed by the HUD, determine the second position description information of the target AR navigation element in the head-up display coordinate system in the virtual image plane.

[0098] It can be understood that the virtual image plane is the virtual image imaging surface of the AR navigation element (the principle of mirror imaging can be referred to). When the light source in the HUD projects the AR navigation element onto the imaging element of the vehicle, the AR navigation element actually observed in the driver's perspective is actually located on this virtual image plane in space. By controlling the imaging size and imaging position of the AR navigation element in the virtual image plane, the final effect presented in the driver's perspective is the integration of the virtual image of the AR navigation element and the road environment in the real world.

[0099] Figure 1cIt is a schematic diagram of the spatial imaging of an optional AR navigation element. In Figure 1c In the example shown, the image X-axis and the image Y-axis are respectively the horizontal axis and the vertical axis of the head-up display coordinate system. The plane where the image X-axis and the image Y-axis are located is the virtual image plane. The three black arrows pointing to the right in the virtual image plane are the target AR navigation elements. The three larger black arrows pointing to the right in front of the virtual image plane are the imaging effects of the target AR navigation elements in the real world. The icon fixed-point coordinates (X1i, Y1i), (X1i, Y1j), (X1j, Y1i), and (X1j, Y1j) are respectively the mapped position coordinates of the four corner points of the AR navigation element in the real world.

[0100] Since the distance between the virtual image plane projected by the vehicle and the vehicle itself is generally a fixed distance, therefore, if you want to change the display position and size of the AR navigation element integrated with the real world in the driver's view, generally, you can change the position and size of the AR navigation element in the head-up display coordinate system in the virtual image plane, so as to change the display position and size of the AR navigation element integrated with the real world in the driver's view.

[0101] The second position description information can be the corner point coordinates of the target AR navigation element in the head-up display coordinate system of the virtual image plane, and the second position description information can be used to describe the specific imaging position of the target AR navigation element in the virtual image plane.

[0102] Optionally, the second position description information can be obtained by calculating the first position description information and the preset optical performance parameters.

[0103] The advantage of this setting is that: by using the optical performance parameters and the first position description information to obtain the second position description information, the specific imaging position of the target AR navigation element in the virtual image plane can be determined in combination with the actual optical performance parameters of the HUD, so that the AR navigation element has a better display effect.

[0104] S130. According to the second position description information, determine the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0105] Among them, the head-up display device in the vehicle can be used to project the target projection image including the target AR navigation element onto the imaging element of the vehicle (for example, a vehicle), so that the user can view the virtual image including the target AR navigation element displayed in the virtual image plane.

[0106] Among them, the imaging element of the vehicle can be the front window of the vehicle (for example, the windshield), the emission film layer or the imaging window disposed on the surface of the front window of the vehicle near the head-up display device. Imaging through the windshield is W-HUD (Windshield-HUD, windshield head-up display), and imaging through the imaging window is C-HUD (Combiner HUD, combined head-up display). Optionally, the imaging window is generally made of a transparent material (transparent to visible light) or an imaging plate with a certain curvature.

[0107] Among them, the target projection image can be an image generated by the head-up display device for display on the imaging element of the vehicle. The target projection image can include projection elements that need to be displayed currently, such as AR navigation elements or navigation prompt animations and other elements. Taking a vehicle as an example, generally speaking, the imaging element of the vehicle can be the front windshield of the vehicle. The four sides of the projection image displayed on the front windshield of the vehicle can coincide with the four sides of a preset rectangular area on the front windshield of the vehicle to reflect the target projection image to the driver's eyes.

[0108] In a specific example, if it is desired to display the AR navigation element in the middle part of the rectangular area, then the AR navigation element should also be arranged at the corresponding middle part of the target projection image.

[0109] It can be understood that according to the existing virtual image plane imaging principle, after obtaining the second position description information of the AR navigation element, the specific position where the AR navigation element should be displayed on the imaging element of the vehicle can be obtained, and further, the specific image layout position of the AR navigation element in the target projection image can also be obtained.

[0110] Furthermore, the standard layout position can be understood as the layout position of the target AR navigation image in the target projection image (it can also be further understood as the pixel point position occupied by the target AR navigation image in the target projection image). Since the relative positions of the virtual image plane, the imaging element of the vehicle, and the head-up display device are generally relatively fixed, therefore, only by obtaining the second position description information of the target AR navigation element, the determined standard layout position of the target AR navigation element in the target projection image can be obtained according to the preset coordinate conversion relationship. The above information all has a certain mapping relationship.

[0111] The technical solution of the embodiment of the present invention obtains the first position description information of the target AR navigation element in the real world, and then determines the second position description information of the target AR navigation element in the head-up display coordinate system of the virtual image plane, and determines the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the second position description information, and displays the target AR navigation element on the imaging element of the vehicle. In this way, according to the actual navigation route and driving conditions, the AR navigation element with the first position description information in the real world that the user feels can be obtained, and the AR navigation element that can bring an immersive navigation element and an enhanced display effect of the integration of the real scene can be obtained and projected onto the imaging element of the vehicle, so that the AR navigation prompt can be accurately fitted to the real world and the user experience can be improved.

[0112] The process of obtaining the first position description information in the above embodiment will be further described in detail below.

[0113] Figure 2a It is a flowchart of another navigation information display method provided by the embodiment of the present invention. On the basis of the above embodiment, the process of obtaining the first position description information is specifically described. As Figure 2a shown, the method includes:

[0114] S210. Obtain the navigation data of the vehicle in real time.

[0115] Among them, taking a vehicle as an example, the navigation data of the vehicle can be sent by the built-in navigation module of the vehicle. The specific route is preset by the user. The navigation data may include information such as the current vehicle speed, the distance to the road ahead, and the specific route planning situation. It can also be understood that the navigation data can provide the current driving state of the vehicle, the predicted driving state of the vehicle, and the road information passed by the vehicle. However, this is only an exemplary description and does not limit the specific content of the navigation data.

[0116] S220. Screen the next expected vehicle action that matches the vehicle in the navigation data, and obtain the distance value that the vehicle needs to travel to execute the next expected vehicle action in real time.

[0117] Among them, the next expected vehicle action can be a new vehicle driving behavior different from the current vehicle driving behavior. In a specific example, the vehicle action can be turning left, turning right, U-turning, etc. According to the vehicle position and navigation information in the vehicle navigation data, the next expected vehicle action that matches the vehicle can be screened out. For example, when the vehicle is currently in a straight-ahead state but needs to turn left at the next intersection, the next expected vehicle action is for the vehicle to turn left, and the distance value that the vehicle needs to travel to execute the next expected action is the distance value between the current vehicle and the next intersection.

[0118] S230. When the distance value reaches the first pre - prompt threshold, determine the first preset layout position of the pre - prompt element in the target projection image, and project the target projection image displayed by the head - up display device onto the imaging element of the vehicle.

[0119] In an embodiment of the present invention, the pre - prompt element can be understood as an element used to represent the current vehicle. Figure 2b For a schematic diagram of an optional pre - prompt element, in Figure 2b taking a vehicle as an example, the pre - prompt element is an arrow, and the direction pointed by the arrow is the current driving direction of the vehicle. Since the vehicle generally goes straight relative to the road, the direction pointed by the arrow is generally in front of the vehicle.

[0120] Among them, the first pre - prompt threshold is a preset distance value. When the distance value that the vehicle needs to travel for the next expected vehicle action reaches the first pre - prompt threshold, the pre - prompt element can be displayed on the front windshield of the vehicle. For example, if the first pre - prompt threshold is 200 meters, when the distance between the vehicle and the next left - turn intersection is 200 meters, the pre - prompt element is projected on the front windshield of the vehicle. Among them, the first layout position is the preset layout position of the pre - prompt element in the target projection image. According to the first layout position, information such as the display size and display position of the pre - prompt element on the front windshield can be determined. Since the pre - prompt element is only used to prompt the driver that the vehicle action is about to change, the projection size and projection position of the pre - prompt element on the front windshield of the vehicle can be preset, independent of the actual state of the vehicle. And since the distance between the vehicle and the head - up display coordinate system in the virtual image plane remains unchanged, the first preset layout position of the pre - prompt element in the target projection image can also be preset.

[0121] S240. When the distance value reaches the second pre - prompt threshold, determine the second preset layout position of the leading - prompt animation of a preset duration in the target projection image, and project the target projection image displayed by the head - up display device onto the imaging element of the vehicle.

[0122] In an embodiment of the present invention, the leading - prompt animation can be understood as a dynamic element used to simply indicate the next expected vehicle action, and the duration of the leading - prompt animation is a preset duration. Figure 2c For a schematic diagram of an optional leading - prompt animation, in Figure 2c in the example shown, the next expected vehicle action is to turn left. The dynamic effect of the leading - prompt animation of the vehicle can be that the pre - prompt element pointing forward flies out along the direction indicated by the long arrow and finally becomes an arrow pointing to the left.

[0123] Among them, the second pre - prompt threshold is a preset distance value. When the distance that the vehicle needs to travel to the next expected vehicle action reaches the second pre - prompt threshold, the leading prompt animation can be displayed on the vehicle's front windshield. For example, if the second pre - prompt threshold is 100 meters, when the vehicle is 100 meters away from the next left - turn intersection, the leading prompt animation is projected on the vehicle's front windshield.

[0124] Among them, the second layout position is the playback position of the leading prompt animation in the target projection image. According to the second layout position, information such as the size, position, and time of projection of each animation frame in the leading prompt animation on the front windshield can be determined. Similarly, since the prompt effect of the leading prompt animation has nothing to do with the actual state of the vehicle, the second layout position preset in the target projection image for the leading prompt animation can be set in advance.

[0125] The advantage of such a setting is that by projecting the pre - prompt element and the leading prompt animation on the imaging element of the vehicle, it is possible to give a pre - prompt to the driver in a period of time before the vehicle action changes, so as to facilitate the driver to prepare for the vehicle action change.

[0126] S250. When it is determined that the leading prompt animation has finished playing, determine the display conditions for detecting the target AR navigation element.

[0127] It can be understood that after the leading prompt animation has finished playing and the vehicle continues to move along the original navigation route, the display conditions for the target AR navigation element can be met.

[0128] Optionally, it can be judged whether the leading prompt animation has finished playing by the set time of the leading prompt animation. In a specific example, if the total duration of the leading prompt animation is 1.7 s (seconds), then the timing can start from the moment when the leading prompt animation starts playing. After reaching 1.7 s, it can be judged that the leading prompt animation has finished playing.

[0129] Figure 2d It is a schematic diagram of an optional AR navigation element. As Figure 2d shown in the example, the AR navigation element for a left - turn can be multiple arrows pointing to the left arranged in order.

[0130] S260. In the pre - established database, obtain the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis.

[0131] Among them, in the pre-established database, obtaining the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis can specifically include:

[0132] According to the navigation data of the vehicle, obtain the target environmental attributes of the current moving environment of the vehicle;

[0133] According to the mapping relationship between the environmental attributes and the position parameters in the database, query and obtain the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis under the target environmental attributes.

[0134] Among them, the environmental attributes can refer to the attribute information of the current moving environment, such as the road grade information of the vehicle, the flight route information of the aircraft, and the port information of the ship, etc. When the vehicle is a car, the reference plane can be the ground, and the height value from the reference plane can be the height value from the ground.

[0135] It can be understood that taking a vehicle as an example, in actual roads, roads in different regions and for different purposes may have different road widths. For example, roads on highway sections are generally wider, and roads in urban and rural sections are generally narrower than those on highway sections. Therefore, in order to ensure that the AR navigation elements can give the driver a more comfortable visual experience on roads of different widths, it is necessary to first obtain the road level of the current driving road, and then further obtain the specific road dimensions of the current driving road.

[0136] The position parameters can refer to the relevant parameters of the position of the AR navigation element in the target moving environment, and specifically can be a set of parameters such as the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis. Different environmental attributes correspond to different position parameters.

[0137] Taking a vehicle as an example, the length and width values of the target AR navigation element in the real world can be understood as the length and width values of the target AR navigation element in the current driving road from the driver's perspective; the height value of the target AR navigation element from the reference plane can be understood as the height value of the target AR navigation element from the ground plane when the ground plane is used as the reference plane; the left offset of the center point of the target AR navigation element relative to the driving axis can be understood as the distance between the target AR navigation element and the vertical center line of the front windshield when the vertical center line of the front windshield is used as the driving axis.

[0138] S270. According to the navigation data of the vehicle, obtain the current distance value that the vehicle needs to travel to perform the next expected vehicle action.

[0139] In a specific example where a vehicle is used as a carrier, if the next expected vehicle action is for the vehicle to turn left at the next intersection, the navigation data of the vehicle can obtain the distance value between the current position of the vehicle and the next intersection. Specifically, the current navigation data can be obtained from the navigation module in the vehicle, and based on the navigation data, the current distance value that the vehicle needs to travel to perform the next expected vehicle action can be obtained.

[0140] S280: Use the combination of the length and width values, the height value from the distance reference plane, the left offset, and the current distance value as the first position description information.

[0141] S290: Determine the second position description information of the target AR navigation element in the head-up display coordinate system on the virtual image plane according to the first position description information of the target AR navigation element in the real world.

[0142] S2100: Determine the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0143] It should be noted that Figure 2a the order between steps S260 and S270 in the flowchart shown is not limited. They can be executed simultaneously, or S270 can be executed first and then S260.

[0144] The technical solution of the embodiment of the present invention can, by projecting the pre-prompt element and the leading prompt animation on the imaging element of the vehicle, give a pre-prompt to the driver in a period of time before the vehicle action changes, so as to facilitate the driver to prepare for the vehicle action change. And by using the navigation data of the vehicle to obtain the position description information of the target AR element in the real world, the relevant information in the real world can be considered before the AR navigation element is displayed on the imaging element, and thus a better head-up display visual experience of vehicle navigation can be provided for the driver.

[0145] Figure 3a This is a flowchart of another method for displaying navigation information provided by an embodiment of the present invention. On the basis of the above embodiment, this embodiment specifically illustrates the process of displaying navigation information. As Figure 3a shown, the method includes:

[0146] S310: When detecting the display condition of the target AR navigation element, obtain the first position description information of the target AR navigation element in the real world.

[0147] S320. Obtain at least one optical performance parameter corresponding to the virtual image plane in a pre-established database.

[0148] Among them, the optical performance parameters may include:

[0149] The horizontal field of view angle of the virtual image plane, the vertical field of view angle of the virtual image plane, the left view angle of the virtual image plane, the lower view angle of the virtual image plane, the horizontal resolution of the virtual image plane, the vertical resolution of the virtual image plane, the virtual image distance, and a preset human eye-to-reference plane distance value.

[0150] For ease of understanding, in Figure 3b a schematic diagram of the horizontal field of view angle of the virtual image plane and the vertical view angle of the virtual image plane is shown with an example of projection on the vehicle front windshield; in Figure 3c a schematic diagram of an optional lower view angle of the virtual image plane is shown; in Figure 3d a schematic diagram of an optional virtual image distance is shown.

[0151] Among them, the horizontal field of view angle of the virtual image plane is as shown in Figure 3b , which can be understood as the angle between the connection lines of the midpoints of the left and right vertical sides of the rectangular virtual image and the central eye point; the vertical view angle of the virtual image plane is as shown in Figure 3b , which can be understood as the angle between the connection lines of the midpoints of the upper and lower horizontal sides of the rectangular virtual image and the central eye point; the left view angle of the virtual image plane can be understood as the left angle between the connection line of the central eye point and the center point of the virtual image and the horizontal line where the center point of the virtual image is located; the lower view angle of the virtual image plane is as shown in Figure 3c , which can be understood as the angle between the connection line of the central eye point and the center point of the virtual image and the horizontal plane; the virtual image distance is as shown in Figure 3d , which can be understood as the distance from the driver's eyes to the projection in the virtual image plane.

[0152] The advantage of such a setting is that: by presetting the optical performance parameters, the projection position and size of the target AR navigation element in the virtual image plane can be calculated more accurately, and at the same time, the calculation efficiency can be effectively improved.

[0153] S330. Determine the corner coordinates of the target AR navigation element in the head-up display coordinate system according to each optical performance parameter and the first position description information, as the second position description information.

[0154] Among them, an optional calculation method for the corner coordinates of the target AR navigation element in the head-up display coordinate system can be:

[0155] Calculate the corner coordinates X i , X j , Y i and Y j of the target AR navigation element in the head-up display coordinate system according to the following formula;

[0156]

[0157]

[0158]

[0159]

[0160] Among them, hr is the horizontal resolution of the virtual image plane, vr is the vertical resolution of the virtual image plane, vid is the virtual image distance, hfov is the horizontal field of view angle, vfov is the vertical field of view angle, irw is the width value of the target AR navigation element in the real world, irh is the length value of the target AR navigation element in the real world, igh is the height value of the target AR navigation element from the distance reference plane in the real world, ild is the left offset of the center point of the target AR navigation element relative to the driving axis, ed is the distance value between the human eye and the ground, loa is the left viewing angle of the virtual image plane, lda is the lower viewing angle of the virtual image plane, and icd is the current distance value that the vehicle needs to travel to perform the next expected vehicle action.

[0161] Figure 3e is an optional schematic diagram of the projection of the AR navigation element in the virtual image plane. In Figure 3e the example shown, the HUD projects the target projection image containing the target AR navigation element onto the windshield, and the driver can observe the target AR navigation element in the virtual image plane. Among them, the three triangles pointing to the left are the target AR navigation elements in the virtual image plane. The four points of the smaller rectangle outside the triangle represent the corner points of the target AR navigation element in the head-up display coordinate system, which is located in the virtual image plane. The specific origin and direction settings of the head-up display coordinate system can be determined by the designer. The corner point coordinates are as Figure 3e shown, and can be the upper left corner coordinates (Xi, Yi), the upper right corner coordinates (Xi, Yj), the lower left corner coordinates (Xj, Yi), and the lower right corner coordinates (Xj, Yj) respectively.

[0162] The advantage of such a setting is that by calculating the corner point coordinates of the target AR navigation element in the head-up display coordinate system by using the optical performance parameters and the first position description information, the corner point coordinates of the target AR navigation element in the head-up display coordinate system can be calculated more accurately, so as to more accurately determine the position of the target AR navigation element in the virtual image plane.

[0163] S340. Determine the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the second position description information.

[0164] S350. Determine multiple refresh time points according to the target projection image refresh rate, and determine the real-time distance value that the vehicle needs to move to execute the next expected vehicle action at each refresh time point according to the real-time vehicle moving speed.

[0165] Continuing with the vehicle as an example, it can be understood that since the vehicle is always in a forward state, the distance between the vehicle and the location where the next expected vehicle action is to be executed gradually decreases. At this time, the road in the driver's perspective gradually magnifies. If the target projection image generated according to the standard layout position is always projected on the front windshield of the vehicle, the display size of the target AR navigation element on the front windshield remains unchanged, and the target AR navigation element in the driver's perspective gradually shrinks relative to the road, which may further lead to the situation where the target AR navigation element does not fit the road. At the same time, in the AR navigation effect designed in this embodiment, the target AR navigation element should increase as the road in the driver's perspective increases, which means that the target AR navigation element should increase as the vehicle moves forward.

[0166] Therefore, the layout position of the target AR navigation element in the target projection image can be adjusted once within a certain period of time, so that the projection of the target AR navigation element on the front windshield gradually increases as the vehicle moves forward, so that the target AR navigation element fits the indicated road, ensuring that the driver can obtain an ideal visual effect.

[0167] Among them, the target projection image refresh rate is the inherent projection performance of the head-up display, representing the number of frames of the target projection image displayed on the front windshield per second. Each refresh time point of the target projection image can be obtained according to the target projection image refresh rate.

[0168] S360. At each refresh time point, determine the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the real-time distance value, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0169] Among them, the corrected layout position is the new layout information of the target AR navigation element in the target projection image obtained by modifying the standard layout position according to the motion information of the vehicle.

[0170] The advantage of this setting is that by combining the use of the real-time vehicle speed, at each refresh time point of the target AR navigation element, the target AR navigation element is displayed on the imaging element of the vehicle according to the corrected layout position, which can achieve the display effect of near-large and far-small on the premise that the target AR navigation element closely fits the real scene only through one calculation of the standard layout position.

[0171] Among them, at each refresh time point, determining the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the real-time distance value, and projecting the target projection image displayed by the head-up display device onto the imaging element of the vehicle may include:

[0172] Obtain the current real-time distance value that matches the current refresh time point;

[0173] According to the current real-time distance value, calculate the current magnification ratio of the target AR navigation element, and according to the standard layout position and the current magnification ratio of the target AR navigation element, calculate the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point;

[0174] At the current refresh time point, project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0175] Optionally, the current real-time distance value that matches the current refresh time point can be obtained during navigation.

[0176] It can be understood that the AR navigation effect to be achieved by the present invention is that when the vehicle gradually approaches the next expected vehicle action execution location, the projection of the target AR navigation element on the imaging element should change from small to large, and finally remain fixed in size and stand above the target road, and there is a certain proportional relationship between the distance and the size of the target AR navigation element.

[0177] Furthermore, projecting the target projection image at the corrected layout position on the imaging element can appropriately change the projection size of the target AR navigation element on the imaging element, thereby adjusting the virtual image of the target AR navigation imaged on the virtual image plane to an appropriate size.

[0178] In a specific example, the magnification ratio of the target AR navigation element can be calculated by the following method: If the vehicle position s(0) when the vehicle first projects the target AR navigation element is 100 meters away from the next expected vehicle action execution location, and the size of the target AR navigation element is a at this time, as the vehicle moves forward, the target AR navigation element gradually enlarges, and the final expected size of the target AR navigation element is b, and the distance from the next expected vehicle action execution location is 20 meters at this time, then at any position point s(t) between 100 meters and 20 meters, the width calculation formula of the target AR navigation element can be

[0179]

[0180] Among them, Xj - Xi is the width of the target AR navigation element at s(t).

[0181] Among them, the target AR navigation element can be a left turn prompt element or a right turn prompt element.

[0182] Optionally, in the above embodiments, various styles of AR navigation elements are shown. The left turn prompt element may be composed of multiple arrows pointing to the left, and the right turn prompt element may be composed of multiple arrows pointing to the right. The specific navigation style is not limited, and whether the target AR navigation element is a left turn prompt element or a right turn prompt element can be determined by the specific navigation content.

[0183] Further, according to the current real-time distance value, the magnification ratio of the current target AR navigation element is calculated, and according to the standard projection parameters and the magnification ratio of the current target AR navigation element, the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point is calculated, which may specifically include:

[0184] According to the current real-time distance value, calculate the current target AR navigation element magnification ratio and the current lateral movement distance;

[0185] According to the standard layout position, the current target AR navigation element magnification ratio and the current lateral movement distance, the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point is calculated.

[0186] It is understandable that as the vehicle moves forward, in addition to the need to enlarge the target AR navigation element, the lateral movement of the AR navigation element also needs to be considered. In an example where the vehicle is a vehicle, if the vehicle needs to turn right at the next intersection, then when the vehicle is 100 meters away from the next intersection, the right turn road at the next intersection may be in the middle of the driver's field of vision. When the vehicle is 50 meters away from the next intersection, the road that the driver can actually see gradually increases, but the field of vision gradually decreases. At this time, the right turn road may be on the right side of the driver's field of vision. Therefore, when correcting the projection parameters of the target AR navigation element, the layout position should also be adjusted so that the target AR navigation element can be adaptively offset to the side of the vehicle action transformation.

[0187] The advantage of this setting is that by using the current real-time distance value of the vehicle, the lateral movement distance of the target AR navigation element under different real-time distance values ​​is determined, so that the target AR navigation element can be displayed in a larger size when it is near and smaller size when it is far away. At the same time, as the vehicle approaches the left turn road or the right turn road, the target AR navigation element can be gradually translated to the left or right side. The technical effect is to further improve the user experience.

[0188] S370: When it is determined that the real-time distance value reaches a preset end display distance threshold, generate an end display animation that matches the target AR navigation element.

[0189] It can be understood that when the distance value of the vehicle from the execution location of the next expected vehicle action is within the preset end range, at this time, the driver can generally confirm the moving environment entered by the next expected vehicle action, and there is no need to use AR navigation anymore, and the target AR navigation element can stop being displayed. However, in order to give the driver a better AR navigation experience, an end display animation matching the target AR navigation element can be generated, and by playing the end display animation, the driver can be prompted that the navigation change for this vehicle action has ended.

[0190] In a specific example, if the next expected action of the vehicle is to turn left, and the target AR navigation element is multiple arrows pointing to the left, then the end display animation matching the target AR navigation element can be that multiple arrows pointing to the left fly out along the indicated direction until they are no longer displayed on the windshield. The specific end display animation can be designed according to actual needs and is not limited here.

[0191] S380. Determine the third layout position preset for the end display animation in the target projection image, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0192] The third layout position can be the specific position of the projected end display animation in the target projection image, generally a preset value, and is not limited in this regard.

[0193] The advantage of such a setting is that: by projecting the end display animation on the imaging element of the vehicle, the driver can be prompted that this AR navigation has temporarily ended, which can improve the driver's experience of using AR navigation to a certain extent.

[0194] Figure 3f It is a hardware structure diagram that can optionally implement the navigation information display method in the present invention. Figure 3f The hardware structure in it can be configured in a vehicle, but Figure 3f it is only for illustrative purposes and is not specifically limited. Figure 3fThe shown hardware structure diagram consists of a controller, a HUD, a camera, a radar, a steering wheel sensor, an Electronic Stability Control (ESC) system of the vehicle, wheel speed sensors, and a six-axis accelerometer (IMU). Among them, the controller can execute the display method of navigation information as described in the embodiments of the present invention; the HUD can be used to project target AR navigation elements onto an imaging element; the camera can be used to capture the road environment; the radar can be used to obtain the position and distance of an object when it is stationary or moving; the steering wheel sensor can detect the steering angle and rotation speed of the steering wheel during vehicle driving; the wheel speed sensors can be used to detect the rotation speed of the vehicle wheels; the Electronic Stability Control system of the vehicle can be used to perform lateral and longitudinal control of the vehicle; the six-axis accelerometer can be used to detect the body motion posture (for example, roll, pitch, and yaw during turning or acceleration / deceleration).

[0195] In the technical solution of the embodiments of the present invention, by calculating the corner coordinates of the target AR navigation element in the head-up display coordinate system using the optical performance parameters and the first position description information, the corner coordinates of the target AR navigation element in the head-up display coordinate system can be calculated more accurately, so as to more accurately determine the position of the target AR navigation element in the virtual image plane. By calculating the current magnification ratio of the target AR navigation element based on the current real-time distance value matching the current refresh time point, and then calculating the current corrected layout position and displaying the target projection image on the imaging element of the vehicle, the projection size of the target AR navigation element on the imaging element can be appropriately changed, so as to adjust the virtual image of the target AR navigation on the virtual image plane to an appropriate size, improving the driver's AR navigation usage experience. At the same time, by considering the forward lateral movement of the vehicle when calculating the current corrected layout position of the vehicle, the problem that the projection position of the target AR navigation element on the imaging element changes due to the forward lateral movement of the vehicle is overcome, effectively improving the accuracy of the projection position of the target AR navigation element on the imaging element of the vehicle, enabling the driver to accurately obtain navigation prompt information.

[0196] Figure 4 It is a schematic structural diagram of a head-up display device provided by the embodiments of the present invention. As Figure 4 shown, the device includes: a first position description information acquisition module 410, a second position description information determination module 420, and a navigation information projection module 430.

[0197] The first position description information acquisition module 410 is configured to acquire the first position description information of the target AR navigation element in the real world when the display condition of the target AR navigation element is detected.

[0198] The second position description information determination module 420 is configured to determine the second position description information of the target AR navigation element in the head-up display coordinate system of the virtual image plane according to the first position description information of the target AR navigation element in the real world.

[0199] The navigation information projection module 430 is configured to determine the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the second position description information, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle according to the standard layout position.

[0200] The technical solution of the embodiment of the present invention can obtain the first position description information of the target AR navigation element in the real world, and then determine the second position description information of the target AR navigation element in the head-up display coordinate system of the virtual image plane, and determine the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the second position description information, and display the target AR navigation element on the imaging element of the vehicle, so as to obtain the AR navigation element with the first position description information in the real world that the user feels according to the actual navigation route and driving conditions, and obtain the AR navigation element that can bring an immersive navigation element and an enhanced display effect of scene fusion to the user and project it onto the imaging element of the vehicle, so that the AR navigation prompt can accurately fit the real world and improve the user experience.

[0201] Based on the above embodiments, the first position description information acquisition module 410 may be specifically configured to:

[0202] Obtain the navigation data of the vehicle in real time;

[0203] Filter the next expected vehicle action that matches the vehicle from the navigation data, and obtain the distance value that the vehicle needs to move to execute the next expected vehicle action in real time;

[0204] When the distance value reaches the first pre-prompt threshold, project the target projection image displayed by the head-up display device onto the imaging element of the vehicle according to the first layout position preset by the pre-prompt element in the target projection image;

[0205] When the distance value reaches the second pre-prompt threshold, determine the second layout position preset by the leading prompt animation of the preset duration in the target projection image, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle;

[0206] When it is determined that the leading prompt animation has ended, determine the display condition for detecting the target AR navigation element.

[0207] Based on the above embodiments, the first position description information acquisition module 410 may specifically be configured to:

[0208] In a pre-established database, obtain the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis;

[0209] According to the navigation data of the vehicle, obtain the current distance value that the vehicle needs to move to perform the next expected vehicle action;

[0210] Use the combination of the length and width values, the height value from the reference plane, the left offset, and the current distance value as the first position description information.

[0211] Based on the above embodiments, the first position description information acquisition module 410 may further be configured to:

[0212] According to the navigation data of the vehicle, obtain the environmental attributes of the current moving environment of the vehicle;

[0213] According to the mapping relationship between the environmental attributes and the position parameters in the database, query and obtain the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis under the environmental attributes.

[0214] Based on the above embodiments, the second position description information determination module 420 may specifically be configured to:

[0215] In a pre-established database, obtain at least one optical performance parameter corresponding to the virtual image plane;

[0216] According to each optical performance parameter and the first position description information, determine the corner coordinates of the target AR navigation element in the head-up display coordinate system as the second position description information.

[0217] Based on the above embodiments, the optical performance parameters may include:

[0218] The horizontal field of view angle of the virtual image plane, the vertical field of view angle of the virtual image plane, the left view angle of the virtual image plane, the lower view angle of the virtual image plane, the horizontal resolution of the virtual image plane, the vertical resolution of the virtual image plane, the virtual image distance, and a preset human eye-to-reference plane distance value.

[0219] Based on the above embodiments, the navigation information projection module 430 may include:

[0220] A real-time distance value determination unit, configured to determine a plurality of refresh time points according to the target projection image refresh rate, and determine the real-time distance value that the vehicle needs to move to perform the next expected vehicle action at each refresh time point according to the real-time vehicle moving speed;

[0221] A navigation element projection unit, configured to, at each refresh time point, determine the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the real-time distance value, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0222] Based on the above embodiments, the navigation element projection unit may specifically be further configured to:

[0223] Obtain the current real-time distance value matching the current refresh time point;

[0224] According to the current real-time distance value, calculate the current target AR navigation element magnification ratio, and according to the standard layout position and the current target AR navigation element magnification ratio, calculate the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point;

[0225] At the current refresh time point, project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0226] Based on the above embodiments, the target AR navigation element is a left turn prompt element or a right turn prompt element.

[0227] Based on the above embodiments, the navigation element projection unit may be further specifically configured to:

[0228] According to the current real-time distance value, calculate the current target AR navigation element magnification ratio and the current lateral movement distance;

[0229] According to the standard layout position, the current target AR navigation element magnification ratio, and the current lateral movement distance, calculate the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point.

[0230] Based on the above embodiments, it may further include an end display animation projection module, specifically configured to:

[0231] When it is determined that the real-time distance value reaches a preset end display distance threshold, generate an end display animation matching the target AR navigation element;

[0232] Determine the preset third layout position of the end display animation in the target projection image, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0233] The head-up display device provided by the embodiments of the present invention can execute the navigation information display method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0234] Figure 5 FIG. shows a schematic structural diagram of a vehicle 50 that can be used to implement the embodiments of the present invention. The vehicle is intended to represent various forms of transportation means. For example, the transportation means may include, but are not limited to, land transportation means such as vehicles, air transportation means such as aircraft (or called flying vehicles), or water or underwater transportation means, etc. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0235] As Figure 5 shown, the vehicle 50 includes at least one processor 51 and a memory communicatively connected to the at least one processor 51, such as a read-only memory (ROM) 52, a random access memory (RAM) 53, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 51 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 52 or the computer program loaded from the storage unit 58 into the random access memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the vehicle 50 can also be stored. The processor 51, the ROM 52, and the RAM 53 are connected to each other through a bus 54. The input / output (I / O) interface 55 is also connected to the bus 54.

[0236] Multiple components in the vehicle 50 are connected to the I / O interface 55, including: an input unit 56, such as a keyboard, a mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a disk, an optical disc, etc.; and a communication unit 59, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 59 allows the vehicle 50 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0237] The processor 51 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 51 executes the various methods and processes described above, such as the method for displaying navigation information as shown in the embodiments of the present invention. That is:

[0238] When the display condition of the target AR navigation element is detected, obtain the first position description information of the target AR navigation element in the real world;

[0239] According to the first position description information of the target AR navigation element in the real world, determine the second position description information of the target AR navigation element in the head-up display coordinate system on the virtual image plane;

[0240] According to the second position description information, determine the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

[0241] In some embodiments, the method for displaying navigation information can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 58. In some embodiments, part or all of the computer program can be loaded and / or installed onto the vehicle 50 via the ROM 52 and / or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the method for displaying navigation information described above can be executed. Alternatively, in other embodiments, the processor 51 can be configured to execute the method for displaying navigation information by any other suitable means (e.g., by means of firmware).

[0242] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0243] The computer program for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer program can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0244] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0245] To provide interaction with a user, the systems and techniques described herein can be implemented on a vehicle that includes: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the vehicle. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0246] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected with each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0247] The computing system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client - server relationship is created by computer programs running on the respective computers and having a client - server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0248] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0249] The above - described specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for displaying navigation information, characterized in that, Including: When detecting the display condition of a target augmented reality (AR) navigation element, obtaining first position description information of the target AR navigation element in the real world; Determining second position description information of the target AR navigation element in a head-up display coordinate system on a virtual image plane according to the first position description information of the target AR navigation element in the real world; Determining a standard layout position of the target AR navigation element in a target projection image formed by a head-up display device in a vehicle according to the second position description information, and projecting the target projection image displayed by the head-up display device onto an imaging element of the vehicle; Detecting the display condition of the target AR navigation element, including: Obtaining navigation data of the vehicle in real time; Filtering a next expected vehicle action that matches the vehicle from the navigation data, and obtaining in real time a distance value that the vehicle needs to move to execute the next expected vehicle action; When the distance value reaches a first pre-prompt threshold, determining a first layout position preset for a pre-prompt element in the target projection image, and projecting the target projection image displayed by the head-up display device onto an imaging element of the vehicle; When the distance value reaches a second pre-prompt threshold, determining a second layout position preset for a leading prompt animation with a preset duration in the target projection image, and projecting the target projection image displayed by the head-up display device onto an imaging element of the vehicle; When it is determined that the leading prompt animation has finished playing, determining that the display condition of the target AR navigation element is detected; Projecting the target projection image displayed by the head-up display device onto an imaging element of the vehicle, including: Determining a plurality of refresh time points according to the target projection image refresh rate, and determining a real-time distance value that the vehicle needs to move to execute the next expected vehicle action at each refresh time point according to the real-time vehicle moving speed; At each refresh time point, determining a corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the real-time distance value, and projecting the target projection image displayed by the head-up display device onto an imaging element of the vehicle.

2. The method according to claim 1, characterized in that, Obtaining the first position description information of the target AR navigation element in the real world, including: In a pre-established database, obtaining the length and width values of the target AR navigation element in the real world, the height value from a reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis; Obtaining a current distance value that the vehicle needs to move to execute the next expected vehicle action according to the navigation data of the vehicle; Taking the combination of the length and width values, the height value from the reference plane, the left offset, and the current distance value as the first position description information.

3. The method according to claim 2, wherein In a pre-established database, obtaining the length and width values of the target AR navigation element in the real world, the height value from a reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis, including: Obtaining a target environment attribute of the current moving environment of the vehicle according to the navigation data of the vehicle; According to the mapping relationship between the environmental attributes and the position parameters in the database, query and obtain the length and width values of the target AR navigation element in the real world, the height value from the reference plane, and the left offset of the center point of the target AR navigation element relative to the driving axis under the target environmental attributes.

4. The method according to any one of claims 1 to 3, characterized in that, According to the first position description information of the target AR navigation element in the real world, determine the second position description information of the target AR navigation element in the head-up display coordinate system on the virtual image plane, including: In the pre-established database, obtain at least one optical performance parameter corresponding to the virtual image plane; According to each optical performance parameter and the first position description information, determine the corner coordinates of the target AR navigation element in the head-up display coordinate system as the second position description information.

5. The method according to claim 4, wherein The optical performance parameters include: The horizontal field of view angle of the virtual image plane, the vertical field of view angle of the virtual image plane, the left view angle of the virtual image plane, the lower view angle of the virtual image plane, the horizontal resolution of the virtual image plane, the vertical resolution of the virtual image plane, the virtual image distance, and a preset human eye-to-reference plane distance value.

6. The method according to claim 1, characterized in that, At each refresh time point, determine the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the real-time distance value, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle, including: Obtain the current real-time distance value matching the current refresh time point; According to the current real-time distance value, calculate the current magnification ratio of the target AR navigation element, and according to the standard layout position and the current magnification ratio of the target AR navigation element, calculate the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point; At the current refresh time point, project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

7. The method according to claim 6, characterized in that, The target AR navigation element is a left-turn prompt element or a right-turn prompt element; According to the current real-time distance value, calculate the current magnification ratio of the target AR navigation element, and according to the standard projection parameters and the current magnification ratio of the target AR navigation element, calculate the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point, including: According to the current real-time distance value, calculate the current magnification ratio of the target AR navigation element and the current lateral movement distance; According to the standard layout position, the current magnification ratio of the target AR navigation element, and the current lateral movement distance, calculate the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle at the current refresh time point.

8. The method according to claim 1, characterized in that, After determining the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the real-time distance value at each refresh time point and projecting the target projection image displayed by the head-up display device onto the imaging element of the vehicle, it further includes: When it is determined that the real-time distance value reaches the preset end display distance threshold, generate an end display animation matching the target AR navigation element; Determine the preset third layout position of the end display animation in the target projection image, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

9. A head-up display device, characterized in that, Including: The first position description information acquisition module is used to acquire the first position description information of the target AR navigation element in the real world when the display condition of the target augmented reality (AR) navigation element is detected; The second position description information determination module is used to determine the second position description information of the target AR navigation element in the head-up display coordinate system of the virtual image plane according to the first position description information of the target AR navigation element in the real world; The navigation information projection module is used to determine the standard layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the second position description information, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle; The first position description information acquisition module is specifically used for: Obtain the navigation data of the vehicle in real time; Filter the next expected vehicle action that matches the vehicle from the navigation data, and obtain the distance value that the vehicle needs to move to execute the next expected vehicle action in real time; When the distance value reaches the first pre-prompt threshold, determine the preset first layout position of the pre-prompt element in the target projection image, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle; When the distance value reaches the second pre-prompt threshold, determine the preset second layout position of the leading prompt animation with a preset duration in the target projection image, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle; When it is determined that the leading prompt animation has finished playing, determine that the display condition of the target AR navigation element is detected; The navigation information projection module is specifically used for: Determine multiple refresh time points according to the target projection image refresh rate, and determine the real-time distance value that the vehicle needs to move to execute the next expected vehicle action at each refresh time point according to the real-time vehicle moving speed; At each refresh time point, determine the corrected layout position of the target AR navigation element in the target projection image formed by the head-up display device in the vehicle according to the real-time distance value, and project the target projection image displayed by the head-up display device onto the imaging element of the vehicle.

10. A vehicle, characterized in that, The vehicle includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the navigation information display method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to implement the navigation information display method according to any one of claims 1-8 when executed.

Images