AR-HUD navigation display method and related device

By fusing high-precision data under the carrier coordinate system with low-precision data under the longitude and latitude high coordinate system, more accurate navigation data is generated, and the problem of deviation between AR-HUD navigation data and actual scenes is solved, improving the user experience.

CN118913306APending Publication Date: 2024-11-08NEUSOFT CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411019316.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The navigation data displayed by AR-HUD is different from the actual scene, which affects the user's user experience.

Method used

By receiving the first position data of the target device under the high coordinate system of the longitude and latitude and latitude and second position data of the carrier coordinate system, the mapping relationship conversion of the carrier coordinate system can be used to obtain more accurate third position data, and then the mapping relationship of the carrier coordinate system can be transformed into the third position data and then combined with the The first pose data is fused to generate more accurate target pose data, thereby improving the accuracy of navigation data and fitting with the actual scene.

Benefits of technology

Improve the accuracy of navigation data, reduce deviation from actual scenes, and improve user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118913306A_ABST
    Figure CN118913306A_ABST
Patent Text Reader

Abstract

The invention discloses a navigation display method of AR-HUD (Auto Register-Head Up Display) and a related device. The method comprises the following steps: receiving first pose data of target equipment in a longitude and latitude coordinate system and second pose data of the target equipment in a carrier coordinate system; the data precision level of the first pose data is lower than that of the second pose data; according to the mapping relation between the longitude, latitude and height coordinate system and the carrier coordinate system, based on the second pose data, third pose data under the longitude, latitude and height coordinate system is obtained through conversion; fusing the first pose data with the third pose data to obtain target pose data of the target equipment; performing conversion based on the target pose data to obtain navigation data; and displaying the navigation data on a display interface of an AR-HUD (Augmented Reality Head-Up Display). Therefore, the data precision of the navigation data is improved, and the deviation between the navigation data and the actual scene is reduced, so that the navigation data displayed by the AR-HUD is more fit with the actual scene, and the use experience of a user is further improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of AR display technology, and in particular to a navigation display method and related devices of an AR-HUD. Background Art

[0002] With the rapid development of science and technology, augmented reality (AR) display technology has gradually matured and is widely used in various fields, such as the automotive industry. Among them, the augmented reality head up display (AR-HUD) system can present virtual information such as navigation data in the driver's field of vision in a more intuitive way, greatly improving the convenience and safety of driving.

[0003] In the related technology, the vehicle can generate navigation data based on the posture data provided by the navigation system, and then project the navigation data on the display interface (such as the front windshield of the vehicle) through AR-HUD. However, the navigation data displayed by AR-HUD is prone to deviation from the actual road scene. Taking the navigation data as a three-dimensional (3D) navigation arrow as an example, it is easy for the navigation direction of the 3D navigation arrow to deviate from the actual direction, which affects the user experience.

[0004] Therefore, how to make the navigation data displayed by AR-HUD more closely match the actual scene has become a problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a navigation display method and related devices of an AR-HUD, with the aim of making the navigation data displayed by the AR-HUD more consistent with the actual scene and improving the user experience.

[0006] In a first aspect, an embodiment of the present application provides a navigation display method of an AR-HUD, the method comprising:

[0007] Receiving first pose data of a target device in a longitude and latitude coordinate system, and second pose data of the target device in a carrier coordinate system; the data accuracy level of the first pose data is lower than the data accuracy level of the second pose data;

[0008] According to the mapping relationship between the longitude and latitude high coordinate system and the carrier coordinate system, based on the second posture data, convert and obtain third posture data in the longitude and latitude high coordinate system;

[0009] Fusing the first posture data with the third posture data to obtain target posture data of the target device;

[0010] Obtaining navigation data based on the target posture data conversion;

[0011] The navigation data is displayed on a display interface of an augmented reality head-up display (AR-HUD).

[0012] As an implementation manner, the sending frequency of the first posture data is lower than the sending frequency of the second posture data; the receiving the first posture data of the target device in the longitude and latitude coordinate system, and the second posture data of the target device in the carrier coordinate system, includes:

[0013] Receiving a set of first pose data and a set of second pose data;

[0014] According to the mapping relationship between the latitude and longitude high coordinate system and the carrier coordinate system, based on the second posture data, converting to obtain third posture data in the latitude and longitude high coordinate system, including:

[0015] Based on the set of first pose data and the set of second pose data, a mapping relationship between the latitude and longitude coordinate system and the carrier coordinate system is calculated, and a new set of second pose data of the target device is continuously received;

[0016] According to the mapping relationship, the third posture data is converted based on the new set of second posture data.

[0017] As an implementation manner, the converting the third pose data based on the new set of second pose data according to the mapping relationship includes:

[0018] Calculate the median of the new set of second posture data and the previous set of second posture data to obtain median second posture data; the previous set of second posture data is the latest set of second posture data received before the new set of second posture data is received;

[0019] Multiplying the time interval between receiving the new set of second posture data and receiving the previous set of second posture data by the median second posture data to obtain incremental second posture data;

[0020] Adding the incremental second pose data and the new set of second pose data to obtain updated second pose data;

[0021] Based on the mapping relationship, the updated second pose data is converted into the third pose data.

[0022] As an implementation manner, the sending frequency of the first posture data is lower than the sending frequency of the second posture data, and the navigation display method of the AR-HUD further includes:

[0023] Before receiving a new set of first pose data, updating the target pose data based on the latest set of second pose data received to obtain updated target pose data;

[0024] Obtaining new navigation data based on the updated target pose data conversion;

[0025] The navigation data on the display interface of the AR-HUD is updated using the new navigation data.

[0026] As an implementation manner, updating the target pose data based on the latest set of second pose data received to obtain updated target pose data includes:

[0027] According to the mapping relationship, based on the latest set of second posture data, fourth posture data in the longitude and latitude high coordinate system is converted;

[0028] The fourth posture data is fused with the target posture data to obtain the updated target posture data.

[0029] As an implementation manner, the navigation display method of the AR-HUD further includes:

[0030] Acquire a historical navigation route of the target device; a historical navigation route includes historical second position data of the target device in the carrier coordinate system;

[0031] If the received second posture data matches the historical second posture data, determining that the driving route of the target device is the historical navigation route;

[0032] The height data in the historical second posture data is added to the navigation data as slope data.

[0033] As an implementation manner, the first posture data includes different types of data, and the third posture data includes different types of data; and fusing the first posture data with the third posture data to obtain the target posture data of the target device includes:

[0034] Performing weighted processing on the data of the same type in the first posture data and the third posture data to obtain weighted data;

[0035] The weighted data, the data in the first posture data that has not been weighted, and the data in the third posture data that has not been weighted are combined into target posture data of the target device.

[0036] As an implementation manner, the weight value corresponding to the third posture data is positively correlated with the number of surrounding objects of the target device; the weight value is used for weighted processing;

[0037] And / or, the first posture data is measured by a navigation system on the target device; the second posture data is measured by a simultaneous localization and mapping (SLAM) system on the target device.

[0038] And / or, the target device includes a target vehicle or a target robot.

[0039] In a second aspect, an embodiment of the present application provides a navigation display device of an AR-HUD, the device comprising:

[0040] A data receiving module, used to receive first pose data of a target device in a longitude and latitude coordinate system, and second pose data of the target device in a carrier coordinate system; the data accuracy level of the first pose data is lower than the data accuracy level of the second pose data;

[0041] A posture data conversion module, configured to convert the second posture data to obtain third posture data in the longitude and latitude high coordinate system according to a mapping relationship between the longitude and latitude high coordinate system and the carrier coordinate system;

[0042] A data fusion module, used for fusing the first posture data with the third posture data to obtain target posture data of the target device;

[0043] A navigation data conversion module, used to convert the target posture data to obtain navigation data;

[0044] The data display module is used to display the navigation data on the display interface of the augmented reality head-up display (AR-HUD).

[0045] In a third aspect, an embodiment of the present application provides an AR-HUD navigation display device, the device comprising a memory and a processor:

[0046] The memory is used to store a computer program and transmit the computer program to the processor;

[0047] The processor is used to execute the computer program so that the device executes the AR-HUD navigation display method described in the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the device running the computer program implements the AR-HUD navigation display method described in the first aspect above.

[0049] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0050] The embodiment of the present application provides a navigation display method and related device of AR-HUD. In this method, the first position data of the target device in the longitude and latitude high coordinate system and the second position data of the target device in the carrier coordinate system are first received, wherein the data accuracy level of the first position data is lower than the data accuracy level of the second position data; then, according to the mapping relationship between the longitude and latitude high coordinate system and the carrier coordinate system, the third position data in the longitude and latitude high coordinate system is obtained based on the second position data conversion; then, the first position data and the third position data in the longitude and latitude high coordinate system are fused to obtain the target position data of the target device; then, navigation data is obtained based on the target position data conversion; finally, the navigation data is displayed on the display interface of the augmented reality head-up display AR-HUD.

[0051] In this way, the present application converts and fuses the second posture data with a higher data accuracy level into the first posture data. The data accuracy of the target posture data obtained after the fusion is greatly improved compared with the data accuracy of the first posture data. Therefore, it is possible to improve the data accuracy of the navigation data, reduce the deviation between the navigation data and the actual scene, and make the navigation data displayed by the AR-HUD more consistent with the actual scene, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 This is an architecture diagram for displaying data through AR-HUD in related technologies;

[0054] Figure 2 It is a schematic diagram of displaying a 3D navigation arrow through an AR-HUD in the related art;

[0055] Figure 3 An application scenario of an AR-HUD navigation display method provided in an embodiment of the present application;

[0056] Figure 4 A flowchart of a navigation display method of an AR-HUD provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of comparison of navigation data provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of a specific AR-HUD navigation display method provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of the structure of an AR-HUD navigation display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] At present, the existing AR-HUD navigation display method is: AR-HUD displays navigation data transmitted by the navigation system on the display interface. Figure 1 , this figure is an architecture diagram of displaying data through AR-HUD in related technologies. The navigation system can send the measured position data and guidance data to AR-HUD, so that AR-HUD can display the navigation data obtained based on the position data and guidance data on the display interface, which is convenient for drivers and other users to watch. However, when AR-HUD displays navigation data, there is sometimes a certain deviation from the actual scene. Taking the AR-HUD and navigation system installed on the car as an example, the 3D navigation arrow can be displayed based on the position data and guidance data (such as lane information, speed limit, etc.) sent by the navigation system, but when the driver turns at the intersection, sometimes there is a problem that the turning direction of the 3D navigation arrow deviates from the actual turning direction of the intersection, which in turn affects the user experience.

[0062] See also Figure 2 , which is a schematic diagram of displaying a 3D navigation arrow through AR-HUD in the related art. Figure 2 As shown, after the vehicle starts moving, the navigation system can measure the vehicle's position data, heading angle data and intersection position data. After receiving these data, it can convert them into AR-HUD display data (that is, the navigation data of the 3D navigation arrow), and then output it to the AR-HUD display interface, and then wait for the navigation system to send the next frame of data (also called the next set of data, including the vehicle's latest position data, the latest heading angle data and the latest intersection position data).

[0063] The inventors have found that the position data measured by the navigation system is in a high-latitude and longitude coordinate system, which is usually developed based on a standard definition map (SD Map). The measured position data is road-level, and the actual intersection position of the vehicle when turning is related to the lane in which the vehicle is located. Therefore, the data accuracy of the position data is insufficient, which can easily lead to deviations from the actual scene. At the same time, the heading angle data measured by the navigation system is also in a high-latitude and longitude coordinate system, which can represent up to 72 directions, that is, 5 degrees for one direction, which can also easily lead to deviations from the actual scene. That is, the posture data measured by the navigation system are all in a high-latitude and longitude coordinate system, and the data accuracy is insufficient, resulting in deviations between the navigation data and the actual lane scene.

[0064] Based on this, in order to solve the above problems, the embodiments of the present application provide an AR-HUD navigation display method and related devices, with the aim of making the navigation data displayed by the AR-HUD more consistent with the actual scene and improving the user experience.

[0065] In the method, a first pose data of a target device in a longitude and latitude high coordinate system and a second pose data of the target device in a carrier coordinate system are received; the data accuracy level of the first pose data is lower than the data accuracy level of the second pose data; according to the mapping relationship between the longitude and latitude high coordinate system and the carrier coordinate system, based on the second pose data, the third pose data in the longitude and latitude high coordinate system is converted; the first pose data and the third pose data are fused to obtain target pose data of the target device; navigation data is obtained based on the target pose data conversion; and the navigation data is displayed on the display interface of an augmented reality head-up display (AR-HUD).

[0066] In this way, compared with the related technology in which the navigation data displayed on the display interface of AR-HUD is only obtained based on the posture data in the longitude and latitude high coordinate system, the present application introduces the second posture data in the carrier coordinate system with a higher data accuracy level, thereby improving the data accuracy of the target posture data obtained by combining the first posture data and the second posture data, thereby reducing the deviation between the navigation data and the actual scene and improving the user experience.

[0067] For example, the embodiments of the present application can be applied to Figure 3The scene shown. The scene includes a navigation system 101, a SLAM system 102, a data processing device 103 and an AR-HUD 104, wherein the navigation system 101 sends the measured first pose data of the target device in the latitude and longitude coordinate system to the data processing device 103, and the SLAM system 102 sends the measured second pose data of the target device in the carrier coordinate system to the data processing device 103, and the data processing device 103 uses the implementation method provided in the embodiment of the present application to perform navigation display of the AR-HUD 104.

[0068] In some embodiments, the navigation system 101, the SLAM system 102, the data processing device 103 and the AR-HUD 104 may all be installed on the target device, which is not limited in this application.

[0069] First, in the above application scenario, although the action description of the implementation method provided in the embodiment of the present application is executed by the data processing device 103; however, the embodiment of the present application is not limited in terms of the execution subject, as long as the actions disclosed in the implementation method provided in the embodiment of the present application are executed.

[0070] Secondly, the above scenario is only an example scenario provided by the embodiment of the present application, and the embodiment of the present application is not limited to this scenario.

[0071] Finally, in the above application scenario, although the first posture data and the second posture data received by the embodiment of the present application are measured by the navigation system 101 and the SLAM system 102 respectively, the embodiment of the present application is not limited to this.

[0072] The specific implementation of the AR-HUD navigation display method and related devices in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0073] See also Figure 4 , which is a flowchart of an AR-HUD navigation display method provided in an embodiment of the present application, combined with Figure 4 As shown, it may specifically include:

[0074] S401: Receive first pose data of a target device in a longitude and latitude coordinate system, and second pose data of the target device in a carrier coordinate system.

[0075] In the embodiment of the present application, the target device refers to any device that needs to display navigation data through AR-HUD.

[0076] In a possible implementation manner of the present application, the target device may include a target vehicle or a target robot.

[0077] For example, when the target device is a target vehicle, the navigation data can be projected in front of the driver's view through AR-HUD on the target vehicle, which is convenient for the driver to watch and then drive the vehicle according to the navigation data. Alternatively, in an autonomous driving task or an assisted driving task, the displayed navigation data can also enable the vehicle to respond and adapt to changes in road conditions more quickly, thereby improving driving performance and facilitating the realization of an autonomous driving task or an assisted driving task.

[0078] As another example, when the target device is a target robot, navigation data can be displayed to the target robot through AR-HUD, so that the target robot can move according to the navigation data and avoid obstacles, thereby better providing the robot with useful navigation and decision-making support, which is conducive to the realization of robot tasks.

[0079] It should be noted that the above-mentioned target device is a target vehicle or a target robot is only an example, that is, the navigation display method of the AR-HUD provided in the embodiment of the present application can be applied to the fields of autonomous driving, assisted driving and robotics. This is only an example, and the present application does not limit this, and AR-HUD can be applied.

[0080] In the embodiment of the present application, the longitude and latitude coordinate system can also be called a custom coordinate system based on WGS84, with the origin being the center of mass of the earth, and mainly composed of three parameters: longitude, latitude, and altitude.

[0081] The carrier coordinate system can also be called the Body coordinate system. It is a right-handed rectangular coordinate system that is fixed to a specific carrier and moves with it. The center of mass of the carrier is taken as the origin, and the three axes are respectively established along the horizontal axis, longitudinal axis, and vertical axis of the carrier. The second pose data of the target device in the carrier coordinate system indicates that the specific carrier is the target device, and the carrier coordinate system will move with the target device.

[0082] The pose data is used to describe the position and attitude of the target device. The first pose data is used to describe the position and attitude of the target device in the longitude and latitude coordinate system, and the second pose data is used to describe the position and attitude of the target device in the carrier coordinate system.

[0083] In some embodiments, the first pose data may include position data of the target device and heading angle data of the target device.

[0084] In some embodiments, the second posture data may include the position data of the target device and the heading angle data, pitch angle data and roll angle data of the target device. The position data includes data on three axes, so the second posture data may also be referred to as 6-DOF data.

[0085] It should be noted that the data content contained in the above-mentioned first posture data and second posture data is only an example and may include more or less content than the above, and this application does not limit this.

[0086] For ease of understanding, the following embodiments are introduced by taking the data content included in the first posture data and the second posture data in the above example as an example.

[0087] In an embodiment of the present application, the data accuracy level of the first pose data is lower than the data accuracy level of the second pose data, wherein the data accuracy level is used to indicate the level of accuracy achieved by the first pose data and the second pose data.

[0088] In some embodiments, the data accuracy level of the position data in the first posture data can be road level, which means that the error between its position data and the actual posture data of the target device (the road on which it is located) covers a few meters to more than ten meters; the data accuracy level of the position data in the second posture data can be centimeter level, which means that the error between its position data and the actual posture data of the target device covers the centimeter accuracy range.

[0089] It should be noted that the above is only an example, and the first pose data and the second pose data may also have other data accuracy levels, for example, the data accuracy level of the first pose data is the lane level, and the data accuracy level of the second pose data is the centimeter level.

[0090] In some embodiments, the posture data in the first posture data can represent 72 directions, that is, the data accuracy level is 5°, and the posture data in the second posture data can represent 360 directions, that is, the data accuracy level is 1°.

[0091] In a possible implementation of the present application, the first posture data may be measured by a navigation system on the target device; and the second posture data may be measured by a simultaneous positioning and mapping SLAM system on the target device.

[0092] In addition, in some embodiments, the second posture data may also be obtained by measuring a visual-inertial odometry (VIO) system, which is not limited in this application.

[0093] S402: According to the mapping relationship between the longitude and latitude high coordinate system and the carrier coordinate system, based on the second pose data, convert and obtain third pose data in the longitude and latitude high coordinate system.

[0094] It should be understood that different coordinate systems use different ways to describe and represent the position information of the target device in space. In order to facilitate the subsequent fusion processing of the data corresponding to the longitude and latitude coordinate system and the carrier coordinate system, it is necessary to ensure the consistency of the coordinate system in which the data is located.

[0095] Therefore, the second posture data in the carrier coordinate system can be converted into the third posture data in the longitude and latitude coordinate system.

[0096] Based on the above introduction, the carrier coordinate system takes the center of mass of the carrier as its origin. When the target device is in the process of moving, the carrier coordinate system is also always moving. Therefore, it is necessary to calculate the real-time mapping relationship between the carrier coordinate system and the longitude and latitude coordinate system based on the received first pose data and second pose data.

[0097] In a possible implementation manner of the present application, the sending frequency of the first pose data is lower than the sending frequency of the second pose data; then the above S401 may specifically include: first receiving a set of first pose data and a set of second pose data.

[0098] Exemplarily, the sending frequency of the first posture data may be once in 1s (also referred to as a frame rate of 1 frame), and the sending frequency of the second posture data may be 30 times in 1s (also referred to as a frame rate of 30 frames), which is not limited in the present application.

[0099] It should be understood that it takes a certain amount of time to calculate the mapping relationship between the two coordinate systems, and the sending frequency of the first pose data is lower than the sending frequency of the second pose data, indicating that when the next set of first pose data is not sent, the second pose data will be sent, and the new set of second pose data is slightly different from the set of second pose data for calculating the mapping relationship, so the mapping relationship does not change much, but the new set of second pose data is more real-time data for the target device, so it can be used to calculate the coordinate system conversion.

[0100] Correspondingly, the above S402 may specifically include: based on the set of first pose data and the set of second pose data, calculating the mapping relationship between the longitude and latitude coordinate system and the carrier coordinate system, and continuously receiving a new set of second pose data of the target device; and then according to the mapping relationship, converting the new set of second pose data to obtain the third pose data.

[0101] In this way, based on a new set of more real-time second posture data, third posture data that is more closely aligned with the current posture of the target device can be converted, which is beneficial to improving the accuracy of navigation data obtained by subsequent calculations.

[0102] In some embodiments, during the process of calculating the mapping relationship between the two coordinate systems, multiple new sets of second pose data may be continuously received. After the mapping relationship calculation is completed, the third pose data can be converted based on the latest set of second pose data received. In this way, the third pose data that is more in line with the actual pose information of the target device can be further obtained, thereby improving the accuracy of subsequent data processing.

[0103] In a possible implementation manner of the present application, the above-mentioned “according to the mapping relationship, converting the new set of second pose data to obtain the third pose data” may specifically include the following steps:

[0104] A1: Calculate the median of the new set of second posture data and the previous set of second posture data to obtain the median second posture data.

[0105] Calculate the average value of the new set of second pose data and the previous set of second pose data.

[0106] Exemplarily, a new set of second posture data can be expressed as (x1, y1, z1, Yaw1, Pitch1, Roll1), and the previous set of second posture data can be expressed as (x2, y2, z2, Yaw2, Pitch2, Roll2), where Yaw, Pitch, and Roll represent heading angle, pitch angle, and roll angle, respectively. The median of the two is calculated, and the obtained median second posture data can be expressed as ((x1+x2) / 2, (y1+y2) / 2, (z1+z2) / 2, (Yaw1+Yaw2) / 2, (Pitch1+Pitch2) / 2, (Roll1+Roll2) / 2).

[0107] Among them, the previous set of second posture data is the latest set of second posture data received before receiving the new set of second posture data, that is, the two sets of second posture data are received adjacently.

[0108] A2: Multiply the time interval between receiving the new set of second posture data and receiving the previous set of second posture data by the median second posture data to obtain incremental second posture data.

[0109] Exemplarily, the time interval between receiving the new set of second pose data and receiving the previous set of second pose data is △t. Based on the above example, the incremental second pose data can be expressed as △t×((x1+x2) / 2, (y1+y2) / 2, (z1+z2) / 2, (Yaw1+Yaw2) / 2, (Pitch1+Pitch2) / 2, (Roll1+Roll2) / 2).

[0110] A3: Add the incremental second pose data and the new set of second pose data to obtain updated second pose data.

[0111] Exemplarily, based on the above example, the updated second pose data can be expressed as △t×((x1+x2) / 2, (y1+y2) / 2, (z1+z2) / 2, (Yaw1+Yaw2) / 2, (Pitch1+Pitch2) / 2, (Roll1+Roll2) / 2)+(x1, y1, z1, Yaw1, Pitch1, Roll1)).

[0112] A4: Based on the mapping relationship, convert the updated second pose data into the third pose data.

[0113] In this way, the noise or extreme points in the second pose data can be eliminated through the above-mentioned A1-A3 method (which can be called median method integration), so as to obtain more stable and reliable updated second pose data. On this basis, the converted third pose data can also be made more reliable and accurate.

[0114] It should be noted that the above A1-A4 are only used to distinguish different steps and are not shown in the figure.

[0115] In addition, considering that the carrier coordinate system changes with the movement of the target device, the mapping relationship between the carrier coordinate system and the latitude and longitude coordinate system will also change. In order to improve the accuracy of the mapping relationship, the mapping relationship between the two coordinate systems can be calculated multiple times in real time.

[0116] In some embodiments, if the sending frequency of the first pose data is lower than the sending frequency of the second pose data, then after receiving a set of first pose data, the mapping relationship between the carrier coordinate system and the longitude and latitude coordinate system can be calculated based on the set of first pose data and the latest set of second pose data; after receiving a new set of first pose data, the mapping relationship between the two coordinate systems is recalculated based on the new set of first pose data and the latest set of second pose data, that is, upon receiving a set of first pose data, the step of "calculating the mapping relationship between the longitude and latitude coordinate system and the carrier coordinate system based on the set of first pose data and the set of second pose data, and continuously receiving a new set of second pose data of the target device" is returned.

[0117] For example, the first pose data is sent once every 1 second, and the second pose data is sent 30 times every 1 second, then the mapping relationship between the longitude and latitude coordinate system and the carrier coordinate system can be calculated once every 1 second. Specifically, a set of the first pose data and the second pose data can be received at the same time, and then the mapping relationship is calculated, and the second pose data is continuously received for subsequent processing.

[0118] In a possible implementation of the present application, the set of first posture data may include first position data; the set of second posture data may include second position data and second posture data; and the updated second posture data may include updated second position data and updated second posture data.

[0119] Accordingly, the above “calculating the mapping relationship between the latitude and longitude coordinate system and the carrier coordinate system based on the set of first pose data and the set of second pose data” may specifically include the following steps:

[0120] B1: Based on the first position data and the second position data, calculate the displacement value between the longitude and latitude coordinate system and the carrier coordinate system.

[0121] In the embodiment of the present application, the displacement value refers to the distance value required to move from the origin of the carrier coordinate system to the origin of the longitude and latitude coordinate system.

[0122] Based on the above introduction, the origin of the latitude and longitude coordinate system is different from the origin of the carrier coordinate system. The origin of the latitude and longitude coordinate system does not move, while the origin of the carrier coordinate system moves with the movement of the target device. However, this application needs to convert the pose data in the carrier coordinate system into the pose data in the latitude and longitude coordinate system.

[0123] Therefore, in some embodiments, the first position data may be converted into position data in a geocentric coordinate system, and then the distance between the first position data and the second position data may be calculated. For example, the position data in a geocentric coordinate system may be expressed as (x3, y3, z3), and the second position data may be expressed as (x4, y4, z4). Then, [(x4-x3) 2 +(y4-y3) 2 +(z4-z3) 2 ] is square rooted to obtain the distance between the two as the displacement value.

[0124] B2: Based on the second posture data, a rotation matrix is ​​calculated.

[0125] In the embodiment of the present application, the rotation matrix is ​​used to indicate the rotation relationship between the longitude and latitude coordinate system and the carrier coordinate system.

[0126] In some embodiments, the second posture data includes heading angle data, pitch angle data, and roll angle data, wherein the heading angle data refers to the rotation angle of the target device around the Z axis, the pitch angle data refers to the rotation angle of the target device around the Y axis, and the roll angle data refers to the rotation angle of the target device around the X axis. Based on these three angle data describing the rotation relationship (as Euler angles), the rotation matrix between the carrier coordinate system and the geocentric coordinate system can be calculated.

[0127] B3: Combining the displacement value and the rotation relationship into the mapping relationship.

[0128] Based on the displacement value between the origins of the two coordinate systems calculated above, and the rotation matrix between the two coordinate systems calculated above, the mapping relationship between the longitude and latitude coordinate system and the carrier coordinate system can be combined to obtain.

[0129] Accordingly, the above A4 may specifically include the following steps:

[0130] B4: Multiply the updated second position data by the rotation matrix to obtain a product value, and add the product value to the displacement value to obtain the third position data in the longitude and latitude coordinate system.

[0131] Exemplarily, the third position data may be calculated by the following formula:

[0132] P W1 =R WB ×P B1 +T0

[0133] Among them, P W1 Indicates the third position data, R WB represents the rotation matrix, P B1 represents the updated second position data, and T0 represents the displacement value.

[0134] That is, the updated second position data is first rotated, and then translated to obtain the third position data in the longitude and latitude coordinate system.

[0135] B5: Multiply the updated second posture data by the rotation matrix to obtain third posture data in the longitude and latitude coordinate system.

[0136] Exemplarily, the third posture data can be calculated by the following formula:

[0137] P W2 =R WB ×P B2

[0138] Among them, P W2 Represents the third posture data, R WB represents the rotation matrix, P B2 Indicates the updated second posture data.

[0139] That is, the updated second posture data is rotated to obtain the third posture data in the longitude and latitude coordinate system.

[0140] B6: Combine the third position data and the third posture data into the third posture data.

[0141] In this way, the data conversion between the coordinate systems is accurately achieved by taking into account the translation relationship and rotation relationship between the latitude and longitude coordinate system and the carrier coordinate system.

[0142] In addition, in some embodiments, the received new set of second posture data may not be processed by the median method, but after executing B3, the new set of second posture data may be directly converted into third posture data based on the mapping relationship, and this application does not limit this.

[0143] S403: Fusing the first posture data with the third posture data to obtain target posture data of the target device.

[0144] The fused pose data obtained by fusing the first pose data with the third pose data is used as the target pose data of the target device.

[0145] In this way, compared with a single first pose data, more information can be integrated to further improve the richness and quality of the target pose data.

[0146] In a possible implementation manner of the present application, the first posture data includes different types of data, the third posture data includes different types of data, and the above S403 may include the following steps:

[0147] C1: weighting the data of the same type in the first posture data and the third posture data to obtain weighted data.

[0148] Exemplarily, based on the above example, the first pose data includes the position data and heading angle data of the target device, and the third pose data includes the position data, heading angle data, pitch angle data and roll angle data of the target device. It can be seen that the first pose data and the third pose data both contain two types of data, namely, position data and heading angle data, and the third pose data also contains pitch angle data and roll angle data.

[0149] Therefore, the position data of the first posture data and the position data of the third posture data can be weighted and fused, and the heading angle data of the first posture data and the heading angle data of the third posture data can also be weighted and fused. The obtained weighted data includes the weighted position data and the weighted heading angle data.

[0150] It should be noted that the data contained in the above-mentioned first pose data and second pose data are only examples. Both may also contain more or less data of the same type. This application does not limit this, and weighted processing can be performed on data of the same type.

[0151] In some embodiments, the first posture data can correspond to a weight value of 1, and the second posture data can correspond to a weight value of 2. Taking the position data in the same type of data as an example, the weight value 1 can be multiplied by the position data of the first posture data, and the weight value 2 can be multiplied by the position data of the third posture data. Finally, the two product values ​​are added to obtain the weighted position data. The weighted processing process of the heading angle data is the same and will not be repeated here.

[0152] Exemplarily, weight value 1 and weight value 2 may both be 0.5; or, weight value 1 may be 0.4, and weight value 2 may be 0.6, which is not limited in the present application.

[0153] In some embodiments, a Kalman filter can be used to perform weighted processing on data of the same type in the first pose data and the third pose data, and the Kalman filter can also filter the first pose data and the third pose data to reduce noise data and further improve data quality.

[0154] C2: combining the weighted data, the data in the first posture data that has not been weighted, and the data in the third posture data that has not been weighted into the target posture data of the target device.

[0155] The data in the first pose data that has not been weighted and the data in the third pose data that has not been weighted refer to the remaining data in the two except for the above-mentioned data belonging to the same category.

[0156] Exemplarily, based on the above example, there is no unweighted data in the first posture data, and the unweighted data in the third posture data includes pitch angle data and roll angle data, then the weighted position data, weighted heading angle data, pitch angle data and roll angle data can be used as the target posture data of the target device.

[0157] In this way, compared with the data with a single data precision (i.e. the first pose data), data with a higher data precision level (i.e. the third pose data) is introduced, that is, the weighted data included in the target pose data fuses data with different data precisions, which can improve the data quality, and the two can verify each other, increasing the credibility of the data; on this basis, the introduced data adds feature information (such as roll angle data and pitch angle data). Therefore, the target pose data can more accurately represent the pose information of the target device, and the navigation data generated based on this is more in line with the actual scene.

[0158] It should be noted that C1-C2 are only used to distinguish different steps and are not shown in the figure.

[0159] In some embodiments, taking the above-mentioned navigation system as an example, the navigation system can measure the intersection position data while measuring the first pose data. Then, the first pose data, the second pose data and the intersection position data can be received, and the intersection position data can also be added to the target pose data.

[0160] It should be noted that the above are only examples, and the navigation system and SLAM system may also measure more different types of data, which can all be added to the target pose data, and this application does not limit this.

[0161] In a possible implementation of the present application, the weight value corresponding to the third posture data is positively correlated with the number of objects around the target device, wherein the weight value is used for weighted processing. Referring to the above example, the weight value is weight 2.

[0162] In the embodiment of the present application, the weight value is dynamically adjustable.

[0163] It should be understood that the second posture data in the carrier coordinate system is data obtained by positioning the target device itself, and therefore has a strong dependence on the surrounding environment.

[0164] Taking the SLAM system mentioned above as an example, it can obtain data about surrounding objects through sensors, and then use these data to calculate the position and posture of the target device to obtain posture data. Therefore, the more surrounding objects there are, the more accurate the posture data obtained.

[0165] Therefore, when there are a large number of objects around the target device, the weight value corresponding to the third posture data can be increased accordingly.

[0166] In this way, dynamically adjusting the corresponding weight value based on the accuracy of the third pose data can further improve the accuracy of the target pose data and balance the data quality with the performance of the target device.

[0167] S404: Obtain navigation data based on the target posture data conversion.

[0168] In the embodiment of the present application, navigation data refers to data used to guide, instruct or assist the target device to achieve a task, and it is also data that can be understood and displayed by the AR-HUD system.

[0169] Taking the fields of autonomous driving and assisted driving as examples, navigation data can be used to control or assist the driving of the target vehicle.

[0170] Taking the field of robotics as an example, navigation data can be used to control the movement of the robot.

[0171] It should be noted that this application does not limit the type of navigation data.

[0172] In some embodiments, the navigation data may be image data and / or text data, etc.

[0173] Taking image data as an example, navigation data may be a 2D image or a 3D image, such as an icon, an arrow, etc. Among them, route planning may be represented by a 3D navigation arrow, and traffic signals may be represented by an image.

[0174] Taking text data as an example, navigation data may include current vehicle speed, current road information, current vehicle speed, and the like.

[0175] S405: Displaying the navigation data on a display interface of an augmented reality head-up display (AR-HUD).

[0176] In some embodiments, when the target device is a target vehicle, the display interface of the AR-HUD can be the front windshield; it can also be outside the target vehicle, at a certain distance from the front windshield.

[0177] In a possible implementation of the present application, the sending frequency of the first posture data is lower than the sending frequency of the second posture data. The navigation display method of the AR-HUD may also include: before receiving a new set of first posture data, first updating the target posture data of the target device based on the latest set of second posture data received to obtain updated target posture data; then converting to obtain new navigation data based on the updated target posture data; and then using the new navigation data to update the navigation data displayed on the display interface of the AR-HUD.

[0178] Among them, the sending frequency of the first pose data is lower than that of the second pose data, indicating that after receiving a set of the first pose data, multiple sets of the second pose data will be received before receiving the next set of the first pose data, and the newly received second pose data is more real-time and can more accurately describe the pose information of the target device. Therefore, the latest set of second pose data can be used to update the fused target pose data to obtain more real-time updated target pose data. Based on this, the new navigation data is further converted and updated and displayed on the display interface of the AR-HUD.

[0179] In this way, the update frequency of the data displayed on the AR-HUD display interface is increased, making the display effect smoother and reducing the display freeze caused by the low frequency of sending the first pose data. At the same time, the real-time performance of the data displayed on the AR-HUD display interface is improved, which is more in line with the actual scene and further reduces the deviation from the actual scene.

[0180] In addition, in some embodiments, before receiving a new set of first pose data, multiple sets of second pose data may be received, so after calculating the target pose data, the target pose data can be continuously updated using the latest set of second pose data received to continuously update the data on the AR-HUD display interface. In this way, the AR-HUD display effect is smoother and more real-time.

[0181] In a possible implementation of the present application, the above-mentioned "updating the target pose data based on the latest set of second pose data received to obtain updated target pose data" may specifically include: first, based on the calculated mapping relationship, based on the latest set of second pose data, converting to obtain the fourth pose data in the longitude and latitude high coordinate system (the specific conversion process can be seen in B4-B6, which will not be repeated here); then fusing the converted fourth pose data with the target pose data to obtain the updated target pose data, for example, the fusion can be performed by weighted processing or the like (the specific fusion process can be seen in the specific implementation of S403, which will not be repeated here).

[0182] In this way, the introduction of more real-time fourth pose data can improve the real-time performance of the updated target pose data, thereby improving its accuracy and achieving better navigation results.

[0183] In a possible implementation of the present application, the navigation display method of the AR-HUD may further include the following steps:

[0184] D1: Obtain the historical navigation route of the target device.

[0185] In the embodiment of the present application, a historical navigation route includes historical second posture data of the target device in the carrier coordinate system, that is, the historical navigation route is composed of historical second posture data.

[0186] In some embodiments, when the target device is traveling on a route, the second position data of the target device on the route may be recorded as a historical navigation route.

[0187] In some embodiments, N historical navigation routes can be pre-stored. For example, taking the target device as a target vehicle, and the SLAM system included in the target vehicle can measure the second posture data as an example, the SLAM system can store N historical navigation routes, and the number N can be set based on the disk size of the SLAM system, which is not limited in this application. For example, N can be 10, that is, 10 historical navigation routes are stored.

[0188] D2: If the received second posture data matches the historical second posture data, determine that the driving route of the target device is the historical navigation route.

[0189] During the movement of the target device, if the received second posture data matches the historical second posture data in the historical navigation route, it can be determined that the target device is traveling on the historical navigation route.

[0190] Exemplarily, assuming that the received multiple sets of second posture data overlap with the multiple sets of historical second posture data, it can be confirmed that the two are consistent.

[0191] D3: adding the height data in the historical second posture data as slope data to the navigation data.

[0192] It should be understood that the height data of the first pose data in the longitude and latitude coordinate system is usually 0, and cannot provide effective height information. The height data of the second pose data in the carrier coordinate system can characterize the height of the target device, that is, the height data can reflect the road type of the road on which the target device is traveling. For example, if the height data in the second pose data is 0, it means that the target device is traveling on a flat road, and if the height data is greater than 0, it means that the target device is traveling on a road with an uneven road surface and a large slope change.

[0193] In some embodiments, the navigation data includes intersection position data, and the slope data may be added to the intersection position data to optimize it.

[0194] In this way, the altitude data is also added to the navigation data as slope data, which can be used to prompt the target device about the flatness of the road on which it is about to travel, so that the navigation data can be further adapted to the actual scene.

[0195] As an example, see Figure 5 , which is a comparative schematic diagram of navigation data provided in an embodiment of the present application, taking the navigation data as a 3D navigation arrow as an example, Figure 5 Part (a) is the navigation data without slope data added. Figure 5 Part (b) is the navigation data with slope data added. Figure 5 (b) adds the curvature of the 3D navigation arrow to simulate the slope of the road, making the display effect more vivid, the directionality more accurate, and more suitable for the road conditions with slopes. In the fields of autonomous driving or assisted driving, it can better control and assist the driving of the target vehicle and improve the user experience.

[0196] Based on the introduction of the above embodiments, the navigation display method of AR-HUD provided in this application can integrate the first posture data and the second posture data with different data accuracy levels and different transmission frequencies to obtain navigation data displayed on the display interface of AR-HUD. Figure 6 , which is a schematic diagram of a specific AR-HUD navigation display method provided in an embodiment of the present application.

[0197] Combination Figure 6 As shown, the target device is the target vehicle, the navigation data is the 3D navigation arrow, the navigation system sends the first pose data, and the SLAM system sends the second pose data as an example. The first pose data sent by the navigation system is received, including the position data and heading angle data of the target vehicle, and the intersection position data sent by the navigation system is also received. At the same time, the second pose data sent by the SLAM system is received, including the position data and pose data (such as heading angle data, roll angle data and pitch angle data); then the longitude and latitude coordinate system and the carrier coordinate system are aligned, that is, the mapping relationship between the two is calculated; then based on the mapping relationship, the first pose data, the second pose data and the intersection position data are fused and converted into the display data of the AR-HUD (that is, the navigation data); then, based on the historical navigation route stored by the SLAM system (that is, the historical second pose data stored by the SLAM system), it is determined whether the target vehicle is traveling on the historical navigation route. If it is, the display data of the AR-HUD is optimized using the historical navigation route, that is, the slope data is added and displayed on the display interface of the AR-HUD; otherwise, the display data is directly displayed on the display interface of the AR-HUD. Then you can continue to wait for the navigation system to send the first pose data and intersection position data (ie, the next frame of data), the SLAM system to send the second pose data (ie, the next frame of data), and repeat the above process.

[0198] The above are some specific implementations of the AR-HUD navigation display method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding AR-HUD navigation display device. The following will introduce the AR-HUD navigation display device provided in the embodiment of the present application from the perspective of functional modularization.

[0199] See also Figure 7 , which is a schematic diagram of the structure of an AR-HUD navigation display device provided in an embodiment of the present application. The AR-HUD navigation display device 700 may include:

[0200] The data receiving module 710 is used to receive the first pose data of the target device in the longitude and latitude coordinate system and the second pose data of the target device in the carrier coordinate system; the data accuracy level of the first pose data is lower than the data accuracy level of the second pose data;

[0201] A posture data conversion module 720 is used to convert the second posture data to obtain third posture data in the longitude and latitude high coordinate system according to the mapping relationship between the longitude and latitude high coordinate system and the carrier coordinate system;

[0202] A data fusion module 730, configured to fuse the first posture data with the third posture data to obtain target posture data of the target device;

[0203] A navigation data conversion module 740 is used to convert the target posture data to obtain navigation data;

[0204] The data display module 750 is used to display the navigation data on the display interface of the augmented reality head-up display AR-HUD.

[0205] As an implementation manner, the sending frequency of the first posture data is lower than the sending frequency of the second posture data; the data receiving module 710 is used to receive a set of first posture data and a set of second posture data;

[0206] Accordingly, the posture data conversion module 720 may specifically include:

[0207] A mapping relationship calculation and data receiving unit, configured to calculate a mapping relationship between the latitude and longitude coordinate system and the carrier coordinate system based on the set of first pose data and the set of second pose data, and continuously receive a new set of second pose data of the target device;

[0208] The first posture data conversion unit is used to obtain the third posture data based on the new set of second posture data according to the mapping relationship.

[0209] As an implementation manner, the first posture data conversion unit is used to:

[0210] Calculate the median of the new set of second posture data and the previous set of second posture data to obtain median second posture data; the previous set of second posture data is the latest set of second posture data received before the new set of second posture data is received;

[0211] Multiplying the time interval between receiving the new set of second posture data and receiving the previous set of second posture data by the median second posture data to obtain incremental second posture data;

[0212] Adding the incremental second pose data and the new set of second pose data to obtain updated second pose data;

[0213] Based on the mapping relationship, the updated second pose data is converted into the third pose data.

[0214] As an implementation manner, the set of first posture data includes first position data; the set of second posture data includes second position data and second posture data; the updated second posture data includes updated second position data and updated second posture data; the mapping relationship calculation and data receiving unit is used to:

[0215] Calculating a displacement value between the latitude and longitude coordinate system and the carrier coordinate system based on the first position data and the second position data;

[0216] Based on the second posture data, a rotation matrix is ​​calculated; the rotation matrix is ​​used to indicate the rotation relationship between the longitude and latitude coordinate system and the carrier coordinate system;

[0217] The displacement value and the rotation relationship are combined into the mapping relationship.

[0218] The step of converting the updated second pose data into the third pose data based on the mapping relationship includes:

[0219] Multiplying the updated second position data by the rotation matrix to obtain a product value, and adding the product value to the displacement value to obtain third position data in the longitude and latitude high coordinate system;

[0220] Multiplying the updated second posture data by the rotation matrix to obtain third posture data in the longitude and latitude coordinate system;

[0221] The third position data and the third posture data are combined into the third posture data.

[0222] As an implementation manner, the sending frequency of the first posture data is lower than the sending frequency of the second posture data. The AR-HUD navigation display device 700 may further include:

[0223] A posture data updating module, used for updating the target posture data based on the latest set of second posture data received before receiving a new set of first posture data, to obtain updated target posture data;

[0224] A new navigation data conversion module, used for converting the updated target posture data to obtain new navigation data;

[0225] A navigation data updating module is used to update the navigation data on the display interface of the AR-HUD using the new navigation data.

[0226] As an implementation manner, the posture data updating module includes:

[0227] A first posture data conversion unit, configured to convert, according to the mapping relationship and based on the latest set of second posture data, obtain fourth posture data in the latitude and longitude high coordinate system;

[0228] The posture data fusion unit is used to fuse the fourth posture data with the target posture data to obtain the updated target posture data.

[0229] As an implementation manner, the AR-HUD navigation display device 700 may further include:

[0230] A route acquisition module, used for acquiring a historical navigation route of the target device; a historical navigation route includes historical second position data of the target device in the carrier coordinate system;

[0231] a route determination module, configured to determine that the driving route of the target device is the historical navigation route if the received second posture data matches the historical second posture data;

[0232] The data adding module is used to add the height data in the historical second posture data as slope data to the navigation data.

[0233] As an implementation manner, the first posture data includes different types of data, and the third posture data includes different types of data; the data fusion module 730 includes:

[0234] A weighted processing unit, used for performing weighted processing on the data of the same type in the first posture data and the third posture data to obtain weighted data;

[0235] A data combining unit is used to combine the weighted data, the data that has not been weighted in the first posture data, and the data that has not been weighted in the third posture data into the target posture data of the target device.

[0236] As an implementation mode, the weight value corresponding to the third posture data is positively correlated with the number of objects around the target device; the weight value is used for weighted processing.

[0237] As an implementation mode, the first posture data is measured by a navigation system on the target device; and the second posture data is measured by a simultaneous localization and mapping SLAM system on the target device.

[0238] As an implementation manner, the target device includes a target vehicle or a target robot.

[0239] The embodiments of the present application also provide a corresponding AR-HUD navigation display device and a computer-readable storage medium for implementing the solution provided in the embodiments of the present application.

[0240] Among them, the navigation display device of the AR-HUD includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program so that the device executes the navigation display method of the AR-HUD described in any embodiment of the present application.

[0241] The computer-readable storage medium stores a computer program. When the computer program is executed, a device executing the computer program implements the navigation display method of the AR-HUD described in any embodiment of the present application.

[0242] The "first" and "second" in the names such as "first" and "second" (if any) mentioned in the embodiments of the present application are only used as name identifiers and do not represent the first or second in order.

[0243] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a readable storage medium, such as a read-only memory (ROM) / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0244] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, in which the unit described as a separate component may or may not be physically separated, and the component prompted as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0245] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A navigation display method of AR-HUD, characterized in that: include: Receiving first pose data of the target device in the longitude and latitude coordinate system and second pose data of the target device in the carrier coordinate system; The data accuracy level of the first posture data is lower than the data accuracy level of the second posture data; According to the mapping relationship between the longitude and latitude high coordinate system and the carrier coordinate system, based on the second posture data, convert and obtain third posture data in the longitude and latitude high coordinate system; Fusing the first posture data with the third posture data to obtain target posture data of the target device; Obtaining navigation data based on the target posture data conversion; The navigation data is displayed on a display interface of an augmented reality head-up display (AR-HUD).

2. The method according to claim 1, characterized in that The sending frequency of the first posture data is lower than the sending frequency of the second posture data; the receiving the first posture data of the target device in the longitude and latitude coordinate system, and the second posture data of the target device in the carrier coordinate system, comprises: Receiving a set of first pose data and a set of second pose data; The converting, based on the mapping relationship between the latitude and longitude high coordinate system and the carrier coordinate system and based on the second pose data, to obtain third pose data in the latitude and longitude high coordinate system includes: Based on the set of first pose data and the set of second pose data, a mapping relationship between the latitude and longitude coordinate system and the carrier coordinate system is calculated, and a new set of second pose data of the target device is continuously received; According to the mapping relationship, the third posture data is converted based on the new set of second posture data.

3. The method according to claim 2, characterized in that The converting the new set of second posture data to obtain the third posture data according to the mapping relationship includes: Calculate the median of the new set of second posture data and the previous set of second posture data to obtain median second posture data; the previous set of second posture data is the latest set of second posture data received before the new set of second posture data is received; Multiplying the time interval between receiving the new set of second posture data and receiving the previous set of second posture data by the median second posture data to obtain incremental second posture data; Adding the incremental second pose data and the new set of second pose data to obtain updated second pose data; Based on the mapping relationship, the updated second pose data is converted into the third pose data.

4. The method according to claim 1, characterized in that: The sending frequency of the first posture data is lower than the sending frequency of the second posture data, further comprising: Before receiving a new set of first pose data, updating the target pose data based on the latest set of second pose data received to obtain updated target pose data; Obtaining new navigation data based on the updated target pose data conversion; The navigation data on the display interface of the AR-HUD is updated using the new navigation data.

5. The method according to claim 4, characterized in that The updating of the target pose data based on the received latest set of second pose data to obtain updated target pose data includes: According to the mapping relationship, based on the latest set of second posture data, fourth posture data in the longitude and latitude high coordinate system is converted; The fourth posture data is fused with the target posture data to obtain the updated target posture data.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: Obtaining a historical navigation route of the target device; A historical navigation route includes historical second position data of the target device in the carrier coordinate system; If the received second posture data matches the historical second posture data, determining that the driving route of the target device is the historical navigation route; The height data in the historical second posture data is added to the navigation data as slope data.

7. The method according to any one of claims 1 to 5, characterized in that: The first posture data includes different types of data, and the third posture data includes different types of data; fusing the first posture data with the third posture data to obtain the target posture data of the target device includes: Performing weighted processing on the data of the same type in the first posture data and the third posture data to obtain weighted data; The weighted data, the data in the first posture data that has not been weighted, and the data in the third posture data that has not been weighted are combined into target posture data of the target device.

8. The method according to claim 7, characterized in that The weight value corresponding to the third posture data is positively correlated with the number of surrounding objects of the target device; the weight value is used for weighted processing; And / or, the first posture data is measured by a navigation system on the target device; the second posture data is measured by a simultaneous localization and mapping (SLAM) system on the target device; And / or, the target device includes a target vehicle or a target robot.

9. An AR-HUD navigation display device, characterized in that: The device comprises: A data receiving module, used to receive first pose data of a target device in a longitude and latitude coordinate system, and second pose data of the target device in a carrier coordinate system; the data accuracy level of the first pose data is lower than the data accuracy level of the second pose data; A posture data conversion module, configured to convert the second posture data to obtain third posture data in the longitude and latitude high coordinate system according to a mapping relationship between the longitude and latitude high coordinate system and the carrier coordinate system; A data fusion module, used for fusing the first posture data with the third posture data to obtain target posture data of the target device; A navigation data conversion module, used to convert the target posture data to obtain navigation data; The data display module is used to display the navigation data on the display interface of the augmented reality head-up display (AR-HUD).

10. An AR-HUD navigation display device, characterized in that: The device comprises a memory and a processor: The memory is used to store a computer program and transmit the computer program to the processor; The processor is used to execute the computer program so that the device performs the steps of the AR-HUD navigation display method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed, a device executing the computer program implements the steps of the AR-HUD navigation display method according to any one of claims 1 to 8.