Method, device, electronic device and program product for pose correction of a navigation device

Abstract

This disclosure provides a method, apparatus, electronic device, and program product for attitude correction of a navigation device. The method includes: acquiring road images collected by the navigation device along a navigation path, and acquiring initial attitude information at the time of road image acquisition; detecting the pixel coordinates of road vanishing points in the road images; correcting the initial attitude information of the navigation device based on the projection relationship between the pixel coordinates of the road vanishing points and the road coordinates of the road vanishing points in a road coordinate system to obtain corrected attitude information; the road coordinates of the road vanishing points in the road coordinate system are predetermined based on the definition of the road coordinate system. This technical solution still outputs correct initial attitude information in areas with magnetic field interference.

Description

Technical Field

[0001] This disclosure relates to the field of geolocation service technology, specifically to a method, apparatus, electronic device, and program product for attitude correction of a navigation device. Background Technology

[0002] AR technology is now widely used in the navigation field. By calculating the position and attitude (including pitch, roll, and yaw) of the navigation device in real time, and with the help of computer graphics technology, virtual navigation signs are integrated into the real-world image on the navigation device's screen, thereby guiding the user to their destination.

[0003] Among these factors, the attitude of the navigation device, especially the calculation of the yaw angle, is particularly important, as it determines the direction indicated by navigation signs. However, existing technologies rely solely on measurements from a magnetometer mounted on the imaging device for yaw angle calculation. In areas with magnetic interference, such as underground parking lots, this can lead to inaccurate yaw angle calculations, resulting in deviations between the direction indicated by navigation signs and the actual direction, or even completely incorrect directions.

[0004] Therefore, a solution is needed to address the problem of inaccurate attitude information calculation by navigation devices due to magnetic field interference and other factors during navigation. Summary of the Invention

[0005] This disclosure provides a method, apparatus, electronic device, and program product for attitude correction of a navigation device.

[0006] In a first aspect, embodiments of this disclosure provide an attitude correction method for a navigation device, comprising:

[0007] Acquire road images captured by the navigation device along the navigation path, and acquire the initial attitude information at the time of acquisition of the road images;

[0008] Detect the pixel coordinates of the road vanishing point in the road image;

[0009] The initial attitude information of the navigation device is corrected based on the projection relationship between the pixel coordinates of the road vanishing point and the road coordinates of the road vanishing point in the road coordinate system to obtain corrected attitude information; the road coordinates of the road vanishing point in the road coordinate system are predetermined based on the definition of the road coordinate system.

[0010] Secondly, embodiments of the present invention provide an AR navigation method, comprising:

[0011] Acquire road images collected by the navigation device during navigation, the location information of the navigation device at the time of road image acquisition, and the initial attitude information of the navigation device at the time of road image acquisition;

[0012] The initial attitude information is corrected using the attitude correction method of the navigation device according to any one of claims 1-8 to obtain the corrected attitude information of the navigation device;

[0013] The virtual navigation markers corresponding to the location information are fused into the road image according to the corrected attitude information and then displayed on the navigation interface of the navigation device.

[0014] Thirdly, embodiments of the present invention provide an attitude correction device for a navigation device, comprising:

[0015] The first acquisition module is configured to acquire road images collected by the navigation device on the navigation path, and to acquire the initial attitude information at the time of acquisition of the road images;

[0016] The detection module is configured to detect the pixel coordinates of the road vanishing point in the road image;

[0017] The first correction module is configured to correct the initial attitude information of the navigation device based on the projection relationship between the pixel coordinates of the road vanishing point and the road coordinates of the road vanishing point in the road coordinate system, so as to obtain corrected attitude information; the road coordinates of the road vanishing point in the road coordinate system are predetermined based on the definition of the road coordinate system.

[0018] Fourthly, embodiments of the present invention provide an AR navigation device, comprising:

[0019] The fourth acquisition module is configured to acquire road images collected by the navigation device during navigation, the location information of the navigation device at the time of road image acquisition, and the initial attitude information of the navigation device at the time of road image acquisition.

[0020] The correction module is configured to correct the initial attitude information using the attitude correction device of the navigation device as described in claim 8, and obtain the corrected attitude information of the navigation device;

[0021] The fusion module is configured to fuse the virtual navigation markers corresponding to the location information into the road image according to the corrected attitude information, and then display them on the navigation interface of the navigation device.

[0022] The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above function.

[0023] In one possible design, the above-described device includes a memory and a processor. The memory stores one or more computer instructions that support the device in performing the corresponding methods described above, and the processor is configured to execute the computer instructions stored in the memory. The device may also include a communication interface for communicating with other devices or communication networks.

[0024] Fifthly, embodiments of this disclosure provide an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in any of the preceding aspects.

[0025] In a sixth aspect, embodiments of this disclosure provide a computer-readable storage medium for storing computer instructions used by any of the above-described devices, which, when executed by a processor, are used to implement the methods described in any of the above aspects.

[0026] In a seventh aspect, embodiments of this disclosure provide a computer program product comprising computer instructions which, when executed by a processor, are used to implement the methods described in any of the preceding aspects.

[0027] The technical solutions provided in this disclosure may have the following beneficial effects:

[0028] In this embodiment, to avoid interference from magnetic fields and other environmental factors on the inertial measurement sensor units (such as magnetometers) of the navigation device, which could lead to inaccurate attitude information calculation, the initial attitude information of the navigation device is calculated based on the sensor information measured by the inertial measurement sensor units. Then, road images acquired by the image sensor units on the navigation device are used to correct this initial attitude information. During the correction process, this embodiment detects the pixel coordinates of the road vanishing point from the road image and corrects the calculated initial attitude information based on the projection relationship between these pixel coordinates and the road coordinates of the road vanishing point in the road coordinate system. In addition to calculating the attitude information of the navigation device based on inertial measurement data, this embodiment also integrates image information acquired by the navigation device, providing additional observations for attitude calculation. This method can still output correct initial attitude information even in areas with magnetic field interference.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0030] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0031] Figure 1A flowchart illustrating an attitude correction method for a navigation device according to an embodiment of the present disclosure is shown;

[0032] Figure 2A This diagram illustrates the effect of a road coordinate system according to an embodiment of the present disclosure.

[0033] Figure 2B A schematic diagram showing the effect of the device coordinate system of a navigation device according to an embodiment of the present disclosure;

[0034] Figure 3 A flowchart illustrating an AR navigation method according to an embodiment of the present disclosure is shown;

[0035] Figure 4 A structural block diagram of an attitude correction device for a navigation device according to an embodiment of the present disclosure is shown.

[0036] Figure 5 A structural block diagram of an AR navigation device according to an embodiment of the present disclosure is shown;

[0037] Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing the attitude correction method and / or AR navigation method of a navigation device according to an embodiment of the present disclosure. Detailed Implementation

[0038] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of the exemplary embodiments have been omitted from the drawings.

[0039] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and do not preclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0040] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] The details of the embodiments of this disclosure are described in detail below through specific examples.

[0042] Figure 1 A flowchart illustrating an attitude correction method for a navigation device according to an embodiment of the present disclosure is shown. Figure 1 As shown, the attitude correction method of this navigation device includes the following steps:

[0043] In step S101, road images collected by the navigation device along the navigation path are acquired, as well as the initial attitude information at the time of acquisition of the road images are acquired.

[0044] In step S102, the pixel coordinates of the road vanishing point in the road image are detected;

[0045] In step S103, the initial attitude information of the navigation device is corrected based on the projection relationship between the pixel coordinates of the road vanishing point and the road coordinates of the road vanishing point in the road coordinate system; the road coordinates of the road vanishing point in the road coordinate system are predetermined based on the definition of the road coordinate system.

[0046] In this embodiment, the attitude correction method of the navigation device can be executed on the navigation device itself. During navigation, the navigation device can obtain the navigation path from the starting position to the ending position from the navigation server, and acquire the position information on the navigation path in real time. Based on the real-time acquired position information and navigation path information, it then provides navigation for the navigable object. During AR navigation, the navigation device can use its image sensing unit to capture road images ahead, and based on the location information of the navigation device at the time of road image acquisition, integrate corresponding virtual navigation signs from the navigation path into the road image to more intuitively guide the navigable object to travel correctly on the navigation path. For this purpose, it is necessary to determine the device attitude information of the navigation device so that, based on the device attitude information, the virtual navigation signs can be integrated into the road image according to the correct guidance direction and then displayed on the navigation screen of the navigation device.

[0047] Navigation devices typically include an inertial measurement unit (IMU) for measuring inertial measurement data. This IMU may include, but is not limited to, a magnetometer, an accelerometer, and a gyroscope, which are used to measure magnetometer data, accelerometer data, and gyroscope data, respectively. The navigation device can calculate its attitude information from the aforementioned magnetometer data, accelerometer data, and gyroscope data using appropriate attitude calculation methods. In some embodiments, the attitude information may include, but is not limited to, the roll angle, yaw angle, and pitch angle of the navigation device.

[0048] As described in the background section, in the presence of magnetic field interference, the yaw angle calculated from the inertial measurement data obtained by the inertial measurement sensor unit may be inaccurate, which may cause the direction indicated by the navigation mark displayed on the navigation screen to deviate from the actual direction.

[0049] Therefore, this disclosure proposes a scheme for correcting the device attitude information, especially the yaw angle, of a navigation device based on road images. In this scheme, the initial attitude information of the navigation device can be calculated based on inertial measurement sensor data obtained from the inertial measurement unit. Then, the vanishing point of the road is detected using road images acquired by the navigation device. The initial attitude information is corrected using the pixel coordinates of the vanishing point in the road image and the road coordinates in the road coordinate system, resulting in the corrected device attitude information.

[0050] In some embodiments, the inertial measurement sensing unit may be a nine-axis IMU unit, and the attitude calculation method may be the AGRS method. Using this AGRS method, the initial attitude information of the navigation device can be calculated based on the input information of the nine-axis IMU unit.

[0051] In some embodiments, the navigation device may use its onboard image sensing unit to acquire road images ahead and detect the pixel coordinates of the road vanishing point in the road image. Methods for detecting the pixel coordinates of the road vanishing point can be found in existing technologies and will not be elaborated upon here.

[0052] In this embodiment of the disclosure, since the acquired data is a road image, and the road coordinate system is determined, the road coordinates of the road vanishing point on the road image are also known in the road coordinate system. For example, as... Figure 2A As shown, in the road coordinate system, the x-axis points to the Earth's center, the y-axis is parallel to the road's widening direction, and the z-axis is parallel to the road's extension direction. The three-dimensional road coordinates of the road's vanishing point in this coordinate system are (x = 0, y = 0, z = 1). This is because the z-axis coordinate of the road's vanishing point is infinite, while the x-axis and y-axis coordinates are negligible relative to the z-axis. Therefore, the x-axis and y-axis coordinates can both be considered 0, and the z-axis coordinate can be considered 1. Of course, it's understandable that if the definition of the road coordinate system differs, the three-dimensional road coordinates of the vanishing point may differ. The specific definition can be determined based on actual needs; no specific restrictions are placed on the definition of the road coordinate system here. It should be noted that the definition of a road coordinate system can include, but is not limited to, the directions of the three coordinate axes (x-axis, y-axis, and z-axis) and the location of the origin. Typically, the directions of the x-axis, y-axis, and z-axis, as well as the location of the origin, change as the road extends.

[0053] When the pixel coordinates of the road vanishing point on the road image and the road coordinates in the road coordinate system are known, a projection relationship between the pixel coordinates of the same point in the image and the road coordinates in the road coordinate system can be established, and then the initial attitude information of the navigation device can be corrected based on this projection relationship.

[0054] It should be noted that the projection transformation of the same location point in different coordinate systems can be achieved based on the relative transformation relationship between the two coordinate systems. In the embodiments of this disclosure, the projection transformation between the pixel coordinates of a certain location point on the road image and the road coordinates of that location point in the road coordinate system can reflect the attitude of the navigation device relative to the road. Therefore, the embodiments of this disclosure utilize the projection relationship between the pixel coordinates of the road vanishing point in the road image and the road coordinates in the road coordinate system to correct the initial attitude information of the navigation device. By using the road image collected by the image sensing unit loaded on the navigation device itself as an observation of attitude information, the attitude information of the navigation device can be corrected after the measurement data of the inertial measurement unit is disturbed by the surrounding environment.

[0055] In this embodiment, to avoid interference from magnetic fields and other environmental factors on the inertial measurement sensor units (such as magnetometers) of the navigation device, which could lead to inaccurate attitude information calculation, the initial attitude information of the navigation device is calculated based on the sensor information measured by the inertial measurement sensor units. Then, road images acquired by the image sensor units on the navigation device are used to correct this initial attitude information. During the correction process, this embodiment detects the pixel coordinates of the road vanishing point from the road image and corrects the calculated initial attitude information based on the projection relationship between these pixel coordinates and the road coordinates of the road vanishing point in the road coordinate system. In addition to calculating the attitude information of the navigation device based on inertial measurement data, this embodiment also integrates image information acquired by the navigation device, providing additional observations for attitude calculation. This method can still output correct initial attitude information even in areas with magnetic field interference.

[0056] In an optional implementation of this embodiment, step S102, namely the step of obtaining the initial pose information at the time of road image acquisition, further includes the following steps:

[0057] The inertial measurement sensor unit installed on the navigation device is used to measure the accelerometer data, gyroscope data, and magnetometer data of the navigation device.

[0058] The original attitude information is obtained by using the attitude calculation algorithm based on the accelerometer data, gyroscope data, and magnetometer data.

[0059] The initial attitude information at the time of road image acquisition is obtained by interpolation based on the calculated original attitude information.

[0060] In this optional implementation, the navigation device is equipped with an inertial measurement unit, which includes an accelerometer, a gyroscope, and a magnetometer, used to measure accelerometer data, gyroscope data, and magnetometer data, respectively. The measured accelerometer data can be input to the navigation device's computing unit, where an attitude calculation method such as the AGRS algorithm is used to calculate the navigation device's original attitude information.

[0061] It should be noted that the accelerometer, gyroscope, and magnetometer data on which the raw attitude information is based may not be the inertial measurement data at the time of road image acquisition. Therefore, the raw attitude information may not reflect the attitude of the navigation device at the time of image acquisition.

[0062] To correct the device attitude information of the navigation device using image information from road images, it is necessary to use both road images and device attitude information from the same moment. Therefore, in this embodiment of the disclosure, the original attitude information calculated from inertial measurement data measured at a time near the image acquisition moment can be interpolated to obtain the initial attitude information at that image acquisition moment. This initial attitude information is then corrected using the image information from the road images.

[0063] In an optional implementation of this embodiment, step S103, which is the step of correcting the initial attitude information of the navigation device based on the projection relationship between the pixel coordinates of the road vanishing point and the road coordinates of the road vanishing point in the road coordinate system, further includes the following steps:

[0064] Transform the pixel coordinates of the road vanishing point to the device coordinate system of the navigation device to obtain the device coordinates of the road vanishing point;

[0065] Establish the projection relationship between the road coordinates and the equipment coordinates at the road vanishing point;

[0066] The corrected attitude information of the navigation device relative to the road coordinate system is determined based on the projection relationship and the initial attitude information.

[0067] In this optional implementation, since the device attitude information of the navigation device is relative to the ground plane, the pixel coordinates of the road vanishing point can be first transformed into the device coordinate system of the navigation device. Then, a projection relationship is established based on the device coordinates in the device coordinate system and the road coordinates in the road coordinate system. This projection relationship reflects the relative attitude between the device coordinate system and the road coordinate system. Since, regardless of how the road coordinate system is defined, one of its planes is parallel to the ground plane in the ground coordinate system, this relative attitude reflects the attitude of the navigation device relative to the ground plane. Therefore, given that the road coordinates and device coordinates of the road vanishing point are known, the initial attitude information can be corrected based on this projection relationship to obtain corrected attitude information. This corrected attitude information can be understood as the attitude information of the navigation device relative to the road coordinate system.

[0068] In some embodiments, the device coordinate system of the navigation device is known, such as... Figure 2B As shown, the device coordinate system of the navigation device is a three-dimensional coordinate system, with its z-axis perpendicular to the navigation device screen, and its x-axis and y-axis parallel to the two sides of the navigation device screen respectively. The origin can be, for example, the center point of the screen.

[0069] In an optional implementation of this embodiment, the step of converting the pixel coordinates of the road vanishing point to the device coordinate system of the navigation device to obtain the device coordinates of the road vanishing point further includes the following steps:

[0070] Obtain the camera intrinsic parameter matrix of the navigation device;

[0071] Based on the camera intrinsic parameter matrix, the pixel coordinates of the road vanishing point are converted into device coordinates in the device coordinate system.

[0072] In this optional implementation, the device coordinate system of the navigation device and the camera intrinsic matrix can be obtained from the navigation device. Based on the camera intrinsic matrix, the pixel coordinates at any location on the road image can be projected onto the device coordinate system of the navigation device to obtain the device coordinates. In some embodiments, the pixel coordinates of the road vanishing point can be multiplied by the transpose of the camera intrinsic matrix to obtain the device coordinates of the road vanishing point in the device coordinate system of the navigation device.

[0073] In an optional implementation of this embodiment, the step of converting the pixel coordinates of the road vanishing point to the device coordinate system of the navigation device to obtain the device coordinates of the road vanishing point further includes the following steps:

[0074] Expand the elements of the rotation matrix in the projection relation into an expression that includes heading angle variables, pitch angle variables, and / or roll angle variables;

[0075] The roll angle of the initial attitude information is substituted into the projection formula as the value of the roll angle variable, and the new heading angle and new pitch angle are calculated from the projection formula based on the pixel coordinates of the vanishing point of the road and the road coordinates.

[0076] The new heading angle, the new pitch angle, and the roll angle in the initial attitude information are used as the corrected attitude information of the navigation device relative to the road coordinate system.

[0077] In this optional implementation, the projection relationship between the road coordinates and the equipment coordinates at the road vanishing point is established based on a rotation matrix between the road coordinate system and the equipment coordinate system. This rotation matrix is ​​a 3x3 matrix, where each of the nine elements can be expanded into an expression that includes heading angle variables, pitch angle variables, and / or roll angle variables.

[0078] The following examples illustrate the projection relation and the expanded expressions for each element in the rotation matrix, where:

[0079]

[0080] in, Here, K represents the pixel coordinates, and K is the camera intrinsic parameter matrix. This is the rotation matrix between the road coordinate system and the equipment coordinate system. The coordinates of the road vanishing point.

[0081] The above projection relationship can also be converted into:

[0082]

[0083] in, The device coordinates of the road vanishing point in the device coordinate system.

[0084] The above projection relationship can also be converted to:

[0085]

[0086] Where r1-r9 are the nine elements of the rotation matrix, it can be seen from the above formula that since the road coordinates of the road vanishing point are (0,0,1), the right side of the equation can ultimately be transformed into a matrix containing only elements r3, r6, and r9. The left side of the equation represents the device coordinates of the road vanishing point in the device coordinate system, which are known quantities. Therefore, the above projection relationship can be transformed into three equations for r3, r6, and r9, as shown below:

[0087] r3=sin(roll)*sin(yaw)+cos(roll)*cos(yaw)*sin(pitch)

[0088] r6=cos(roll)*sin(yaw)-cos(yaw)*sin(pitch)*sin(roll)

[0089] r9 = cos(pitch) * cos(yaw)

[0090] Where roll is the roll angle variable, yaw is the yaw angle variable, and pitch is the pitch angle variable.

[0091] The values ​​on the left side of the three equations above (i.e., the values ​​of r3, r9, and r6) can be determined based on the product of the transpose of the camera intrinsic parameter matrix K and the pixel coordinates of the road vanishing point in the projection relation. The right side of the equations includes the yaw angle, pitch angle, and / or roll angle variables. Considering the multiplicative relationship between the three variables in the above three equations, the roll angle in the initial attitude information is substituted as a known quantity into the above three equations for calculation to obtain the new yaw angle and new pitch angle. It should be noted that since the environment has a relatively small impact on the roll angle in the initial attitude information calculated from the inertial measurement sensor data, the roll angle in this initial attitude information is basically accurate. Therefore, this roll angle in the initial attitude information can be used as the final information, and substituted into the above equations to calculate the yaw angle and pitch angle, which have a greater impact from the environment, as the corrected information.

[0092] Therefore, the corrected attitude information, calculated from the initial attitude information using image information and the inertial measurement sensor data, includes the new heading angle, the new pitch angle, and the roll angle from the initial attitude information.

[0093] In an optional implementation of this embodiment, the corrected attitude information includes yaw angle, pitch angle, and roll angle; the method further includes the following steps:

[0094] Obtain the road direction at the location of the navigation device at the time of image acquisition;

[0095] After processing the heading angle in the corrected attitude information based on the road direction, the resulting heading angle, pitch angle, and roll angle serve as the corrected attitude information of the navigation device relative to the geodetic coordinate system.

[0096] In this optional implementation, the corrected attitude information obtained after correcting the initial attitude information based on the projection relationship between the pixel coordinates of the road vanishing point in the road image and the road coordinates in the road coordinate system can be understood as the attitude of the navigation device relative to the road coordinate system. However, since the road coordinate system is constantly changing as the navigation device moves, it would increase the computational difficulty to fuse virtual navigation signs with the road image in AR navigation based on the corrected attitude information of the navigation device relative to the road coordinate system. Therefore, this embodiment can also obtain the road direction at the location of the navigation device at the time of image acquisition, and obtain the corrected attitude information of the navigation device relative to the geodetic coordinate system (i.e., the world coordinate system) based on the road direction and the corrected attitude information of the navigation device relative to the road coordinate system.

[0097] As described above, one plane of the road coordinate system is parallel to the geodetic coordinate system. Unlike the geodetic coordinate system, the road coordinate system has a certain rotation angle relative to the geodetic coordinate system along the z-axis, and the z-axis direction in the road coordinate system represents the road direction, which changes continuously as the navigation device moves on the road. This z-axis direction affects the heading angle of the navigation device relative to the geodetic coordinate system, while the pitch and roll angles relative to the geodetic coordinate system are the same as those relative to the road coordinate system. Therefore, in this embodiment, the heading angle can be corrected based on the road direction at the location of the navigation device at the time of road image acquisition, thus obtaining the corrected attitude information of the navigation device relative to the geodetic coordinate system. For example, when both the heading angle and the heading angle corresponding to the road direction are angles relative to a certain direction, such as north, they can be added together to obtain the heading angle in the corrected attitude information of the navigation device relative to the geodetic coordinate system.

[0098] In an optional implementation of this embodiment, the step of obtaining the road direction at the location of the navigation device at the time of image acquisition further includes the following steps:

[0099] Obtain the navigation path information of the navigation device and the location information of the navigation device at the time of image acquisition;

[0100] Obtain the road direction at the location information from the navigation path information.

[0101] In this optional implementation, the embodiment of this disclosure corrects the attitude information of the navigation device during navigation, and the road images collected by the navigation device are also images along the navigation path. Therefore, the location information of the navigation device at the moment of collecting the road images can be used to obtain the road direction from the navigation path information based on this location information. The navigation path information can be navigation path information planned by the navigation server based on the starting address and destination address provided by the navigation device, and the navigation path information can include information such as the road direction at different locations along the navigation path.

[0102] In an optional implementation of this embodiment, the method further includes the following steps:

[0103] Obtain the definition of the road coordinate system;

[0104] The road coordinates of the road vanishing point in the road coordinate system are determined based on the definition of the road coordinate system.

[0105] In this optional implementation, the definition of the road coordinate system can vary depending on the application environment. Therefore, when implementing the solution disclosed herein, the definition of the road coordinate system in the current application environment can be obtained in advance, for example... Figure 2A The road coordinate system definition is shown below. Once the road coordinate system definition is established, the road coordinates of the road vanishing point can be determined based on this definition. As mentioned above, the position coordinates of the road vanishing point in the road direction (such as the z-axis) are infinite, therefore the coordinates in this direction can be set to 1, while the coordinates in the other two directions (such as the x-axis and y-axis) can be set to 0.

[0106] Figure 3 A flowchart illustrating an AR navigation method according to an embodiment of this disclosure is shown. Figure 3 As shown, this AR navigation method includes the following steps:

[0107] In step S301, the following are obtained: road images collected by the navigation device during navigation, the position information of the navigation device at the time of road image collection, and the initial attitude information of the navigation device at the time of road image collection.

[0108] In step S302, the initial attitude information is corrected using the attitude correction method of the navigation device described above, to obtain the corrected attitude information of the navigation device;

[0109] In step S303, the virtual navigation marker corresponding to the location information is fused into the road image according to the corrected attitude information and then displayed on the navigation interface of the navigation device.

[0110] In this embodiment, the navigation device can be an AR navigation device, which is equipped with an image sensing unit and an inertial measurement sensing unit. During AR navigation, the navigation device receives navigation path information from the navigation server and obtains its location information in real time. AR navigation devices can include, but are not limited to, mobile phones, in-vehicle navigation terminals, tablets, etc.

[0111] Navigation devices can capture road images and obtain location information in real time during navigation.

[0112] After acquiring road images, the attitude correction method described above can be used to correct the attitude information of the navigation device, resulting in corrected attitude information. The navigation device then uses this corrected attitude information to fuse the virtual navigation markers corresponding to the location information at the time of road image acquisition into the road image, which is then displayed on the navigation device's interface.

[0113] In some embodiments, the virtual navigation marker corresponding to the location information can be obtained from the navigation path information. It is generated in advance by the navigation server and recorded in the navigation path information after being bound to the location information.

[0114] In other embodiments, virtual navigation signs include, but are not limited to, ground guide arrows, air guide signs, and destination signs.

[0115] For specific details regarding the correction of the initial attitude information of the navigation device in this embodiment, please refer to the description of the attitude correction method of the navigation device above, which will not be repeated here.

[0116] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein.

[0117] Figure 4 This diagram illustrates a structural block diagram of an attitude correction device for a navigation apparatus according to an embodiment of the present disclosure. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 4 As shown, the attitude correction device of the navigation equipment includes:

[0118] The first acquisition module 401 is configured to acquire road images collected by the navigation device on the navigation path, and to acquire the initial attitude information at the time of acquisition of the road images;

[0119] Detection module 402 is configured to detect the pixel coordinates of the road vanishing point in the road image;

[0120] The first correction module 403 is configured to correct the initial attitude information of the navigation device based on the projection relationship between the pixel coordinates of the road vanishing point and the road coordinates of the road vanishing point in the road coordinate system, so as to obtain corrected attitude information; the road coordinates of the road vanishing point in the road coordinate system are predetermined based on the definition of the road coordinate system.

[0121] In this embodiment, the attitude correction device of the navigation device can be executed on the navigation device itself. During navigation, the navigation device can obtain the navigation path from the starting position to the ending position from the navigation server, and acquire the position information on the navigation path in real time. Based on the real-time acquired position information and navigation path information, it then provides navigation for the navigable object. During AR navigation, the navigation device can use the image sensing unit mounted on the device to capture road images ahead. Based on the position information of the navigation device at the time of road image acquisition, it integrates corresponding virtual navigation signs from the navigation path into the road image to more intuitively guide the navigable object to travel correctly on the navigation path. For this purpose, it is necessary to determine the device attitude information of the navigation device so that, based on the device attitude information, the virtual navigation signs can be integrated into the road image according to the correct guidance direction and then displayed on the navigation screen of the navigation device.

[0122] Navigation devices typically include an inertial measurement unit (IMU) for measuring inertial measurement data. This IMU may include, but is not limited to, a magnetometer, an accelerometer, and a gyroscope, which are used to measure magnetometer data, accelerometer data, and gyroscope data, respectively. The navigation device can calculate its attitude information from the aforementioned magnetometer data, accelerometer data, and gyroscope data using a corresponding attitude calculation device. In some embodiments, the attitude information may include, but is not limited to, the roll angle, yaw angle, and pitch angle of the navigation device.

[0123] As described in the background section, in the presence of magnetic field interference, the yaw angle calculated from the inertial measurement data obtained by the inertial measurement sensor unit may be inaccurate, which may cause the direction indicated by the navigation mark displayed on the navigation screen to deviate from the actual direction.

[0124] Therefore, this disclosure proposes a scheme for correcting the device attitude information, especially the yaw angle, of a navigation device based on road images. In this scheme, the initial attitude information of the navigation device can be calculated based on inertial measurement sensor data obtained from the inertial measurement unit. Then, the vanishing point of the road is detected using road images acquired by the navigation device. The initial attitude information is corrected using the pixel coordinates of the vanishing point in the road image and the road coordinates in the road coordinate system, resulting in the corrected device attitude information.

[0125] In some embodiments, the inertial measurement sensing unit may be a nine-axis IMU unit, and the attitude calculation device may be an AGRS device. The AGRS device can be used to calculate the initial attitude information of the navigation device based on the input information from the nine-axis IMU unit.

[0126] In some embodiments, the navigation device may use an onboard image sensing unit to acquire a road image ahead and detect the pixel coordinates of the road vanishing point in the road image. The device for detecting the pixel coordinates of the road vanishing point can be found in existing technology and will not be described in detail here.

[0127] In this embodiment of the disclosure, since the acquired data is a road image, and the road coordinate system is determined, the road coordinates of the road vanishing point on the road image are also known in the road coordinate system. For example, as... Figure 2A As shown, in the road coordinate system, the x-axis points to the Earth's center, the y-axis is parallel to the road's widening direction, and the z-axis is parallel to the road's extension direction. The three-dimensional road coordinates of the road's vanishing point in this coordinate system are (x = 0, y = 0, z = 1). This is because the z-axis coordinate of the road's vanishing point is infinite, while the x-axis and y-axis coordinates are negligible relative to the z-axis. Therefore, the x-axis and y-axis coordinates can both be considered 0, and the z-axis coordinate can be considered 1. Of course, it's understandable that if the definition of the road coordinate system differs, the three-dimensional road coordinates of the vanishing point may differ. The specific definition can be determined based on actual needs; no specific restrictions are placed on the definition of the road coordinate system here. It should be noted that the definition of a road coordinate system can include, but is not limited to, the directions of the three coordinate axes (x-axis, y-axis, and z-axis) and the location of the origin. Typically, the directions of the x-axis, y-axis, and z-axis, as well as the location of the origin, change as the road extends.

[0128] When the pixel coordinates of the road vanishing point on the road image and the road coordinates in the road coordinate system are known, a projection relationship between the pixel coordinates of the same point in the image and the road coordinates in the road coordinate system can be established, and then the initial attitude information of the navigation device can be corrected based on this projection relationship.

[0129] It should be noted that the projection transformation of the same location point in different coordinate systems can be achieved based on the relative transformation relationship between the two coordinate systems. In the embodiments of this disclosure, the projection transformation between the pixel coordinates of a certain location point on the road image and the road coordinates of that location point in the road coordinate system can reflect the attitude of the navigation device relative to the road. Therefore, the embodiments of this disclosure utilize the projection relationship between the pixel coordinates of the road vanishing point in the road image and the road coordinates in the road coordinate system to correct the initial attitude information of the navigation device. By using the road image collected by the image sensing unit loaded on the navigation device itself as an observation of attitude information, the attitude information of the navigation device can be corrected after the measurement data of the inertial measurement unit is disturbed by the surrounding environment.

[0130] In this embodiment, to avoid interference from magnetic fields and other environmental factors on the inertial measurement sensor units (such as magnetometers) of the navigation device, which could lead to inaccurate attitude information calculation, the initial attitude information of the navigation device is calculated based on the sensor information measured by the inertial measurement sensor units. Then, road images acquired by the image sensor units on the navigation device are used to correct this initial attitude information. During the correction process, this embodiment detects the pixel coordinates of the road vanishing point from the road image and corrects the calculated initial attitude information based on the projection relationship between these pixel coordinates and the road coordinates of the road vanishing point in the road coordinate system. In addition to calculating the attitude information of the navigation device based on inertial measurement data, this embodiment also integrates image information acquired by the navigation device, providing additional observations for attitude calculation. This method can still output correct initial attitude information even in areas with magnetic field interference.

[0131] In an optional implementation of this embodiment, the first acquisition module includes:

[0132] The measurement submodule is configured to measure the accelerometer data, gyroscope data, and magnetometer data of the navigation device using the inertial measurement sensor unit provided on the navigation device;

[0133] The first calculation submodule is configured to use an attitude calculation algorithm to calculate the original attitude information based on the accelerometer data, gyroscope data and magnetometer data.

[0134] The interpolation submodule is configured to interpolate the original attitude information based on the solved information to obtain the initial attitude information at the time of road image acquisition.

[0135] In this optional implementation, the navigation device is equipped with an inertial measurement unit, which includes an accelerometer, a gyroscope, and a magnetometer, used to measure accelerometer data, gyroscope data, and magnetometer data, respectively. The measured accelerometer data can be input to the navigation device's computing unit, where an attitude calculation device, such as the AGRS algorithm, calculates the navigation device's raw attitude information.

[0136] It should be noted that the accelerometer, gyroscope, and magnetometer data on which the raw attitude information is based may not be the inertial measurement data at the time of road image acquisition. Therefore, the raw attitude information may not reflect the attitude of the navigation device at the time of image acquisition.

[0137] To correct the device attitude information of the navigation device using image information from road images, it is necessary to use both road images and device attitude information from the same moment. Therefore, in this embodiment of the disclosure, the original attitude information calculated from inertial measurement data measured at a time near the image acquisition moment can be interpolated to obtain the initial attitude information at that image acquisition moment. This initial attitude information is then corrected using the image information from the road images.

[0138] In an optional implementation of this embodiment, the first correction module includes:

[0139] The first transformation submodule is configured to transform the pixel coordinates of the road vanishing point to the device coordinate system of the navigation device to obtain the device coordinates of the road vanishing point;

[0140] A submodule is configured to establish a projection relationship between the road coordinates and the device coordinates at the road vanishing point.

[0141] The first determining submodule is configured to determine the corrected attitude information of the navigation device relative to the road coordinate system based on the projection relationship and the initial attitude information.

[0142] In this optional implementation, since the device attitude information of the navigation device is relative to the ground plane, the pixel coordinates of the road vanishing point can be first transformed into the device coordinate system of the navigation device. Then, a projection relationship is established based on the device coordinates in the device coordinate system and the road coordinates in the road coordinate system. This projection relationship reflects the relative attitude between the device coordinate system and the road coordinate system. Since, regardless of how the road coordinate system is defined, one of its planes is parallel to the ground plane in the ground coordinate system, this relative attitude reflects the attitude of the navigation device relative to the ground plane. Therefore, given that the road coordinates and device coordinates of the road vanishing point are known, the initial attitude information can be corrected based on this projection relationship to obtain corrected attitude information. This corrected attitude information can be understood as the attitude information of the navigation device relative to the road coordinate system.

[0143] In some embodiments, the device coordinate system of the navigation device is known, such as... Figure 2B As shown, the device coordinate system of the navigation device is a three-dimensional coordinate system, with its z-axis perpendicular to the navigation device screen, and its x-axis and y-axis parallel to the two sides of the navigation device screen respectively. The origin can be, for example, the center point of the screen.

[0144] The first conversion submodule includes:

[0145] The first acquisition submodule is configured to acquire the camera intrinsic parameter matrix of the navigation device;

[0146] The second transformation submodule is configured to transform the pixel coordinates of the road vanishing point into device coordinates in the device coordinate system based on the camera intrinsic parameter matrix.

[0147] In this optional implementation, the device coordinate system of the navigation device and the camera intrinsic matrix can be obtained from the navigation device. Based on the camera intrinsic matrix, the pixel coordinates at any location on the road image can be projected onto the device coordinate system of the navigation device to obtain the device coordinates. In some embodiments, the pixel coordinates of the road vanishing point can be multiplied by the transpose of the camera intrinsic matrix to obtain the device coordinates of the road vanishing point in the device coordinate system of the navigation device.

[0148] In an optional implementation of this embodiment, the projection relationship includes a rotation matrix between the road coordinate system and the equipment coordinate system; the first determining submodule includes:

[0149] The expansion submodule is configured to expand the elements of the rotation matrix in the projection relation into an expression that includes heading angle variables, pitch angle variables, and / or roll angle variables;

[0150] The second calculation submodule is configured to substitute the roll angle of the initial attitude information as the value of the roll angle variable into the projection relation, and calculate the new heading angle and new pitch angle from the projection relation based on the pixel coordinates of the vanishing point of the road and the road coordinates;

[0151] The second determining submodule is configured to use the new heading angle, the new pitch angle, and the roll angle in the initial attitude information as the corrected attitude information of the navigation device relative to the road coordinate system.

[0152] In this optional implementation, the projection relationship between the road coordinates and the equipment coordinates at the road vanishing point is established based on a rotation matrix between the road coordinate system and the equipment coordinate system. This rotation matrix is ​​a 3x3 matrix, where each of the nine elements can be expanded into an expression that includes heading angle variables, pitch angle variables, and / or roll angle variables.

[0153] Therefore, the corrected attitude information, calculated from the initial attitude information using image information and the inertial measurement sensor data, includes the new heading angle, the new pitch angle, and the roll angle from the initial attitude information.

[0154] In an optional implementation of this embodiment, the corrected attitude information includes yaw angle, pitch angle, and roll angle; the device further includes:

[0155] The second acquisition module is configured to acquire the road direction at the location of the navigation device at the time of image acquisition.

[0156] The processing module is configured to process the heading angle in the corrected attitude information based on the road direction, and the resulting heading angle, pitch angle, and roll angle are used as the corrected attitude information of the navigation device relative to the geodetic coordinate system.

[0157] In this optional implementation, the corrected attitude information obtained after correcting the initial attitude information based on the projection relationship between the pixel coordinates of the road vanishing point in the road image and the road coordinates in the road coordinate system can be understood as the attitude of the navigation device relative to the road coordinate system. However, since the road coordinate system is constantly changing as the navigation device moves, it would increase the computational difficulty to fuse virtual navigation signs with the road image in AR navigation based on the corrected attitude information of the navigation device relative to the road coordinate system. Therefore, this embodiment can also obtain the road direction at the location of the navigation device at the time of image acquisition, and obtain the corrected attitude information of the navigation device relative to the geodetic coordinate system (i.e., the world coordinate system) based on the road direction and the corrected attitude information of the navigation device relative to the road coordinate system.

[0158] As described above, one plane of the road coordinate system is parallel to the geodetic coordinate system. Unlike the geodetic coordinate system, the road coordinate system has a certain rotation angle relative to the geodetic coordinate system along the z-axis, and the z-axis direction in the road coordinate system represents the road direction, which changes continuously as the navigation device moves on the road. This z-axis direction affects the heading angle of the navigation device relative to the geodetic coordinate system, while the pitch and roll angles relative to the geodetic coordinate system are the same as those relative to the road coordinate system. Therefore, in this embodiment, the heading angle can be corrected based on the road direction at the location of the navigation device at the time of road image acquisition, thus obtaining the corrected attitude information of the navigation device relative to the geodetic coordinate system. For example, when both the heading angle and the heading angle corresponding to the road direction are angles relative to a certain direction, such as north, they can be added together to obtain the heading angle in the corrected attitude information of the navigation device relative to the geodetic coordinate system.

[0159] In an optional implementation of this embodiment, the second acquisition module includes:

[0160] The second acquisition submodule is configured to acquire the navigation path information of the navigation device and the location information of the navigation device at the time of image acquisition.

[0161] The third acquisition submodule is configured to acquire the road direction at the location information from the navigation path information.

[0162] In this optional implementation, the embodiment of this disclosure corrects the attitude information of the navigation device during navigation, and the road images collected by the navigation device are also images along the navigation path. Therefore, the location information of the navigation device at the moment of collecting the road images can be used to obtain the road direction from the navigation path information based on this location information. The navigation path information can be navigation path information planned by the navigation server based on the starting address and destination address provided by the navigation device, and the navigation path information can include information such as the road direction at different locations along the navigation path.

[0163] In an optional implementation of this embodiment, the apparatus further includes:

[0164] The third acquisition module is configured to acquire the definition of the road coordinate system;

[0165] The determination module is configured to determine the road coordinates of the road vanishing point in the road coordinate system based on the definition of the road coordinate system.

[0166] In this optional implementation, the definition of the road coordinate system can vary depending on the application environment. Therefore, when implementing the solution disclosed herein, the definition of the road coordinate system in the current application environment can be obtained in advance, for example... Figure 2AThe road coordinate system definition is shown below. Once the road coordinate system definition is established, the road coordinates of the road vanishing point can be determined based on this definition. As mentioned above, the position coordinates of the road vanishing point in the road direction (such as the z-axis) are infinite, therefore the coordinates in this direction can be set to 1, while the coordinates in the other two directions (such as the x-axis and y-axis) can be set to 0.

[0167] Figure 5 A structural block diagram of an AR navigation device according to an embodiment of the present disclosure is shown. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 5 As shown, the AR navigation device includes:

[0168] The fourth acquisition module 501 is configured to acquire road images collected by the navigation device during navigation, the position information of the navigation device at the time of road image acquisition, and the initial attitude information of the navigation device at the time of road image acquisition.

[0169] The correction module 502 is configured to correct the initial attitude information using the attitude correction device of the navigation device to obtain the corrected attitude information of the navigation device;

[0170] The fusion module 503 is configured to fuse the virtual navigation markers corresponding to the location information into the road image according to the corrected attitude information, and then display them on the navigation interface of the navigation device.

[0171] In this embodiment, the navigation device can be an AR navigation device, which is equipped with an image sensing unit and an inertial measurement sensing unit. During AR navigation, the navigation device receives navigation path information from the navigation server and obtains its location information in real time. AR navigation devices can include, but are not limited to, mobile phones, in-vehicle navigation terminals, tablets, etc.

[0172] Navigation devices can capture road images and obtain location information in real time during navigation.

[0173] After acquiring road images, the attitude correction device of the navigation device described above can be used to correct the attitude information of the navigation device, obtaining corrected attitude information. The navigation device also uses this corrected attitude information to fuse the virtual navigation markers corresponding to the location information at the time of road image acquisition into the road image, and then displays it on the navigation interface of the navigation device.

[0174] In some embodiments, the virtual navigation marker corresponding to the location information can be obtained from the navigation path information. It is generated in advance by the navigation server and recorded in the navigation path information after being bound to the location information.

[0175] In other embodiments, virtual navigation signs include, but are not limited to, ground guide arrows, air guide signs, and destination signs.

[0176] For specific details regarding the correction of the initial attitude information of the navigation device in this embodiment, please refer to the description of the attitude correction device of the navigation device above, which will not be repeated here.

[0177] Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing the attitude correction method and / or AR navigation method of a navigation device according to an embodiment of the present disclosure.

[0178] like Figure 6 As shown, the electronic device 600 includes a processing unit 601, which can be implemented as a CPU, GPU, FPGA, NPU, or other processing unit. The processing unit 601 can execute various processes according to any of the methods described above in this disclosure, based on a program stored in the read-only memory (ROM) 602 or a program loaded from the storage portion 608 into the random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0179] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0180] In particular, according to embodiments of this disclosure, any of the methods described above in the embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing any of the methods in the embodiments of this disclosure. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611.

[0181] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0182] The units or modules described in the embodiments of this disclosure can be implemented in software or hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0183] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the apparatus described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs that are used by one or more processors to perform the methods described in this disclosure.

[0184] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. An attitude correction method for a navigation device, wherein, include: Acquire road images captured by the navigation device along the navigation path, and acquire the initial attitude information at the time of acquisition of the road images; Detect the pixel coordinates of the road vanishing point in the road image; The initial attitude information of the navigation device is corrected based on the projection relationship between the pixel coordinates of the road vanishing point and the road coordinates of the road vanishing point in the road coordinate system to obtain corrected attitude information; the road coordinates of the road vanishing point in the road coordinate system are predetermined based on the definition of the road coordinate system.

2. The method according to claim 1, wherein, The process of obtaining the initial pose information at the time of road image acquisition includes: The inertial measurement sensor unit installed on the navigation device is used to measure the accelerometer data, gyroscope data, and magnetometer data of the navigation device. The original attitude information is obtained by using the attitude calculation algorithm based on the accelerometer data, gyroscope data, and magnetometer data. The initial attitude information at the time of road image acquisition is obtained by interpolation based on the calculated original attitude information.

3. The method according to claim 1 or 2, wherein, The initial attitude information of the navigation device is corrected based on the projection relationship between the pixel coordinates of the road vanishing point and the road coordinates of the road vanishing point in the road coordinate system to obtain corrected attitude information, including: Transform the pixel coordinates of the road vanishing point to the device coordinate system of the navigation device to obtain the device coordinates of the road vanishing point; Establish the projection relationship between the road coordinates and the equipment coordinates at the road vanishing point; The corrected attitude information of the navigation device relative to the road coordinate system is determined based on the projection relationship and the initial attitude information.

4. The method according to claim 3, wherein, Transforming the pixel coordinates of the road vanishing point to the device coordinate system of the navigation device to obtain the device coordinates of the road vanishing point includes: Obtain the camera intrinsic parameter matrix of the navigation device; Based on the camera intrinsic parameter matrix, the pixel coordinates of the road vanishing point are converted into device coordinates in the device coordinate system.

5. The method according to claim 3, wherein, The projection relationship includes the rotation matrix between the road coordinate system and the equipment coordinate system; Determining the corrected attitude information of the navigation device relative to the road coordinate system based on the projection relationship and the initial attitude information includes: Expand the elements of the rotation matrix in the projection relation into an expression that includes heading angle variables, pitch angle variables, and / or roll angle variables; The roll angle of the initial attitude information is substituted into the projection formula as the value of the roll angle variable, and the new heading angle and new pitch angle are calculated from the projection formula based on the pixel coordinates of the vanishing point of the road and the road coordinates. The new heading angle, the new pitch angle, and the roll angle in the initial attitude information are used as the corrected attitude information of the navigation device relative to the road coordinate system.

6. The method according to claim 1, wherein, The corrected attitude information includes heading angle, pitch angle, and roll angle; the method further includes: Obtain the road direction at the location of the navigation device at the time of image acquisition; After processing the heading angle in the corrected attitude information based on the road direction, the resulting heading angle, pitch angle, and roll angle serve as the corrected attitude information of the navigation device relative to the geodetic coordinate system.

7. The method according to claim 5, wherein, Obtaining the road direction at the location of the navigation device at the time of image acquisition includes: Obtain the navigation path information of the navigation device and the location information of the navigation device at the time of image acquisition; Obtain the road direction at the location information from the navigation path information.

8. The method according to any one of claims 1-2 and 4-5, wherein, The method further includes: Obtain the definition of the road coordinate system; The road coordinates of the road vanishing point in the road coordinate system are determined based on the definition of the road coordinate system.

9. An AR navigation method, wherein, include: Acquire road images collected by the navigation device during navigation, the location information of the navigation device at the time of road image acquisition, and the initial attitude information of the navigation device at the time of road image acquisition; The initial attitude information is corrected using the attitude correction method of the navigation device according to any one of claims 1-8 to obtain the corrected attitude information of the navigation device; The virtual navigation markers corresponding to the location information are fused into the road image according to the corrected attitude information and then displayed on the navigation interface of the navigation device.

10. An attitude correction device for a navigation equipment, wherein, include: The first acquisition module is configured to acquire road images collected by the navigation device on the navigation path, and to acquire the initial attitude information at the time of acquisition of the road images; The detection module is configured to detect the pixel coordinates of the road vanishing point in the road image; The first correction module is configured to correct the initial attitude information of the navigation device based on the projection relationship between the pixel coordinates of the road vanishing point and the road coordinates of the road vanishing point in the road coordinate system, so as to obtain corrected attitude information; the road coordinates of the road vanishing point in the road coordinate system are predetermined based on the definition of the road coordinate system.

11. An AR navigation device, wherein, include: The fourth acquisition module is configured to acquire road images collected by the navigation device during navigation, the location information of the navigation device at the time of road image acquisition, and the initial attitude information of the navigation device at the time of road image acquisition. The correction module is configured to correct the initial attitude information using the attitude correction device of the navigation device as described in claim 8, and obtain the corrected attitude information of the navigation device; The fusion module is configured to fuse the virtual navigation markers corresponding to the location information into the road image according to the corrected attitude information, and then display them on the navigation interface of the navigation device.

12. An electronic device, wherein, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method of any one of claims 1-9.

13. A computer program product comprising computer instructions, wherein, When executed by a processor, the computer instructions implement the method described in any one of claims 1-9.

Images