Correction method of vio map, electronic device and storage medium

Abstract

Embodiments of the present application provide a VIO map correction method, electronic equipment and storage medium, the method comprises: obtaining a to-be-corrected map of a visual-inertial odometry (VIO) in a camera coordinate system; determining an attitude angle of a measurement device relative to a geographic coordinate system, the measurement device comprising an acceleration sensor and a magnetic force sensor; obtaining magnetic field data of the measurement device; obtaining an azimuth angle of the measurement device relative to the geographic coordinate system by combining the magnetic field data with the attitude angle; and converting the to-be-corrected map from the camera coordinate system to the geographic coordinate system based on the attitude angle, the azimuth angle, and a camera external parameter corresponding to the camera coordinate system, to obtain a corrected map. The corrected map of the present embodiment is more intuitive than the map in the camera coordinate system, thereby improving the user experience.

Description

Technical Field

[0001] This application relates to the field of augmented reality technology, and in particular to a VIO map correction method, electronic device, and storage medium. Background Technology

[0002] Visual Inertial Odometry (VIO) is an indispensable component of the AR field. Mapping provides a reference for VIO positioning, reduces computational burden, and lays the groundwork for later visualization. On one hand, current VIO map correction technologies are mostly based on IMU sensors, including gyroscopes and accelerometers. The essence of this innovation is to achieve VIO map correction solely through accelerometer and magnetometer data, without relying on IMU sensors or gyroscopes. On the other hand, current VIO mapping lacks absolute azimuth observation, making it difficult for VIO algorithms to align the generated map with the geographic coordinate system. Aligning with the geographic coordinate system results in a map that better reflects human intuition. Due to the lack of absolute azimuth observation, while roll and pitch angles can be well corrected when aligning the map to the geographic coordinate system using accelerometers and gravity direction, the azimuth angle can be rotated to any position, causing significant confusion for clients during visualization. Summary of the Invention

[0003] This application provides a method for correcting a VIO map, including:

[0004] Obtain the map to be calibrated in the camera coordinate system using the Visual Inertial Odometry (VIO) system;

[0005] Determine the attitude angle of the measuring device relative to the geographic coordinate system; the measuring device includes an accelerometer and a magnetometer.

[0006] Acquire the magnetic field data of the measuring device;

[0007] The magnetic field data is combined with the attitude angle to obtain the azimuth angle of the measuring device relative to the geographic coordinate system;

[0008] Based on the attitude angle, azimuth angle, and camera extrinsic parameters corresponding to the camera coordinate system, the map to be calibrated is transformed from the camera coordinate system to the geographic coordinate system to obtain the calibrated map.

[0009] In some embodiments, determining the attitude angle of the measuring device relative to a geographic coordinate system includes:

[0010] Obtain the average acceleration value measured by the measuring device;

[0011] The attitude angle of the measuring device relative to the geographic coordinate system is determined based on the mean acceleration and the gravitational acceleration vector.

[0012] In some embodiments, the magnetic field data includes X-axis component data, Y-axis component data, and Z-axis component data;

[0013] The step of obtaining the azimuth angle of the measuring device relative to the geographic coordinate system by combining the magnetic field data with the attitude angle includes: combining the attitude angle with the X-axis component data, Y-axis component data and Z-axis component data to determine the azimuth angle of the measuring device relative to the geographic coordinate system.

[0014] In some embodiments, the step of transforming the map to be calibrated from the camera coordinate system to the geographic coordinate system based on the attitude angle, azimuth angle, and camera extrinsic parameters corresponding to the camera coordinate system to obtain a calibrated map includes:

[0015] The offset attitude angle of the measuring device relative to the geographic coordinate system is determined based on the attitude angle and azimuth angle;

[0016] An offset matrix is ​​constructed based on the offset attitude angle, and a transformation matrix is ​​determined based on the offset matrix and the camera extrinsic parameters corresponding to the camera coordinate system.

[0017] Based on the transformation matrix, the map to be corrected is transformed from the camera coordinate system to the geographic coordinate system to obtain the corrected map.

[0018] In some embodiments, determining the attitude angle of the measuring device relative to the geographic coordinate system based on the mean acceleration and the gravitational acceleration vector includes:

[0019] The X-direction acceleration, Y-direction acceleration, and Z-direction acceleration of the measuring device relative to the geographic coordinate system are determined based on the mean acceleration and the gravitational acceleration vector.

[0020] The roll angle of the measuring device relative to the geographic coordinate system is determined based on the Y-direction acceleration and the Z-direction acceleration, and the pitch angle of the measuring device relative to the geographic coordinate system is determined based on the Y-direction acceleration and the Z-direction acceleration.

[0021] The attitude angle is determined based on the roll angle and the pitch angle.

[0022] In some embodiments, determining the roll angle of the measuring device relative to the geographic coordinate system based on the Y-direction acceleration and the Z-direction acceleration, and determining the pitch angle of the measuring device relative to the geographic coordinate system based on the Y-direction acceleration and the Z-direction acceleration, includes:

[0023] The first vector product is obtained based on the acceleration in the Y direction and the acceleration in the Z direction. The arctangent of the first vector product is then used to obtain the roll angle.

[0024] The second vector product is obtained based on the X-direction acceleration and the Z-direction acceleration. The pitch angle is obtained by taking the arctangent of the second vector product.

[0025] In some embodiments, the determination of the azimuth angle of the measuring device relative to the geographic coordinate system by combining the attitude angle with the X-axis component data, Y-axis component data, and Z-axis component data is made by the following formula:

[0026]

[0027] in, These are the X-axis component data, Y-axis component data, and Z-axis component data, respectively. r is the roll angle, p is the pitch angle, and y is the azimuth angle.

[0028] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the VIO map correction method described above.

[0029] This application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the VIO map correction method as described above.

[0030] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the VIO map correction method as described above. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram illustrating the effect of existing technology;

[0033] Figure 2 This is one of the flowcharts illustrating a VIO map correction method provided in an embodiment of this application;

[0034] Figure 3 This is a schematic diagram illustrating the effect of a VIO map correction method provided in one embodiment of this application;

[0035] Figure 4 This is a second schematic flowchart of a VIO map correction method provided in one embodiment of this application;

[0036] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Reference Figure 1 Typically, converting an image from a camera coordinate system to a geographic coordinate system involves aligning the image with the geographic coordinate system using the image's roll and pitch angles. While aligning a map to the geographic coordinate system using accelerometers and gravity direction data can effectively correct the roll and pitch angles, the azimuth angle can be rotated to any position, causing significant confusion for users during visualization. To address this technical problem, this application provides a VIO map correction method.

[0039] The VIO map correction method provided in this application can be applied to terminal devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). It can also be applied to databases, servers, and service response systems based on terminal artificial intelligence. This application does not impose any restrictions on the specific type of terminal device.

[0040] The mobile terminal (terminal device) in this application embodiment includes various handheld devices, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem with wireless communication capabilities, such as mobile phones, tablets, desktop laptops, and smart devices capable of running applications, including the central control console of a smart car. Specifically, it can refer to user equipment (UE), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user device. Terminal devices can also be satellite phones, cellular phones, smartphones, wireless data cards, wireless modems, machine-type communication devices, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, in-vehicle devices or wearable devices, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical care, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, terminal devices in 5G networks, or terminal devices in future communication networks. Mobile terminals can be battery-powered or attached to and powered by the power system of a vehicle or vessel. The power system of a vehicle or ship can also charge the battery of a mobile terminal to extend the communication time of the mobile terminal.

[0041] Reference Figure 2 This application provides a VIO map correction method, including but not limited to the following steps:

[0042] Step 210: Obtain the map to be calibrated in the camera coordinate system using the visual inertial odometry (VIO).

[0043] Step 220: Determine the attitude angle of the measuring device relative to the geographic coordinate system. The measuring device includes an accelerometer and a magnetometer.

[0044] Step 230: Obtain the magnetic field data of the measuring device;

[0045] Step 240: Combine the magnetic field data with the attitude angle to obtain the azimuth angle of the measuring device relative to the geographic coordinate system;

[0046] Step 250: Based on the attitude angle, azimuth angle and the camera extrinsic parameters corresponding to the camera coordinate system, the map to be corrected is transformed from the camera coordinate system to the geographic coordinate system to obtain the corrected map.

[0047] The steps of this embodiment are described in detail below.

[0048] In step 210 above, a map to be calibrated in the camera coordinate system using visual inertial odometry (VIO) is obtained. Visual inertial odometry (VIO) is an indispensable component in the field of augmented reality (AR). VIO mainly estimates the body state (position and velocity) and offset at different times using measurements from the camera and the sensors of the measuring device. The offset is used to compensate for the measurement deviations of the sensors of the measuring device.

[0049] In this embodiment, the VIO map to be corrected is established in the camera coordinate system. The camera coordinate system in this embodiment is a three-dimensional rectangular coordinate system with the camera's focus center as the origin and the optical axis as the Z-axis.

[0050] In step 220 above, the attitude angle of the measuring device relative to the geographic coordinate system is determined. The measuring device includes an accelerometer and a magnetometer. The measuring device can be fixed to the camera, i.e., both are in the same coordinate system, i.e., the camera coordinate system. The measuring device includes a rigidly connected accelerometer (accelerometer sensor) and a magnetometer (magnetic sensor), which are used to measure acceleration values ​​and magnetic data, respectively.

[0051] A geographic coordinate system is a coordinate system that uses a three-dimensional sphere to define positions on the Earth's surface, enabling the reference of points on the Earth's surface via latitude and longitude. A geographic coordinate system consists of three parts: a unit of angle measurement, the prime meridian, and a reference ellipsoid. In a spherical system, horizontal lines are lines of equal latitude or parallels of parallel, and vertical lines are lines of equal longitude or parallels of longitude.

[0052] In this embodiment, determining the attitude angle of the measuring device in the geographic coordinate system is equivalent to determining the attitude angle of the accelerometer relative to the geographic coordinate system, which can be denoted as R. rp Attitude angles typically include roll and pitch.

[0053] It should be noted that the roll angle (roll) is the X-axis direction g measured by the accelerometer.x relative to the direction of gravity g z The angle (in the Z-axis direction), the pitch angle is the angle measured by the accelerometer in the Y-axis direction g. y relative to the direction of gravity g z The angle (in the Z-axis direction).

[0054] In steps 230 and 240 above, the magnetic field data of the measuring device is obtained, and then the magnetic field data is combined with the attitude angle to obtain the azimuth angle of the measuring device relative to the geographic coordinate system.

[0055] In this embodiment, the azimuth angle is measured relative to the geographic coordinate system using the magnetometer of the measuring device. This embodiment combines the data measured by the magnetometer with the roll and pitch angles obtained in step 220 above to obtain the azimuth angle.

[0056] Finally, through the above step 250, based on the attitude angle, azimuth angle and the camera extrinsic parameters corresponding to the camera coordinate system, the map to be corrected is transformed from the camera coordinate system to the geographic coordinate system to obtain the corrected map.

[0057] It should be noted that this embodiment utilizes the roll angle, pitch angle, and azimuth angle of the measuring device relative to the geographic coordinate system, combined with the camera's extrinsic parameters, to convert the map to be calibrated and obtain a calibrated map in the geographic coordinate system.

[0058] It should be noted that in order to achieve localization and mapping through the fusion of camera and measuring equipment, the extrinsic parameters of both the camera and the measuring equipment need to reach a certain level of accuracy. The camera and the measuring equipment are complementary: the camera is prone to failure under conditions of rapid movement or changes in lighting. The measuring equipment, on the other hand, can acquire high-frequency motion information from within the robot and is unaffected by the surrounding environment, thus compensating for the limitations of the camera; simultaneously, the camera can obtain rich environmental information and perform loop closure detection and correction through visual matching, thereby effectively correcting the cumulative drift error of the measuring equipment.

[0059] The external parameters between the camera and the measuring device include two parts: first, the relative pose between the camera and the measuring device, which refers to the transformation between the camera coordinate system and the measuring device coordinate system, including the relative rotation angle and the relative translation amount.

[0060] Specifically, the camera coordinate system coordinates and the measuring device coordinate system coordinates satisfy the following transformation relationship:

[0061] T iw =T ic ·T cw

[0062] Among them, T iw T is the coordinate of the measuring device in the world coordinate system. cwLet T be the coordinates of the camera in the world coordinate system. ic This is the transformation matrix between the coordinate system of the measuring equipment and the coordinate system of the camera, also known as the camera extrinsic parameters. Using the transformation matrix corresponding to the camera extrinsic parameters, the relative pose between the camera and the measuring equipment can be determined, thus completing the calibration of the camera and the measuring equipment.

[0063] Secondly, due to the existence of trigger delay and transmission delay, the sampling time and timestamp of the sensor do not match, resulting in a time difference between the camera and the measuring device.

[0064] In this embodiment, camera extrinsic parameters refer to the relative attitude between the camera and other sensors (i.e., measuring devices), which are typically rigid body connections. This means their relative attitude remains constant and does not change over time. For example, the relative attitude of the camera and measuring device on AR glasses is also their mechanically mounted relative position. Since they are rigidly connected, their relative position does not change over time. The extrinsic parameter calibration in this embodiment only involves the calibration of relative pose and does not involve the correction of time differences.

[0065] Reference Figure 3 This embodiment utilizes map corrections such as roll angle, pitch angle, and azimuth angle to prevent the azimuth angle from being rotated to other positions, ensuring it remains consistent with the camera coordinate system. This allows users to observe the map more intuitively when using AR glasses to obtain the corrected map.

[0066] The VIO map calibration method provided in this application calibrates the VIO map by acquiring data (attitude angle and azimuth angle) solely through accelerometer and magnetometer data without relying on IMU sensors or gyroscopes. This simplifies the measurement process while ensuring the accuracy of map calibration. By acquiring the attitude angle and azimuth angle of the measuring device relative to the geographic coordinate system, and using the attitude angle, azimuth angle, and camera extrinsic parameters, the VIO map to be calibrated can be transformed from the camera coordinate system to the geographic coordinate system to obtain the calibrated VIO map. Users can display the calibrated map on devices such as AR glasses. Since the calibrated map is a map in the geographic coordinate system, it is more intuitive than a map in the camera coordinate system, thus improving the user experience.

[0067] In some embodiments, determining the attitude angle of the measuring device relative to a geographic coordinate system includes:

[0068] Obtain the average acceleration value measured by the measuring device;

[0069] The attitude angle of the measuring device relative to the geographic coordinate system is determined based on the mean acceleration and the gravitational acceleration vector.

[0070] It is understood that this embodiment provides a method for determining the attitude angle.

[0071] First, collect a certain amount of accelerometer data, sum and average them, and denot the average as g. obervation Based on the acceleration vector in the constructed geographic coordinate system, it is denoted as g(0,0,9.8), which is the gravitational acceleration vector.

[0072] Then, based on the mean acceleration and the gravitational acceleration vector, the acceleration components of the measuring device in each direction under the geographic coordinate system are obtained.

[0073] Finally, the attitude angles are obtained using the acceleration components in each direction.

[0074] Specifically, this can be manifested in the following embodiments: determining the attitude angle of the measuring device relative to the geographic coordinate system based on the mean acceleration and the gravitational acceleration vector includes:

[0075] The X-direction acceleration, Y-direction acceleration, and Z-direction acceleration of the measuring device relative to the geographic coordinate system are determined based on the mean acceleration and the gravitational acceleration vector.

[0076] The roll angle of the measuring device relative to the geographic coordinate system is determined based on the Y-direction acceleration and the Z-direction acceleration, and the pitch angle of the measuring device relative to the geographic coordinate system is determined based on the Y-direction acceleration and the Z-direction acceleration.

[0077] The attitude angle is determined based on the roll angle and the pitch angle.

[0078] Further, determining the roll angle of the measuring device relative to the geographic coordinate system based on the Y-direction acceleration and the Z-direction acceleration, and determining the pitch angle of the measuring device relative to the geographic coordinate system based on the Y-direction acceleration and the Z-direction acceleration, includes:

[0079] The first vector product is obtained based on the acceleration in the Y direction and the acceleration in the Z direction. The arctangent of the first vector product is then used to obtain the roll angle.

[0080] The second vector product is obtained based on the X-direction acceleration and the Z-direction acceleration. The pitch angle is obtained by taking the arctangent of the second vector product.

[0081] It should be further explained that the X-axis acceleration, Y-axis acceleration, and Z-axis acceleration represent the acceleration components of the measuring device in each direction under the geographic coordinate system, denoted as g. x g y and g z Then for g y and g zThe roll angle is obtained by taking the arctangent of the vector product, as shown in the following formula:

[0082] roll = arctan(g) y ×g z )

[0083] Similarly, for g x and g z The pitch angle is obtained by taking the arctangent of the vector product, as shown in the following formula:

[0084] pitch = arctan(g) x ×g z )

[0085] The VIO map correction method provided in this application determines the acceleration vector through the accelerometer of the measuring device, thereby obtaining the attitude angle of the measuring device relative to the geographic coordinate system. This makes it easier to use the attitude angle to correct the VIO map and obtain a corrected map in a geographic coordinate system that is more convenient for observation.

[0086] In some embodiments, the magnetic field data includes X-axis component data, Y-axis component data, and Z-axis component data;

[0087] The step of obtaining the azimuth angle of the measuring device relative to the geographic coordinate system by combining the magnetic field data with the attitude angle includes: combining the attitude angle with the X-axis component data, Y-axis component data and Z-axis component data to determine the azimuth angle of the measuring device relative to the geographic coordinate system.

[0088] It is understood that this embodiment describes the method for determining the azimuth angle.

[0089] First, it is necessary to acquire the magnetic field data of the measuring equipment. The magnetic field data includes X-axis component data, Y-axis component data, and Z-axis component data, which can be denoted as follows:

[0090] Then, by combining the attitude angle with the X-axis component data, Y-axis component data, and Z-axis component data, the azimuth angle of the measuring device relative to the geographic coordinate system is determined.

[0091] Specifically, it is determined by the following formula:

[0092]

[0093] in, These are the X-axis component data, Y-axis component data, and Z-axis component data, respectively. r is the roll angle, p is the pitch angle, and y is the azimuth angle.

[0094] The VIO map correction method provided in this application determines magnetic data through the magnetometer of the measuring device, and then calculates the azimuth angle of the measuring device relative to the geographic coordinate system by combining the magnetic data with the roll angle and pitch angle, thereby facilitating coordinate system transformation of the image to be corrected using the azimuth angle.

[0095] In some embodiments, the step of transforming the map to be calibrated from the camera coordinate system to the geographic coordinate system based on the attitude angle, azimuth angle, and camera extrinsic parameters corresponding to the camera coordinate system to obtain a calibrated map includes:

[0096] The offset attitude angle of the measuring device relative to the geographic coordinate system is determined based on the attitude angle and azimuth angle;

[0097] An offset matrix is ​​constructed based on the offset attitude angle, and a transformation matrix is ​​determined based on the offset matrix and the camera extrinsic parameters corresponding to the camera coordinate system.

[0098] Based on the transformation matrix, the map to be corrected is transformed from the camera coordinate system to the geographic coordinate system to obtain the corrected map.

[0099] Specifically, this embodiment describes the specific process of coordinate system transformation.

[0100] First, the offset attitude angle of the measuring equipment relative to the geographic coordinate system is determined based on the attitude angle and azimuth angle, as shown in the following formula:

[0101] R offset =R rp *R y

[0102] Among them, R offset This is the offset attitude angle.

[0103] Then, the offset matrix is ​​constructed based on the offset attitude angle, as shown in the following formula:

[0104]

[0105] Among them, T offset This is the offset matrix;

[0106] The transformation matrix is ​​determined based on the offset matrix and the camera extrinsic parameters corresponding to the camera coordinate system, as shown in the following formula:

[0107] T = T offset *T ic

[0108] Where T is the transformation matrix, T ic This refers to the camera's external parameters.

[0109] Finally, the transformation matrix transforms the map to be calibrated from the camera coordinate system to the geographic coordinate system, resulting in the calibrated map.

[0110] The VIO map correction method provided in this application determines the offset attitude angle based on the attitude angle and azimuth angle, and then uses the offset attitude angle to determine the offset matrix and transformation matrix. The transformation matrix can be used to transform the VIO map to be corrected from the camera coordinate system to the geographic coordinate system to obtain the corrected VIO map. Users can display the corrected map on devices such as AR glasses. Since the corrected map is a map in the geographic coordinate system, it is more intuitive than the map in the camera coordinate system, thus improving the user experience.

[0111] It should be noted that in actual calculations, inaccurate or unacquisitionable magnetic data may cause azimuth calculations to fail. Since roll and pitch angles also vary based on magnetic data, new roll and pitch angles need to be recalculated until the azimuth calculation is successful.

[0112] The VIO map correction method provided in this application reduces the error value of the calculated azimuth angle by eliminating problems in the measurement data, thereby improving the correction effect of the VIO map and further enhancing the user experience.

[0113] Reference Figure 4 , Figure 4 A complete flowchart illustrating the VIO map correction method provided in this application embodiment includes the following steps:

[0114] Step 410: Input accelerometer data;

[0115] Step 420: Calculate the roll angle and pitch angle;

[0116] Step 430: Determine if the calculation was successful; if yes, proceed to step 440; otherwise, return to step 420.

[0117] Step 440: Calculate the azimuth using a magnetometer;

[0118] Step 450: Determine if the calculation was successful. If yes, proceed to step 460; otherwise, return to step 420.

[0119] Step 460: Construct the matrix of the measurement unit for the ground coordinate system;

[0120] Step 470: Construct the camera coordinate system attitude relative to the geographic coordinate system using camera extrinsic parameters;

[0121] Step 480: Rotate the VIO map to the geographic coordinate system.

[0122] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a VIO map correction method, which includes:

[0123] Obtain the map to be calibrated in the camera coordinate system using the Visual Inertial Odometry (VIO) system;

[0124] Determine the attitude angle of the measuring device relative to the geographic coordinate system; the measuring device includes an accelerometer and a magnetometer.

[0125] Acquire the magnetic field data of the measuring device;

[0126] The magnetic field data is combined with the attitude angle to obtain the azimuth angle of the measuring device relative to the geographic coordinate system;

[0127] Based on the attitude angle, azimuth angle, and camera extrinsic parameters corresponding to the camera coordinate system, the map to be calibrated is transformed from the camera coordinate system to the geographic coordinate system to obtain the calibrated map.

[0128] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the VIO map correction method provided by the above methods, the method including:

[0130] Obtain the map to be calibrated in the camera coordinate system using the Visual Inertial Odometry (VIO) system;

[0131] Determine the attitude angle of the measuring device relative to the geographic coordinate system; the measuring device includes an accelerometer and a magnetometer.

[0132] Acquire the magnetic field data of the measuring device;

[0133] The magnetic field data is combined with the attitude angle to obtain the azimuth angle of the measuring device relative to the geographic coordinate system;

[0134] Based on the attitude angle, azimuth angle, and camera extrinsic parameters corresponding to the camera coordinate system, the map to be calibrated is transformed from the camera coordinate system to the geographic coordinate system to obtain the calibrated map.

[0135] Furthermore, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the VIO map correction method provided by the methods described above, the method comprising:

[0136] Obtain the map to be calibrated in the camera coordinate system using the Visual Inertial Odometry (VIO) system;

[0137] Determine the attitude angle of the measuring device relative to the geographic coordinate system; the measuring device includes an accelerometer and a magnetometer.

[0138] Acquire the magnetic field data of the measuring device;

[0139] The magnetic field data is combined with the attitude angle to obtain the azimuth angle of the measuring device relative to the geographic coordinate system;

[0140] Based on the attitude angle, azimuth angle, and camera extrinsic parameters corresponding to the camera coordinate system, the map to be calibrated is transformed from the camera coordinate system to the geographic coordinate system to obtain the calibrated map.

[0141] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for calibrating a VIO map, characterized in that, include: Obtain the map to be calibrated in the camera coordinate system using the Visual Inertial Odometry (VIO) system; Determine the attitude angle of the measuring device relative to the geographic coordinate system; the measuring device includes an accelerometer and a magnetometer. Acquire the magnetic field data of the measuring device; The magnetic field data is combined with the attitude angle to obtain the azimuth angle of the measuring device relative to the geographic coordinate system; Based on the attitude angle, azimuth angle, and camera extrinsic parameters corresponding to the camera coordinate system, the map to be corrected is transformed from the camera coordinate system to the geographic coordinate system to obtain the corrected map. The magnetic field data includes X-axis component data, Y-axis component data, and Z-axis component data; The step of combining the magnetic field data with the attitude angle to obtain the azimuth angle of the measuring device relative to the geographic coordinate system includes: combining the attitude angle with the X-axis component data, Y-axis component data and Z-axis component data to determine the azimuth angle of the measuring device relative to the geographic coordinate system; The step of transforming the map to be corrected from the camera coordinate system to the geographic coordinate system based on the attitude angle, azimuth angle, and camera extrinsic parameters corresponding to the camera coordinate system to obtain the corrected map includes: The offset attitude angle of the measuring device relative to the geographic coordinate system is determined based on the attitude angle and azimuth angle; An offset matrix is ​​constructed based on the offset attitude angle, and a transformation matrix is ​​determined based on the offset matrix and the camera extrinsic parameters corresponding to the camera coordinate system. Based on the transformation matrix, the map to be corrected is transformed from the camera coordinate system to the geographic coordinate system to obtain the corrected map.

2. The VIO map correction method according to claim 1, characterized in that, Determining the attitude angle of the measuring device relative to the geographic coordinate system includes: Obtain the average acceleration value measured by the measuring device; The attitude angle of the measuring device relative to the geographic coordinate system is determined based on the mean acceleration and the gravitational acceleration vector.

3. The VIO map correction method according to claim 2, characterized in that, Determining the attitude angle of the measuring device relative to the geographic coordinate system based on the mean acceleration and the gravitational acceleration vector includes: The X-direction acceleration, Y-direction acceleration, and Z-direction acceleration of the measuring device relative to the geographic coordinate system are determined based on the mean acceleration and the gravitational acceleration vector. The roll angle of the measuring device relative to the geographic coordinate system is determined based on the Y-direction acceleration and the Z-direction acceleration, and the pitch angle of the measuring device relative to the geographic coordinate system is determined based on the Y-direction acceleration and the Z-direction acceleration. The attitude angle is determined based on the roll angle and the pitch angle.

4. The VIO map correction method according to claim 3, characterized in that, Determining the roll angle of the measuring device relative to the geographic coordinate system based on the Y-direction acceleration and the Z-direction acceleration, and determining the pitch angle of the measuring device relative to the geographic coordinate system based on the Y-direction acceleration and the Z-direction acceleration, includes: The first vector product is obtained based on the acceleration in the Y direction and the acceleration in the Z direction. The arctangent of the first vector product is then used to obtain the roll angle. The second vector product is obtained based on the X-direction acceleration and the Z-direction acceleration. The pitch angle is obtained by taking the arctangent of the second vector product.

5. The VIO map correction method according to claim 1, characterized in that, The azimuth angle of the measuring device relative to the geographic coordinate system is determined by combining the attitude angle with the X-axis component data, Y-axis component data, and Z-axis component data, using the following formula: ； in, These are the X-axis component data, Y-axis component data, and Z-axis component data, respectively. For roll angle, The pitch angle, For direction.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the VIO map correction method as described in any one of claims 1 to 5.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the VIO map correction method as described in any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the VIO map correction method as described in any one of claims 1 to 5.

Images