Mobile phone inertial navigation rapid alignment method based on camera pose information

Through data acquisition and calculation of mobile phone cameras and IMU, the initial attitude of the inertial navigation system is quickly obtained, which solves the problems of slow inertial navigation alignment speed and poor real-time performance in the existing technology, and realizes the rapid initialization and high precision of the inertial navigation system in mobile phones. alignment.

CN118999620APending Publication Date: 2024-11-22SOUTHEAST UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411085376.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing mobile phone inertial navigation positioning system has problems of slow speed and poor real-time performance during the initial alignment process, especially on low-cost MEMS inertial sensors, which cannot be effectively applied, especially when satellite signals are obscured or require rapid start-up.

Method used

Take pictures of the target through a mobile phone camera, use IMU and magnetometer to collect data, calculate the rotation matrix from the camera coordinate system to the navigation coordinate system and the conversion matrix from the navigation coordinate system to the IMU coordinate system, and obtain the Convert relationships to achieve fast initial alignment of mobile phone inertial navigation.

Benefits of technology

It realizes the completion of inertial guidance poses within a few seconds, improves alignment accuracy and real-time system, meets the needs of fast start, simplifies the operation process, and is suitable for most pedestrian positioning applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118999620A_ABST
    Figure CN118999620A_ABST
Patent Text Reader

Abstract

The invention discloses a camera pose information-based mobile phone inertial navigation rapid alignment method, which comprises the following steps of: firstly, photographing a target object through a mobile phone camera, and acquiring data for seconds by using a mobile phone IMU (Inertial Measurement Unit) and a magnetometer; the method comprises the following steps: performing pose calculation on a mobile phone camera based on a target object image to obtain a rotation matrix # imgabs0 # from a camera coordinate system to a navigation coordinate system; performing IMU pose calculation based on data acquired by a mobile phone IMU and a magnetometer; obtaining a conversion matrix # imgabs1 # from a navigation coordinate system to an IMU coordinate system; calculating based on # imgabs2 # and # imgabs3 # to obtain a shafting conversion matrix # imgabs4 # from a camera coordinate system to the IMU coordinate system; finally, when initial alignment is carried out on a mobile phone IMU, obtaining a rotation matrix # imgabs5 # from the camera coordinate system to the navigation coordinate system, and carrying out attitude transmission through a shafting conversion matrix Ccb; the method provided by the invention is simple and effective, can instantly complete attitude determination, and quickly realizes initial alignment of mobile phone inertial navigation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of mobile phone inertial navigation positioning, and mainly relates to a mobile phone inertial navigation rapid alignment method based on camera posture information. Background Art

[0002] At present, inertial navigation using smartphones equipped with low-cost MEMS inertial devices has been widely used. The inertial navigation system needs to be initially aligned before startup to obtain accurate initial navigation information, including position, velocity and attitude information. Position and velocity information can be provided by external measurement tools such as global navigation satellite systems, and the accuracy of the initial attitude is crucial to the accuracy of inertial navigation solutions.

[0003] Existing dual-vector attitude determination methods cannot be effectively applied to low-cost MEMS inertial sensors because these sensors are insensitive to the Earth's rotation speed. The main methods for determining the heading angle in the initial alignment of MEMS inertial sensors are: (1) using GNSS observations to calculate the heading angle, which can achieve high alignment accuracy within seconds, but cannot be used when the satellite signal is blocked; (2) using magnetometers to assist MEMS sensors in alignment. To ensure alignment accuracy, the alignment process needs to last for several minutes; (3) using high-precision inertial navigation to transfer alignment of low-cost MEMS inertial navigation. The alignment takes a short time and significantly improves the accuracy of inertial navigation solution, but requires a high-precision main inertial navigation to provide a reference.

[0004] In the inertial navigation positioning application of mobile phones, fast alignment speed, high real-time performance and easy operation are important considerations, but the existing alignment methods have the problems of long alignment time and slow startup. With the development of MEMS systems, there is an urgent need for a fast alignment method for mobile phone inertial navigation based on camera posture information. By positioning each sensor and using the conversion relationship between the sensor axis systems obtained by calibration, the initial posture of the mobile phone inertial navigation can be quickly determined, thereby realizing the fast initialization of the mobile phone inertial navigation system. Summary of the invention

[0005] The present invention aims at the problems existing in the existing mobile phone inertial navigation alignment method and provides a mobile phone inertial navigation fast alignment method based on camera posture information. First, the target object is photographed by the mobile phone camera, and the mobile phone IMU and magnetometer are used to collect data for several seconds; the mobile phone camera posture is solved based on the target image to obtain the rotation matrix from the camera coordinate system to the navigation coordinate system. Then, based on the data collected by the mobile phone IMU and magnetometer, the IMU attitude is solved to obtain the conversion matrix from the navigation coordinate system to the IMU coordinate system. based on and Calculate the axis transformation matrix from the camera coordinate system to the IMU coordinate system Finally, when the mobile phone IMU is initially aligned, the rotation matrix from the camera coordinate system to the navigation coordinate system is obtained Through the axis transformation matrix Perform attitude transfer to obtain the transformation matrix of the mobile phone I MU in the navigation coordinate system The method of the invention is simple and effective, can complete attitude determination instantly, and quickly realize the initial alignment of the mobile phone inertial navigation.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is: a mobile phone inertial navigation rapid alignment method based on camera posture information, comprising the following steps:

[0007] S1: When the mobile phone is stationary, take a photo of the target object through the mobile phone camera, and the target object has n feature points in the three-dimensional navigation coordinate system, where n≥3; collect data for several seconds using the mobile phone IMU and magnetometer;

[0008] S2: Based on the target image obtained in step S1, the position of the mobile phone camera is solved to obtain the rotation matrix from the camera coordinate system to the navigation coordinate system.

[0009] S3: Perform IMU attitude calculation based on the data collected by the mobile phone IMU and magnetometer to obtain the conversion matrix from the navigation coordinate system to the IMU coordinate system

[0010] S4: Based on the rotation matrix obtained in step S2 and the transformation matrix obtained in step S3 Calculate the axis transformation matrix from the camera coordinate system to the IMU coordinate system

[0011] S5: When initially aligning the mobile phone IMU, use the mobile phone camera to take a photo of the target object with m feature points in the three-dimensional navigation coordinate system, m ≥ 3; obtain the rotation matrix from the camera coordinate system to the navigation coordinate system The axis system transformation matrix obtained by step S4 Perform attitude transfer to obtain the transformation matrix of the mobile phone I MU in the navigation coordinate system

[0012] As an improvement of the present invention, the rotation matrix from the camera coordinate system to the navigation coordinate system in step S2 is The specific way to obtain is:

[0013] Establish the equation system for the i-th characteristic point in n:

[0014]

[0015] In the formula, (ui ,v i ) is the coordinate of the i-th feature point in the pixel coordinate system; f u ,f v ,u0,v0 are the camera internal parameters; t1,t2,t3 are the translation vectors; (X i ,Y i ,Z i ) is the coordinate of the i-th feature point in the navigation coordinate system;

[0016] Rotation Matrix

[0017] As another improvement of the present invention, the conversion matrix from the navigation coordinate system to the IMU coordinate system in step S3 is The calculation method is:

[0018] Solving the transformation matrix

[0019]

[0020] Where, κ, θ, φ are the navigation coordinate system around y n , x n , z n The angles at which the axes rotate in sequence are the pitch angle, roll angle, and heading angle.

[0021] As another improvement of the present invention, in step S3, the pitch angle κ and the roll angle θ are calculated as follows:

[0022]

[0023] In the formula, is the output value of the accelerometer around the three axes of the IMU coordinate system; g is a scalar;

[0024] The heading angle φ is calculated as:

[0025]

[0026] In the formula, are the geomagnetic triaxial components in the I MU coordinate system; are the three-axis components of the magnetometer when I MU is in the horizontal state; △φ is the magnetic declination.

[0027] As another improvement of the present invention, the axis conversion matrix from the camera coordinate system to the IMU coordinate system is The calculation method is:

[0028]

[0029] As a further improvement of the present invention, the transformation matrix of the mobile phone I MU in the navigation coordinate system in step S5 is The calculation method is:

[0030]

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) The method of the present invention only needs to calibrate the axis relationship between the mobile phone camera and the inertial sensor once. When the mobile phone inertial navigation needs to be initially aligned later, the camera posture can be quickly acquired through the visual positioning method, and the posture can be transferred using the axis conversion matrix. The inertial navigation posture can be instantly determined, which is fast and accurate, and the alignment accuracy is improved.

[0033] (2) Compared with the existing alignment methods, the method provided by the present invention can complete the alignment process within a few seconds, greatly speeding up the alignment speed and meeting the demand for quick startup of mobile phone inertial navigation positioning applications.

[0034] (3) The method of the present invention can not only quickly complete the alignment process, but also realize real-time attitude transmission, so that the mobile phone inertial navigation system can be quickly initialized in a real-time scenario, thereby improving the real-time performance of the system.

[0035] (4) Compared with the existing magnetometer-assisted alignment method that requires several minutes of static waiting, the method provided by the present invention is more convenient to operate, does not require additional waiting time, and is suitable for most pedestrian positioning applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flowchart of the steps of a method for rapid alignment of mobile phone inertial navigation based on camera pose information. DETAILED DESCRIPTION

[0037] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0038] Example 1

[0039] A mobile phone inertial navigation fast alignment method based on camera pose information, such as Figure 1 As shown, the following steps are included:

[0040] Step S1: When the mobile phone is stationary, the mobile phone camera takes a photo of the target object, which has n (n≥3) feature points in the three-dimensional navigation coordinate system. In this state, the mobile phone IMU and magnetometer are used to collect data for several seconds;

[0041] Step S2: Calculate the position of the mobile phone camera based on the target image to obtain the rotation matrix from the camera coordinate system to the navigation coordinate system

[0042] Establish the equation system for the i-th characteristic point in n:

[0043]

[0044] In the formula, (u i ,v i ) is the coordinate of the i-th feature point in the pixel coordinate system; f u ,f v ,u0,v0 are the camera internal parameters; is the rotation matrix from the camera coordinate system to the navigation coordinate system; t1, t2, t3 are translation vectors; (X i ,Y i ,Z i ) is the coordinate of the i-th feature point in the navigation coordinate system;

[0045] The rotation matrix Under the condition that the coordinates of n feature points in the navigation coordinate system and the corresponding pixel coordinates are known, the PnP algorithm can be used to obtain them.

[0046] Step S3: Perform IMU attitude calculation based on the data collected by the mobile phone IMU and magnetometer to obtain the conversion matrix from the navigation coordinate system to the IMU coordinate system Transformation Matrix Specifically:

[0047]

[0048] Where, κ, θ, φ are the navigation coordinate system around y n , x n , z n The angles at which the axes rotate in sequence are the pitch angle, roll angle, and heading angle.

[0049] κ, θ, φ are solved using the following formula:

[0050]

[0051] In the formula, is the output value of the accelerometer around the three axes of the I MU coordinate system; g is a scalar, the magnitude of which is the local gravitational acceleration;

[0052] The three-axis components of the geomagnetic field in the IMU coordinate system are output by the magnetometer Calculate the three-axis components of the magnetometer in the I MU horizontal state

[0053]

[0054] In the formula, φ is the heading angle, and △φ is the magnetic declination, which can be obtained based on the international geomagnetic reference field and the location of the magnetometer;

[0055] Step S4: Based on and Calculate the axis transformation matrix from the camera coordinate system to the IMU coordinate system

[0056]

[0057] Step S5: When the mobile phone IMU is initially aligned, the mobile phone camera is used to take a photo of a target object with m (m ≥ 3) feature points in the three-dimensional navigation coordinate system to obtain the rotation matrix from the camera coordinate system to the navigation coordinate system. Through the axis transformation matrix Perform attitude transfer to obtain the transformation matrix of the mobile phone I MU in the navigation coordinate system

[0058]

[0059] Example 2

[0060] The data in this embodiment takes the HUAWE I Mate 40Pro smartphone as an example to quickly determine the attitude of the mobile phone inertial navigation. The smartphone is equipped with a 2.7mm focal length camera with a pixel resolution of 1706*1280. The camera intrinsic parameter matrix is ​​obtained by calibration:

[0061]

[0062] In this camera positioning scene, the photography depth is 3m. Four control points are selected according to the design size of the QR code, and the coordinate information of the control points in the navigation coordinate system is measured and written into the QR code, as shown in Table 1.

[0063] Table 1 Coordinates of the corner points of the QR code in the navigation coordinate system / (m)

[0064] Top left Lower left Bottom right Top right -0.045,5.195,1.200 -0.045,5.195,1.200 0.045,4.805,0.800 0.045,4.805,0.800

[0065] The pixel coordinates of the corner points of the QR code in the image are obtained by using the feature point extraction method, and the average value of the feature point pixel coordinate extraction results of each image is taken, as shown in Table 2.

[0066] Table 2 Pixel coordinates of corner points of QR code / (pi xe l)

[0067] serial number Top left Lower left Bottom right Top right 1 925,363 933,563 1138,552 1125,348 2 925,362 933,562 1137,550 1125,347 …… …… …… …… …… Mean 925,362 933,562 1138,551 1126,346

[0068] The inertial sensor model is InvenSense i cm-20690, the magnetometer model is akm09918, and the sampling rate is set to 100Hz. The latitude of the experimental site measured by external measurement tools is 31°53′35″N, the longitude is 118°49′06″E, and the altitude is 50m.

[0069] Place the smartphone in the setup position and start the accelerometer and magnetometer. After the sensor readings are stable, collect data for 600 seconds and take the average of the sampled data. Table 3 shows the statistical values ​​of the static sampling results of the accelerometer and magnetometer.

[0070] Table 3 Average values ​​of 600s static sampling data of accelerometer and magnetometer

[0071] <![CDATA[Accelerometer / (m·s -2 )]]> Magnetometer / (μT) x-axis 9.818 -2.5 y-axis 0.316 -3.8 z-axis 0.030 -38.5

[0072] The mobile phone inertial navigation fast alignment method based on camera posture information specifically includes the following steps:

[0073] Step S1: Set up the smartphone at a fixed position, start the smartphone inertial sensor, and read 600s of data. At the same time, start the camera to take pictures of the target object with n (n≥3) feature points in the three-dimensional navigation coordinate system, with an interval of 6s, and take a total of 100 pictures.

[0074] Step S2: Use the target image to calculate the position of the mobile phone camera and obtain the rotation matrix from the camera coordinate system to the navigation coordinate system.

[0075] Use the PnP method to calculate the rotation matrix from the camera coordinate system to the navigation coordinate system

[0076]

[0077] Step S3: Perform IMU attitude calculation on the data collected by the mobile phone inertial sensor and magnetometer to obtain the conversion matrix from the navigation coordinate system to the IMU coordinate system

[0078]

[0079] φ=φ m +△φ=267.264°

[0080]

[0081] Step S4: Based on and Calculate the axis transformation matrix from the camera coordinate system to the IMU coordinate system

[0082]

[0083] Step S5: Use the mobile phone camera to take a photo of a target object with m (m ≥ 3) feature points in the three-dimensional navigation coordinate system, and obtain the rotation matrix from the camera coordinate system to the navigation coordinate system. Through the axis transformation matrix Perform attitude transfer to obtain the transformation matrix of the mobile phone I MU in the navigation coordinate system

[0084]

[0085] In summary, the present invention proposes a method for rapid alignment of a mobile phone inertial navigation system based on camera posture information. By performing posture determination on each sensor and utilizing the conversion relationship between the sensor axis systems obtained through calibration, the initial posture of the mobile phone inertial navigation system is rapidly determined, thereby achieving rapid initialization of the mobile phone inertial navigation system. The method of the present invention is simple and effective, can complete posture determination instantly, and can rapidly achieve initial alignment of the mobile phone inertial navigation system.

[0086] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A mobile phone inertial navigation rapid alignment method based on camera posture information, characterized in that: The steps include: S1: When the mobile phone is stationary, take a photo of the target object through the mobile phone camera, and the target object has n feature points in the three-dimensional navigation coordinate system, where n≥3; collect data for several seconds using the mobile phone IMU and magnetometer; S2: Based on the target image obtained in step S1, the position of the mobile phone camera is solved to obtain the rotation matrix from the camera coordinate system to the navigation coordinate system. S3: Perform IMU attitude calculation based on the data collected by the mobile phone IMU and magnetometer to obtain the conversion matrix from the navigation coordinate system to the IMU coordinate system S4: Based on the rotation matrix obtained in step S2 and the transformation matrix obtained in step S3 Calculate the axis transformation matrix from the camera coordinate system to the IMU coordinate system S5: When initially aligning the mobile phone IMU, use the mobile phone camera to take a photo of the target object containing m feature points in the three-dimensional navigation coordinate system, where m ≥ 3; Get the rotation matrix from the camera coordinate system to the navigation coordinate system The axis system transformation matrix obtained by step S4 Perform attitude transfer to obtain the transformation matrix of the mobile phone IMU in the navigation coordinate system 2. A mobile phone inertial navigation rapid alignment method based on camera posture information as claimed in claim 1, characterized in that: The rotation matrix from the camera coordinate system to the navigation coordinate system in step S2 The specific way to obtain is: Establish the equation system for the i-th characteristic point in n: In the formula, (u i ,v i ) is the coordinate of the i-th feature point in the pixel coordinate system; f u ,f v ,u0,v0 are the camera internal parameters; t1,t2,t3 are the translation vectors; (X i ,Y i ,Z i ) is the coordinate of the i-th feature point in the navigation coordinate system; Rotation Matrix 3. A mobile phone inertial navigation rapid alignment method based on camera posture information as claimed in claim 2, characterized in that: The conversion matrix from the navigation coordinate system to the IMU coordinate system in step S3 is The calculation method is: solve the transformation matrix Where, κ, θ, φ are the navigation coordinate system around y n , x n , z n The angles at which the axes rotate in sequence are the pitch angle, roll angle, and heading angle.

4. A mobile phone inertial navigation rapid alignment method based on camera posture information as claimed in claim 3, characterized in that: In step S3, the pitch angle κ and the roll angle θ are calculated as follows: In the formula, is the output value of the accelerometer around the three axes of the IMU coordinate system; g is a scalar; The heading angle φ is calculated as: In the formula, are the three-axis components of the geomagnetic field in the IMU coordinate system; are the three-axis components of the magnetometer when the IMU is in the horizontal state; △φ is the magnetic declination.

5. A mobile phone inertial navigation rapid alignment method based on camera posture information as claimed in claim 2 or 4, characterized in that: The axis transformation matrix from the camera coordinate system to the IMU coordinate system The calculation method is:

6. A mobile phone inertial navigation rapid alignment method based on camera posture information as claimed in claim 5, characterized in that: The conversion matrix of the mobile phone IMU in the navigation coordinate system in step S5 The calculation method is: