Method for determining relative attitude of inertial navigation and camera

By collecting picture information and inertial navigation acceleration of inertial navigation cameras in different axes angle directions, combining reference error formulas and optimization algorithms, the relative postures of inertial navigation and the camera are determined, and the problems of insufficient accuracy and complex operation in the existing technology are solved. It achieves higher measurement accuracy and stability and is suitable for a variety of application scenarios.

CN119085641APending Publication Date: 2024-12-06TIANJIN JINHANG INST OF TECH PHYSICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411180781.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art has problems of insufficient accuracy, complex operation and great environmental impact when determining the relative posture of inertial navigation and cameras, and cannot meet the increasingly improved accuracy and reliability requirements, especially in complex motion scenarios and variable environmental conditions. precise.

Method used

Acquire the image information set of cameras equipped with inertial guides in different axial angle directions, including the captured images with specific marks at different positions in the image and the corresponding inertial guide acceleration. By obtaining the reference error formula and the camera's internal parameter matrix, use optimization The algorithm solves the rotation matrix with the smallest error to determine the relative pose of the inertial guide and the camera.

Benefits of technology

It significantly improves measurement accuracy, reduces the impact of a single factor on measurement results, makes measurement results more stable and reliable in complex scenarios and environmental conditions, simplifies operating procedures, improves work efficiency, and is suitable for the fields of autonomous driving and aerospace. Accuracy measurement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119085641A_ABST
    Figure CN119085641A_ABST
Patent Text Reader

Abstract

The invention provides a method for determining the relative attitude of inertial navigation and a camera, which comprises the following steps: acquiring a picture information set corresponding to the camera equipped with inertial navigation in different axial angle directions, the picture information set at least comprising a plurality of shot images shot by the camera and having specific marks at different positions in the image, the inertial navigation acceleration of the inertial navigation corresponding to the shooting moment of each shot image is obtained; obtaining a reference formula, obtaining an internal parameter matrix of the camera, and obtaining the relative attitude of the inertial navigation system and the camera according to the position of the specific mark of each shot image in the image, the inertial navigation acceleration of the inertial navigation system and the internal parameter matrix of the camera. The relative attitude of the inertial navigation and the camera can be determined more comprehensively and accurately, and the measurement precision is remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of determining relative posture of an inertial navigation system and a camera, and in particular to a method for determining the relative posture of an inertial navigation system and a camera. Background Art

[0002] In many fields involving precise measurement and positioning, such as autonomous driving, aerospace, robotics, etc., it is of vital importance to accurately determine the relative posture of the inertial navigation system and the camera. Traditional measurement methods often have problems such as insufficient accuracy, complex operation, and great environmental influence.

[0003] In the past, the integrated application of inertial navigation and cameras usually relied on rough estimation or a single measurement method to determine the relative posture between them, which could not meet the increasing accuracy and reliability requirements. Due to the different working principles and characteristics of cameras and inertial navigation, how to effectively fuse the data of the two to accurately obtain the relative posture has always been a technical challenge. In addition, the existing technical solutions are often not stable and accurate enough when dealing with complex motion scenes and changing environmental conditions, which limits their widespread promotion in practical applications. Summary of the invention

[0004] In view of the above defects or deficiencies in the prior art, the present application aims to provide a method for determining the relative attitude of an inertial navigation system and a camera, comprising the following steps:

[0005] Collecting a picture information set corresponding to different axial angle directions of a camera equipped with an inertial navigation system, wherein the picture information set includes at least a number of images taken by the camera with a specific mark at different positions in the image, and the inertial navigation acceleration of the inertial navigation system corresponding to each of the images at the time of taking the images;

[0006] A reference formula is obtained, and an intrinsic parameter matrix of a camera is obtained, and a relative posture between the inertial navigation system and the camera is obtained according to a position of the specific mark in each captured image, the inertial navigation acceleration of the inertial navigation system, and the intrinsic parameter matrix of the camera.

[0007] According to the technical solution provided in the embodiment of the present application, the acquisition of the picture information set corresponding to the camera with inertial navigation in different axial angle directions at least includes the following steps:

[0008] Installing a camera equipped with an inertial navigation system on a turntable directly facing the collimator, the camera equipped with an inertial navigation system directly facing the collimator, and installing a cross mark between the turntable and the collimator;

[0009] When the camera equipped with an inertial navigation system is in the initial axis angle direction, the azimuth angle and the pitch angle of the turntable are rotated so that the specific mark displayed by the cross marker is respectively located at the center of the image, the upper position of the image, the lower position of the image, the left position of the image, and the right position of the image, and 5 captured images corresponding to the initial axis angle direction are collected;

[0010] The camera equipped with an inertial navigation system is rotated around the optical axis by N degrees, 2N degrees, and 3N degrees respectively to obtain a plurality of rotation rear axis angular directions, and 5 photographed images corresponding to each rotation rear axis angular direction are collected;

[0011] The 20 captured images are aggregated, and the pixel coordinates of the specific mark in each captured image and the inertial navigation acceleration of the inertial navigation are extracted to obtain a picture information set corresponding to different axis angle directions of the camera with inertial navigation.

[0012] According to the technical solution provided in the embodiment of the present application, the pixel coordinates include the pixel x coordinates in the x-axis direction and the pixel y coordinates in the y-axis direction; the inertial navigation acceleration includes the acceleration component in the x-axis direction, the acceleration component in the y-axis direction and the acceleration component in the z-axis direction;

[0013] The step of obtaining the relative posture between the inertial navigation system and the camera according to the reference error formula, the position of the specific mark of each captured image in the image, the inertial navigation acceleration of the inertial navigation system and the intrinsic parameter matrix of the camera comprises at least the following steps:

[0014] Substituting the pixel coordinates of each of the captured images, the inertial navigation acceleration of the inertial navigation system, and the intrinsic parameter matrix of the camera into a reference error formula to obtain an image reference error formula corresponding to each of the captured images, wherein all of the image reference error formulas constitute an equation group;

[0015] Based on the equation group, an optimization algorithm is used to solve the rotation matrix with the minimum error, and the rotation matrix with the minimum error is used as the relative posture of the inertial navigation and the camera, wherein the solved rotation matrix must satisfy unit orthogonality.

[0016] According to the technical solution provided in the embodiment of the present application, the reference error formula is:

[0017]

[0018] Among them, v n ,u n is the position of the specific mark in the nth captured image in the image, X n ,Y n ,Z n is the inertial navigation acceleration during the n-th shooting, K is the intrinsic parameter matrix of the camera, and R is the relative attitude.

[0019] According to the technical solution provided in the embodiment of the present application, the rotation matrix with the smallest error is

[0020]

[0021] Among them, argmin represents the value of R that minimizes the total error function.

[0022] According to the technical solution provided in the embodiment of the present application, the method further includes obtaining an intrinsic parameter matrix of the camera, wherein obtaining the intrinsic parameter matrix of the camera includes at least the following steps:

[0023] Place a standard checkerboard calibration plate on a flat surface;

[0024] Use the camera to take multiple experimental photos from different angles and distances, and input all the experimental photos into the camera calibration software to obtain the camera's intrinsic parameter matrix.

[0025] According to the technical solution provided in the embodiment of the present application, the use of an optimization algorithm to solve the rotation matrix with the minimum error includes at least the following steps:

[0026] The sum of squares of the reference error formulas corresponding to all the captured images is used as a total error function;

[0027] An optimization algorithm is used to solve a rotation matrix that minimizes the total error function, and the rotation matrix is ​​used as the relative posture of the inertial navigation system and the camera.

[0028] According to the technical solution provided in the embodiment of the present application, the total error function is

[0029] Wherein, N is the total number of the captured images, which is 20.

[0030] In summary, the present application proposes a method for determining the relative posture of an inertial navigation system and a camera, comprising the following steps: collecting a picture information set corresponding to a camera equipped with an inertial navigation system in different axial angle directions, the picture information set comprising at least a number of captured images taken by the camera with specific marks at different positions in the image, and the inertial navigation acceleration of the inertial navigation system corresponding to each captured image at the time of shooting; obtaining a reference error formula, and obtaining an intrinsic parameter matrix of the camera, and obtaining the relative posture of the inertial navigation system and the camera according to the reference error formula, the position of the specific mark of each captured image in the image, the inertial navigation acceleration of the inertial navigation system and the intrinsic parameter matrix of the camera.

[0031] Compared with the prior art, the beneficial effect of the present application is that the present application can more comprehensively and accurately determine the relative posture of the inertial navigation and the camera by collecting picture information in multiple different axial angle directions and the corresponding inertial navigation acceleration, and significantly improve the measurement accuracy. At the same time, a variety of different shooting angles and positions are taken into account, reducing the impact of a single factor on the measurement results, making the measurement results more stable and reliable in complex motion scenes and environmental conditions. Moreover, this solution does not require complex equipment and cumbersome operating procedures. It only needs to collect picture information and inertial navigation data according to the prescribed steps for calculation, which improves work efficiency and is suitable for a variety of different application scenarios. Whether it is real-time posture determination in autonomous driving or high-precision measurement in the field of aerospace, it can play a good role, providing strong technical support for the further development of related fields, and helping to promote the progress of technologies such as intelligent driving and drone navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of the steps of a method for determining the relative posture of an inertial navigation system and a camera provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of the positions of a camera equipped with an inertial navigation system, a cross marker, and a collimator provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of the structure of the captured image provided by the embodiment of the present application;

[0035] Figure 4 A schematic diagram of the turntable rotating to change the angular direction of the shaft provided in an embodiment of the present application.

[0036] The text annotations in the figure represent:

[0037] 1. Camera; 2. Turntable; 3. Specific mark; 4. Cross mark; 5. Collimator; 6. Capture image. DETAILED DESCRIPTION

[0038] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0039] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] Example 1

[0041] As mentioned in the background technology, in order to solve the problems in the prior art, this application proposes a method for determining the relative posture of an inertial navigation system and a camera, please refer to Figure 1 As shown, the following steps are included:

[0042] S1. Collecting a picture information set corresponding to different axial angle directions of a camera equipped with an inertial navigation system, wherein the picture information set includes at least a number of images taken by the camera with specific marks at different positions in the image, and the inertial navigation acceleration of the inertial navigation system corresponding to each of the images at the time of shooting;

[0043] S2, obtaining a reference error formula, and obtaining an intrinsic parameter matrix of the camera 1, and obtaining a relative posture between the inertial navigation system and the camera according to the reference error formula, the position of the specific mark 3 of each captured image 6 in the image, the inertial navigation acceleration of the inertial navigation system, and the intrinsic parameter matrix of the camera 1;

[0044] Furthermore, the reference error formula is:

[0045] Among them, v n ,u n is the position of the specific mark in the nth captured image 6 in the image, X n ,Y n ,Z n is the inertial navigation acceleration during the n-th shooting, K is the intrinsic parameter matrix of the camera, and R is the relative attitude.

[0046] In a preferred embodiment, please refer to Figure 2 As shown, the acquisition of the picture information set corresponding to the camera with inertial navigation in different axial angle directions at least includes the following steps:

[0047] Installing a camera equipped with an inertial navigation system on a turntable directly facing the collimator, the camera equipped with an inertial navigation system directly facing the collimator, and installing a cross mark between the turntable and the collimator;

[0048] Please refer to Figure 3 As shown, when the camera equipped with an inertial navigation system is in the initial axis angle direction, the azimuth angle and the pitch angle of the turntable are rotated so that the specific mark displayed by the cross marker is located at the center of the image, the upper position of the image, the lower position of the image, the left position of the image, and the right position of the image, respectively, and the five captured images corresponding to the initial axis angle direction are acquired;

[0049] Specifically, the captured image is a rectangle, the center of the image is the intersection of the two diagonals of the rectangle, the upper position of the image is the upper position of the image and the distance between it and the center of the image is one-quarter of the width of the rectangle, the upper position of the image is the lower position of the image and the distance between it and the center of the image is one-quarter of the width of the rectangle, the left position of the image is the left position of the image and the distance between it and the center of the image is one-quarter of the length of the rectangle, and the right position of the image is the right position of the image and the distance between it and the center of the image is one-quarter of the length of the rectangle; other cross mark positions that can evenly cover the image can also be used. The position does not need to be precise and does not affect the calibration accuracy.

[0050] Specifically, the initial axis angle direction is 0 degrees.

[0051] The camera equipped with an inertial navigation system is rotated around the optical axis by N degrees, 2N degrees, and 3N degrees respectively to obtain a plurality of rotation rear axis angular directions, and 5 photographed images corresponding to each rotation rear axis angular direction are collected;

[0052] Specifically, please refer to Figure 4 As shown, the direction indicated by the arrow is the direction of rotation of the turntable. On the basis of the initial axis angle direction of 0 degree, the camera equipped with the inertial navigation is rotated around the optical axis by N degrees, which is 90 degrees, 2N is 180 degrees, and 3N is 270 degrees.

[0053] The 20 captured images are aggregated, and the pixel coordinates of the specific mark in each captured image and the inertial navigation acceleration of the inertial navigation are extracted to obtain a picture information set corresponding to different axis angle directions of the camera with inertial navigation.

[0054] In a preferred embodiment, the pixel coordinates include pixel x coordinates in the x-axis direction and pixel y coordinates in the y-axis direction; the inertial navigation acceleration includes an acceleration component in the x-axis direction, an acceleration component in the y-axis direction, and an acceleration component in the z-axis direction;

[0055] The step of obtaining the relative posture between the inertial navigation system and the camera according to the reference error formula, the position of the specific mark 3 of each captured image 6 in the image, the inertial navigation acceleration of the inertial navigation system and the intrinsic parameter matrix of the camera 1 comprises at least the following steps:

[0056] Substitute the pixel coordinates of each of the captured images 6, the inertial navigation acceleration of the inertial navigation system, and the intrinsic parameter matrix of the camera 1 into the reference error formula to obtain the image reference error formula corresponding to each of the captured images 6, and all the image reference error formulas constitute an equation group;

[0057] Based on the equation group, an optimization algorithm is used to solve the rotation matrix with the minimum error, and the rotation matrix with the minimum error is used as the relative posture of the inertial navigation and the camera, wherein the solved rotation matrix must satisfy unit orthogonality.

[0058] Furthermore, the method of using an optimization algorithm to solve the rotation matrix with the minimum error includes at least the following steps:

[0059] The sum of squares of the reference error formulas corresponding to all the captured images 6 is used as a total error function;

[0060] An optimization algorithm is used to solve a rotation matrix that minimizes the total error function, and the rotation matrix is ​​used as the relative posture of the inertial navigation system and the camera.

[0061] Specifically, the optimization algorithm may be selected as Levenberg-Marquardt algorithm, and the relative posture that minimizes the total error function is solved, which is the relative posture of the inertial navigation and the camera.

[0062] In a preferred embodiment, the reference error formula is:

[0063] In a preferred embodiment, the total error function is Wherein, N is the total number of the captured images 6, which is 20.

[0064] Furthermore, the rotation matrix with the minimum error obtained according to the total error function is

[0065] In a preferred embodiment, the method further comprises obtaining an intrinsic parameter matrix of the camera 1, wherein obtaining the intrinsic parameter matrix of the camera 1 comprises at least the following steps:

[0066] Place a standard checkerboard calibration plate on a flat surface;

[0067] Use the camera to take multiple experimental photos from different angles and distances, and input all the experimental photos into the camera calibration software to obtain the camera's intrinsic parameter matrix.

[0068] Specifically, after all the experimental photos are input into the camera calibration software, feature point extraction and matching are performed first. The software automatically detects specific feature points from the input photos, such as the corner points of the chessboard, matches the same feature points in different photos, determines their position correspondence in different images, and sets the initial values ​​of the camera internal parameters according to the pinhole camera model and prior knowledge. At the same time, an initial model of lens distortion, such as initial parameters of radial distortion and tangential distortion, is established. Based on the image coordinates of the feature points and their known coordinates in the world coordinate system, the error between the predicted image coordinates and the actually detected image coordinates is calculated. The errors of all feature points are combined to construct a total error function. An optimization algorithm (such as the Levenberg-Marquardt algorithm) is used to adjust the camera internal parameters and distortion parameters to minimize the error function. In each iteration, the error is recalculated according to the change of the parameters, and it is determined whether the convergence condition is reached. Check whether the error is small enough or whether the change of the parameters is less than a certain threshold. If the convergence condition is met, the optimization is stopped to obtain the final camera internal parameter matrix and distortion coefficient. Use some evaluation metrics to judge the quality of the calibration results, such as reprojection error. Additional validation images may be used to verify the accuracy of the calibration results.

[0069] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and its core ideas of this application. The above is only the preferred implementation method of this application. It should be pointed out that due to the limitations of textual expression and the objective existence of infinite specific structures, ordinary technicians in this technical field can make several improvements, modifications or changes without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the scope of protection of this application.

Claims

1. A method for determining the relative attitude of an inertial navigation system and a camera, characterized in that: The following steps are involved: Collecting a set of picture information corresponding to a camera (1) equipped with an inertial navigation system in different axial angle directions, the set of picture information at least comprising a plurality of images (6) taken by the camera (1) with a specific marker (3) at different positions in the image, and an inertial navigation acceleration of the inertial navigation system corresponding to each of the images (6) at the time of taking the images; A reference error formula is obtained, and an intrinsic parameter matrix of the camera (1) is obtained, and the relative posture of the inertial navigation system and the camera is obtained according to the reference error formula, the position of the specific mark (3) in each of the captured images (6), the inertial navigation acceleration of the inertial navigation system, and the intrinsic parameter matrix of the camera (1).

2. The method for determining the relative attitude between an inertial navigation system and a camera according to claim 1, characterized in that: The acquisition of picture information sets corresponding to the camera (1) with inertial navigation in different axial angle directions comprises at least the following steps: A camera (1) equipped with an inertial navigation system is mounted on a turntable (2) facing a collimator (5), wherein the camera (1) equipped with an inertial navigation system faces the collimator (5), and a cross marker (4) is mounted between the turntable (2) and the collimator (5); When the inertial navigation camera (1) is in the initial axis angle direction, the azimuth angle and the pitch angle of the turntable (2) are rotated so that the specific mark (3) displayed by the cross marker (4) is located at the center of the image, the upper position of the image, the lower position of the image, the left position of the image, and the right position of the image, respectively, and the five captured images (6) corresponding to the initial axis angle direction are collected; The camera (1) equipped with an inertial navigation system is rotated around the optical axis by N degrees, 2N degrees, and 3N degrees respectively to obtain a plurality of rotation rear axis angular directions, and five captured images (6) corresponding to each rotation rear axis angular direction are collected; The 20 captured images (6) are aggregated, and the pixel coordinates of the specific mark (3) in each captured image (6) and the inertial navigation acceleration of the inertial navigation are extracted to obtain a picture information set corresponding to the camera (1) with inertial navigation in different axial angle directions.

3. The method for determining the relative attitude between an inertial navigation system and a camera according to claim 2, characterized in that: The pixel coordinates include the pixel x coordinates in the x-axis direction and the pixel y coordinates in the y-axis direction; the inertial navigation acceleration includes the acceleration component in the x-axis direction, the acceleration component in the y-axis direction and the acceleration component in the z-axis direction; The step of obtaining the relative posture between the inertial navigation system and the camera according to the reference error formula, the position of the specific mark (3) in each of the captured images (6), the inertial navigation acceleration of the inertial navigation system and the intrinsic parameter matrix of the camera (1) comprises at least the following steps: Substituting the pixel coordinates of each captured image (6), the inertial navigation acceleration of the inertial navigation system and the intrinsic parameter matrix of the camera (1) into a reference error formula, obtaining an image reference error formula corresponding to each captured image (6), wherein all the image reference error formulas constitute an equation group; Based on the equation group, an optimization algorithm is used to solve the rotation matrix with the minimum error, and the rotation matrix with the minimum error is used as the relative posture of the inertial navigation and the camera, wherein the solved rotation matrix must satisfy unit orthogonality.

4. The method for determining the relative attitude between an inertial navigation system and a camera according to claim 3, characterized in that: The reference error formula is: Among them, v n ,u n is the position of the specific mark in the nth captured image (6) in the image, X n ,Y n ,Z n is the inertial navigation acceleration during the n-th shooting, K is the intrinsic parameter matrix of the camera, and R is the relative attitude.

5. The method for determining the relative attitude between an inertial navigation system and a camera according to claim 3, characterized in that: The rotation matrix with the smallest error is Among them, argmin represents the value of R that minimizes the total error function.

6. The method for determining the relative attitude between an inertial navigation system and a camera according to claim 3, characterized in that: The method further comprises obtaining an internal parameter matrix of the camera (1), wherein obtaining the internal parameter matrix of the camera (1) comprises at least the following steps: Place a standard checkerboard calibration plate on a flat surface; The camera (1) is used to take a plurality of experimental photos from different angles and distances, and all the experimental photos are input into the camera (1) calibration software to obtain the intrinsic parameter matrix of the camera (1).

7. The method for determining the relative attitude between an inertial navigation system and a camera according to claim 4, characterized in that: The method of using an optimization algorithm to solve the rotation matrix with the minimum error includes at least the following steps: The sum of squares of the reference error formulas corresponding to all the captured images (6) is used as a total error function; An optimization algorithm is used to solve a rotation matrix that minimizes the total error function, and the rotation matrix is ​​used as the relative posture of the inertial navigation system and the camera.

8. The method for determining the relative attitude between an inertial navigation system and a camera according to claim 7, characterized in that: The total error function is Wherein, N is the total number of the captured images (6), which is 20.