Visual-inertial calibration system and calibration method based on precise angle reference

Abstract

The present invention discloses a visual inertial calibration system based on a precise angle reference, comprising a high-precision three-axis turntable, a visual inertial system, a planar target and a controller; the visual inertial system comprises multiple cameras with non-overlapping fields of view and an inertial test unit, which are usually arranged on a specific tooling, such as a wearable device, a micro aerial vehicle (MAVs), a mobile robot, etc.; the visual inertial system is arranged on an inner frame; the planar target is within the field of view that can be photographed during the rotation of the camera; the inertial test unit is used to measure the acceleration and angular velocity data of the visual inertial system during the rotation of the three-axis turntable; the controller is used to perform a calibration method of the visual inertial calibration system according to the visual inertial system, including: defining the initial position of the precise three-axis turntable coordinate system, calibrating the external parameter rotation matrix and translation vector of the inertial test unit, and calibrating the internal and external parameters of the multiple cameras. A calibration method of a visual inertial calibration system based on a precise angle reference is also disclosed.

Description

Technical Field

[0001] The present invention relates to the field of precision measurement and positioning in industrial large-scale environments, and in particular to a visual inertial calibration system based on a precision angle reference and a calibration method thereof. Background Art

[0002] In recent years, the visual-inertial combined positioning system has developed rapidly and has been widely used in the fields of augmented reality, spatial posture measurement, simultaneous positioning and map construction. Visual sensors have high measurement accuracy and rich information, and can achieve large-scale multi-target tracking. The inertial measurement unit (IMU) can quickly sense the object's own motion state and has good real-time performance. The combination of the two can achieve complementary advantages, overcome the problems of low visual measurement frequency and accumulated inertial measurement errors, and effectively improve the system accuracy and robustness.

[0003] Visual inertial combined positioning requires the use of the posture relationship between each sensor, so the accuracy of system calibration will directly affect the accuracy of combined positioning. In addition, different calibration methods will lead to differences in calibration efficiency. Improving calibration efficiency will inevitably improve engineering efficiency. Patent CN113781579A discloses a panoramic infrared camera geometric calibration method, which controls the rotation of the camera through a high-precision turntable and uses a two-step estimation method to achieve precise calibration of camera parameters. Patent CN108921901A discloses a large field of view camera calibration method based on a precision two-axis turntable and a laser tracker. The turntable drives the camera to move in two dimensions, and the camera shoots optical reference points to achieve calibration. Both methods use a turntable to achieve high-precision calibration of camera parameters, but both are single-camera calibrations and are suitable for large-field-of-view cameras. Patent CN110873578B discloses a hexahedral prism and IMU installation error calibration method based on turntable transmission. The relative angle relationship between the IMU and the turntable axis system is calibrated through a three-axis turntable, but this method cannot achieve the calibration of the position relationship between the IMU and the turntable. Patent CN108645428A discloses an integrated calibration method for large-size 6D laser targets, which uses the outer frame and middle frame of a high-precision three-axis turntable to drive the 6D laser target to rotate in front of the outgoing light of the total station in front to construct a measurement control field, calculate the projection of the light vector in the camera coordinate system, and the projection position of the camera is obtained by detection, thereby obtaining the attitude relationship between the inclinometer coordinate system and the camera coordinate system. Since the light vector contains direction information but not position information, the translation relationship between the camera and the turntable cannot be obtained.

[0004] Therefore, an integrated, high-precision calibration method for the unknown parameters of the visual inertial calibration system is needed to realize the calibration of the internal and external parameters of multiple cameras without a common field of view, and the external parameters between cameras and IMU. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art, propose a calibration method for a visual-inertial combined positioning system based on a precise angle reference, introduce a precise three-axis turntable to provide an angle reference, and utilize the advantages of the turntable such as high precision and programmable control to improve the calibration accuracy and efficiency; at the same time, an inertial test unit and multiple cameras are combined as a visual-inertial device and placed on the turntable, and the three-axis rotation of the turntable is utilized to solve the rotational-translational relationship between the camera and the turntable, as well as the rotational-translational relationship between the inertial test unit and the turntable; at the same time, the problem of no common field of view between multiple cameras is overcome, and the integrated calibration of the position and posture relationship of the visual-inertial sensor is realized.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] A visual inertial calibration system based on a precise angle reference comprises a high-precision three-axis turntable, a visual inertial system, a plane target and a controller; the visual inertial system comprises multiple cameras with non-overlapping fields of view and an inertial test unit; the high-precision three-axis turntable comprises an outer frame, a middle frame and an inner frame, and the rotation of the three-axis turntable is controlled by programming; the visual inertial system is arranged on the inner frame; the position of the plane target is arranged in a site to be tested, and is within the range of a photographable field of view during the rotation of the camera; the inertial test unit is used to measure the acceleration and angular velocity data of the visual inertial system during the rotation of the three-axis turntable; the controller is used to execute a calibration method of the visual inertial calibration system according to the visual inertial system, comprising: defining an initial position of the coordinate system of the precise three-axis turntable, calibrating an external parameter rotation matrix and a translation vector of the inertial test unit, and calibrating internal and external parameters of the multiple cameras.

[0008] Furthermore, the visual inertial system is arranged on a wearable device, a micro aerial vehicle or a mobile robot.

[0009] A calibration method for a visual inertial calibration system based on a precise angle reference, comprising:

[0010] A precision three-axis turntable is placed in the field, and a visual inertial system is fixed on the inner frame of the three-axis turntable. The visual inertial system includes multiple cameras with non-overlapping fields of view and an inertial test unit. The multiple cameras and the inertial test unit are both arranged on a wearable helmet, and the wearable helmet is fixed on the inner frame. The placement position should ensure that the multiple cameras have a good field of view during the rotation process.

[0011] The initial position of the three-axis turntable is defined as: T is the rotation center of the turntable, and the X-axis, Y-axis and Z-axis of the turntable coordinate system coincide with the outer frame rotation axis, middle frame rotation axis and inner frame rotation axis of the turntable respectively;

[0012] Calibrate IMU external parameters:

[0013] For the IMU external parameter rotation matrix calibration, the gravity vector is used as the intermediate variable to establish the attitude relationship equation between the IMU coordinate system and the turntable coordinate system. With the initial position as the center, the outer frame, the middle frame and the inner frame of the three-axis turntable are rotated between -50° and +50°, respectively, with a step length of 25°, driving the visual inertial system to rotate accordingly. At each calibration position, the inertial test unit is used to collect accelerometer data in a static state, and the attitude relationship equation is solved using the invariance of the gravity vector to obtain the external parameter rotation matrix from the inertial test unit to the three-axis turntable;

[0014] For the calibration of the IMU external reference translation vector, the middle frame and the inner frame of the three-axis turntable are used to drive the turntable coordinate system to rotate, so that the turntable coordinate system is adjusted to: the X-axis vertically coincides with the outer frame rotation axis (i.e., the initial position of the three-axis turntable), the Y-axis vertically coincides with the outer frame rotation axis (i.e., the inner frame rotates 90° based on the initial position), and the Z-axis vertically coincides with the outer frame rotation axis (i.e., the middle frame rotates 90° based on the initial position). In the above three positions, the three-axis turntable is controlled to be in a static state and a uniform rotation state of the outer frame respectively. The inertial test unit is used to collect the accelerometer output value in the static state and the accelerometer output value when rotating at a uniform speed in the horizontal plane. The rotation radius is solved by the centripetal acceleration, and a geometric constraint equation of the rotation radius and the translation relationship between the IMU and the turntable is established, thereby obtaining the IMU translation vector in the turntable coordinate system;

[0015] Calibrate the camera’s internal and external parameters:

[0016] Rotate the turntable outer frame and the middle frame in sequence, first rotate the outer frame horizontally to cover the horizontal field of view of the multi-camera, then rotate it vertically once, and then rotate it horizontally, and rotate it repeatedly to cover all the fields of view of the multi-camera, use the multi-camera to shoot the planar target, and record the rotation angle of the high-precision three-axis turntable; establish the transformation matrix of the multi-camera relative to the turntable coordinate system, and obtain the transformation relationship between the camera coordinate systems at different positions before and after the rotation of the high-precision three-axis turntable; introduce the turntable coordinate system into the camera imaging model, and optimize the internal parameters of the multi-camera and the external parameters between the multi-camera and the high-precision three-axis turntable by minimizing the reprojection error;

[0017] Finally, the turntable coordinate system is used as a transfer to obtain the posture relationship between the camera coordinate system and the IMU coordinate system.

[0018] Furthermore, the “obtaining the external parameter rotation matrix from the IMU to the turntable” includes the following steps:

[0019] The turntable rotates to n different positions and accumulates the gravity vector equation measured by n IMUs as follows:

[0020]

[0021] Among them, R (i)The transformation matrix representing the turntable coordinate system; b g (i) Represents the gravity vector in the IMU coordinate system after the turntable rotates, T g0 represents the gravity vector in the turntable coordinate system at the initial position, which is a constant value;

[0022] Among them, the transformation matrix R of the turntable coordinate system (i) for:

[0023]

[0024] Among them, α represents the rotation angle of the outer frame, β represents the rotation angle of the middle frame, and γ represents the rotation angle of the inner frame;

[0025] Construct the minimization objective function:

[0026]

[0027] The Levenberg-Marquardt algorithm is used to obtain the unit orthogonal matrix The least squares solution of is obtained to obtain the external parameter rotation matrix from IMU to the turntable.

[0028] Furthermore, the “obtaining the IMU translation vector in the turntable coordinate system” specifically includes the following steps:

[0029] The translation vector solution utilizes the centripetal acceleration during uniform rotation. To separate the centripetal acceleration from the gravitational acceleration, the inertial test unit is controlled to perform uniform circular motion in the horizontal plane (i.e. around the outer frame rotation axis), and the rotation radius is:

[0030]

[0031] in, b a n is the centripetal acceleration, b g is the gravitational acceleration value collected when the inertial test unit is in a stationary state; b a is the accelerometer output value collected when the inertial test unit rotates at a constant speed; ω is the angular velocity value when the inertial test unit rotates at a constant speed;

[0032] When the X-axis, Y-axis, and Z-axis of the turntable coordinate system are vertically aligned with the outer frame axis, the rotation radius r of the uniform rotation around the outer frame axis at the three positions is calculated. X 、r Y 、r Z ;

[0033] Define the translation vector between the IMU and the turntable in the turntable coordinate system There are the following geometric constraint relationships between the magnitudes of the components of the translation vector and the three rotation radii:

[0034]

[0035] Among them, [t X t Y t Z ] is the translation component in the turntable coordinate system; and the direction of the translation vector can be determined based on the direction of the centripetal acceleration in the IMU coordinate system during rotation.

[0036] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0037] The visual inertial system of the present invention is fixedly connected to the inner frame of the turntable, and the turntable drives the visual inertial system to move, which conforms to the actual measurement mode of the calibration object in this patent - the wearable helmet, that is, the camera on the helmet shoots the external fixed control points during the movement, and the IMU senses the motion information of the helmet. The calibration scheme is designed according to the device measurement method, and the calibration results and accuracy are more suitable for the actual use scenario;

[0038] The camera and the inertial test unit of the present invention are connected according to the relationship between the IMU coordinate system b, the turntable coordinate system T, and the camera coordinate system C, and the use of fewer coordinate system conversions reduces the links of error introduction. At the same time, the posture relationship between bT and between TC is obtained by calibration with the help of the precise angle reference and standard motion trajectory of the turntable, thereby ensuring the calibration accuracy.

[0039] Moreover, the present invention uses a high-precision three-axis turntable (positioning accuracy is generally ±1") as a reference datum, which is more accurate and the calibration result is more reliable, thereby improving the measurement accuracy. During the calibration process, the visual inertial system does not need to be disassembled, which overcomes the problem of no common field of view between multiple cameras, realizes the integrated calibration of the internal parameters of each camera, the external parameters of the camera without a common field of view, and the external parameters between the camera and the IMU, thereby improving the calibration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a structural schematic diagram of the visual inertial calibration system based on a precise angle reference of the present invention;

[0041] Figure 2 Three postures for calibrating the IMU translation vector; (a) shows the X-axis O of the turntable coordinate system T X T The vertical axis coincides with the outer frame axis; (b) shows the Y axis O T Y T The vertical axis coincides with the outer frame axis; (c) shows the Z axis O T Z T The vertical axis coincides with the outer frame axis;

[0042] Figure 3Schematic diagram for solving the translation vector between IMU and turntable;

[0043] Figure 4 This is a schematic diagram of the camera imaging model;

[0044] In the figure:

[0045] 101: Precision three-axis turntable, 102: Outer frame, 103: Middle frame, 104: Inner frame, 105: Plane target, 106: Inertial sensor, 107: Multi-camera DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution, beneficial effects and significant progress of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention is clearly and completely described below in combination with the drawings provided in the examples of the present invention. Obviously, all the described embodiments are only partial embodiments of the present invention, rather than all embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0047] like Figure 1 As shown, a visual inertial calibration system based on a precise angle reference includes a wearable helmet, a plane target 105, a high-precision three-axis turntable 101, a multi-camera 107 with no overlapping fields of view, an inertial measurement unit (hereinafter referred to as IMU) 106, and a controller. The precise three-axis turntable 101 includes an outer frame 102, a middle frame 103, and an inner frame 104. The wearable helmet is fixed on the inner frame 104. The multi-camera 107 is arranged on the wearable helmet and is three positioning cameras with no overlapping fields of view. The IMU 106 is arranged on the top of the wearable helmet; the inertial measurement unit of this embodiment has a built-in accelerometer and a gyroscope. The position of the plane target 105 is set in the test site, placed at a working distance where the camera can clearly shoot, and during the rotation of the camera, ensure that the plane target is in the camera field of view; the controller is used to solve the external reference relationship between the IMU 106 and the turntable, and between the multi-camera 107 and the turntable according to the calibration method of the visual inertial combined positioning system executed by the multi-camera and the inertial measurement unit. The present invention takes a wearable helmet as a typical carrier, on which are installed three non-overlapping field-of-view positioning cameras and an inertial measurement unit. The calibration method is also applicable to other carriers or systems with relevant characteristics.

[0048] The present invention uses IMU and multi-camera as calibration objects, takes high-precision three-axis turntable 101 as the core, uses the turntable to drive the visual inertial test unit to rotate, and solves the external parameter relationship between IMU106 and the turntable, and between multi-camera 107 and the turntable. For the calibration of IMU external parameters, the gravity vector is used as an intermediate variable to establish the attitude relationship equation between the IMU coordinate system and the turntable coordinate system. The turntable outer frame 102, the middle frame 103 and the inner frame 104 drive the inertial test unit 106 to rotate, and the attitude relationship equation is solved by using the invariance of the gravity vector to obtain the rotation transformation matrix between the IMU 106 and the turntable; the turntable drives the system to perform uniform circular motion around the outer frame rotation axis, and the rotation radius is solved by the centripetal acceleration. The geometric constraint equation of the rotation radius and the translation relationship between the IMU and the turntable is established to solve the IMU external parameter translation vector; for the calibration of camera internal and external parameters, the turntable outer frame and the middle frame drive the system to rotate in front of the plane target 105 to construct an angle control field, and the turntable coordinate system is introduced into the camera imaging model. The camera internal parameters and the external parameters between the camera and the turntable are optimized by minimizing the reprojection error; finally, the turntable coordinate system is used as a transfer to obtain the attitude relationship between the camera coordinate system and the IMU coordinate system. At this point, the external parameters of the visual inertial sensor are calibrated. The arrangement of the visual-inertial combined positioning system and the specific steps of the controller are as follows:

[0049] Step 1: Place the precision three-axis turntable 101 in the field, fix the wearable helmet on the inner frame 104 of the three-axis turntable 101, and set the IMU 106 and the multi-camera 107 on the wearable helmet. The placement position should ensure that the multi-camera 107 has a good field of view during the rotation process;

[0050] Step 2:

[0051] S201: Define the initial position of the precision three-axis turntable coordinate system as follows: Figure 2 (a) shows: Origin O T is the rotation center of the turntable, and the X axis O of the turntable coordinate system T X T , Y axis O T Y T and Z axis O T Z T They coincide with the outer frame 102, the middle frame 103 and the inner frame 104 of the turntable respectively; in the initial position, the outer frame rotation angle α = 0°, the middle frame rotation angle β = 0°, and the inner frame rotation angle γ = 0° are defined; during the calibration process, each frame of the three-axis turntable needs to rotate to multiple positions. When the outer frame rotates by angle α, the middle frame rotates by angle β, and the inner frame rotates by angle γ, the coordinate system of the three-axis turntable changes relative to the initial position. Since the rotation angles of the three frames correspond exactly to the three Euler angles, the transformation matrix R of the turntable coordinate system (i) for:

[0052]

[0053] S202: Calibrate IMU external parameter rotation matrix

[0054] The IMU coordinate system is the coordinate system of the IMU106 system. It is calibrated and aligned before leaving the factory. The origin of the IMU coordinate system is on the IMU device, and the directions of the three coordinate axes are marked on the IMU package. The IMU106 can measure and output acceleration and angular velocity data in different states. The accelerometer output value in the static state is b f=[f x f y f z ] T , which represents the reaction force exerted by gravity on the inertial device. The gravity vector in the IMU coordinate system is obtained by normalizing it:

[0055]

[0056] According to the rotation matrix between IMU and turntable (amount to be optimized), the gravity vector in the IMU coordinate system can be converted to the turntable coordinate system, with the initial position as the center, the three-axis turntable outer frame 102, the middle frame 103 and the inner frame 104 are rotated between -50° and +50°, with a step length of 25°, and the outer frame, the middle frame and the inner frame are rotated in sequence, one frame is rotated each time, thereby generating 5×5×5 125 calibration positions in total, and the IMU106 accelerometer data is collected at each calibration position for a certain period of time, and the three-axis rotation angle of the turntable is recorded accordingly. The rotation of the three-axis turntable is controlled by programming.

[0057] When the turntable moves to different positions, the turntable coordinate system rotates R relative to its initial position. (i) , then the gravity vector measured by the IMU is converted to the initial position turntable coordinate system and can be expressed as:

[0058]

[0059] in, b g (i) Represents the gravity vector in the IMU coordinate system after the turntable rotates, T g0 represents the gravity vector in the turntable coordinate system at the initial position;

[0060] When the turntable rotates to n different positions, n equations can be accumulated as shown in equation (4):

[0061]

[0062] Gravity vector in the turntable coordinate system at the initial position Tg0 is a constant, but it will be affected by the turntable leveling error, thus affecting the accuracy of the solved rotation matrix. Therefore, the minimization objective function is constructed:

[0063]

[0064] The Levenberg-Marquardt algorithm is used to obtain the unit orthogonal matrix The least squares solution of is obtained to obtain the external parameter rotation matrix from IMU to the turntable.

[0065] S203: Calibrate IMU extrinsic translation vector

[0066] The solution of the translation vector utilizes the consistency of the centripetal acceleration value under uniform circular motion. In order to remove the gravity acceleration from the acceleration data, the system needs to rotate around the vertical axis. According to the definition of the turntable coordinate system, the turntable middle frame 103 and the inner frame 104 are rotated to make the turntable coordinate system X-axis O T X T The vertical axis coincides with the outer frame axis (such as Figure 2 (a)), Y axis O T Y T The vertical axis coincides with the outer frame axis (such as Figure 2 (b)), Z axis O T Z T The vertical axis coincides with the outer frame axis (such as Figure 2 (c) shows that the accelerometer data in the static state are collected at three positions.

[0067] When the turntable rotates around the outer frame axis at a uniform angular velocity ω of 60° / s, the IMU106 performs uniform circular motion in the horizontal plane and is subjected to the centripetal acceleration b a n The acceleration due to gravity b g, the accelerometer output value b a is the vector synthesis of the two:

[0068] b a= b a n + b g (6)

[0069] Among them, the gravity acceleration can be obtained from the output value of the accelerometer in a static state, and the rotation radius of the IMU when it performs uniform circular motion around the rotation axis of the turntable outer frame is:

[0070]

[0071] exist Figure 2 In the three postures shown, the output values ​​of the accelerometer in the static state and the output values ​​of the accelerometer when rotating at a constant speed around the outer frame axis are collected respectively, and the three rotation radii r are obtained.X 、r Y 、r Z ;

[0072] Define the translation vector between the IMU and the turntable in the turntable coordinate system T b T=[t X t Y t Z ],like Figure 3 As shown, there is a geometric constraint relationship between the magnitude of each component of the translation vector and the three rotation radii:

[0073]

[0074] The size of the translation vector between the IMU and the turntable can be optimized according to the above formula, where [t X t Y t Z ] is the translation component in the turntable coordinate system, which is not affected by the three-axis orientation of the IMU coordinate system. The direction of the translation vector can be determined according to the direction of the centripetal acceleration in the IMU coordinate system during rotation;

[0075] S204: Calibrate camera internal and external parameters

[0076] Define the initial position of the camera calibration:

[0077] Use the three-axis turntable 101 to calibrate the initial position of the multi-camera 107, such as Figure 2 As shown in (c), the Z axis O of the turntable coordinate system T Z T , Y axis O T Y T The same as the outer frame rotation axis and the middle frame rotation axis of the turntable, the outer frame rotation angle α=0° and the middle frame rotation angle β=0° are defined at this position.

[0078] Calibrate the camera’s internal and external parameters:

[0079] Then, the outer frame and the middle frame of the turntable are rotated in sequence, that is, the outer frame is rotated horizontally to cover the horizontal field of view of the camera first, then rotated vertically once, and then rotated horizontally, and this rotation is repeated to cover the entire field of view of the camera, a total of 4×4 rotations. During the rotation process, the camera shoots the plane target 105 and records the rotation angle of the turntable. The transformation matrix R of the multi-camera relative to the turntable coordinate system is (k) It can be expressed as:

[0080]

[0081] The present invention uses spatial object points with known three-dimensional coordinates to minimize the reprojection error for multi-camera intrinsic calibration and external calibration between cameras and turntables, and has spatial direction and position information, so the rotation and translation relationship between the camera and the turntable can be obtained at the same time. In addition, compared with the traditional camera calibration method using planar targets, the camera internal and external parameter calibration in the present invention introduces a turntable coordinate system into the object point image point coordinate conversion relationship, and the angle of turntable rotation constitutes an angle control field, thereby providing constraints on the camera internal and external parameters.

[0082] The multi-camera 107 calibration uses a pinhole imaging model, such as Figure 4 As shown, a point P in the world coordinate system W (X W ,Y W ,Z W ) is transformed to the image pixel plane through projection, and its projection transformation relationship is:

[0083]

[0084] Among them, Z C is the Z-axis coordinate of point P in the camera coordinate system, (u, v) is the image point coordinate, f is the focal length of the camera, d x d y is the distance of unit pixel in x and y directions, (u0, v0) is the coordinate of the origin of the camera physical coordinate system in the pixel coordinate system, K is the camera intrinsic parameter matrix, is the rotation and translation relationship between the camera and the world coordinate system; in actual imaging, the position of the spatial point projected onto the imaging plane will be nonlinearly distorted, and radial distortion parameters (k1, k2, k3) and tangential distortion parameters (p1, p2) are introduced to perform distortion correction on the normalized coordinates in the physical coordinate system of the image;

[0085] Camera calibration only requires rotating the turntable outer frame 102 and the middle frame 103, and the external reference between the camera and the turntable is and the turntable rotation matrix R (k) , we can get the transformation relationship between the camera coordinate systems at different positions before and after the turntable rotates. Combined with the camera imaging model, the relationship between the corner point coordinates in the image at each position after the turntable rotates and the world coordinates is:

[0086]

[0087] Among them, (u k ,v k ) is the pixel coordinate of the midpoint of the image captured by the camera when the turntable rotates for the kth time, R (k) is the rotation matrix of the turntable relative to its initial position after the kth rotation, Represents the rotation and translation between the initial position camera and the world coordinate system;

[0088] Rotate the turntable multiple times to construct a minimized objective function:

[0089]

[0090] F represents the sum of squares of all image point reprojection errors, (u k ,v k ) is the actual image point coordinate, (u' k ,v' k ) is the coordinate of the reprojected image point after distortion compensation, and the Levenberg-Marquardt algorithm is used to optimize the camera intrinsic parameters and the extrinsic parameters between the camera and the turntable;

[0091] According to the above principle, the external parameters from the IMU coordinate system to the turntable coordinate system are obtained. External reference of each camera coordinate system to the turntable coordinate system Using the turntable coordinate system as the intermediate point, the external parameters between the camera and IMU can be obtained And the external parameters between cameras Right now:

[0092]

[0093]

[0094] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention. The non-essential improvements, adjustments or replacements made by those skilled in the art based on the contents of this specification are all within the scope of protection required by the present invention.

Claims

1. A calibration method for a visual inertial calibration system based on a precise angle reference, characterized in that: A visual inertial calibration system based on a precise angle reference comprises a high-precision three-axis turntable (101), a visual inertial system, a planar target (105) and a controller; the visual inertial system comprises multiple cameras (107) with non-overlapping fields of view and an inertial test unit (106); The high-precision three-axis turntable (101) comprises an outer frame (102), a middle frame (103) and an inner frame (104); the rotation of the high-precision three-axis turntable (101) is controlled by programming; the visual inertial system is arranged on the inner frame (104); The position of the planar target (105) is set in the field to be tested, within the field of view that can be photographed during the rotation of the camera; The inertial testing unit (106) is used to measure the acceleration and angular velocity data of the visual inertial system during the rotation of the high-precision three-axis turntable (101); The controller is used to execute a calibration method of a visual inertial calibration system according to the visual inertial system, comprising: defining an initial position of the precision three-axis turntable coordinate system, calibrating an external parameter rotation matrix and a translation vector of the inertial test unit (106), and calibrating internal and external parameters of multiple cameras (107); The calibration method comprises: A high-precision three-axis turntable (101) is placed in a field, and a visual inertial system is fixed on an inner frame (104) of the high-precision three-axis turntable (101), wherein the visual inertial system comprises a multi-camera (107) with non-overlapping fields of view and an inertial test unit (106), wherein the multi-camera and the inertial test unit are both arranged on a wearable helmet, and the wearable helmet is fixed on the inner frame; the placement position must ensure that the multi-camera (107) has a good field of view during the rotation process; The initial position of the three-axis turntable is defined as: T The X axis, Y axis and Z axis of the turntable coordinate system coincide with the outer frame (102), middle frame (103) and inner frame (104) rotation axes of the turntable respectively; Calibrate IMU external parameters: For the calibration of the IMU external parameter rotation matrix, the gravity vector is used as an intermediate variable to establish an attitude relationship equation between the IMU coordinate system and the turntable coordinate system. With the initial position as the center, the three-axis turntable outer frame (102), the middle frame (103) and the inner frame (104) are respectively rotated between -50° and +50°, with a step length of 25°, driving the visual inertial system to rotate accordingly. At each calibration position, the inertial test unit is used to collect accelerometer data in a static state, and the attitude relationship equation is solved using the invariance of the gravity vector to obtain the external parameter rotation matrix from the inertial test unit (106) to the high-precision three-axis turntable (101); For the calibration of the IMU external parameter translation vector, the three-axis turntable middle frame (103) and the inner frame (104) are used to drive the turntable coordinate system to rotate, so that the turntable coordinate system is adjusted to: the X-axis vertically coincides with the rotation axis of the outer frame (102), that is, the initial position of the high-precision three-axis turntable (101); the Y-axis vertically coincides with the rotation axis of the outer frame (102), that is, the inner frame (104) is rotated 90° based on the initial position; the Z-axis vertically coincides with the rotation axis of the outer frame (102), that is, the middle frame (103) is rotated 90° based on the initial position; in the above three The high-precision three-axis turntable (101) is controlled to be in a stationary state and a uniformly rotating state of the outer frame (102) respectively under the same position, the accelerometer output value in the stationary state and the accelerometer output value when rotating at a uniform speed in a horizontal plane are collected by using the inertial test unit (106), the rotation radius is solved by the centripetal acceleration, and a geometric constraint equation of the rotation radius and the translation relationship between the inertial test unit (106) and the high-precision three-axis turntable (101) is established, thereby obtaining the IMU translation vector in the turntable coordinate system; Calibrate the camera’s internal and external parameters: The initial position of the multi-camera (107) is calibrated using the high-precision three-axis turntable (101), and the Z axis and Y axis of the turntable coordinate system are the same as the rotation axis of the outer frame (102) and the middle frame (103) of the turntable; The turntable outer frame (102) and the middle frame (103) are rotated in sequence, the outer frame (102) is first rotated horizontally to cover the horizontal field of view of the multi-camera (107), then rotated vertically once, and then rotated horizontally, and the rotation is repeated to cover all fields of view of the multi-camera (107), the multi-camera (107) is used to shoot the planar target (105), and the rotation angle of the high-precision three-axis turntable (101) is recorded; a transformation matrix of the multi-camera (107) relative to the turntable coordinate system is established, and the transformation relationship between the camera coordinate systems at different positions before and after the rotation of the high-precision three-axis turntable (101) is obtained; the turntable coordinate system is introduced into the camera imaging model, and the internal parameters of the multi-camera (107) and the external parameters between the multi-camera (107) and the high-precision three-axis turntable (101) are optimized by minimizing the reprojection error; Finally, the turntable coordinate system is used as a transfer to obtain the posture relationship between the camera coordinate system and the IMU coordinate system.

2. The calibration method according to claim 1, characterized in that: The visual inertial system is arranged on a wearable device, a micro aerial vehicle or a mobile robot.

3. The calibration method according to claim 1, characterized in that: The calibrating the IMU external parameters to obtain the external parameter rotation matrix from the IMU to the turntable also includes the following steps: The turntable rotates to n different positions, and the gravity vector equation measured by the n inertial test units (106) is accumulated as follows: Among them, R (i) The transformation matrix representing the turntable coordinate system; b g (i) Represents the gravity vector in the IMU coordinate system after the turntable rotates, T g0 represents the gravity vector in the turntable coordinate system at the initial position, which is a constant value; Among them, the transformation matrix R of the turntable coordinate system ( i) for: Among them, α represents the rotation angle of the outer frame, β represents the rotation angle of the middle frame, and γ represents the rotation angle of the inner frame; Construct the minimization objective function: The Levenberg-Marquardt algorithm is used to obtain the unit orthogonal matrix The least squares solution of is obtained, thereby obtaining the external parameter rotation matrix from the inertial test unit (106) to the high-precision three-axis turntable (101).

4. The calibration method of the visual inertial calibration system based on the precise angle reference according to claim 1, characterized in that: The specific steps of calibrating the IMU external parameters to obtain the IMU translation vector in the turntable coordinate system are as follows: The translation vector is solved by using the centripetal acceleration during uniform rotation. To separate the centripetal acceleration from the gravitational acceleration, the inertial test unit (106) is controlled to perform uniform circular motion in a horizontal plane, and the rotation radius is: in, b a n is the centripetal acceleration, b g is the gravity acceleration value collected by the inertial testing unit (106) in a stationary state; b a is the accelerometer output value collected when the inertial test unit (106) rotates at a uniform speed; ω is the angular velocity value when the inertial test unit (106) rotates at a uniform speed; When the X-axis, Y-axis, and Z-axis of the turntable coordinate system are respectively vertical and coincide with the rotation axis of the outer frame (102), the rotation radius r of the uniform rotation around the rotation axis of the outer frame at the three positions is obtained. X 、r Y 、r Z ; Define the translation vector between the inertial test unit (106) and the high-precision three-axis turntable (101) in the turntable coordinate system There are the following geometric constraint relationships between the magnitudes of the components of the translation vector and the three rotation radii: Among them, [t X t Y t Z ] is the translation component in the turntable coordinate system; and the direction of the translation vector can be determined based on the direction of the centripetal acceleration in the IMU coordinate system during rotation.

Images