Unmanned aerial vehicle strapdown camera and imu extrinsic parameter offline calibration method based on landmark re-projection

Abstract

A landmark re-projection-based off-line calibration method for the external parameters of a UAV strapdown camera and an IMU, comprising the following steps: (1) selecting multiple landmark points on the flight path of the UAV and recording their geographic coordinates; acquiring image data, high-frequency GPS data and IMU attitude angle data of the UAV, and recording the original pixel coordinates of each landmark point in the image; (2) converting the geographic coordinates of each landmark point into the northeast celestial coordinate system; (3) constructing a northeast celestial-camera coordinate system conversion matrix; (4) converting the northeast celestial coordinates of each landmark point into predicted coordinates in the camera coordinate system by using the northeast celestial-camera coordinate system conversion matrix, and obtaining the predicted pixel coordinates of each landmark point; (5) constructing the re-projection error of each landmark point, taking the sum of the re-projection errors of all landmark points as an optimization objective function, and iteratively optimizing the optimal external parameters of the camera and the IMU by using an iterative algorithm to complete the external parameter calibration between the camera and the IMU.

Description

Technical Field

[0001] This invention relates to an offline calibration method for UAV strapdown cameras and IMU extrinsic parameters based on landmark reprojection, belonging to the fields of UAV photogrammetry, remote sensing and navigation positioning technology. Background Technology

[0002] In the process of precise vision-inertial fusion navigation and positioning for UAVs, due to manufacturing and assembly errors, as well as factors such as thermal deformation and vibration during flight, there is a deviation between the actual installation relationship (true extrinsic parameters) of the camera and the inertial measurement unit (IMU) and their design values ​​(nominal extrinsic parameters), which directly leads to systematic errors in geolocation. Therefore, it is necessary to calibrate the extrinsic parameters of the camera and the IMU.

[0003] There are two main types of existing extrinsic parameter calibration techniques. The first is the laboratory calibration method based on a calibration board. This method requires a precision turntable and a calibration board for joint calibration, resulting in high overall laboratory setup costs. Furthermore, it cannot account for dynamic flight scenarios during extrinsic parameter calibration, and the calibration results may be inaccurate in practical applications. The second is the online calibration method based on algorithms. While this method can adapt to dynamic environments, the algorithms are complex, have specific requirements for flight trajectories, and the initialization process is unstable. Therefore, for drones that have completed aerial photography missions, if the positioning accuracy is found to be unsatisfactory, there is currently a lack of a simple, high-precision calibration method that requires no additional hardware and has a wide range of applicability. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of existing technologies and provide an offline calibration method for the extrinsic parameters of UAV strapdown cameras and IMUs based on landmark reprojection. This method constructs and minimizes the reprojection error, iteratively solving for high-precision camera and IMU extrinsic parameters, thereby achieving accurate calibration of the strapdown camera and IMU and solving the problems of complex operation, high implementation cost, and low positioning accuracy of existing methods.

[0005] The technical solution of this invention is: An offline calibration method for UAV strapdown cameras and IMU extrinsics based on landmark reprojection, comprising the following steps: (1) Select multiple landmarks evenly along the flight path of the UAV and record the geographic coordinates of each landmark; acquire the image data captured by the UAV in real time during flight, as well as the high-frequency GPS data and IMU attitude angle data of the UAV when each image data is captured; define the image data containing landmarks as the target image and record the original pixel coordinates of each landmark in the target image; (2) Convert the geographic coordinates of each landmark to northeast-sky coordinates; (3) Based on the external parameters of the camera and IMU, the IMU installation position and the UAV attitude angle data, construct the northeast sky-camera coordinate system transformation matrix; (4) Convert the high-frequency GPS data corresponding to the UAV when the target image is captured into the northeast sky coordinates in the same coordinate system as the landmark; based on the high-frequency GPS data in the northeast sky coordinates, use the northeast sky-camera coordinate system transformation matrix to convert the northeast sky coordinates of each landmark into the predicted coordinates in the camera coordinate system. (5) By using the camera intrinsic parameter matrix, the predicted coordinates of each landmark are projected onto the image plane to obtain the predicted pixel coordinates of each landmark; (6) Based on the predicted pixel coordinates and original pixel coordinates of each landmark, construct the reprojection error of each landmark; (7) Using the sum of the reprojection errors of all landmarks as the objective function, the optimal external parameters of the camera and IMU are iteratively optimized using an iterative algorithm, and the camera and IMU are calibrated using the optimal external parameters of the camera and IMU.

[0006] Furthermore, the image data of the UAV includes a timestamp; the high-frequency GPS data includes a timestamp, as well as the longitude, latitude, and altitude data of the UAV at the time corresponding to the timestamp; the IMU attitude angle data includes a timestamp, as well as the pitch, roll, and heading data of the UAV at the time corresponding to the timestamp. In step (1), obtaining the high-frequency GPS data and IMU attitude angle data corresponding to the UAV when each image data is captured is specifically as follows: performing time synchronization processing on the image data, high-frequency GPS data and IMU attitude angle data, and aligning the time using the timestamps contained in the image data, high-frequency GPS data and IMU attitude angle data, thereby obtaining the high-frequency GPS data and IMU attitude angle data of the UAV when each image data is captured.

[0007] Furthermore, the step of uniformly selecting multiple landmark points along the flight path of the UAV specifically involves selecting stable points on the ground with stable intersection characteristics as landmark points along the flight path of the UAV; the stable points include road intersections, building corners, and isolated trees.

[0008] Furthermore, the specific process of constructing the northeast-camera coordinate system transformation matrix in step (3) is as follows: (3.1) Based on IMU attitude angle data, calculate the northeast-north-skin rotation matrix from the northeast-north-sky coordinate system to the body coordinate system. ; (3.2) Based on the extrinsic parameters of the IMU, the body-IMU rotation matrix is ​​obtained. ; (3.3) Based on the camera's extrinsic parameters, obtain the IMU-camera rotation matrix. ; (3.4) Based on the Northeast Sky-body Rotation Matrix Body-IMU rotation matrix and IMU-camera rotation matrix Construct the Northeast Sky-Camera Coordinate System Transformation Matrix The expression is .

[0009] Furthermore, in step (3.1), the northeast sky-body rotation matrix The calculation formula is:

[0010] in, This is the rotation matrix of the northeast-central coordinate system about the Z-axis; The rotation matrix of the heading coordinate system about the new Y-axis; This is the rotation matrix of the pitch coordinate system about the new X-axis; The calculation formula is:

[0011] in, The yaw angle of the UAV in the northeast celestial coordinate system; The calculation formula is:

[0012] in, The pitch angle of the UAV in the northeast celestial coordinate system; The calculation formula is:

[0013] in, The roll angle of the UAV in the northeast celestial coordinate system; In step (3.2), the body-IMU rotation matrix The calculation formula is

[0014] In step (3.3), the IMU-camera rotation matrix The calculation formula is

[0015] in, This is the rotation matrix around the Z-axis; This is the rotation matrix around the new Y-axis; For the rotation matrix about the new X-axis; The calculation formula is:

[0016] in, The yaw angle of the camera in the IMU coordinate system; The calculation formula is:

[0017] in, The camera's pitch angle in the IMU coordinate system; The calculation formula is:

[0018] in, This is the roll angle of the camera in the IMU coordinate system.

[0019] Furthermore, in step (7), the Levenberg-Marquardt algorithm is used to iteratively optimize the optimal extrinsic parameters of the camera and IMU; the condition for the termination of the Levenberg-Marquardt algorithm iteration is: the change in the sum of the reprojection errors of all landmarks calculated in two consecutive iterations is less than a preset threshold or the number of iterations of the Levenberg-Marquardt algorithm reaches the maximum number of iterations.

[0020] Secondly, this invention also proposes an offline calibration system for UAV strapdown cameras and IMU extrinsic parameters based on landmark reprojection, including: a data acquisition module, a preprocessing module, a coordinate system transformation relationship construction module, a reprojection error calculation module, and an optimization calculation module; The data acquisition module uniformly selects multiple landmarks along the UAV's flight path and records the geographical coordinates of each landmark; it acquires real-time image data captured by the UAV during flight, as well as the high-frequency GPS data and IMU attitude angle data of the UAV corresponding to each image data capture time; the image data containing landmarks is defined as the target image, and the data acquisition module records the original pixel coordinates of each landmark in the target image; The preprocessing module converts the geographic coordinates of each landmark point into northeast-sky coordinates. The coordinate system transformation module constructs a northeast-sky-camera coordinate system transformation matrix based on the external parameters of the camera and IMU, the IMU installation position, and the UAV attitude angle data. At the same time, it transforms the high-frequency GPS data corresponding to the UAV when the target image is captured into northeast-sky coordinates in the same coordinate system as the landmark. Based on the high-frequency GPS data in the northeast-sky coordinates, the northeast-sky-camera coordinate system transformation matrix is ​​used to transform the northeast-sky coordinates of each landmark into predicted coordinates in the camera coordinate system. The reprojection error calculation module constructs the reprojection error of each landmark point based on the predicted pixel coordinates and the original pixel coordinates of each landmark point. The optimization calculation module uses the sum of reprojection errors of all landmarks as the optimization objective function, and uses an iterative algorithm to iteratively optimize the optimal extrinsic parameters of the camera and IMU. The optimal extrinsic parameters of the camera and IMU are then used to calibrate the camera and IMU.

[0021] Furthermore, the offline calibration system also includes a result output module; the result output module is used to visualize the results of the camera and IMU after calibration using optimal extrinsic parameters.

[0022] The advantages of this invention compared to the prior art are: (1) This invention constructs and minimizes the reprojection error, iteratively solves the high-precision camera and IMU extrinsic parameters, and realizes the accurate calibration of strapdown camera and IMU. The whole process is simple to operate, has low implementation cost, and the final calibration result is highly accurate.

[0023] (2) The present invention directly uses known geographic coordinates on the ground as a reference, so no additional calibration equipment is needed. By minimizing the reprojection error, the calibration can be completed after the UAV has finished its normal flight, reducing the calibration cost and complexity.

[0024] (3) This invention utilizes the high-frequency GPS and IMU data collected during aerial photography missions, which can be used to evaluate the accuracy of completed aerial photography projects and correct external parameters, thereby improving the efficiency of comprehensive data utilization.

[0025] (4) This invention can be applied to different models of cameras and IMUs by adjusting the corresponding parameters, and has good scalability. Attached Figure Description

[0026] Figure 1 This is a flowchart of an offline calibration method for UAV strapdown camera and IMU extrinsic parameters based on landmark reprojection according to the present invention; Figure 2 This is a flowchart of data acquisition and preprocessing in an offline calibration method for UAV strapdown cameras and IMU extrinsics based on landmark reprojection, as described in this invention. Figure 3 This is a flowchart illustrating the optimization function design and iterative calculation process in an offline calibration method for UAV strapdown cameras and IMU extrinsics based on landmark reprojection, as described in this invention. Figure 4 This is a schematic diagram of the coordinate system transformation relationship in an offline calibration method for UAV strapdown cameras and IMU extrinsic parameters based on landmark reprojection according to the present invention. Figure 5This is a visualization of the calibration results of an offline calibration method for UAV strapdown cameras and IMU extrinsics based on landmark reprojection according to the present invention. Figure 6 This is a diagram illustrating the architecture of an offline calibration system for UAV strapdown cameras and IMU extrinsic parameters based on landmark reprojection, as described in this invention. Detailed Implementation

[0027] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings.

[0028] like Figure 1 As shown, this invention provides an offline calibration method for the extrinsic parameters of a UAV strapdown camera and IMU based on landmark reprojection, the steps of which are as follows: (1) such as Figure 2 As shown, multiple landmarks are evenly selected along the flight path of the UAV, and the geographical coordinates of each landmark are recorded; real-time image data captured by the UAV during flight is acquired, as well as the high-frequency GPS data and IMU attitude angle data of the UAV corresponding to each image data capture time; the image data containing landmarks is defined as the target image, and the original pixel coordinates of each landmark in the target image are recorded; The image data of the UAV includes a timestamp; the high-frequency GPS data includes a timestamp, as well as the longitude, latitude, and altitude data of the UAV at the time corresponding to the timestamp; the IMU attitude angle data includes a timestamp, as well as the pitch, roll, and heading data of the UAV at the time corresponding to the timestamp. In step (1), obtaining the high-frequency GPS data and IMU attitude angle data corresponding to the UAV when each image data is captured is specifically as follows: performing time synchronization processing on the image data, high-frequency GPS data and IMU attitude angle data, and aligning the time using the timestamps contained in the image data, high-frequency GPS data and IMU attitude angle data, thereby obtaining the high-frequency GPS data and IMU attitude angle data of the UAV when each image data is captured.

[0029] The step of uniformly selecting multiple landmarks along the UAV's flight path specifically involves selecting stable points on the ground along the UAV's flight path that have stable intersection characteristics as landmarks; the stable points include road intersections, building corners, and isolated trees.

[0030] Based on step (1), this invention directly uses known geographic coordinates on the ground as a reference, so no additional calibration equipment is needed. By minimizing the reprojection error, the calibration can be completed after the UAV has finished its normal flight, reducing calibration costs and complexity. At the same time, this invention utilizes the high-frequency GPS and IMU data already collected in the aerial photography mission, which can be used to evaluate the accuracy of completed aerial photography projects and correct external parameters, thereby improving the overall efficiency of data utilization.

[0031] (2) such as Figure 2 As shown, the geographic coordinates of each landmark are converted to northeast celestial coordinates; (3) such as Figure 4 As shown, a northeast-to-camera coordinate system transformation matrix is ​​constructed based on the extrinsic parameters of the camera and IMU, the IMU installation location, and the UAV attitude angle data. The specific process of constructing the northeast sky-camera coordinate system transformation matrix in step (3) is as follows: (3.1) Based on IMU attitude angle data, calculate the northeast-north-skin rotation matrix from the northeast-north-sky coordinate system to the body coordinate system. The calculation formula is:

[0032] in, This is the rotation matrix of the northeast-central coordinate system about the Z-axis; The rotation matrix of the heading coordinate system about the new Y-axis; This is the rotation matrix of the pitch coordinate system about the new X-axis; The calculation formula is:

[0033] in, The yaw angle of the UAV in the northeast celestial coordinate system; The calculation formula is:

[0034] in, The pitch angle of the UAV in the northeast celestial coordinate system; The calculation formula is:

[0035] in, The roll angle of the UAV in the northeast celestial coordinate system; In summary, the Northeast Sky-body rotation matrix The rotation sequence is yaw-pitch-roll, and the rotation angles are the aircraft's attitude angles, namely yaw angle, pitch angle, and roll angle. First, rotate the northeast celestial coordinate system to yaw. Adjust the angle so that its X-axis aligns with the "heading line" to obtain the heading coordinate system. Then rotate and pitch the heading coordinate system around its Y-axis. The angle is used to obtain the pitch coordinate system, and finally the pitch coordinate system is rotated and rolled around the X-axis. Angles can be aligned with the body coordinate system.

[0036] (3.2) Based on the extrinsic parameters of the IMU, the body-IMU rotation matrix is ​​obtained. The calculation formula is:

[0037] Machine-IMU Rotation Matrix This indicates the actual mounting location of the IMU on the unit; (3.3) Based on the camera's extrinsic parameters, obtain the IMU-camera rotation matrix. The calculation formula is:

[0038] in, This is the rotation matrix around the Z-axis; This is the rotation matrix around the new Y-axis; For the rotation matrix about the new X-axis; The calculation formula is:

[0039] in, It is the yaw angle of the camera in the IMU coordinate system. In terms of physical meaning, it is the projection of the camera's X-axis onto the horizontal plane from the X-axis of the IMU to the X-axis of the camera. Counterclockwise is positive (viewed from the positive direction of the Z-axis). The calculation formula is:

[0040] in, The pitch angle of the camera in the IMU coordinate system is the rotation around the rotated Y-axis, with pitch being positive (i.e., the camera rotates upward along the X-axis). The calculation formula is:

[0041] in, It represents the roll angle of the camera in the IMU coordinate system. In physical terms, it means the rotation around the rotated X-axis, with right roll being positive (i.e., the camera rotates to the right along the Y-axis). In summary, the IMU-camera rotation matrix The rotation from the IMU to the camera is counterclockwise, with the rotation sequence being yaw-pitch-roll. The IMU is mounted at the same origin by default, with no translation. First, it rotates around the IMU's Z-axis, projecting the IMU's X-axis onto the horizontal plane as the camera's X-axis. Then, it rotates around the rotated Y-axis to align the IMU's X-axis vertically with the camera's X-axis. Finally, it rotates around the rotated X-axis to make the two coordinate systems completely coincident.

[0042] (3.4) Based on the Northeast Sky-body Rotation Matrix Body-IMU rotation matrix and IMU-camera rotation matrix Construct the Northeast Sky-Camera Coordinate System Transformation Matrix The expression is .

[0043] (4) Convert the high-frequency GPS data corresponding to the UAV when the target image is captured into the northeast sky coordinates in the same coordinate system as the landmark; based on the high-frequency GPS data in the northeast sky coordinates, use the northeast sky-camera coordinate system transformation matrix to convert the northeast sky coordinates of each landmark into the predicted coordinates in the camera coordinate system. (5) By using the camera intrinsic parameter matrix, the predicted coordinates of each landmark are projected onto the image plane to obtain the predicted pixel coordinates of each landmark; (6) Based on the predicted pixel coordinates and original pixel coordinates of each landmark, construct the reprojection error of each landmark; (7) For example Figure 3 As shown, the sum of reprojection errors of all landmarks is used as the objective function. An iterative algorithm is employed to iteratively optimize the optimal extrinsic parameters of the camera and IMU. The optimal extrinsic parameters are then used to calibrate the camera and IMU. The calibration results are shown below. Figure 5 As shown; In step (7), the Levenberg-Marquardt algorithm is used to iteratively optimize the optimal extrinsic parameters of the camera and IMU. The Levenberg-Marquardt algorithm is terminated when the sum of the reprojection errors of all landmarks calculated in two consecutive iterations is less than a preset threshold or the number of iterations of the Levenberg-Marquardt algorithm reaches the maximum number of iterations.

[0044] Based on the above process, this invention can be broadly divided into two main parts, such as... Figure 6 As shown, this invention constructs and minimizes reprojection errors, iteratively solving for high-precision camera and IMU extrinsic parameters, thus achieving accurate calibration of strapdown cameras and IMUs. The entire process is simple to operate, has low implementation costs, and produces high-precision calibration results. Furthermore, this invention can be applied to different models of cameras and IMUs by adjusting the corresponding parameters, demonstrating good scalability.

[0045] Secondly, this invention also proposes an offline calibration system for UAV strapdown cameras and IMU extrinsic parameters based on landmark reprojection, including: a data acquisition module, a preprocessing module, a coordinate system transformation relationship construction module, a reprojection error calculation module, and an optimization calculation module; The data acquisition module uniformly selects multiple landmarks along the UAV's flight path and records the geographical coordinates of each landmark; it acquires real-time image data captured by the UAV during flight, as well as the high-frequency GPS data and IMU attitude angle data of the UAV corresponding to each image data capture time; the image data containing landmarks is defined as the target image, and the data acquisition module records the original pixel coordinates of each landmark in the target image; The preprocessing module converts the geographic coordinates of each landmark point into northeast-sky coordinates. The coordinate system transformation module constructs a northeast-sky-camera coordinate system transformation matrix based on the external parameters of the camera and IMU, the IMU installation position, and the UAV attitude angle data. At the same time, it transforms the high-frequency GPS data corresponding to the UAV when the target image is captured into northeast-sky coordinates in the same coordinate system as the landmark. Based on the high-frequency GPS data in the northeast-sky coordinates, the northeast-sky-camera coordinate system transformation matrix is ​​used to transform the northeast-sky coordinates of each landmark into predicted coordinates in the camera coordinate system. The reprojection error calculation module constructs the reprojection error of each landmark point based on the predicted pixel coordinates and the original pixel coordinates of each landmark point. The optimization calculation module uses the sum of reprojection errors of all landmarks as the optimization objective function, and uses an iterative algorithm to iteratively optimize the optimal extrinsic parameters of the camera and IMU. The optimal extrinsic parameters of the camera and IMU are then used to calibrate the camera and IMU.

[0046] Furthermore, the offline calibration system also includes a result output module; the result output module is used to visualize the results of the camera and IMU after calibration using optimal extrinsic parameters.

[0047] The parts of this invention not described in detail are common knowledge to those skilled in the art.

Claims

1. An offline calibration method for UAV strapdown cameras and IMU extrinsics based on landmark reprojection, characterized in that... include: Multiple landmarks are evenly selected along the drone's flight path, and the geographical coordinates of each landmark are recorded. Acquire real-time image data captured by the drone during flight, as well as the high-frequency GPS data and IMU attitude angle data of the drone at the time of each image capture; define image data containing landmarks as target images, and record the original pixel coordinates of each landmark in the target image; The geographic coordinates of each landmark are converted to northeast-sky coordinates. Based on the extrinsic parameters of the camera and IMU, the IMU installation location, and the UAV attitude angle data, a transformation matrix between the northeast sky and the camera coordinate system is constructed. The high-frequency GPS data corresponding to the drone when the target image was captured is converted into northeast sky coordinates in the same coordinate system as the landmark; based on the high-frequency GPS data in the northeast sky coordinates, the northeast sky-camera coordinate system transformation matrix is ​​used to convert the northeast sky coordinates of each landmark into the predicted coordinates in the camera coordinate system. By using the camera intrinsic parameter matrix, the predicted coordinates of each landmark are projected onto the image plane to obtain the predicted pixel coordinates of each landmark. Based on the predicted pixel coordinates and the original pixel coordinates of each landmark, the reprojection error of each landmark is constructed. Using the sum of reprojection errors of all landmarks as the objective function, an iterative algorithm is used to iteratively optimize the optimal extrinsic parameters of the camera and IMU, and the camera and IMU are calibrated using the optimal extrinsic parameters.

2. The method for offline calibration of UAV strapdown camera and IMU extrinsic parameters based on landmark reprojection according to claim 1, characterized in that: The image data of the UAV includes a timestamp; the high-frequency GPS data includes a timestamp, as well as the longitude, latitude, and altitude data of the UAV at the time corresponding to the timestamp; the IMU attitude angle data includes a timestamp, as well as the pitch, roll, and heading data of the UAV at the time corresponding to the timestamp. The specific steps for obtaining the high-frequency GPS data and IMU attitude angle data corresponding to the UAV at the time of each image data capture are as follows: Time synchronization processing is performed on the image data, high-frequency GPS data, and IMU attitude angle data, and time alignment is performed using the timestamps contained in the image data, high-frequency GPS data, and IMU attitude angle data, thereby obtaining the high-frequency GPS data and IMU attitude angle data of the UAV at the time of each image data capture.

3. The method for offline calibration of UAV strapdown camera and IMU extrinsic parameters based on landmark reprojection according to claim 1, characterized in that: The step of uniformly selecting multiple landmarks along the UAV's flight path specifically involves selecting stable points on the ground along the UAV's flight path that have stable intersection characteristics as landmarks; the stable points include road intersections, building corners, and isolated trees.

4. The method for offline calibration of UAV strapdown camera and IMU extrinsic parameters based on landmark reprojection according to claim 1, characterized in that: The specific process for constructing the northeast sky-camera coordinate system transformation matrix is ​​as follows: (3.1) Based on IMU attitude angle data, calculate the northeast-north-skin rotation matrix from the northeast-north-sky coordinate system to the body coordinate system. ; (3.2) Based on the extrinsic parameters of the IMU, the body-IMU rotation matrix is ​​obtained. ; (3.3) Based on the camera's extrinsic parameters, obtain the IMU-camera rotation matrix. ; (3.4) Based on the Northeast Sky-body Rotation Matrix Body-IMU rotation matrix and IMU-camera rotation matrix Construct the Northeast Sky-Camera Coordinate System Transformation Matrix The expression is 。 5. The method for offline calibration of UAV strapdown camera and IMU extrinsic parameters based on landmark reprojection according to claim 4, characterized in that: In step (3.1), the northeast sky-body rotation matrix The calculation formula is: in, This is the rotation matrix of the northeast-central coordinate system about the Z-axis; The rotation matrix of the heading coordinate system about the new Y-axis; This is the rotation matrix of the pitch coordinate system about the new X-axis; The calculation formula is: in, The yaw angle of the UAV in the northeast celestial coordinate system; The calculation formula is: in, The pitch angle of the UAV in the northeast celestial coordinate system; The calculation formula is: in, The roll angle of the UAV in the northeast celestial coordinate system; In step (3.2), the body-IMU rotation matrix The calculation formula is In step (3.3), the IMU-camera rotation matrix The calculation formula is in, This is the rotation matrix around the Z-axis; This is the rotation matrix around the new Y-axis; For the rotation matrix about the new X-axis; The calculation formula is: in, The yaw angle of the camera in the IMU coordinate system; The calculation formula is: in, The camera's pitch angle in the IMU coordinate system; The calculation formula is: in, This is the roll angle of the camera in the IMU coordinate system.

6. The method for offline calibration of UAV strapdown camera and IMU extrinsic parameters based on landmark reprojection according to claim 1, characterized in that: The optimal extrinsic parameters of the camera and IMU are optimized iteratively using the Levenberg-Marquardt algorithm. The Levenberg-Marquardt algorithm is terminated when the sum of the reprojection errors of all landmarks calculated in two consecutive iterations is less than a preset threshold or the number of iterations of the Levenberg-Marquardt algorithm reaches the maximum number of iterations.

7. An offline calibration system for UAV strapdown cameras and IMU extrinsic parameters based on landmark reprojection, characterized in that... include: The module includes a data acquisition module, a preprocessing module, a coordinate system transformation relationship construction module, a reprojection error calculation module, and an optimization calculation module. The data acquisition module uniformly selects multiple landmarks along the UAV's flight path and records the geographical coordinates of each landmark; it acquires real-time image data captured by the UAV during flight, as well as the high-frequency GPS data and IMU attitude angle data of the UAV corresponding to each image data capture time; the image data containing landmarks is defined as the target image, and the data acquisition module records the original pixel coordinates of each landmark in the target image; The preprocessing module converts the geographic coordinates of each landmark point into northeast-sky coordinates. The coordinate system transformation module constructs a northeast-sky-camera coordinate system transformation matrix based on the external parameters of the camera and IMU, the IMU installation position, and the UAV attitude angle data. At the same time, it transforms the high-frequency GPS data corresponding to the UAV when the target image is captured into northeast-sky coordinates in the same coordinate system as the landmark. Based on the high-frequency GPS data in the northeast-sky coordinates, the northeast-sky-camera coordinate system transformation matrix is ​​used to transform the northeast-sky coordinates of each landmark into predicted coordinates in the camera coordinate system. The reprojection error calculation module constructs the reprojection error of each landmark point based on the predicted pixel coordinates and the original pixel coordinates of each landmark point. The optimization calculation module uses the sum of reprojection errors of all landmarks as the optimization objective function, and uses an iterative algorithm to iteratively optimize the optimal extrinsic parameters of the camera and IMU. The optimal extrinsic parameters of the camera and IMU are then used to calibrate the camera and IMU.

8. The offline calibration system for UAV strapdown camera and IMU extrinsic parameters based on landmark reprojection according to claim 7, characterized in that: The offline calibration system also includes a result output module; the result output module is used to visualize the results of the camera and IMU after calibration using optimal extrinsic parameters.

Images