Automatic Camera Intrinsic Parameter Calibration Method, System, and Electronic Equipment Based on UAVs

Abstract

This invention discloses an automatic camera intrinsic parameter calibration method, system, and electronic device based on a drone, relating to the field of camera parameter calibration technology. The method includes: setting up a calibration scene based on the size information of a checkerboard calibration board and the camera's field of view; moving a target drone along a preset trajectory within the calibration scene to obtain the target drone's pose data and the camera's photographic data of the checkerboard calibration board; wherein the target drone is a drone with the checkerboard calibration board fixed in place; the preset trajectory is determined based on camera intrinsic parameter errors and an iterative optimization algorithm; and calibrating the camera's intrinsic parameters based on the installation position of the checkerboard calibration board relative to the drone, the camera's installation position in the world coordinate system, the target drone's pose data, and the camera's photographic data of the checkerboard calibration board. This invention solves the problem of requiring manual intervention during the calibration of large batches of camera intrinsic parameters.

Description

Technical Field

[0001] This invention relates to the field of camera parameter calibration technology, and in particular to an automatic calibration method, system and electronic device for camera intrinsic parameters based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Current camera intrinsic parameter calibration methods are based on detecting and matching corner points on a checkerboard pattern, estimating the camera's intrinsic and distortion parameters by minimizing reprojection error. First, the checkerboard pattern on the calibration board is captured in the calibration image, and corner points are detected. Corner detection algorithms are typically based on feature point detection methods, such as Harris corner detection or Shi-Tomasi corner detection. The detected corner points are then matched with theoretical corner points on the calibration board. The geometry and known dimensions of the calibration board make corner matching possible, and the camera's intrinsic and distortion parameters are estimated by minimizing the reprojection error. The reprojection error refers to the difference between projecting the theoretical corner points back to the image plane using calibration parameters and calculating this difference from the actually detected corner points. Nonlinear optimization algorithms (such as the Levenberg-Marquardt algorithm) are used to iteratively adjust the intrinsic and distortion parameters to minimize the reprojection error.

[0003] During calibration, the need for a person to hold the calibration board and capture calibration images introduces instability in the board's position and angle. Since corner detection and matching rely on the accurate position of the calibration board, this instability introduces uncertainty and error into corner location. Furthermore, the relative posture between the camera and the calibration board can change when held by hand. Factors such as hand tremors, rotation or tilting of the calibration board, and its placement can all cause posture variations. These posture changes affect corner detection and matching, thus impacting the estimation results of intrinsic parameters and distortion parameters. Additionally, the need for manual adjustment of the calibration board's position and angle during handheld calibration, especially for large-scale camera production, necessitates significant manpower and time for intrinsic parameter calibration. These issues increase the complexity and time-consuming nature of calibration, limiting its efficiency and accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and electronic device for automatic calibration of camera intrinsic parameters based on unmanned aerial vehicles (UAVs), which solves the problem that human intervention is required during the calibration of a large number of camera intrinsic parameters.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] In a first aspect, the present invention provides an automatic camera intrinsic parameter calibration method based on a UAV, comprising:

[0007] Determine the dimensions of the checkerboard calibration plate, its installation position relative to the UAV, and the camera's installation position in the world coordinate system;

[0008] Based on the size information of the checkerboard calibration board and the field of view information of the camera, the calibration scene is set up;

[0009] In the calibration scenario, the target drone is moved along a preset trajectory to obtain the pose data of the target drone and the photo data of the checkerboard calibration board captured by the camera; wherein, the target drone is a drone with the checkerboard calibration board fixed; the preset trajectory is determined based on the camera intrinsic error and iterative optimization algorithm;

[0010] Based on the installation position of the checkerboard calibration board relative to the UAV, the installation position of the camera in the world coordinate system, the pose data of the target UAV, and the photo data of the checkerboard calibration board taken by the camera, the camera's intrinsic parameters are calibrated.

[0011] Optionally, based on the size information of the checkerboard calibration board and the camera's field of view, a calibration scene is arranged, specifically including:

[0012] Based on the size information of the checkerboard calibration plate and the field of view information of the camera, the placement range of the target drone is determined; the checkerboard calibration plate has a flat and non-reflective surface and is rigidly connected to the drone.

[0013] Optionally, based on the installation position of the checkerboard calibration board relative to the UAV, the installation position of the camera in the world coordinate system, the pose data of the target UAV, and the photo data of the checkerboard calibration board captured by the camera, the camera's intrinsic parameters are calibrated, specifically including:

[0014] Based on the installation position of the checkerboard calibration board relative to the UAV, the pose data of the target UAV, and the photo data of the checkerboard calibration board taken by the camera, the coordinates of the checkerboard corner points in the pixel coordinate system and the coordinates in the world coordinate system are determined.

[0015] Based on the camera's installation position in the world coordinate system and the target UAV's pose data, determine the target UAV's rotation matrix and translation vector relative to the camera;

[0016] Based on the rotation matrix and translation vector of the target UAV relative to the camera, the coordinate relationship between the checkerboard corner points in the pixel coordinate system and the world coordinate system, and the coordinates of the checkerboard corner points in the pixel coordinate system and the world coordinate system, calculate the camera's intrinsic parameter matrix and distortion parameters.

[0017] Optionally, based on the installation position of the checkerboard calibration board relative to the UAV, the pose data of the target UAV, and the photo data of the checkerboard calibration board captured by the camera, the coordinates of the checkerboard corner points in the pixel coordinate system and the coordinates in the world coordinate system are determined, specifically including:

[0018] Synchronize the photo data of the checkerboard calibration board captured by the camera with the pose data of the drone in time;

[0019] Based on the time-synchronized photo data, determine the coordinates of the checkerboard corner points in the pixel coordinate system;

[0020] Based on the pose data after time synchronization and the installation position of the checkerboard calibration board relative to the UAV, the coordinates of each corner point in the checkerboard calibration board in the world coordinate system are determined.

[0021] Optionally, the preset trajectory determination process is as follows:

[0022] Obtain the trajectory of the target drone determined by the current iteration number;

[0023] Based on the trajectory of the target UAV determined by the current iteration number, determine the camera in-camera parameters determined by the current iteration number, and based on the camera in-camera parameters determined by the current iteration number, determine the camera in-camera parameter error for the current iteration number.

[0024] Compare the camera intrinsic error obtained in the current iteration with the camera intrinsic error obtained in the previous iteration.

[0025] If the camera intrinsic error obtained in the current iteration is better than the camera intrinsic error obtained in the previous iteration, then the trajectory of the target UAV determined in the current iteration is retained.

[0026] If the camera intrinsic error obtained in the previous iteration is better than the camera intrinsic error obtained in the current iteration, then the trajectory of the target UAV determined in the previous iteration is retained.

[0027] Determine if the current iteration count has reached the preset iteration count;

[0028] If the current iteration count has not reached the preset iteration count, then increment the current iteration count by 1, update the trajectory of the target UAV determined by the current iteration count, determine the trajectory of the target UAV retained by the current iteration count as the trajectory of the target UAV determined by the previous iteration count, and determine the camera intrinsic parameter error corresponding to the trajectory of the target UAV retained by the current iteration count as the camera intrinsic parameter error obtained by the previous iteration count. Then return to the steps of determining the camera intrinsic parameters determined by the current iteration count based on the trajectory of the target UAV determined by the current iteration count, and determining the camera intrinsic parameter error of the current iteration count based on the camera intrinsic parameters determined by the current iteration count.

[0029] If the current iteration count reaches the preset iteration count, the iterative optimization process ends, and the trajectory of the target UAV retained in the current iteration count is determined as the preset trajectory.

[0030] Optionally, the process of determining the trajectory of the target UAV is as follows:

[0031] Based on the pose data of the target UAV, combined with the camera's field of view constraints and front-to-back distance constraints, the trajectory of the target UAV is determined.

[0032] Secondly, the present invention provides an automatic camera intrinsic parameter calibration system based on a UAV, comprising:

[0033] The information acquisition module is used to determine the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the UAV, and the installation position of the camera in the world coordinate system.

[0034] The calibration scene setup module is used to set up the calibration scene based on the size information of the checkerboard calibration board and the field of view information of the camera;

[0035] The pose data and photo data determination module is used to move the target drone along a preset trajectory in the calibration scene to obtain the pose data of the target drone and the photo data of the checkerboard calibration board captured by the camera; wherein, the target drone is a drone with the checkerboard calibration board fixed; the preset trajectory is determined based on the camera intrinsic error and iterative optimization algorithm;

[0036] The calibration module is used to calibrate the camera's intrinsic parameters based on the installation position of the checkerboard calibration board relative to the UAV, the camera's installation position in the world coordinate system, the pose data of the target UAV, and the photo data of the checkerboard calibration board taken by the camera.

[0037] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the automatic camera intrinsic parameter calibration method based on a UAV as described in the first aspect.

[0038] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0039] To address the shortcomings of traditional camera intrinsic parameter calibration, this invention proposes an automatic camera intrinsic parameter calibration method, system, and electronic equipment based on a UAV. This invention achieves automatic camera intrinsic parameter calibration by using a UAV carrying a calibration board, thereby improving calibration efficiency, reducing errors, and saving manpower. It also introduces iterative optimization of the UAV's route planning. By continuously optimizing the UAV's path planning, the calibration board is positioned in the optimal pose for maximizing camera intrinsic parameter calibration accuracy and efficiency. This minimizes calibration errors, improves the accuracy of camera intrinsic parameters, and ultimately achieves high-precision, high-efficiency, large-scale camera intrinsic parameter calibration. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This invention provides an automatic camera intrinsic parameter calibration method based on an unmanned aerial vehicle (UAV).

[0042] Figure 2 This is a schematic diagram of a calibration scenario provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the internal parameter calibration data acquisition process provided in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the camera intrinsic parameter calibration data calculation process provided in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of the drone trajectory optimization process provided in an embodiment of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] Example 1

[0049] When a LiDAR-based perception system is in operation, it is necessary to calibrate the camera's focal length, principal point position, distortion parameters, etc., i.e., the intrinsic parameters (camera intrinsic parameter matrix: focal length f, physical lengths dx and dy of a pixel in the x and y directions on the camera's photosensitive plate, coordinates u0 and v0 of the center of the camera's photosensitive plate in the pixel coordinate system; camera distortion parameters: radial distortion parameters k1, k2, and k3, tangential distortion parameters p1 and p2). Currently, camera intrinsic parameter calibration is quite mature, and the Zhang Zhengyou calibration method is generally used. The Zhang Zhengyou calibration method involves first printing a template and pasting it onto a plane, then taking several images of the template from different angles, detecting feature points in the images, solving for the camera's intrinsic and extrinsic parameters under ideal distortion-free conditions, and using maximum likelihood estimation to improve accuracy. Then, the least squares method is applied to calculate the actual radial distortion coefficients. Combining the intrinsic, extrinsic, and distortion coefficients, the maximum likelihood method is used to optimize the estimation, improving the estimation accuracy, and finally, the camera intrinsic parameter matrix and camera distortion parameters are obtained. However, the calibration process requires manual movement and rotation of the calibration plate. Especially when the accuracy requirements for camera intrinsic parameter calibration are very high, a large-size calibration plate can generally be used to improve the accuracy of intrinsic parameter calibration. In this case, it is time-consuming and labor-intensive to manually move the calibration plate. Furthermore, when calibrating the intrinsic parameters of a large number of cameras, manually placing and moving the calibration plate is also time-consuming and labor-intensive.

[0050] To address the aforementioned issues, this embodiment provides an automatic camera intrinsic parameter calibration method based on unmanned aerial vehicles (UAVs), comprising the following steps.

[0051] Step 100: Determine the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the UAV, and the installation position of the camera in the world coordinate system.

[0052] In this embodiment, it is first necessary to confirm the size of the checkerboard calibration plate, such as the size of each square and the specifications of the checkerboard corner points. Secondly, it is necessary to confirm the installation position of the checkerboard calibration plate relative to the UAV, such as rotation and translation transformations. Finally, it is necessary to confirm the installation position of the camera in the world coordinate system.

[0053] Step 200: Set up the calibration scene based on the size information of the checkerboard calibration board and the field of view information of the camera.

[0054] In this embodiment, as Figure 2 As shown, place the drone (i.e., the target drone) with the fixed checkerboard calibration plate within a range of 0.5 to 1.5 meters in front of the fixed camera (depending on the size of the selected checkerboard calibration plate and the field of view of the camera). The checkerboard calibration plate must have a flat surface, be free of bending, and be non-reflective to ensure that the checkerboard pattern on the calibration plate is clearly and completely imaged in the camera. The checkerboard calibration plate must be rigidly connected to the drone to ensure that there is no relative displacement or rotation during movement.

[0055] Step 300: In the calibration scenario, the target drone is moved along a preset trajectory to obtain the pose data of the target drone required for calibration and the photo data of the checkerboard calibration board taken by the camera; the target drone is a drone with the checkerboard calibration board fixed.

[0056] In this embodiment, after the calibration scene is set up, raw data is acquired, such as... Figure 3 As shown. The target drone is guided within the camera's field of view (FOV) to move at a low speed (less than 1 m / s) along a preset trajectory to acquire the target drone's pose data and the camera's photographs of the checkerboard calibration board required for calibration. During this process, it is crucial to ensure the calibration program can detect a sufficient number of checkerboard corner points in the x, y, and z directions, and that these corner points form a relatively horizontal rectangular network. If these conditions are not met, a new motion trajectory needs to be planned and data re-acquired to prevent excessive image capture at the same camera location, which could increase errors.

[0057] Step 400: Based on the installation position of the checkerboard calibration board relative to the UAV, the installation position of the camera in the world coordinate system, the pose data of the target UAV, and the photo data of the checkerboard calibration board taken by the camera, calibrate the camera's intrinsic parameters.

[0058] In this embodiment, as Figure 4 As shown, step 400 specifically includes:

[0059] Step 1: Based on the installation position of the checkerboard calibration board relative to the UAV, the pose data of the target UAV, and the photo data of the checkerboard calibration board taken by the camera, determine the coordinates of the checkerboard corner points in the pixel coordinate system and the coordinates in the world coordinate system. The detailed process is as follows.

[0060] 1) Data synchronization: The photo data of the chessboard calibration board captured by the camera and the pose data of the UAV are synchronized in time.

[0061] 2) Based on the time-synchronized photo data, determine the coordinates (u, v) of the corner points of the chessboard in the pixel coordinate system. Based on the time-synchronized pose data and the installation position of the chessboard calibration board relative to the UAV, determine the coordinates (X, Y, Z) of each corner point in the chessboard calibration board in the world coordinate system.

[0062] The second step is to determine the rotation matrix and translation vector (R, T) of the target UAV relative to the camera, based on the camera's installation position in the world coordinate system and the target UAV's pose data. Here, R represents the rotation matrix and T represents the translation vector.

[0063] The third step involves calculating the camera's intrinsic parameter matrix A and distortion parameters based on the target UAV's rotation matrix and translation vector relative to the camera, the coordinate relationship between the checkerboard corner points in the pixel coordinate system and the world coordinate system, and the coordinates of the checkerboard corner points in the pixel coordinate system and the world coordinate system.

[0064] The single-point distortion-free camera imaging model is as follows:

[0065]

[0066] In the above formula, (X, Y, Z) are the physical coordinates of a point in the world coordinate system, (u, v) are the pixel coordinates of the corresponding pixel coordinate system, and Z is the scale factor.

[0067] The intrinsic parameter matrix of the camera depends on the camera's internal parameters, where f is the image distance, dx and dy represent the physical length of a pixel on the camera's image sensor in the x and y directions, respectively (i.e., how many millimeters a pixel is on the image sensor), u0 and v0 represent the coordinates of the center of the image sensor in the pixel coordinate system, and θ represents the angle between the horizontal and vertical edges of the image sensor. This is called the camera's extrinsic parameter matrix.

[0068] Fixing the world coordinate system on the chessboard grid, the physical coordinates of any point on the chessboard grid are Z=0. Let the intrinsic parameter matrix be A, and R1 and R2 be the first two columns of the rotation matrix R. Therefore, the original single-point distortion-free imaging model can be written as:

[0069]

[0070] For different images, the intrinsic parameter matrix A is a constant. For the same image, the intrinsic parameter matrix A and the extrinsic parameter matrix (R1, R2, T) are constants. For a single point on the same image, the intrinsic parameter matrix A, the extrinsic parameter matrix (R1, R2, T), and the scale factor Z are constants. Let A(R1, R2, T) be denoted as matrix H, where H is the product of the intrinsic and extrinsic parameter matrices. Let the three columns of matrix H be (H1, H2, H3), then we have:

[0071]

[0072] From the above formula, we can obtain:

[0073]

[0074] In the above formula, X and Y can be output by the UAV, while u and v can be output by the camera, thus allowing us to obtain matrix H, where matrix H = A(R1, R2, T), and R1, R2, and T are the relevant data output by the UAV, thus yielding the camera intrinsic parameter matrix. Typically, 15 to 20 images of the chessboard calibration board are captured, and the least squares method is generally used for fitting to obtain the optimal intrinsic parameter matrix A.

[0075] The formulas for calculating radial distortion parameters k1, k2, k3 and tangential distortion parameters p1, p2 (3rd order) are as follows:

[0076]

[0077] The formula for tangential distortion is as follows:

[0078]

[0079] Where (x0, y0) are the ideal distortion-free normalized image coordinates, which can be obtained from the coordinates of the calibration board output by the target UAV and then derived from the original single-point distortion-free imaging model. (x, y) are the distorted normalized image coordinates, which can be obtained directly from the coordinates of the corner points output by the camera, i.e. (x0, y0). Since (x, y) are known, and r2 = x2 + y2, the radial distortion parameters k1, k2, k3 and the tangential distortion parameters p1, p2 can be obtained.

[0080] Fourth step: Through the second and third steps above, all camera intrinsic parameters (camera intrinsic parameter matrix: focal length f, physical length dx, dy of a pixel in the x and y directions on the camera sensor, coordinates u0, v0 of the center of the camera sensor in the pixel coordinate system; camera distortion parameters: radial distortion parameters k1, k2, k3, tangential distortion parameters p1, p2) have been calculated, and calibration is complete.

[0081] Since the pose of the checkerboard calibration board has a significant impact on the calibration results during the camera intrinsic parameter calibration process, we optimize the optimal pose of the checkerboard calibration board by iteratively optimizing the UAV planning path, so that the large-scale camera intrinsic parameter calibration can achieve the effect of low error and high efficiency.

[0082] like Figure 5 As shown, the preset number of iterations is first set to N. In each iteration, the target UAV performs camera intrinsic parameter calibration according to the planned trajectory, and performs camera intrinsic parameter calibration error analysis.

[0083] Specifically, the process involves: obtaining the trajectory of the target drone determined by the current iteration number; determining the camera's intrinsic parameters based on the trajectory, and determining the camera's intrinsic parameter error for the current iteration number; comparing the camera's intrinsic parameter error obtained in the current iteration number with that obtained in the previous iteration number; retaining the trajectory of the target drone determined by the current iteration number if the current iteration number is better than the previous iteration number; and retaining the trajectory of the target drone determined by the previous iteration number if the previous iteration number is better than the current iteration number. Finally, determining whether the current iteration number has reached the preset iteration number. If the previous iteration count has not reached the preset iteration count, the current iteration count is incremented by 1, and the trajectory of the target UAV determined by the current iteration count is updated. The trajectory of the target UAV retained in the current iteration count is determined as the trajectory of the target UAV determined in the previous iteration count, and the camera intrinsic parameter error corresponding to the trajectory of the target UAV retained in the current iteration count is determined as the camera intrinsic parameter error obtained in the previous iteration count. The process returns to the steps of determining the camera intrinsic parameters determined by the current iteration count based on the trajectory of the target UAV determined by the current iteration count, and determining the camera intrinsic parameter error of the current iteration count based on the camera intrinsic parameters determined by the current iteration count. If the current iteration count reaches the preset iteration count, the iterative optimization process ends, and the trajectory of the target UAV retained in the current iteration count is determined as the preset trajectory.

[0084] By setting a preset number of iterations and an optimized trajectory selection mechanism, this embodiment can automatically select a better target UAV trajectory in each iteration to maximize the improvement of camera intrinsic parameter calibration error. Through continuous execution of the iterative optimization process, the accuracy and precision of camera intrinsic parameter calibration can be gradually improved. Therefore, this invention has high efficiency and accuracy in large-scale camera intrinsic parameter calibration tasks, while also reducing labor costs.

[0085] The iterative optimization process also involves the UAV trajectory output part. In order to achieve constrained trajectory planning and ensure that the chessboard calibration board does not fly out of the camera's field of view and meets the front and rear distance requirements, this embodiment will determine the target UAV's travel trajectory based on the target UAV's pose data, combined with the camera's field of view (FOV) constraints and front and rear distance constraints.

[0086] The goal of trajectory planning is to ensure that the target UAV does not repeat its pose during flight and to satisfy the constraints.

[0087] First, the space surrounding the target drone is divided into 27 small squares, including the square containing the drone itself. Then, numbers between 0 and 1 are randomly generated from the 26 squares surrounding the drone. The target square for the drone's next flight is determined based on the magnitude of these random numbers. This method allows for a degree of randomness in trajectory planning, increasing trajectory diversity. Furthermore, considering variations in pitch, roll, and yaw angles, numbers are used to represent the drone's attitude in these three directions. For example, the number 452034 represents a pitch angle of 45 degrees, a roll angle of 20 degrees, and a yaw angle of 34 degrees. Random generation in each direction allows for variations in the drone's attitude within a certain range, further increasing trajectory diversity. By combining camera field-of-view constraints and forward / backward distance constraints, the drone's flight area is confined to a specific region, ensuring the checkerboard calibration board remains within the camera's field of view and avoiding situations where it is too close or too far away, thus guaranteeing the accuracy of corner point identification.

[0088] Example 2

[0089] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, an automatic camera intrinsic parameter calibration system based on UAV is provided below.

[0090] This embodiment provides an automatic camera intrinsic parameter calibration system based on a drone, including:

[0091] The information acquisition module is used to determine the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the UAV, and the installation position of the camera in the world coordinate system.

[0092] The calibration scene setup module is used to set up the calibration scene based on the size information of the checkerboard calibration board and the field of view information of the camera.

[0093] The pose data and photo data determination module is used to move the target drone along a preset trajectory in the calibration scene to obtain the pose data of the target drone and the photo data of the checkerboard calibration board captured by the camera. The target drone is a drone with the checkerboard calibration board fixed. The preset trajectory is determined based on the camera intrinsic error and iterative optimization algorithm.

[0094] The calibration module is used to calibrate the camera's intrinsic parameters based on the installation position of the checkerboard calibration board relative to the UAV, the camera's installation position in the world coordinate system, the pose data of the target UAV, and the photo data of the checkerboard calibration board taken by the camera.

[0095] Example 3

[0096] This invention provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to execute an automatic camera intrinsic parameter calibration method based on a drone, as described in Embodiment 1.

[0097] Alternatively, the aforementioned electronic device may be a server.

[0098] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements an automatic camera intrinsic parameter calibration method based on a UAV according to Embodiment 1.

[0099] Compared with the prior art, the innovation of this invention is as follows:

[0100] 1. This invention establishes an automatic camera intrinsic parameter calibration method using a drone carrying a calibration board. 2. This invention implements a process and model for converting the corner coordinates on the calibration board from the world coordinate system to the pixel coordinate system using data output by the drone and the relative position of the calibration board and the drone. 3. This invention uses an iterative optimization method to optimize the drone trajectory to improve the accuracy of camera intrinsic parameter calibration. 4. The drone retrieval and deployment methods used in this invention are simple and require no human intervention, saving manpower and improving efficiency.

[0101] Compared with the prior art, the advantages of the present invention are as follows:

[0102] 1. Traditional camera intrinsic parameter calibration methods require manual intervention, and the calibration process is cumbersome and time-consuming. This invention, however, utilizes a drone carrying a calibration board to automate the calibration process, eliminating the need for human intervention, saving significant manpower, and greatly shortening calibration time, thereby improving calibration efficiency.

[0103] 2. This invention utilizes the attitude and position information output by the UAV to calculate the position and attitude of the calibration board in the camera, ensuring the accuracy of corner points. By iteratively optimizing the UAV's planned route, the optimal trajectory can be selected to place the calibration board in the optimal position and attitude, thereby further improving the accuracy of calibration.

[0104] 3. This invention utilizes a drone carrying a calibration board for automated calibration, eliminating the need for manual operation. By iteratively optimizing the drone's route planning, the process of calibrating large batches of camera intrinsic parameters is automated. This automation saves manpower and improves calibration efficiency and batch processing capabilities.

[0105] 4. Compared to traditional camera intrinsic parameter calibration methods, this invention simplifies the calibration process by using a drone to retrieve and deploy the calibration board. No human intervention is required during the drone's carrying, deployment, and retrieval of the calibration board, reducing labor costs and time consumption. This simplified process significantly improves calibration efficiency and is particularly suitable for large-scale camera intrinsic parameter calibration tasks.

[0106] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0107] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for automatic calibration of camera intrinsic parameters based on unmanned aerial vehicles (UAVs), characterized in that, include: Determine the dimensions of the checkerboard calibration plate, its installation position relative to the UAV, and the camera's installation position in the world coordinate system; Based on the size information of the checkerboard calibration board and the field of view information of the camera, the calibration scene is set up; In the calibration scenario, the target drone moves along a preset trajectory to obtain the pose data of the target drone and the photo data of the checkerboard calibration board captured by the camera. The target drone is a drone with the checkerboard calibration board fixed in place. The preset trajectory is determined based on camera intrinsic error and an iterative optimization algorithm, specifically including: obtaining the target drone's trajectory determined by the current iteration number; determining the camera intrinsic parameters determined by the current iteration number based on the target drone's trajectory, and determining the camera intrinsic parameter error for the current iteration number based on the camera intrinsic parameters determined by the current iteration number; comparing the camera intrinsic parameter error obtained by the current iteration number with the camera intrinsic parameter error obtained by the previous iteration number; if the camera intrinsic parameter error obtained by the current iteration number is better than the camera intrinsic parameter error obtained by the previous iteration number, then the target drone's trajectory determined by the current iteration number is retained; if the camera intrinsic parameter error obtained by the previous iteration number is better than the current iteration number... If the camera intrinsic error is obtained from the current iteration, the trajectory of the target UAV determined in the previous iteration is retained. It is then determined whether the current iteration count has reached the preset iteration count. If the current iteration count has not reached the preset iteration count, the current iteration count is incremented by 1, and the trajectory of the target UAV determined in the current iteration count is updated. The trajectory of the target UAV retained in the current iteration count is determined as the trajectory of the target UAV determined in the previous iteration count, and the camera intrinsic error corresponding to the trajectory of the target UAV retained in the current iteration count is determined as the camera intrinsic error obtained in the previous iteration count. The process then returns to the steps of determining the camera intrinsic parameters determined in the current iteration count based on the trajectory of the target UAV determined in the current iteration count, and determining the camera intrinsic parameter error of the current iteration count based on the camera intrinsic parameters determined in the current iteration count. If the current iteration count has reached the preset iteration count, the iterative optimization process ends, and the trajectory of the target UAV retained in the current iteration count is determined as the preset trajectory. Based on the installation position of the checkerboard calibration board relative to the UAV, the camera's installation position in the world coordinate system, the target UAV's pose data, and the photo data of the checkerboard calibration board captured by the camera, the camera's internal parameters are calibrated. Specifically, this includes: determining the coordinates of the checkerboard corner points in the pixel coordinate system and in the world coordinate system based on the installation position of the checkerboard calibration board relative to the UAV, the target UAV's pose data, and the photo data of the checkerboard calibration board captured by the camera; synchronizing the photo data of the checkerboard calibration board captured by the camera with the UAV's pose data in time; determining the coordinates of the checkerboard corner points in the pixel coordinate system based on the time-synchronized photo data; and determining the coordinates of each corner point in the checkerboard calibration board in the world coordinate system based on the time-synchronized pose data and the installation position of the checkerboard calibration board relative to the UAV. Based on the camera's installation position in the world coordinate system and the target UAV's pose data, determine the target UAV's rotation matrix and translation vector relative to the camera; Based on the rotation matrix and translation vector of the target UAV relative to the camera, the coordinate relationship between the checkerboard corner points in the pixel coordinate system and the world coordinate system, and the coordinates of the checkerboard corner points in the pixel coordinate system and the world coordinate system, calculate the camera's intrinsic parameter matrix and distortion parameters.

2. The automatic camera intrinsic parameter calibration method based on a UAV according to claim 1, characterized in that, Based on the size information of the checkerboard calibration board and the camera's field of view, the calibration scene is set up, specifically including: Based on the size information of the checkerboard calibration plate and the field of view information of the camera, the placement range of the target drone is determined; the checkerboard calibration plate has a flat and non-reflective surface and is rigidly connected to the drone.

3. The automatic camera intrinsic parameter calibration method based on a UAV according to claim 1, characterized in that, The process of determining the trajectory of the target drone is as follows: Based on the pose data of the target UAV, combined with the camera's field of view constraints and front-to-back distance constraints, the trajectory of the target UAV is determined.

4. An automatic camera intrinsic parameter calibration system based on unmanned aerial vehicles (UAVs), characterized in that, include: The information acquisition module is used to determine the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the UAV, and the installation position of the camera in the world coordinate system. The calibration scene setup module is used to set up the calibration scene based on the size information of the checkerboard calibration board and the field of view information of the camera; The pose data and image data determination module is used to move a target UAV along a preset trajectory in a calibration scene to obtain the pose data of the target UAV and the image data of the checkerboard calibration board captured by the camera. The target UAV is a UAV with the checkerboard calibration board fixed in place. The preset trajectory is determined based on camera intrinsic error and an iterative optimization algorithm, specifically including: obtaining the trajectory of the target UAV determined by the current iteration number; determining the camera intrinsic parameters determined by the current iteration number based on the trajectory of the target UAV determined by the current iteration number, and determining the camera intrinsic parameter error for the current iteration number based on the camera intrinsic parameters determined by the current iteration number; comparing the camera intrinsic parameter error obtained by the current iteration number with the camera intrinsic parameter error obtained by the previous iteration number; if the camera intrinsic parameter error obtained by the current iteration number is better than the camera intrinsic parameter error obtained by the previous iteration number, then retaining the trajectory of the target UAV determined by the current iteration number; if the camera intrinsic parameter error obtained by the previous iteration number is better than the previous iteration number, then retaining the trajectory of the target UAV determined by the current iteration number; if the camera intrinsic parameter error obtained by the previous iteration number is better than the previous iteration number, then retaining the trajectory of the target UAV determined by the current iteration number. If the difference between the current iteration number and the camera intrinsic error obtained in the current iteration number is better than that obtained in the current iteration number, then the trajectory of the target UAV determined in the previous iteration number is retained. It is then determined whether the current iteration number has reached the preset iteration number. If the current iteration number has not reached the preset iteration number, the current iteration number is incremented by 1, and the trajectory of the target UAV determined in the current iteration number is updated. The trajectory of the target UAV retained in the current iteration number is determined as the trajectory of the target UAV determined in the previous iteration number, and the camera intrinsic error corresponding to the trajectory of the target UAV retained in the current iteration number is determined as the camera intrinsic error obtained in the previous iteration number. The process then returns to the steps of determining the camera intrinsic parameters determined in the current iteration number based on the trajectory of the target UAV determined in the current iteration number, and determining the camera intrinsic parameter error of the current iteration number based on the camera intrinsic parameters determined in the current iteration number. If the current iteration number has reached the preset iteration number, then the iterative optimization process ends, and the trajectory of the target UAV retained in the current iteration number is determined as the preset trajectory. The calibration module is used to calibrate the camera's intrinsic parameters based on the installation position of the checkerboard calibration board relative to the UAV, the camera's installation position in the world coordinate system, the target UAV's pose data, and the photo data of the checkerboard calibration board captured by the camera. Specifically, it includes: determining the coordinates of the checkerboard corner points in the pixel coordinate system and the coordinates in the world coordinate system based on the installation position of the checkerboard calibration board relative to the UAV, the target UAV's pose data, and the photo data of the checkerboard calibration board captured by the camera; synchronizing the photo data of the checkerboard calibration board captured by the camera with the UAV's pose data in time; determining the coordinates of the checkerboard corner points in the pixel coordinate system based on the time-synchronized photo data; and determining the coordinates of each corner point in the checkerboard calibration board in the world coordinate system based on the time-synchronized pose data and the installation position of the checkerboard calibration board relative to the UAV. Based on the camera's installation position in the world coordinate system and the target UAV's pose data, determine the target UAV's rotation matrix and translation vector relative to the camera; Based on the rotation matrix and translation vector of the target UAV relative to the camera, the coordinate relationship between the checkerboard corner points in the pixel coordinate system and the world coordinate system, and the coordinates of the checkerboard corner points in the pixel coordinate system and the world coordinate system, calculate the camera's intrinsic parameter matrix and distortion parameters.

5. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to enable the electronic device to perform an automatic camera intrinsic parameter calibration method based on a drone according to any one of claims 1 to 3.

Images