Unmanned aerial vehicle multi-camera extrinsic parameter online calibration space fusion positioning method and system

Abstract

This application discloses a spatial fusion positioning method and system for online calibration of extrinsic parameters of multiple cameras on a UAV, belonging to the field of UAV technology. The UAV multi-camera extrinsic parameter online calibration spatial fusion positioning method includes: online extrinsic parameter calibration provides an accurate parameter basis for the entire positioning method, ensuring the correctness of the camera projection relationship; multi-view triangulation obtains the initial 3D coordinates of the target based on accurate extrinsic parameters, but its results are affected by noise and have deviations; EKF uses the target motion model and historical observations to perform noise smoothing and optimization on the initial 3D coordinates, reducing positioning errors. The entire process achieves dynamic parameter correction through periodic extrinsic parameter calibration, and fuses multi-view data from 8 cameras in real time to form a closed-loop optimized spatial fusion positioning system, taking into account parameter accuracy, positioning precision, and real-time performance.

Description

Technical Field

[0001] This application belongs to the field of unmanned aerial vehicle (UAV) technology, and in particular relates to a method and system for online calibration of spatial fusion positioning of multiple camera extrinsic parameters of UAVs. Background Technology

[0002] With its advantages of being contactless, highly accurate, and adaptable to various scenarios, multi-camera visual positioning technology for drones has been widely applied in civilian and industrial fields such as drone logistics delivery, power line inspection, geographic information mapping, and security area monitoring. The core technology of multi-camera spatial positioning relies on the accuracy of the camera's extrinsic parameters (rotation matrix R, translation vector T). These extrinsic parameters quantitatively describe the spatial transformation relationship between the camera coordinate system and the world coordinate system. Their accuracy directly determines the reliability of the subsequent 3D positioning results and is the foundation of the multi-camera visual positioning system.

[0003] Existing multi-camera spatial positioning technology for drones still faces numerous technical challenges in engineering applications, failing to meet the practical requirements of high precision, real-time performance, and robustness. The main problems are as follows: 1. The external parameter calibration mode has inherent defects: Mainstream technologies all adopt the laboratory offline calibration mode, which completes the initial calibration of camera external parameters through professional calibration boards. However, during the actual flight of the drone, environmental and mechanical factors such as fuselage vibration, outdoor temperature and humidity changes, and slight deformation of the camera installation structure will cause the external parameter parameters to drift gradually. The parameters calibrated offline will become invalid in a short time, resulting in a significant decrease in positioning accuracy. 2. Large error in 3D coordinate calculation and weak anti-interference ability: Most technologies use triangulation algorithms to solve the 3D coordinates of the target. Affected by interference factors such as image target detection noise, camera optical distortion, and outdoor lighting changes, the solution results of single triangulation have obvious random noise, which easily leads to positioning deviation and cannot meet the requirements of high-precision positioning. 3. Difficulty in balancing real-time performance and computational overhead: Some technologies employ complex multi-algorithm fusion schemes to improve positioning accuracy, but this results in a huge amount of computation and excessively long positioning time per operation, which cannot meet the real-time positioning requirements of UAVs in dynamic flight. At the same time, frequent complex calculations will excessively consume the UAV's onboard computing power, affecting the normal operation of other core systems such as flight control and navigation. 4. High complexity of on-site operation: Offline calibration mode relies on specialized equipment such as professional calibration boards and precision measuring instruments. In scenarios without calibration equipment, such as outdoor inspections and emergency mapping, it is impossible to complete the recalibration of external parameters, resulting in poor technical implementation.

[0004] To address the aforementioned technical issues, it is necessary to provide a novel method and system for online calibration of extrinsic parameters of multiple UAV cameras and spatial fusion positioning. Summary of the Invention

[0005] The purpose of this disclosure is to provide a spatial fusion positioning method and system for online calibration of extrinsic parameters of multiple cameras on unmanned aerial vehicles (UAVs) in order to solve the above-mentioned problems.

[0006] This disclosure achieves the above objectives through the following technical solutions: A method for online calibration of extrinsic parameters of multiple cameras on a drone and spatial fusion positioning includes the following steps: In the drone flight scenario, a static target of known size is selected, ensuring that the target is visible simultaneously in the field of view of at least two cameras. The target is then detected, and the pixel coordinates of the center point of the target's 2D bounding box are obtained. Using the target as a calibration object, the camera extrinsic parameter deviation is solved and online calibration is completed; Based on the calibrated extrinsic parameters, the pixel coordinates are calculated to obtain the initial 3D coordinates of the target; The initial 3D coordinates are optimized to obtain the final positioning result.

[0007] As a further optimization of this disclosure, using the target as a calibration object, the camera extrinsic parameter deviation is solved and online calibration is completed, including: Based on the actual size of the target, construct the coordinates of the target's 3D feature points in the world coordinate system; Input the 3D feature point coordinates and the 2D pixel coordinates into the solvePnP function to solve for the deviation of the rotation matrix R and translation vector T of each camera relative to the initial extrinsic parameters; The coordinates of the 3D feature points are projected onto the camera imaging plane through the current extrinsic parameters to obtain the projected 2D pixel coordinates. These are then compared with the 2D pixel coordinates to calculate the reprojection error. A reprojection error threshold is set. If the calculated reprojection error is greater than the reprojection error threshold, the rotation matrix R and translation vector T are optimized and adjusted using the least squares method until the reprojection error is no greater than the reprojection error threshold. The optimized extrinsic parameters are then updated to the camera parameter library. If the reprojection error is no greater than the reprojection error threshold, it is determined that the current extrinsic parameters have no significant deviation and the current extrinsic parameters are retained.

[0008] As a further optimization of this disclosure, the pixel coordinates are calculated based on the calibrated extrinsic parameters to obtain the initial 3D coordinates of the target, including: For the same target, obtain clear 2D bounding box center point pixel coordinates from all cameras respectively; A camera projection matrix is ​​constructed by combining the calibrated camera extrinsic parameters, and a system of linear equations for the DLT algorithm is constructed based on the camera projection matrix and the pixel coordinates. Singular value decomposition is performed on the coefficient matrix of the linear equation system to solve the least squares solution of the linear equation system, thereby obtaining the initial 3D coordinates of the target.

[0009] As a further optimization of this disclosure, the initial 3D coordinates are optimized to obtain the final positioning result, including: Define the state vector and the observation vector; Based on the state vector and the observation vector, construct the state equation and the observation equation respectively: Based on the state vector and covariance matrix of the previous frame, predict the state vector and covariance matrix of the current frame; combine the observation vector of the current frame, calculate the Kalman gain and correct the predicted state to obtain the updated state vector and covariance matrix. The 3D position coordinates of the target are extracted from the updated state vector as the final spatial positioning result.

[0010] A spatial fusion positioning system for online calibration of extrinsic parameters of multiple drone cameras includes: The target detection module is used to select a static target of known size in the drone flight scene, ensure that the target is visible in the field of view of at least two cameras at the same time, detect the target and obtain the pixel coordinates of the center point of the target's 2D bounding box; The external parameter online calibration module is used to solve the camera's external parameter deviation and complete online calibration using the target as a calibration object; The multi-view triangulation module calculates the pixel coordinates based on the calibrated extrinsic parameters to obtain the initial 3D coordinates of the target. The EKF optimization module is used to optimize the initial 3D coordinates to obtain the final positioning result.

[0011] As a further optimization of this disclosure, the online extrinsic parameter calibration module uses the target as a calibration object to solve for the camera extrinsic parameter deviation and complete online calibration, including: Based on the actual size of the target, construct the coordinates of the target's 3D feature points in the world coordinate system; Input the 3D feature point coordinates and the 2D pixel coordinates into the solvePnP function to solve for the deviation of the rotation matrix R and translation vector T of each camera relative to the initial parameters; The 3D feature points are projected onto the camera imaging plane through the current extrinsic parameters to obtain the projected 2D pixel coordinates. These coordinates are then compared with the detected actual 2D pixel coordinates to calculate the reprojection error. A reprojection error threshold is set. If the calculated reprojection error is greater than the reprojection error threshold, the rotation matrix R and translation vector T are optimized and adjusted using the least squares method until the reprojection error is no greater than the reprojection error threshold. The optimized extrinsic parameters are then updated to the camera parameter library. If the reprojection error is no greater than the reprojection error threshold, it is determined that the current extrinsic parameters have no significant deviation, and the original parameters are retained.

[0012] As a further optimization of this disclosure, the multi-view triangulation module calculates the pixel coordinates based on calibrated extrinsic parameters to obtain the initial 3D coordinates of the target, including: For the same target, obtain clear 2D bounding box center point pixel coordinates from all cameras respectively; A camera projection matrix is ​​constructed by combining the calibrated camera extrinsic parameters, and a system of linear equations for the DLT algorithm is constructed based on the camera projection matrix and the pixel coordinates. Singular value decomposition is performed on the coefficient matrix of the linear equation system to solve the least squares solution of the linear equation system, thereby obtaining the initial 3D coordinates of the target.

[0013] As a further optimization of this disclosure, the EKF optimization module optimizes the initial 3D coordinates to obtain the final positioning result, including: Define the state vector and the observation vector; Based on the state vector and the observation vector, construct the state equation and the observation equation respectively: Based on the state vector and covariance matrix of the previous frame, predict the state vector and covariance matrix of the current frame; combine the observation vector of the current frame, calculate the Kalman gain and correct the predicted state to obtain the updated state vector and covariance matrix. The 3D position coordinates of the target are extracted from the updated state vector as the final spatial positioning result.

[0014] An electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to execute the program stored in the memory to implement the above-described online calibration spatial fusion positioning method for multiple camera extrinsics of the UAV.

[0015] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned online calibration spatial fusion positioning method for multi-camera extrinsic parameters of a UAV.

[0016] The beneficial effects of this disclosure are as follows: 1. Achieve online dynamic calibration of extrinsic parameters and completely solve the problem of parameter drift failure in offline calibration: This disclosure does not require professional calibration equipment. It can complete the extrinsic parameter calibration by selecting common static targets of known size in the scene. The extrinsic parameter deviation is back-calculated and optimized in real time by using the solvePnP function. It can correct the extrinsic parameter drift caused by vibration and environmental changes during the flight of the UAV in real time. After calibration, the extrinsic parameter error is ≤5%, which ensures the accuracy of multi-camera positioning from the parameter foundation level. 2. High positioning accuracy and strong system robustness: This disclosure uses multiple cameras for positioning; and when some cameras lose the target due to obstruction of view, the remaining cameras can still complete effective positioning, which greatly improves the system's adaptability to complex scenes; 3. Excellent real-time performance, perfectly adapted to the dynamic flight requirements of UAVs: This disclosure features lightweight design and optimization of calibration and positioning algorithms. The time for a single online calibration of external parameters is ≤10ms, and the time for a single positioning operation is ≤5ms with multi-view triangulation combined with EKF optimization, which is far lower than the interval time of conventional frame rates of UAVs and has no significant computational overhead. At the same time, the periodic calibration design avoids the waste of resources caused by frequent calculations and can meet the real-time positioning engineering requirements of UAV dynamic flight. 4. Low on-site operation complexity and adaptable to various outdoor uncalibrated scenarios: This invention eliminates the reliance on specialized equipment such as professional calibration boards and precision measuring instruments. It can complete external parameter calibration by selecting common static targets of known size in the scenario (such as vehicles, poles, guardrails, etc.), without the need for additional manual calibration facilities. It can quickly adapt to the positioning needs of various uncalibrated environments such as outdoor inspection, emergency mapping, and security monitoring, and has strong technical applicability. 5. Constructing a full-process closed-loop optimization system with high long-term system stability: This disclosure constructs a full-process closed-loop optimization system consisting of online external parameter calibration, multi-view triangulation, and EKF optimization. Online external parameter calibration provides an accurate parameter basis for multi-view triangulation, and EKF optimization corrects and smooths the noise results of triangulation. At the same time, the calibration results are continuously updated with the positioning process, realizing bidirectional correction of parameter calibration and positioning results, which greatly improves the long-term operational stability of the entire positioning system. 6. Modular algorithm design, easy integration and secondary development: This disclosure divides the positioning system into independent modules such as target detection, extrinsic parameter calibration, triangulation, and EKF optimization. The modules transmit information through standardized data interfaces, which is easy to integrate with the existing flight control and navigation systems of UAVs. At the same time, the modular design also facilitates secondary development and optimization of the algorithm according to actual application needs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the method flow in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the system structure in an embodiment of this disclosure; Figure 3 This is a block diagram of the device structure in an embodiment of this disclosure. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] like Figure 1 As shown, this embodiment uses an 8-camera drone. A method for online calibration of spatial fusion positioning of multi-camera extrinsic parameters of a drone includes the following steps: S1. Select a static target of known size in the drone flight scene, ensuring that the target is simultaneously visible in the field of view of at least two cameras. Detect the target and obtain the pixel coordinates of the center point of the target's 2D bounding box. Specifically, this includes: Select a static target of known size (such as a vehicle 5m long or a pole 3m high) in the drone flight scenario, and ensure that the target is visible in the field of view of at least two cameras at the same time to provide effective observation data for external parameter calibration. The images captured by eight cameras are input into the improved YOLOv8n object detection model to detect the known-size target and obtain the pixel coordinates of its 2D bounding box center point, denoted as [the model name is missing here]. ,in This is the camera index, with values ​​from 1, 2, ..., 8, corresponding to 8 cameras.

[0021] S2. Using the target as a calibration object, solve for the camera's extrinsic parameter deviation and complete online calibration, specifically including: Constructing known 3D points: Based on the actual size of the selected known static target, construct the coordinates of the target's 3D feature points in the world coordinate system; Solve for extrinsic parameter bias: Compare the constructed known 3D points with the corresponding 2D pixels detected in step 1. Input the solvePnP function to solve for the deviation of the rotation matrix R and translation vector T of each camera relative to the initial extrinsic parameters; Calculate reprojection error: Project the known 3D points onto the camera imaging plane using the current extrinsic parameters to obtain the projected 2D pixel coordinates. Compare it with the actual 2D pixel coordinates obtained from the detection. The reprojection error e is calculated by comparing the two values ​​using the following formula:

[0022] Where ||·|| is the Euclidean norm, used to calculate the Euclidean distance between two 2D pixels; Extrinsic parameter optimization and update: Set the reprojection error threshold to 5%. If the calculated reprojection error e > 5%, then optimize and adjust the rotation matrix R and translation vector T using the least squares method until the reprojection error e ≤ 5%, and update the optimized extrinsic parameters to the camera parameter library; if e ≤ 5%, then it is determined that the current extrinsic parameters have no obvious deviation, and the original parameters are retained. Calibration frequency control: To balance calibration accuracy and computing power, an online external parameter calibration process is performed every 30 frames to avoid wasting UAV onboard resources due to frequent calculations.

[0023] The code is written in Python and calls the solvePnP function of OpenCV to solve for and calibrate the extrinsic parameter deviation. The time taken for a single calibration is ≤10ms.

[0024] S3. Based on the calibrated extrinsic parameters, the pixel coordinates are calculated to obtain the initial 3D coordinates of the target, specifically including: For the same target, obtain the pixel coordinates of its clear 2D bounding box center point from 8 cameras. (i=1,2,...,8); The camera projection matrix is ​​constructed by combining the calibrated camera extrinsic parameters (rotation matrix R_i, translation vector T_i). Based on the projection matrix and the 2D pixel coordinates, a system of linear equations for the DLT algorithm is constructed:

[0025] Where A is the coefficient matrix of the DLT algorithm, with a dimension of 2n×4 (n=8, which is the number of cameras participating in the triangulation), and X is the three-dimensional world homogeneous coordinate vector of the target; Perform SVD (Singular Value Decomposition) on the coefficient matrix A, solve the least-squares solution of the above linear equation system, and obtain the initial 3D coordinates of the target, denoted as . .

[0026] S4. Optimize the initial 3D coordinates to obtain the final positioning result, specifically including: S401. Define the state vector and observation vector: State vector X: Contains the target's 3D position coordinates and three-axis motion velocity, with dimensions of 6×1, and its expression is: ,in Let be the 3D position coordinates of the target in the world coordinate system. The velocity of the target along the X, Y, and Z coordinate axes; Observation vector Z: The initial 3D coordinates of the target obtained from multi-view triangulation, with dimensions 3×1, expressed as follows: .

[0027] S402. Establish the state equation and observation equation: State equation: Constructed based on a uniform motion model, describing the state transition relationship of the target from frame (k-1) to frame k, the formula is:

[0028] in, Let k be the state vector of the k-th frame. Let F be the state vector of the (k-1)th frame, and F be a 6×6 motion transition matrix. The process noise vector of the kth frame (following a zero-mean Gaussian distribution) has its covariance matrix Q adaptively adjusted according to the actual speed of the target. Observation equation: describes the projection transformation relationship from the state vector to the observation vector, and the formula is:

[0029] in Let k be the observation vector of the k-th frame. The observation matrix is ​​3×6. Let R be the observation noise vector of the k-th frame (which follows a zero-mean Gaussian distribution), and let its covariance matrix R be set according to the actual error of the triangulation.

[0030] S403 and EKF iterative optimization: This involves two steps: prediction and update, optimizing the target state frame by frame. Prediction steps: Based on the state vector and covariance matrix of the previous frame, predict the state vector and covariance matrix of the current frame. The formula is:

[0031]

[0032] in, This is the predicted state vector for the current frame. The prediction covariance matrix for the current frame. This is the updated state vector from the previous frame. This is the updated covariance matrix from the previous frame. Motion transition matrix The transpose of the matrix; Update steps: Combine the observation vector of the current frame, calculate the Kalman gain and correct the predicted state to obtain the updated state vector and covariance matrix, as shown in the formula:

[0033]

[0034]

[0035] in, Here is the Kalman gain matrix. Observation matrix The transpose of the matrix, This is the inverse operation of a matrix. It is a 6×6 identity matrix. The updated state vector, This is the updated covariance matrix.

[0036] S404. Output the final positioning result: Extract the target's 3D position coordinates (x, y, z) from the EKF-updated state vector X as the final spatial positioning result. The positioning error of this result is ≤8%. Error verification is achieved by comparing simulated data with real coordinates.

[0037] The code was written in C++ and implemented using the Eigen library to perform matrix operations and triangulation of the DLT algorithm. At the same time, the EKF state equation and observation equation were written and iterated, with a single positioning time of ≤5ms.

[0038] like Figure 2 As shown, embodiments of this disclosure provide a spatial fusion positioning system for online calibration of extrinsic parameters of multiple drone cameras, comprising: Unmanned aerial vehicle carrier: located in the attached Figure 2 On the far left is the basic hardware module, labeled "UAV carrier (hardware layer) - 8-channel distributed camera installation carrier"; 8-channel distributed camera module: Located adjacent to the right side of the drone carrier, this is a hardware layer module labeled "8-channel distributed camera module (hardware) - evenly distributed around the drone body, providing full-view coverage, with at least 2 channels capable of displaying targets of known size". This module is connected vertically to the drone carrier, receives power and synchronization commands from the drone carrier, and outputs "synchronously acquired image data" to the next module. Target detection module: Located to the right of the 8-channel distributed camera module, it is the first module in the software layer and is labeled "Target Detection Module (Software) - Improved YOLOv8n Target Detection"; it receives "image data" from the previous module and outputs "2D bounding box center point pixel coordinates". The signal is then split into two paths and transmitted to the subsequent two modules. External parameter online calibration module: Located to the lower right of the target detection module, this is a software layer module labeled "External parameter online calibration module (software) - based on OpenCV solvePnP function"; it receives data from the target detection module. , output "external parameter deviation" to the camera parameter library; Camera parameter library: Located to the right of the external parameter online calibration module, it is a software layer data storage module labeled "Camera parameter library (software) - storing calibrated external parameters R, T"; it receives the "external parameter deviation" from the external parameter online calibration module and completes parameter updates, and outputs "calibrated external parameters R, T" to the multi-view triangulation module; Multi-view triangulation module: Located above the target detection module, this is a software layer module labeled "Multi-view triangulation module (software) - based on DLT algorithm + Eigen library"; it receives data from the target detection module. The system retrieves the "calibrated extrinsic parameters R and T" from the camera parameter library and outputs the "initial 3D coordinates (x_0, y_0, z_0)" to the next module. EKF Optimization Module: Located to the right of the Multi-view Triangulation Module, this is a software layer module labeled "EKF Optimization Module (Software) - Extended Kalman Filter Iterative Optimization"; it receives the "initial 3D coordinates (x_0, y_0, z_0)" from the previous module and outputs the "final 3D coordinates (x, y, z)" to the last module. Positioning result output module: Located on the far right of the attached diagram, this is a software-level terminal module labeled "Positioning result output module (software) - feeding back positioning results to the flight control system"; it receives the "final 3D coordinates (x, y, z)" from the EKF optimization module, completing the entire positioning process.

[0039] The implementation process of the functions and roles of each module in the above system is detailed in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0040] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0041] See Figure 3 The electronic device provided in the embodiments of this disclosure includes a processor 1110, a communication interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120 and the memory 1130 communicate with each other through the communication bus 1140. Memory 1130 is used to store computer programs; The processor 1110, when executing the program stored in the memory 1130, implements the above-mentioned online calibration spatial fusion positioning method for multi-camera extrinsic parameters of a UAV.

[0042] The aforementioned communication bus 1140 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0043] The communication interface 1120 is used for communication between the above-mentioned electronic device and other devices.

[0044] The memory 1130 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1130 may also be at least one storage device located remotely from the aforementioned processor 1110.

[0045] Embodiments of this disclosure also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the UAV multi-camera extrinsic parameter online calibration spatial fusion positioning method as described above.

[0046] The embodiments described above are merely examples of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent disclosure. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these modifications and improvements all fall within the protection scope of this disclosure.

Claims

1. A spatial fusion positioning method for online calibration of extrinsic parameters of multiple cameras on a UAV, characterized in that, Includes the following steps: In the drone flight scenario, a static target of known size is selected, ensuring that the target is visible simultaneously in the field of view of at least two cameras. The target is then detected, and the pixel coordinates of the center point of the target's 2D bounding box are obtained. Using the target as a calibration object, the camera extrinsic parameter deviation is solved and online calibration is completed; Based on the calibrated extrinsic parameters, the pixel coordinates are calculated to obtain the initial 3D coordinates of the target; The initial 3D coordinates are optimized to obtain the final positioning result.

2. The method for online calibration of extrinsic parameters of multiple cameras in a UAV and spatial fusion positioning according to claim 1, characterized in that, Using the target as a calibration object, the camera extrinsic parameter deviation is solved and online calibration is completed, including: Based on the actual size of the target, construct the coordinates of the target's 3D feature points in the world coordinate system; Input the 3D feature point coordinates and the 2D pixel coordinates into the solvePnP function to solve for the deviation of the rotation matrix R and translation vector T of each camera relative to the initial extrinsic parameters; The coordinates of the 3D feature points are projected onto the camera imaging plane through the current extrinsic parameters to obtain the projected 2D pixel coordinates. These are then compared with the 2D pixel coordinates to calculate the reprojection error. A reprojection error threshold is set. If the calculated reprojection error is greater than the reprojection error threshold, the rotation matrix R and translation vector T are optimized and adjusted using the least squares method until the reprojection error is no greater than the reprojection error threshold. The optimized extrinsic parameters are then updated to the camera parameter library. If the reprojection error is no greater than the reprojection error threshold, it is determined that the current extrinsic parameters have no significant deviation and the current extrinsic parameters are retained.

3. The method for online calibration of extrinsic parameters of multiple cameras in a UAV and spatial fusion positioning according to claim 1, characterized in that, Based on the calibrated extrinsic parameters, the pixel coordinates are calculated to obtain the initial 3D coordinates of the target, including: For the same target, obtain clear 2D bounding box center point pixel coordinates from all cameras respectively; A camera projection matrix is ​​constructed by combining the calibrated camera extrinsic parameters, and a system of linear equations for the DLT algorithm is constructed based on the camera projection matrix and the pixel coordinates. Singular value decomposition is performed on the coefficient matrix of the linear equation system to solve the least squares solution of the linear equation system, thereby obtaining the initial 3D coordinates of the target.

4. The method for online calibration of extrinsic parameters of multiple cameras in a UAV and spatial fusion positioning according to claim 1, characterized in that, The initial 3D coordinates are optimized to obtain the final positioning result, including: Define the state vector and the observation vector; Based on the state vector and the observation vector, construct the state equation and the observation equation respectively: Based on the state vector and covariance matrix of the previous frame, predict the state vector and covariance matrix of the current frame; combine the observation vector of the current frame, calculate the Kalman gain and correct the predicted state to obtain the updated state vector and covariance matrix. The 3D position coordinates of the target are extracted from the updated state vector as the final spatial positioning result.

5. A spatial fusion positioning system for online calibration of extrinsic parameters of multiple cameras on unmanned aerial vehicles (UAVs), characterized in that, include: The target detection module is used to select a static target of known size in the drone flight scene, ensure that the target is visible in the field of view of at least two cameras at the same time, detect the target and obtain the pixel coordinates of the center point of the target's 2D bounding box; The external parameter online calibration module is used to solve the camera's external parameter deviation and complete online calibration using the target as a calibration object; The multi-view triangulation module calculates the pixel coordinates based on the calibrated extrinsic parameters to obtain the initial 3D coordinates of the target. The EKF optimization module is used to optimize the initial 3D coordinates to obtain the final positioning result.

6. A spatial fusion positioning system for online calibration of extrinsic parameters of multiple cameras on a UAV according to claim 5, characterized in that, The online extrinsic parameter calibration module uses the target as a calibration object to solve for the camera's extrinsic parameter deviation and complete online calibration, including: Based on the actual size of the target, construct the coordinates of the target's 3D feature points in the world coordinate system; Input the 3D feature point coordinates and the 2D pixel coordinates into the solvePnP function to solve for the deviation of the rotation matrix R and translation vector T of each camera relative to the initial parameters; The 3D feature points are projected onto the camera imaging plane through the current extrinsic parameters to obtain the projected 2D pixel coordinates. These coordinates are then compared with the detected actual 2D pixel coordinates to calculate the reprojection error. A reprojection error threshold is set. If the calculated reprojection error is greater than the reprojection error threshold, the rotation matrix R and translation vector T are optimized and adjusted using the least squares method until the reprojection error is no greater than the reprojection error threshold. The optimized extrinsic parameters are then updated to the camera parameter library. If the reprojection error is no greater than the reprojection error threshold, it is determined that the current extrinsic parameters have no significant deviation, and the original parameters are retained.

7. A spatial fusion positioning system for online calibration of extrinsic parameters of multiple cameras on a UAV according to claim 5, characterized in that, The multi-view triangulation module calculates the pixel coordinates based on calibrated extrinsic parameters to obtain the initial 3D coordinates of the target, including: For the same target, obtain clear 2D bounding box center point pixel coordinates from all cameras respectively; A camera projection matrix is ​​constructed by combining the calibrated camera extrinsic parameters, and a system of linear equations for the DLT algorithm is constructed based on the camera projection matrix and the pixel coordinates. Singular value decomposition is performed on the coefficient matrix of the linear equation system to solve the least squares solution of the linear equation system, thereby obtaining the initial 3D coordinates of the target.

8. A spatial fusion positioning system for online calibration of extrinsic parameters of multiple cameras on a UAV according to claim 5, characterized in that, The EKF optimization module optimizes the initial 3D coordinates to obtain the final positioning result, including: Define the state vector and the observation vector; Based on the state vector and the observation vector, construct the state equation and the observation equation respectively: Based on the state vector and covariance matrix of the previous frame, predict the state vector and covariance matrix of the current frame; combine the observation vector of the current frame, calculate the Kalman gain and correct the predicted state to obtain the updated state vector and covariance matrix. The 3D position coordinates of the target are extracted from the updated state vector as the final spatial positioning result.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor is used to execute a program stored in a memory to implement the UAV multi-camera extrinsic parameter online calibration spatial fusion positioning method as described in any one of claims 1-4.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV multi-camera extrinsic parameter online calibration spatial fusion positioning method as described in any one of claims 1-4.

Images