A natural feature control point loop iteration camera array external parameter timing correction method

Abstract

The application discloses a natural feature control point loop iteration camera array external parameter timing correction method, which comprises the following steps: acquiring original image data of a moving platform camera array; obtaining a natural feature control point set through a multi-scale cascaded natural feature control point optimization; constructing a normalized geometric cost model, establishing a space-time correlation loop of a current frame and a historical frame based on the natural feature control point set, adopting a descriptor similarity confidence and a geometric consistency verification dynamic weight correction, generating a weighted feature point set as an a priori condition to input a target optimization function, adopting an LM algorithm to iteratively solve an optimal external parameter, updating a historical frame database through a current optimization result, forming an error suppression loop, and obtaining an optimized camera array timing external parameter. The application realizes high-precision timing correction of the external parameter of the moving platform camera array, significantly improves the precision and stability of the camera external parameter in a long-time dynamic environment, and breaks through the measurement bottleneck caused by the existing feature matching error accumulation and attitude drift.

Description

Technical Field

[0001] This invention relates to the field of timing correction technology, and in particular to a timing correction method for extrinsic parameters of a camera array with natural feature control point loop iteration. Background Technology

[0002] Conducting high-precision dynamic attitude testing of high-speed moving objects in complex environments places higher demands on the field of view coverage, measurement accuracy, and attitude robustness of the testing system. Especially in applications such as deep space probe testing and evaluation of new long-range precision strike weapons, the target object is often in a state of high-speed movement and multi-dimensional attitude changes. Traditional fixed-position high-speed video measurement systems are difficult to accurately observe the entire process of the aircraft during descent, landing, leap, and ballistic flight due to limited viewing angle and insufficient spatial coverage.

[0003] To overcome this limitation, the flexible deployment of high-speed cameras on mobile platforms (such as aerial work platforms, rotary-wing UAVs, cable trolley systems, and shipborne platforms) has become a key approach. These mobile platforms offer advantages such as rapid deployment, wide-area tracking, and multi-angle data acquisition, enabling the construction of a 3D testing network with extensive coverage, varied angles, and flexible field of view. This provides more comprehensive data support for reconstructing the dynamic behavior of targets in complex environments. Especially in environments with obstructed views, complex terrain, or temporary testing conditions, mobile platforms significantly improve system adaptability and testing efficiency, making them a necessary path to building high-performance 3D vision measurement systems.

[0004] However, during operation, the moving platform is affected by factors such as wind, platform vibration, and disturbances from its own propulsion mechanism, often keeping its onboard camera array in a non-static state. While minute attitude jitters or displacements occurring during camera acquisition may be difficult to detect in the short term, they can lead to the accumulation of extrinsic parameter errors over long periods of high-frame-rate image sequences, manifesting as a "false displacement" phenomenon. This error not only interferes with the extraction of true displacement but also amplifies the nonlinear deviations in trajectory reconstruction and attitude estimation, directly affecting the accuracy and stability of pose measurement for high-speed moving bodies. Therefore, how to perform high-precision dynamic correction of the temporal extrinsic parameters of the moving platform camera array is a core technical challenge for the engineering application of such systems.

[0005] Currently, mainstream camera attitude compensation methods can be divided into two categories: one is the extrinsic parameter correction method based on the fusion of inertial and satellite navigation; the other is the visual self-calibration method based on image features. The former typically fuses IMU (Inertial Measurement Unit) and GNSS data for attitude calculation. IMUs have the advantages of high frequency and fast response (typical data rate 100–1000Hz), and can reflect the camera's angular velocity and acceleration information in real time, making them suitable for capturing dynamic changes. However, GNSS has a low update frequency (generally 1–10Hz), which does not match the frame rate of high-speed cameras (often 200–2000Hz), causing a lag in attitude correction accuracy. More importantly, GNSS signals are easily obstructed or interfered with in complex environments, and their positioning drift will be directly transmitted to the extrinsic parameter estimation, causing error accumulation. Visual self-calibration methods, on the other hand, do not rely on external equipment and utilize natural feature points in the image sequence for attitude inversion. Typical strategies include optical flow-deep learning fusion methods and feature-matching-based geometric optimization methods. Optical flow methods can achieve pixel-level displacement estimation. Combined with deep learning models, they can adapt to nonlinear distortions, forming an end-to-end automated correction process suitable for complex and variable environments. However, this method is highly dependent on the scale and representativeness of the training data and struggles to guarantee robustness across different scenes. Feature-matching methods utilize stable natural structures in images (such as road edges, corners, and contours) for inter-frame matching, optimizing camera extrinsic parameters through 3D point cloud inversion. Their advantages include high computational efficiency, strong adaptability, sub-pixel accuracy, and strong anti-perturbation capabilities. However, the compensation accuracy of this method is limited by the geometric stability and distribution density of natural features. When feature points are sparse or matching errors are large, extrinsic parameter errors will be difficult to converge effectively, affecting the overall stability of the system. Summary of the Invention

[0006] The purpose of this invention is to provide a method for timing correction of extrinsic parameters of a camera array using a loop-loop iterative method for natural feature control points, in order to solve the problems existing in the prior art.

[0007] To achieve the above objectives, the present invention is implemented according to the following technical solution: On one hand, the present invention includes the following steps: Acquire raw image data from the moving platform camera array; A set of natural feature control points is obtained through multi-scale cascaded natural feature control point optimization. A normalized geometric cost model is constructed, and a spatiotemporal correlation closed loop between the current frame and historical frames is established based on the natural feature control point set. The dynamic weight correction is verified by descriptor similarity confidence and geometric consistency. A weighted feature point set is generated as a priori condition input to the objective optimization function. The optimal extrinsic parameters are solved iteratively using the LM algorithm. The historical frame database is updated through the current optimization results to form an error suppression closed loop, and the optimized temporal extrinsic parameters of the camera array are obtained.

[0008] Furthermore, a set of natural feature control points is obtained through multi-scale cascaded natural feature control point optimization, specifically including: ① Multi-scale layer geometric stability enhancement: Construct a Gaussian pyramid with 5 scales for the original image, calculate the Hessian matrix of natural feature points in the Gaussian image at each scale, determine geometric stability based on the eigenvalues ​​of the Hessian matrix, and select a set of control points for geometric stability enhancement. ② Optimize the local entropy spatial distribution of the control point set: Divide the sequence image into an initial uniform grid, calculate the local entropy value of each grid, subdivide the grids with local entropy values ​​exceeding the threshold, generate the Voronoi map of the natural feature control point set, and supplement feature points and remove clustered points through the gravity-repulsion balance model to obtain a uniformly distributed feature point set; ③ Obtain the set of natural feature control points by constraining feature quality with similarity: Extract image blocks centered on the left image points, calculate their SSIM similarity with the corresponding image blocks in the right image, select high-quality feature points that meet the similarity threshold, and combine the same operation with the right image to obtain the final set of natural feature control points.

[0009] Furthermore, the initial uniform grid of the grid with local entropy exceeding the threshold is 8×6, and the subdivided grid is 16×12; the threshold of the local entropy is 0.7 times the global entropy of the image; in the gravity-repulsion balance model, the condition for determining the aggregation point is that the distance between feature points is less than 0.5 times the median of the global Voronoi cell side length.

[0010] Furthermore, in the multi-scale layer geometric stability enhancement, specific measures are taken for each scale layer. Establish geometric stability criteria and obtain the set of natural feature control points that enhance geometric stability. : in, , It is a scale layer eigenvalues, The empirical threshold is 10.

[0011] Furthermore, when constructing the normalized geometric cost model, the temporal extrinsic parameters of the machine array are: in, For the normalization operation, the cost of epipolar geometric constraints is used to identify and remove feature points that do not conform to the geometric consistency of the static scene: in, For the fundamental matrix of epipolar geometry, These are two-dimensional feature points in adjacent frames.

[0012] in, For a homography matrix containing only rotational components, The distance is Euclidean.

[0013] in, The global optical flow field caused by camera motion. For point The local optical flow vector.

[0014] Furthermore, descriptor similarity confidence and geometric consistency are used to verify dynamic weight correction, and the appearance consistency and geometric consistency of feature points are jointly evaluated to generate weights. :

[0015]

[0016] in, For the current frame and history Mean cosine similarity of frames The reprojection error is calculated from the temporal extrinsic parameters of the camera to be optimized. for The corresponding matching point pairs yield a weighted feature point set. And will The objective function is input as a priori condition, and the LM algorithm is used to iteratively solve for the optimal extrinsic parameters. The current optimization result updates the historical frame database, and the updated historical data participates in the weight calculation of subsequent frames. The optimization direction is dynamically adjusted through weights to form an error suppression closed loop, and finally the optimized high-precision camera array temporal extrinsic parameters are obtained. .

[0017] The beneficial effects of this invention are: This invention is a method for timing correction of extrinsic parameters of a camera array using loop-loop iterative methods based on natural feature control points. Compared with existing technologies, this invention has the following technical advantages: This invention addresses the issues of insufficient geometric stability and uneven spatial distribution of feature points by employing multi-scale cascaded natural feature control point optimization, accurately acquiring the coordinates of three-dimensional control points. Furthermore, it constructs a geometric cost-driven loop-based iterative correction mechanism, combined with a normalized geometric cost model, to achieve high-precision temporal correction of the extrinsic parameters of the moving platform camera array. This significantly improves the accuracy and stability of the camera extrinsic parameters under long-term dynamic environments, overcoming the measurement bottlenecks caused by existing feature matching error accumulation and attitude drift. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the adjustment system framework for a natural feature control point loop-loop iterative camera array extrinsic parameter timing correction method according to the present invention; Detailed Implementation

[0019] The present invention will be further described below through specific embodiments. The illustrative embodiments and descriptions herein are used to explain the present invention, but are not intended to limit the present invention.

[0020] like Figure 1 As shown, a method for timing correction of extrinsic parameters of a camera array using loop-closure iterative methods based on natural feature control points includes the following steps: Acquire raw image data from the moving platform camera array; A set of natural feature control points is obtained through multi-scale cascaded natural feature control point optimization. A normalized geometric cost model is constructed, and a spatiotemporal correlation closed loop between the current frame and historical frames is established based on the natural feature control point set. The dynamic weight correction is verified by descriptor similarity confidence and geometric consistency. A weighted feature point set is generated as a priori condition input to the objective optimization function. The optimal extrinsic parameters are solved iteratively using the LM algorithm. The historical frame database is updated through the current optimization results to form an error suppression closed loop, and the optimized temporal extrinsic parameters of the camera array are obtained.

[0021] (1) Multi-scale cascaded natural feature control point optimization: To improve the accuracy and stability of camera extrinsic parameter correction in moving platform image sequences, it is necessary to strictly screen and optimize natural feature points. As a reference benchmark for extrinsic parameter compensation, the geometric stability, spatial distribution rationality, and matching quality of control points directly determine the correction effect. Low-quality or unevenly distributed feature points not only easily cause mismatches but also amplify geometric solution errors and reduce the overall system robustness. Therefore, a multi-scale cascaded natural feature control point optimization method considering geometric stability, spatial distribution, and feature quality is constructed to achieve progressive constraints from local geometric analysis to global distribution optimization.

[0022] First, define It is the initial set of natural feature control points extracted through the SuperPoint network, and the image plane coordinates of each feature control point are... .

[0023] ① Multi-scale layer geometric stability enhancement: Gaussian blurring is used to preserve the original image. The advantage of resolution allows for the construction of original images. The five-scale Gaussian pyramid ( ), at each scale layer Gaussian Image In the process of calculating natural feature points Hessian matrix : in, , , The second derivative of the image.

[0024] For each scale layer Establish geometric stability criteria and obtain the set of natural feature control points that enhance geometric stability. : in, , It is a scale layer eigenvalues, The empirical threshold is 10.

[0025] ② Local entropy spatial distribution optimization: Divide the image sequence into an initial uniform grid. (8×6), calculate the local entropy value for each grid cell: in, For grid Internal grayscale value The pixel probability. If The grid is subdivided into 16×12. , This is the statistical value of the global entropy of the image.

[0026] Generate a set of natural feature control points The Voronoi diagram is used to supplement feature points and remove cluster points by constructing a gravity-repulsion balance model of the Voronoi diagram.

[0027] Gravitational supplementary feature points: each point Corresponding to one Voronoi unit ,like unit area Larger than all grids If the median area of ​​the Voronoi cells is obtained, then new feature points are randomly added within that grid.

[0028] Repulsive eliminator clustering: For each point Obtain the set of nearest neighbors by radius search. If it exists And satisfy Marked as aggregation point, It is 0.5 times the median side length of the global Voronoi cells. This is achieved by removing clusters. Points with low local entropy result in a uniformly distributed set of feature points. .

[0029] ③ Similarity-based feature quality constraint: Improve the quality of natural feature control points by constraining the similarity of local image patches. (Left image point) Centered on, the extraction window size is Image blocks Calculate its corresponding image block with the right image. SSIM similarity yields a high-quality left image point set. : Simultaneously, the same operation is performed on the right image points, ultimately yielding a set of natural feature control points that satisfy the conditions of geometric stability, spatial distribution, and feature quality. .

[0030] 2) Geometric cost-driven loop-based iterative timing correction of camera array extrinsics Due to factors such as environmental disturbances, sensor noise, rapid target movement, and long-term error accumulation, existing methods for calculating the temporal extrinsic parameters of moving platform camera arrays still suffer from large accuracy fluctuations and unstable results, making it difficult to meet the application requirements of high-precision dynamic measurement. Especially during long-term, multi-frame continuous observations, errors gradually amplify, leading to a decrease in the overall stability of the measurement system. Therefore, it is necessary to construct a geometry-cost-driven loop-based iterative temporal correction method for camera array extrinsic parameters. By introducing a loop structure to suppress error propagation and using geometric consistency as the optimization objective, this method systematically improves the robustness and consistency of the camera array's temporal extrinsic parameters, providing crucial support for achieving highly reliable pose measurement of moving platforms in complex dynamic environments.

[0031] ① Construction of Normalized Geometric Cost Model: Comprehensive utilization of epipolar geometric constraint cost Rotational Uniformity Cost Cost of motion consistency Advantages of constructing natural feature control points Geometric cost model Dynamic outliers are gradually eliminated. Finally, the extrinsic parameter estimation is optimized using RANSAC and iterative reweighted least squares method to calculate the temporal extrinsic parameters of the camera array. .

[0032] in, To normalize the operation and address the dimensionality issue of the geometric cost model, ensuring that the detection results are both robust and accurate.

[0033] Epipolar geometry constraint cost: Primarily identifies and removes feature points that do not conform to the geometric consistency of the static scene. in, For the fundamental matrix of epipolar geometry, These are two-dimensional feature points from adjacent frames. Low algebraic values ​​indicate that the feature points conform to epipolar geometry constraints, while dynamic points are assigned higher algebraic values.

[0034] Rotation homography cost: Primarily addresses the failure of epipolar geometry constraints in scenarios with small translations and large rotations. It detects dynamic points through the reprojection error of the rotation homography matrix. in, For a homography matrix containing only rotational components, The distance is Euclidean.

[0035] Motion consistency cost: The motion of static points is caused only by camera motion, while the motion direction or velocity of dynamic points is inconsistent with the background. Based on this, a motion consistency cost model is constructed: in, The global optical flow field caused by camera motion. For point The local optical flow vector.

[0036] ② Loop-based iterative camera array extrinsic parameter timing correction: The main purpose is to achieve cross-module parameter collaborative optimization. Through iterative feedback of camera array extrinsic parameter optimization, the accumulation of positioning errors in long-term images of the airborne moving platform is suppressed. The iterative optimization function is designed as follows: in, For the camera timing extrinsic parameters to be optimized, The three-dimensional coordinates of the natural feature control points. For the corresponding image point coordinates, For projection function, The camera extrinsic parameters are the observation frames to which the i-th feature point belongs.

[0037] The core of this function is how to determine the dynamic weight of the loop. By establishing a spatiotemporal correlation closed loop between the current frame and historical frames, dynamic weight correction is verified using descriptor similarity confidence and geometric consistency. The weights are generated by jointly evaluating the appearance consistency (cross-frame similarity) and geometric consistency (projection error) of feature points. :

[0038]

[0039] in, For the current frame and history Mean cosine similarity of frames The reprojection error is calculated from the temporal extrinsic parameters of the camera to be optimized. for The corresponding matching point pairs yield a weighted feature point set. And will The objective function is input as a priori condition, and the LM algorithm is used to iteratively solve for the optimal extrinsic parameters. The current optimization result updates the historical frame database, and the updated historical data participates in the weight calculation of subsequent frames. The optimization direction is dynamically adjusted through weights to form an error suppression closed loop, and finally the optimized high-precision camera array temporal extrinsic parameters are obtained. .

[0040] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for time-series correction of extrinsic parameters of a camera array using loop-loop iterative correction of natural feature control points, characterized in that, Includes the following steps: Acquire raw image data from the moving platform camera array; A set of natural feature control points is obtained through multi-scale cascaded natural feature control point optimization. A normalized geometric cost model is constructed, and a spatiotemporal correlation closed loop between the current frame and historical frames is established based on the natural feature control point set. The dynamic weight correction is verified by descriptor similarity confidence and geometric consistency. A weighted feature point set is generated as a priori condition input to the target optimization function. The optimal extrinsic parameters are solved iteratively by using the LM algorithm. The historical frame database is updated by the current optimization result to form an error suppression closed loop and obtain the optimized temporal extrinsic parameters of the camera array. Dynamic weight correction is verified using descriptor similarity confidence and geometric consistency, and weights are generated by jointly evaluating the appearance consistency and geometric consistency of feature points. ; in, For the current frame and history Mean cosine similarity of frames The reprojection error is calculated from the temporal extrinsic parameters of the camera to be optimized. for The corresponding matching point pairs yield a weighted feature point set. ; and will The objective function is input as a priori condition, and the LM algorithm is used to iteratively solve for the optimal extrinsic parameters. The current optimization result updates the historical frame database, and the updated historical data participates in the weight calculation of subsequent frames. The optimization direction is dynamically adjusted through weights to form an error suppression closed loop, and finally the optimized high-precision camera array temporal extrinsic parameters are obtained. .

2. The method for correcting the timing of extrinsic parameters of a camera array using loop-loop iterative camera arrays based on natural feature control points according to claim 1, characterized in that, The set of natural feature control points is obtained through multi-scale cascaded natural feature control point optimization, specifically including: ① Multi-scale layer geometric stability enhancement: Construct a Gaussian pyramid with 5 scales for the original image, calculate the Hessian matrix of natural feature points in the Gaussian image at each scale, determine geometric stability based on the eigenvalues ​​of the Hessian matrix, and select a set of control points for geometric stability enhancement. ② Optimize the local entropy spatial distribution of the control point set: Divide the sequence image into an initial uniform grid, calculate the local entropy value of each grid, subdivide the grids with local entropy values ​​exceeding the threshold, generate the Voronoi map of the natural feature control point set, and supplement feature points and remove clustered points through the gravity-repulsion balance model to obtain a uniformly distributed feature point set; ③ Use similarity to constrain feature quality Obtain the set of natural feature control points: Extract image blocks centered on the left image points, calculate their SSIM similarity with the corresponding image blocks in the right image, select high-quality feature points that meet the similarity threshold, and combine the same operation with the right image to obtain the final set of natural feature control points.

3. The natural feature control point loop-loop iterative camera array extrinsic parameter timing correction method according to claim 2, characterized in that, The initial uniform grid of the grid whose local entropy exceeds the threshold is 8×6, and the subdivided grid is 16×12; the threshold of the local entropy is 0.7 times the global entropy of the image; in the gravity-repulsion balance model, the condition for determining the aggregation point is that the distance between feature points is less than 0.5 times the median of the side length of the global Voronoi cell.

4. The method for correcting the timing of extrinsic parameters of a camera array using loop-loop iterative camera arrays based on natural feature control points according to claim 1, characterized in that... In multi-scale layer geometric stability enhancement, specific measures are taken for each scale layer. Establish geometric stability criteria and obtain the set of natural feature control points that enhance geometric stability. : ； in, , It is a scale layer eigenvalues, The empirical threshold is 10.

Images