Camera distortion correction method based on spectral correction iterative LM algorithm

Abstract

The application provides a camera distortion correction method based on a spectral correction iterative LM algorithm, and belongs to the technical field of distortion correction. In view of the fact that in the existing nonlinear distortion correction model solving process, the nonlinear parameter estimation is not accurate, thereby directly affecting the correction result, the application innovatively provides a novel camera distortion correction method based on the spectral correction iterative LM algorithm. Specifically, a nonlinear function model of a camera imaging system is constructed according to a distortion correction model, and an observation value vector is obtained; an error equation of a nonlinear least square problem is constructed by using a Taylor formula, an iterative formula of a to-be-estimated parameter of the nonlinear least square problem is constructed according to a spectral correction iteration method; spectral correction estimation is calculated, an actual-to-estimated drop ratio is calculated, and the parameter estimation value of the secondary iteration is updated; and a judgment threshold is set, and when the convergence condition is met, the optimal parameter estimation value is output.

Description

Technical Field

[0001] This application relates to the field of distortion correction, and specifically proposes a camera distortion correction method based on the spectral correction iterative LM algorithm. Background Technology

[0002] To improve camera image quality, distortion correction models are typically used to correct camera distortion. In solving nonlinear distortion correction models, the instability and uncertainty of parameter estimation are significant factors that can easily affect the correction results. When using numerical iterative algorithms to solve nonlinear least squares problems, the unstable iterative sequence generated by the iterative formula can affect the accuracy of the parameter correction values, ultimately leading to unreliable nonlinear parameter estimates.

[0003] Existing techniques, based on iterative algorithms such as Newton-type algorithms, gradient descent, and the Gauss-Newton algorithm, have yielded numerous hybrid improved algorithms. These improvements primarily focus on enhancing iterative computation efficiency, avoiding matrix inversion during iteration, and enhancing practical applications in specific disciplines. Among existing numerical iterative algorithms, the LM algorithm is a widely used approach for solving such problems.

[0004] In recent years, various LM algorithms have been proposed for nonlinear least squares problems in different disciplines. However, research on combining the LM algorithm with methods for handling ill-conditioned linear problems is relatively limited. Spectral correction iteration is a commonly used method for handling ill-conditioned linear problems, and its iterative formula is similar to that of the LM algorithm. Spectral correction iteration has wide applications in various fields, and its connection with the LM algorithm gives it specific application value.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] This application addresses the problem of inaccurate estimation of nonlinear parameters during the solution process of existing nonlinear distortion correction models, which directly affects the missing steps and deficiencies of the correction results. Based on the spectral correction iterative LM algorithm, a novel camera distortion correction method is proposed.

[0007] To achieve the aforementioned objectives, this application proposes a camera distortion correction method based on a spectral correction iterative LM algorithm. This method constructs a nonlinear function model of the camera imaging system based on the distortion correction model and obtains the observation vector. It then uses Taylor's formula to construct the error equation for the nonlinear least squares problem and constructs iterative formulas for the parameters to be estimated in the nonlinear least squares problem based on the spectral correction iterative method. Finally, it calculates the spectral correction estimates and the ratio of the actual to the estimated decrease. ,renew The parameter estimates for the next iteration are obtained; a threshold is set, and the optimal parameter estimate is output when the convergence condition is met.

[0008] The implementation steps include the following:

[0009] S1. Construct a nonlinear function model of the camera imaging system based on the distortion correction model, and obtain the observation vector;

[0010] S2. Based on the nonlinear function model and the observation vector, the error equation of the nonlinear least squares problem is constructed using Taylor's formula.

[0011] S3. Construct the iterative formula for the parameters to be estimated in the nonlinear least squares problem based on the spectral correction iterative method, and calculate the spectral correction estimate by combining the initial value of the LM with the spectral correction iteration.

[0012] S4. Calculate the initial damping factor and the ratio of actual descent to estimated descent. And based on the initial damping factor, spectral correction estimate and renew The parameter estimates for the next iteration;

[0013] S5. Set a judgment threshold and make a judgment based on the change in the residual vector between two adjacent outer iterations;

[0014] If the residual vectors of two consecutive outer iterations are greater than or equal to the judgment threshold, return to the spectral correction estimation step; if the residual vectors of two consecutive outer iterations are less than the judgment threshold, output... The parameter estimates for the next iteration are the optimal estimates of the parameters.

[0015] S1, as described above, involves constructing a nonlinear function model of the camera imaging system based on the distortion correction model. And obtain the observation vector. , as follows:

[0016] ;

[0017] ;

[0018] in, for The dimension of, let for index, , for The One component; Let be the vector of parameters to be estimated. for The dimension of, let for index, This is the transpose symbol.

[0019] The aforementioned S2 utilizes Taylor's formula in initial approximation Expanding the equation to the first-order term, we obtain the error equation for the nonlinear least squares problem as follows:

[0020] ;

[0021] ;

[0022] in, The residual vector of the observed values. for The parameter estimates, For Jacobian matrices, For incremental operators, This is a parameter correction value.

[0023] The S3 mentioned above includes,

[0024] S3.1 The iterative formula for the parameters to be estimated in the nonlinear least squares problem is constructed based on the spectral correction iteration as follows:

[0025] ;

[0026] ;

[0027] In the formula, The damping factor, It is the identity matrix;

[0028] S3.2, Let Index of the outer iteration count, The index for the inner iteration number is used to calculate the initial value of the spectral correction iteration. as follows:

[0029] ;

[0030] in, for Jacobian matrix at the location, for The predicted value of the nonlinear function model at that point. These are the estimated values ​​of the current parameter vector to be estimated;

[0031] S3.3, No. Subspectral Correction Valuation as follows:

[0032] ;

[0033] in, This represents the number of inner iterations.

[0034] Step S4 includes,

[0035] S4.1 Calculate the actual descent amount as follows:

[0036] ;

[0037] in, It is a function of the sum of squared residuals;

[0038] Calculate the estimated decrease as follows:

[0039] ;

[0040] ;

[0041] ;

[0042] in, It is a quadratic approximation function;

[0043] S4.2 Calculate the initial damping factor as follows:

[0044] ;

[0045] in, Scaling factor For diagonal operators, for Jacobian matrix at the location;

[0046] Calculate the ratio of actual to estimated decrease as follows:

[0047] ;

[0048] like ,but ,Will times value assigned to ;

[0049] like ,but ,Will value assigned to ;

[0050] like ,but ,Will The value assigned to ;

[0051] in, For the first The parameter estimates of the next outer iteration.

[0052] Step S5 includes,

[0053] Set the judgment threshold ,set up For the first The residual vector of the observations from the next outer iteration, if Output ;

[0054] like Then let Increase by 1 and return to obtain the spectral correction estimate.

[0055] In step S5, a threshold for the number of corrections is set. ,when When the time comes, stop the spectral correction iteration;

[0056] The distortion correction model can be the Brown model, the polynomial model, the Fourier model, or the Legendre model.

[0057] This application proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned camera distortion correction method based on the spectral correction iterative LM algorithm.

[0058] This application proposes a computer-readable storage medium storing a computer program that, when executed, can implement the aforementioned camera distortion correction method based on the spectral correction iterative LM algorithm.

[0059] In summary, this application has the following advantages and beneficial effects compared with the prior art:

[0060] 1. This application solves the parameter correction number by spectral correction iterative stability and combines it with an adaptive damping factor adjustment mechanism, which can significantly improve the numerical stability, convergence speed and estimation accuracy of camera distortion correction, effectively reduce the dependence on initial parameter values ​​and has good engineering applicability.

[0061] 2. Specifically, in self-calibrated bundle adjustment, this application can significantly improve the consistency of estimation accuracy for the same distortion model, thereby enhancing the applicability of the improved LM algorithm in self-calibration problems;

[0062] 3. Specifically, from the perspective of the line element error norm, the LM algorithm based on spectral correction proposed in this application can achieve LM... TR+CCVMaximizing solution accuracy; specifically, under each distortion model, the solution accuracy is improved by 75.2%, 84.8%, and 84.8% respectively compared to existing trust region algorithms. Attached Figure Description

[0063] The implementation process of the present application will now be further explained and illustrated with reference to the following figures.

[0064] Figure 1 This is a schematic diagram of the flight path in Example 1;

[0065] Figure 2 For LM TR +Brown's reprojection error map of corresponding image points;

[0066] Figure 3 For LM TR +FRP reprojection error map of corresponding image points;

[0067] Figure 4 For LM TR +LRP image point reprojection error map;

[0068] Figure 5 For LM TR Reprojection error map of corresponding image points of +CCV+Brown;

[0069] Figure 6 For LM TR Reprojection error map of corresponding image points of +CCV+FRP;

[0070] Figure 7 For LM TR Reprojection error map of corresponding image points of +CCV+LRP;

[0071] Figure 8 This is a flowchart of the camera distortion correction method based on the spectral correction iterative LM algorithm described in this application. Detailed Implementation

[0072] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be further described below in conjunction with the accompanying drawings and embodiments. Many specific details are set forth in the following description to provide a thorough understanding of this application; however, this application may be implemented in other ways than those described herein, and therefore, this application is not limited to the specific embodiments disclosed below.

[0073] Example 1, as Figures 1 to 8As shown, this application proposes a camera distortion correction method based on the spectral correction iterative LM algorithm. It constructs a nonlinear function model of the camera imaging system based on the distortion correction model and obtains the observation vector; it uses Taylor's formula to construct the error equation for the nonlinear least squares problem, and constructs the iterative formula for the parameters to be estimated in the nonlinear least squares problem based on the spectral correction iterative method; it calculates the spectral correction estimate and the ratio of the actual to the estimated drop. ,renew The parameter estimates for the next iteration are set, and the optimal parameter estimates are output when the convergence condition is met.

[0074] In response, this application solves for parameter corrections through spectral correction iterative stability and combines it with an adaptive damping factor adjustment mechanism, which can significantly improve the numerical stability, convergence speed and estimation accuracy of camera distortion correction, effectively reduce the dependence on initial parameter values, and thus have good engineering applicability.

[0075] Specifically, the method includes the following implementation steps:

[0076] S1. Construct a nonlinear function model of the camera imaging system based on the distortion correction model, and obtain the observation vector;

[0077] This includes constructing a nonlinear function model of the camera imaging system based on the distortion correction model. And obtain the observation vector. , as follows:

[0078] ;

[0079] ;

[0080] in, for The dimension of, let for index, , for The One component; Let be the vector of parameters to be estimated. for The dimension of, let for index, It is the transpose symbol;

[0081] S2. Based on the nonlinear function model and the observation vector, the error equation for the nonlinear least squares problem is constructed using Taylor's formula; including,

[0082] Using Taylor's formula initial approximation Expanding the equation to the first-order term, we obtain the error equation for the nonlinear least squares problem as follows:

[0083] ;

[0084] ;

[0085] in, The residual vector of the observed values. for The parameter estimates, For Jacobian matrices, For incremental operators, This is a parameter correction value;

[0086] S3. Construct iterative formulas for the parameters to be estimated in the nonlinear least squares problem based on the spectral correction iterative method, and calculate the spectral correction estimate using LM combined with the initial values ​​of the spectral correction iteration; including,

[0087] S3.1 The iterative formula for the parameters to be estimated in the nonlinear least squares problem is constructed based on the spectral correction iteration as follows:

[0088] ;

[0089] ;

[0090] In the formula, The damping factor, It is the identity matrix;

[0091] Based on the above spectral correction iterative process, the normal equations are... Spectral corrections are applied to both sides simultaneously. An improved LM iteration formula based on spectral correction was derived, which is superior to the standard LM iteration formula. The right-hand item becomes From the perspective of matrix eigenvalues, after expanding the improved iterative formula, for the i-th eigendirection, the matrix eigenvalues ​​are... The gradient is The iterative formula is For ill-conditioned matrices with large condition numbers, the spectral correction LM algorithm proposed in this application smooths the ill-conditioned direction through historical updates, which is equivalent to regularizing the solution space and effectively reducing the influence of the condition number.

[0092] S3.2, Let Index of the outer iteration count, The index for the inner iteration number is used to calculate the initial value of the spectral correction iteration. as follows:

[0093] ;

[0094] in, for Jacobian matrix at the location, for The predicted value of the nonlinear function model at that point. These are the estimated values ​​of the current parameter vector to be estimated;

[0095] S3.3, No. Subspectral Correction Valuation as follows:

[0096] ;

[0097] in, This represents the number of inner iterations.

[0098] In the standard LM iteration formula of the existing technology, when the eigenvalues ​​of the normal matrix... , Almost entirely dependent on the damping factor The value, if If the value is large, then the update in that direction is almost non-existent; if If the value is smaller, the direction may be unstable; however, this application improves the LM iterative formula based on spectral correction, making... This means that updates in the direction of small eigenvalues ​​remain continuous and will not be affected by changes in the eigenvalue direction. Tiny changes cause dramatic fluctuations in that direction, and can make full use of the effective information accumulated in previous iterations.

[0099] Furthermore, the improved iterative formula proposed in this application has stability guarantees in the direction of small eigenvalues.

[0100] That is, spectral correction term In fact, implicit second-order information is introduced as follows:

[0101]

[0102] Therefore, it can not only effectively improve the ill-conditioned nature of the equation, but also maintain the inertia of the update direction, enabling the improved LM algorithm to reach a high-precision solution more quickly;

[0103] S4. Calculate the initial damping factor and the ratio of actual descent to estimated descent. And based on the initial damping factor, spectral correction estimate and renew The parameter estimates for the next iteration include,

[0104] S4.1 Calculate the actual descent amount as follows:

[0105] ;

[0106] in, It is a function of the sum of squared residuals;

[0107] Calculate the estimated decrease as follows:

[0108] ;

[0109] ;

[0110] ;

[0111] in, It is a quadratic approximation function;

[0112] S4.2 Calculate the initial damping factor as follows:

[0113] ;

[0114] in, Scaling factor For diagonal operators, for Jacobian matrix at the location;

[0115] Calculate the ratio of actual to estimated decrease as follows:

[0116] ;

[0117] like ,but ,Will times The value assigned to ;

[0118] like ,but ,Will The value assigned to ;

[0119] like ,but ,Will The value assigned to ;

[0120] in, For the first Parameter estimates for the next outermost iteration;

[0121] S5. Set a judgment threshold and make a judgment based on the change in the residual vector between two adjacent outer iterations;

[0122] If the residual vectors of two consecutive outer iterations are greater than or equal to the judgment threshold, return to the spectral correction estimation step; if the residual vectors of two consecutive outer iterations are less than the judgment threshold, output... The parameter estimates for the next iteration are the optimal parameter estimates; including,

[0123] Set the judgment threshold ,set up For the first The residual vector of the observations from the next outer iteration, if Output ;

[0124] like Then let Increase by 1 and return to obtain the spectral correction estimate;

[0125] Furthermore, a threshold for the number of corrections can be set. ,when When the time comes, stop the spectral correction iteration;

[0126] Furthermore, the distortion correction model can be the Brown model, the polynomial model, the Fourier model, or the Legendre model.

[0127] Based on the above design concept, the following specific implementation scheme for self-calibrated bundle adjustment experiment can be applied to this application.

[0128] Specifically, such as Figure 1 The flight path shown is based on recent aerial imagery acquired using a UAV. The flight direction is north-south, with an average absolute flight altitude of approximately 112 meters. The aerial scale is 1:10000. The forward overlap of the survey area is set at 80%, and the lateral overlap at 60%. The imagery consists of 28 images along three flight paths. In actual aerial photogrammetry work, the true values ​​of the image's internal and external orientation elements and distortion parameters are unknown.

[0129] This application utilizes position and attitude data measured during UAV operations to obtain approximate exterior orientation elements for each image after error correction and rotation transformation, and then evaluates the parameter estimation results based on these approximate values. In the embodiment, the SURF algorithm is used for image matching to obtain corresponding image points on the images, followed by relative-to-absolute orientation. During this process, spatial forward intersection is performed based on the collinearity equation, and a spatial similarity transformation is applied to obtain the NED coordinates of the object points. These coordinates are then used as initial values ​​for self-calibrated bundle adjustment under conditions without ground control. The analysis uses the condition number method to measure the ill-conditioning of the iteration matrix, and compares and analyzes the improvement effect of different algorithms on the ill-conditioning by combining the parameter iteration process and calculation results.

[0130] This embodiment uses LM. TR(The Trust Region-Based LM Algorithm) and LM TR+CCV (This application combines the method of the LM algorithm based on the trust region) with the Brown model, the Fourier and radial distortion-quadratic polynomial hybrid model (FRP), and the Legendre and radial distortion-quadratic polynomial hybrid model (LRP), respectively, and sets... , A self-calibrated bundle adjustment method is used to iteratively calculate the image's interior and exterior orientation elements and distortion parameters. The accuracy of the parameter estimation is evaluated using approximate values ​​of the image's exterior orientation elements. Based on the L-curve, the iteration count corresponding to the point of maximum curvature of the L-curve is selected as the optimal threshold. Image data (ImageData) is iteratively calculated to obtain the interior and exterior orientation elements, distortion parameters, and NED coordinates of the corresponding object points for each image. The reprojection error distribution of corresponding image points for each algorithm is plotted, as shown below. Figures 2 to 7 As shown in Table 1, the line element error norm, corner element error norm, and reprojection error norm of each algorithm were calculated.

[0131] Table 1. Line element error norm, corner element error norm, and reprojection error norm for each algorithm

[0132] .

[0133] This application also proposes a novel electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the aforementioned camera distortion correction method based on the spectral correction iterative LM algorithm.

[0134] This application also proposes a novel computer-readable storage medium storing a computer program that, when executed, enables the implementation of the aforementioned camera distortion correction method based on the spectral correction iterative LM algorithm.

[0135] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A camera distortion correction method based on the spectral correction iterative LM algorithm, characterized in that: The implementation steps include the following: S1. Construct a nonlinear function model of the camera imaging system based on the distortion correction model, and obtain the observation vector; A nonlinear function model of the camera imaging system is constructed based on the distortion correction model. And obtain the observation vector of the same image point on the image. , as follows: ； ； in, for The dimension of, let for index, , for The One component; The vector of parameters to be estimated includes the interior and exterior orientation elements and distortion parameters of the image calculated iteratively. for The dimension of, let for index, It is the transpose symbol; S2. Based on the nonlinear function model and the observation vector, construct the error equation for the nonlinear least squares problem using Taylor's formula; apply Taylor's formula to... initial approximation Expanding the equation to the first-order term, we obtain the error equation for the nonlinear least squares problem as follows: ； ； in, The residual vector of the observed values. for The parameter estimates, For Jacobian matrices, For incremental operators, This is a parameter correction value; S3. Construct iterative formulas for the parameters to be estimated in the nonlinear least squares problem based on the spectral correction iterative method, and calculate the spectral correction estimate using LM combined with the initial values ​​of the spectral correction iteration; including, S3.1 The iterative formula for the parameters to be estimated in the nonlinear least squares problem is constructed based on the spectral correction iteration as follows: ； ； In the formula, The damping factor, It is the identity matrix; S3.2, Let Index of the outer iteration count, The index for the inner iteration number is used to calculate the initial value of the spectral correction iteration. as follows: ； in, for Jacobian matrix at the location, for The predicted value of the nonlinear function model at that point. These are the estimated values ​​of the parameters in the current parameter vector to be estimated. S3.3, No. Subspectral Correction Valuation as follows: ； in, This represents the number of inner iterations. S4. Calculate the initial damping factor and the ratio of actual descent to estimated descent. And based on the initial damping factor, spectral correction estimate and renew The parameter estimates for the next iteration include, S4.1 Calculate the actual descent amount as follows: ； in, It is a function of the sum of squared residuals; Calculate the estimated decrease as follows: ； ； ； in, It is a quadratic approximation function; S4.2 Calculate the initial damping factor as follows: ； in, Scaling factor For diagonal operators, for Jacobian matrix at the location; Calculate the ratio of actual to estimated decrease as follows: ； like ,but ,Will times value assigned to ; like ,but ,Will value assigned to ; like ,but ,Will value assigned to ; in, For the first Parameter estimates for the next outermost iteration; S5. Set a judgment threshold and make a judgment based on the change in the residual vector between two adjacent outer iterations; If the residual vectors of two consecutive outer iterations are greater than or equal to the judgment threshold, return to the spectral correction estimation step; if the residual vectors of two consecutive outer iterations are less than the judgment threshold, output... The parameter estimates for the next iteration are the optimal estimates of the parameters.

2. The camera distortion correction method based on the spectral correction iterative LM algorithm according to claim 1, characterized in that: Step S5 includes, Set the judgment threshold ,set up For the first The residual vector of the observations from the next outer iteration, if Output ; like Then let Increase by 1 and return to obtain the spectral correction estimate.

3. The camera distortion correction method based on the spectral correction iterative LM algorithm according to claim 1, characterized in that: In step S5, a threshold for the number of corrections is set. ,when When the time comes, stop the spectral correction iteration; The distortion correction model can be the Brown model, the polynomial model, the Fourier model, or the Legendre model.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the camera distortion correction method based on the spectral correction iterative LM algorithm as described in any one of claims 1 to 3.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, it can implement the camera distortion correction method based on the spectral correction iterative LM algorithm as described in any one of claims 1 to 3.

Images