Camera lens calibration method and device, computer equipment and medium

Abstract

The invention provides a camera lens calibration method and device, computer equipment and a medium. The method comprises the following steps: acquiring calibration plate image data acquired by a distortion camera; determining a first calibration internal reference of the distortion camera by adopting a Zhang's method based on the calibration plate image data, and determining a second calibration internal reference of the distortion camera by adopting an analytical method based on the calibration plate image data; weight matching is carried out on the first calibration internal reference and the second calibration internal reference, and a mixed calibration internal reference of the distortion camera is determined based on the first calibration internal reference, the calibration weight corresponding to the first calibration internal reference, the second calibration internal reference and the calibration weight corresponding to the second calibration internal reference; acquiring initial calibration external parameters of the distortion camera; and performing staged nonlinear optimization on the mixed calibration internal reference of the distortion camera and the initial calibration external reference of the distortion camera to obtain a target calibration internal reference and a target calibration external reference of the distortion camera so as to perform lens calibration on the distortion camera. Therefore, the accuracy of the calibration result is effectively improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of image processing technology, and in particular, to a lens calibration method, apparatus, computer equipment, and medium applicable to a camera. Background Art

[0002] When performing photogrammetric 3D modeling, conventional camera lenses are often insufficiently polished, resulting in uneven surfaces. This can cause varying degrees of distortion in the captured images, compromising 3D model accuracy. Lens correction can significantly improve measurement accuracy, image quality, and consistency. Furthermore, when stitching multiple photos together, it can eliminate inconsistencies caused by lens issues, resulting in a more accurate stitched image.

[0003] In the related art, lens calibration is mainly performed through camera calibration of a single plane checkerboard. This method uses a plane checkerboard pattern as a calibration plate and takes multiple images from different angles to calculate the camera's intrinsic and extrinsic parameters, as well as the distortion coefficient.

[0004] However, using the existing method, the calibration result is inaccurate. Summary of the Invention

[0005] The embodiments described herein provide a camera lens calibration method, apparatus, computer device, and medium, which overcome the above-mentioned problems.

[0006] In a first aspect, according to the present disclosure, a camera lens calibration method is provided, comprising:

[0007] Obtain the calibration plate image data collected by the distortion camera;

[0008] Based on the calibration plate image data, a first calibration intrinsic parameter of the distorted camera is determined using Zhang's method, and based on the calibration plate image data, a second calibration intrinsic parameter of the distorted camera is determined using an analytical method; the Zhang's method is implemented using the calibrateCamera function of OpenCV, and the analytical method is implemented through homography matrix decomposition;

[0009] performing weighted ratio matching on the first calibration internal parameter and the second calibration internal parameter respectively, and determining a hybrid calibration internal parameter of the anamorphic camera based on the first calibration internal parameter, the calibration weight corresponding to the first calibration internal parameter, the second calibration internal parameter, and the calibration weight corresponding to the second calibration internal parameter;

[0010] Obtaining initial calibration extrinsic parameters of the distortion camera;

[0011] The hybrid calibration intrinsic parameters of the distorted camera and the initial calibration extrinsic parameters of the distorted camera are nonlinearly optimized in stages to obtain target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera, so as to perform lens calibration on the distorted camera.

[0012] In a second aspect, according to the present disclosure, a camera lens calibration device is provided, comprising:

[0013] A first acquisition module is used to acquire calibration plate image data collected by the distortion camera;

[0014] a first determination module, configured to determine a first calibration intrinsic parameter of the anamorphic camera using a Zhang method based on the calibration plate image data, and to determine a second calibration intrinsic parameter of the anamorphic camera using an analytical method based on the calibration plate image data;

[0015] a second determination module, configured to weight the first calibration internal parameter and the second calibration internal parameter, respectively, and determine a hybrid calibration internal parameter of the anamorphic camera based on the first calibration internal parameter, the calibration weight corresponding to the first calibration internal parameter, the second calibration internal parameter, and the calibration weight corresponding to the second calibration internal parameter;

[0016] A second acquisition module is used to obtain the initial calibration extrinsic parameters of the distortion camera;

[0017] The optimization module is used to perform phased nonlinear optimization on the hybrid calibration intrinsic parameters of the distorted camera and the initial calibration extrinsic parameters of the distorted camera to obtain the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera, so as to perform lens calibration on the distorted camera.

[0018] In a third aspect, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the camera lens calibration method in any of the above embodiments are implemented.

[0019] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the camera lens calibration method in any of the above embodiments are implemented.

[0020] The camera lens calibration method provided in an embodiment of the present application obtains calibration plate image data collected by a distortion camera; based on the calibration plate image data, a first calibration intrinsic parameter of the distortion camera is determined using Zhang's method, and based on the calibration plate image data, a second calibration intrinsic parameter of the distortion camera is determined using an analytical method; the first calibration intrinsic parameter and the second calibration intrinsic parameter are weighted, and based on the first calibration intrinsic parameter, the calibration weight corresponding to the first calibration intrinsic parameter, the second calibration intrinsic parameter, and the calibration weight corresponding to the second calibration intrinsic parameter, a mixed calibration internal parameter of the distortion camera is determined; initial calibration extrinsic parameters of the distortion camera are obtained; and the mixed calibration internal parameters of the distortion camera and the initial calibration extrinsic parameters of the distortion camera are subjected to staged nonlinear optimization to obtain target calibration intrinsic parameters and target calibration extrinsic parameters of the distortion camera, so as to perform lens calibration on the distortion camera. In this way, by combining Zhang's method and analytical method to calculate initial parameters, and improving the accuracy of the initial parameters by setting different weights, and then optimizing the initial parameters, the accuracy of the calibration results is effectively improved.

[0021] The above description is only an overview of the technical solutions of the embodiments of the present application. In order to more clearly understand the technical means of the embodiments of the present application, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below. It should be noted that the drawings described below only relate to some embodiments of the present disclosure and are not intended to limit the present disclosure.

[0023] Figure 1 The present invention provides a flow chart of a camera lens calibration method.

[0024] Figure 2 Schematic diagram of a standard chessboard provided by the present disclosure.

[0025] Figure 3 This is a schematic diagram obtained by photographing a standard chessboard with a camera provided by the present disclosure.

[0026] Figure 4 It is an original distortion image of a standard chessboard provided by the present disclosure.

[0027] Figure 5 This is a calibration result diagram of a standard chessboard provided by the present invention.

[0028] Figure 6 This is a calibration result diagram of another standard chessboard provided by the present disclosure.

[0029] Figure 7 It is a structural schematic diagram of a camera lens calibration device provided by the present disclosure.

[0030] Figure 8 It is a structural diagram of a computer device provided by the present disclosure.

[0031] It should be noted that the elements in the drawings are schematic and not drawn to scale. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work also fall within the scope of protection of the present disclosure.

[0033] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the specification and the relevant art, and will not be interpreted in an idealized or overly formal manner unless otherwise explicitly defined herein. As used herein, a statement that two or more parts are "connected" or "coupled" together shall mean that the parts are joined together either directly or through one or more intermediate components.

[0034] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase "embodiment" in various places in the specification does not necessarily refer to the same embodiment, nor does it necessarily refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0035] The term "and / or" in this document simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists, A and B exist simultaneously, and B exists. Additionally, the character " / " in this document generally indicates that the related objects are in an "or" relationship. Terms such as "first" and "second" are used solely to distinguish one component (or portion of a component) from another component (or portion of a component).

[0036] In the description of this application, unless otherwise specified, "plurality" means more than two (including two), and similarly, "multiple groups" means more than two (including two).

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0038] Figure 1 FIG. 1 is a flow chart of a camera lens calibration method provided by an embodiment of the present disclosure, such as Figure 1 As shown in FIG, the specific process of the camera lens calibration method includes:

[0039] S110: Obtain calibration plate image data captured by the distortion camera.

[0040] The calibration plate used in this embodiment is in a standard chessboard template format, and its specification is that each chessboard grid is a 22 mm rectangle.

[0041] S120 : Based on the calibration plate image data, use Zhang's method to determine a first calibration internal parameter of the anamorphic camera, and based on the calibration plate image data, use an analytical method to determine a second calibration internal parameter of the anamorphic camera.

[0042] The first calibration intrinsic parameter is the first intrinsic parameter matrix calculated by Zhang's method, and the second calibration intrinsic parameter is the second intrinsic parameter matrix calculated by the analytical method. Zhang's method is implemented using the calibrateCamera function in OpenCV, while the analytical method is implemented using homography matrix decomposition.

[0043] In some embodiments, an analytical method is used to determine a second calibration intrinsic parameter of the distortion camera based on the calibration plate image data, including:

[0044] Obtain a homography matrix representation corresponding to the calibration plate image data; and determine a second calibration intrinsic parameter of the distortion camera based on the homography matrix representation corresponding to the calibration plate image data.

[0045] Among them, each calibration plate image can be calculated to obtain a homography matrix representation H, as shown in the following formula (1).

[0046]

[0047] Furthermore, the DLT matrix can be constructed by normalizing the coordinate points, and H can be solved through singular value decomposition (SVD). The constraint equation can be constructed using the H matrix of multiple calibration plate images to solve the initial intrinsic parameter matrix K2 (i.e., the second calibration intrinsic parameter).

[0048] In some embodiments, obtaining a homography matrix representation corresponding to the calibration plate image data includes:

[0049] Based on the two-dimensional coordinates of the points corresponding to the calibration plate image data, a first linear constraint equation and a second linear constraint equation are constructed.

[0050] For example, defining the homography matrix H=K2[r1, r2, t], we have the following formula (2).

[0051]

[0052] In formula (2), S = Zc, which is the scale factor; f x and f y are the focal lengths in the horizontal and vertical directions, respectively; H is the transformation matrix that converts the points in the three-dimensional world of the calibration plate image data into the two-dimensional world coordinate system. Then, expand the equation and eliminate S to obtain two (i.e., the first linear constraint equation and the second linear constraint equation), as shown in equation (3).

[0053]

[0054] Based on the first linear constraint equation and the second linear constraint equation, a linear constraint matrix is ​​determined.

[0055] Combined with the above example, each (X i ,Y i )<->(u i ,v i ) is substituted into equation (3) and rearranged into the form of AH=0, as shown in equation (4).

[0056]

[0057] In formula (4), A is the linear constraint matrix.

[0058] After stacking, it is shown in the following formula (5).

[0059] Ah=0,h=[h 11 ,h 12 ,h 13 ,h 21 ,h 22 ,h 23 ,h 31 ,h 32 ,h 33 ] T (5)

[0060] The linear constraint matrix is ​​decomposed by SVD to obtain multiple target singular vectors, and the multiple target singular vectors are arranged to obtain the homography matrix representation corresponding to the calibration plate image data.

[0061] Among them, the matrix A is decomposed into singular values, and the last column of V (corresponding to the right singular vector with the minimum singular value, i.e., the target singular vector) is taken as h. h is rearranged into a 3×3 matrix H, and the homography matrix representation H corresponding to the calibration plate image data is obtained.

[0062] In some embodiments, determining a second calibration intrinsic parameter of the distortion camera based on a homography matrix representation corresponding to the calibration plate image data includes:

[0063] Orthogonal constraints are imposed on the homography matrix representation corresponding to the calibration plate image data, and linear constraint parameters are obtained through Cholesky decomposition.

[0064] Combined with the above example, according to the properties of the rotation matrix: r T 1r2=0,||r1||=||r2|=r T 1r1=r T 2r2=1,r3=r1×r2。

[0065] Let h1, h2 be the first two columns of H Substituting the orthogonal constraint, we get the following equation (6).

[0066]

[0067] make Substituting into formula (6) yields the following formula (7).

[0068]

[0069] After expansion, the linear equation system is obtained, as shown in the following equation (8).

[0070]

[0071] In formula (8), b is the linear constraint parameter.

[0072] Based on the linear constraint parameter recombination, a reorganization constraint matrix is ​​obtained, and the reorganization constraint matrix is ​​decomposed to obtain the second calibration intrinsic parameter of the distortion camera.

[0073] Among them, the reorganization constraint matrix B obtained by reorganization of b is decomposed into B=K2 using Cholesky -T K2 -1 , we get the second calibration internal parameter K2 of the distortion camera, as shown in the following formula (9).

[0074]

[0075] S130 , weighting the first calibration internal parameter and the second calibration internal parameter respectively, and determining a hybrid calibration internal parameter of the distortion camera based on the first calibration internal parameter, the calibration weight corresponding to the first calibration internal parameter, the second calibration internal parameter, and the calibration weight corresponding to the second calibration internal parameter.

[0076] Among them, by comparing different K1 ratios, it can be found that as the K1 ratio increases, the number of function evaluations, the number of iterations, and the initial cost all decrease significantly. When K1 ≥ 0.7, the number of function evaluations decreases by more than 50%, indicating that a high K1 ratio can significantly reduce computational costs. Therefore, in this embodiment, the calibration weight corresponding to the first calibration intrinsic parameter K1 is selected as 0.7, and the calibration weight corresponding to the second calibration intrinsic parameter K2 is selected as 0.3, resulting in a hybrid calibration intrinsic parameter K3 = 0.7K1 + 0.3K2 for the distorted camera. This makes the camera's initial intrinsic parameters more accurate, effectively reducing the impact of the initial value on subsequent optimization, and improving the stability and convergence speed of the algorithm.

[0077] Although a higher K1 ratio can increase optimization efficiency, save computational costs, and enable rapid calibration, it may not be fully adaptable to complex distortion patterns. Therefore, this embodiment chooses to add a certain proportion of analytical K2. Although compared to a high-ratio solution, it retains the flexibility of customized optimization and is more adaptable to complex distortion patterns. After discriminant analysis, a balance between efficiency and cost is achieved when K1 accounts for 0.7 and analytical K2 accounts for 0.3. This is shown in the following equation (10).

[0078]

[0079] In formula (10), and These are the weighted fusion focal length solutions of Zhang's method and analytical method respectively; and is the initial focal length parameter obtained based on Zhang’s calibration method; and The initial focal length parameters calculated by the homography matrix analytical method.

[0080] S140: Obtain initial calibration extrinsic parameters of the distortion camera.

[0081] Among them, the following formula (11) is obtained from the properties of the homography matrix H and the standard rotation matrix.

[0082]

[0083] In formula (11), λ = ||A -1 h1||=||A -1 h2||.

[0084] The distortion coefficient is estimated as shown in the following equation (12).

[0085]

[0086] In formula (12), x 2 +y 2 =r 2; x and y represent the distorted horizontal and vertical coordinates respectively; X c and Y c They represent the restored abscissa and ordinate respectively; k1, k2, k3 represent radial distortion parameters.

[0087] Tangential distortion correction is shown in equation (13).

[0088]

[0089] In formula (13), p1 and p2 represent the tangential distortion parameters.

[0090] S150 , performing phased nonlinear optimization on the hybrid calibration intrinsic parameters of the distorted camera and the initial calibration extrinsic parameters of the distorted camera to obtain the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera, so as to perform lens calibration on the distorted camera.

[0091] In some embodiments, a hybrid calibration intrinsic parameter of the distorted camera and an initial calibration extrinsic parameter of the distorted camera are subjected to staged nonlinear optimization to obtain a target calibration intrinsic parameter and a target calibration extrinsic parameter of the distorted camera, including:

[0092] The trust region reflection algorithm is used to perform initial optimization on the hybrid calibration intrinsic parameters of the distorted camera. The Levenberg-Marquardt algorithm is used to fine-tune the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera after initial optimization to obtain the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera.

[0093] The Trust Region Reflective (TRF) algorithm is used for rough optimization. The TRF algorithm selects a trust region within the domain and continuously approaches the optimal solution. It dynamically adjusts the trust region based on the optimization quality during the optimization process, and reflects back when the optimization approaches the boundary. The TRF algorithm is robust, suitable for large-scale problems, and has boundary constraints to ensure global convergence. It is suitable for the initial stage of large-scale solution convergence quickly, avoiding local optimization.

[0094] The Levenberg-Marquardt (LM) algorithm, based on the initial convergence of the TRF algorithm, avoids the LM algorithm's tendency to converge poorly or locally. The LM algorithm combines the advantages of gradient descent and Gauss-Newton methods, ensuring both rapid convergence and improved algorithm stability. It is widely used in camera calibration and is suitable for second-stage refinement.

[0095] In some embodiments, a trust region reflection algorithm is used to initially optimize the hybrid calibration internal parameters of a distortion camera, including: defining an initialization parameter range of an objective function to be optimized and setting an objective function, where the objective function is calculated through the reprojection error of the calibration plate image; defining a trust region around the current parameter value to limit the amplitude of parameter change at each step; dynamically adjusting according to the optimization effect during the iteration process, and solving the optimal parameter adjustment direction of the approximate model under the trust region constraint; if the optimal parameter adjustment direction exceeds the physical boundary of the parameter, a reflection strategy is used to adjust the optimal parameter adjustment direction to ensure that the parameter always changes within a reasonable range; comparing the actual error reduction with the model prediction value, and when the parameter approaches the preset physical limit, correcting the optimal parameter adjustment direction through reflection or truncation operations to ensure that the iteration is always performed within the feasible domain; when the objective function value changes very little, the parameter adjustment amplitude is lower than a threshold, or the maximum number of iterations is reached, the optimization is terminated, and the current parameters are output as the result of the initial optimization.

[0096] Among them, the following formula (14) is the trust region reflective optimization model.

[0097]

[0098] In formula (14), x is the parameter vector to be optimized (such as camera internal parameters, distortion coefficients, etc.); f i (x) is the i-th residual term (reprojection error); l and u are the lower and upper bounds of the parameters, respectively. For example, the focal length must be positive and the distortion coefficient has a physical range limit; TRF can be called using the least squares function.

[0099] In numerical optimization, limiting the range of parameters enables the parameters to converge more quickly and ensures the stability of parameter optimization and the rationality of the final value to avoid over-optimization. The following formula (15) is used for boundary constraints, and the boundary constraint range is θ min <θ<θ max .

[0100] θ=[f x ,f y ,c x ,c y ,k1,k2,k3,p1,p2,…] (15)

[0101] Specifically, the internal parameter matrix constraint is implemented using the following formula (16).

[0102]

[0103] The following formula (17) is used to implement the distortion coefficient constraint.

[0104]

[0105] The external parameter constraint is: R∈R 3 ,T∈R 3 .

[0106] When calculating the homography matrix H by analytical method and the initial intrinsic parameter matrix by analytical solution, normalization is used to scale the range of coordinate points to [-1,1], avoiding numerical instability problems caused by coordinate values ​​that are too large or too small and simplifying the calculation.

[0107] In some embodiments, the Levenberg-Marquardt algorithm is used to fine-tune the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera after initial optimization to obtain the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera, including:

[0108] For the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera after initial optimization, the corresponding Jacobian matrix and residual vector are calculated by the finite difference method; the linear equation is solved based on the Jacobian matrix and residual vector to obtain the iteration step size; the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera are updated based on the iteration step size, and the objective function values ​​corresponding to the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera are evaluated; based on the objective function values ​​corresponding to the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera, the damping factor is adjusted to control the iteration direction of the iteration step size until the convergence condition is reached, and the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera are obtained.

[0109] Among them, the Levenberg-Marquardt algorithm is a hybrid method of the Gauss-Newton method and the gradient descent method, which dynamically adjusts the parameter update direction by introducing a damping factor λ.

[0110] By taking the TRF optimization results as the starting point and fine-tuning the internal parameters, distortion coefficients, and external parameters, SciPy automatically calculates the Jacobian matrix through the finite difference method and dynamically adjusts λ to balance the convergence speed and stability: (J T J+λI)δ=-J T r. Where J is the Jacobian matrix of the residual function; r is the residual vector, output by the objective function in the code; λ is the damping factor that controls the direction of the step size. λ→0: close to the Gauss-Newton method (converges quickly but may be unstable), λ→∞: close to the gradient descent method (stable but slow to converge).

[0111] During the iteration process, calculate the current residual r and Jacobian matrix J; solve the linear equation (J T J+λI)δ=-J T r gets the step size δ; calculate the new parameter θ new=θ+δ; evaluate the objective function value of the new parameters. If the error decreases, accept the update and reduce λ (increase the step size). If the error increases, reject the update and increase λ (reduce the step size); repeat until convergence (such as ftol or xtol conditions are met). The LM algorithm can be implemented using SciPy's least squares function (least squares method).

[0112] In this embodiment, calibration plate image data captured by an anamorphic camera is obtained; based on the calibration plate image data, a first calibration intrinsic parameter of the anamorphic camera is determined using Zhang's method, and based on the calibration plate image data, a second calibration intrinsic parameter of the anamorphic camera is determined using an analytical method; the first calibration intrinsic parameter and the second calibration intrinsic parameter are weighted, and a hybrid calibration intrinsic parameter of the anamorphic camera is determined based on the first calibration intrinsic parameter, the calibration weight corresponding to the first calibration intrinsic parameter, the second calibration intrinsic parameter, and the calibration weight corresponding to the second calibration intrinsic parameter; initial calibration extrinsic parameters of the anamorphic camera are obtained; and the hybrid calibration intrinsic parameters and the initial calibration extrinsic parameters of the anamorphic camera are subjected to staged nonlinear optimization to obtain target calibration intrinsic parameters and target calibration extrinsic parameters for the anamorphic camera, thereby performing lens calibration for the anamorphic camera. In this manner, by combining Zhang's method and the analytical method to calculate initial parameters, improving the accuracy of the initial parameters by setting different weights, and then optimizing the initial parameters, the accuracy of the calibration results is effectively improved.

[0113] In addition, the combined calibration of Zhang's method and analytical method in this embodiment was experimentally verified. Figure 2 The chessboard shown is a standard chessboard pattern. The black and white interlaced points are selected as calibration points. There are 6*6 calibration points (corner points). By adjusting the camera orientation and angle through rotation and translation, the chessboard scenes under different orientation conditions are continuously photographed for calibration. A total of 19 pictures are taken in this experiment (only the following pictures are shown here). Figure 3 The one shown is used as an example) calibration and image correction were performed using a custom algorithm and OpenCV algorithm, respectively. Each set of experiments was repeated 5 times to verify stability.

[0114] The test results and accuracy analysis of the distortion image correction are shown in Table 1.

[0115] Table 1 Distortion image correction test results and accuracy analysis

[0116]

[0117] It can be concluded that the error of the customized method is reduced by 29.6% compared with the traditional Zhang Zhengyou algorithm.

[0118] Long-distance error analysis is shown in Tables 2 and 3.

[0119] Table 2 Average physical error analysis table

[0120]

[0121] Table 3 Maximum physical error analysis table

[0122]

[0123]

[0124] It can be concluded that the average physical actual distance error is calculated by the custom method σ=0.011, and OpenCV σ=0. ​​It can be seen that the two σ errors are close, and the stability of the custom method is slightly lower but within an acceptable range.

[0125] In order to more intuitively demonstrate the correction effect of image distortion, this paper selected two representative groups of correction images for comparison and explanation. The experimental results are as follows Figure 4-Figure 6 shown.

[0126] Figure 4 The original distorted image shows obvious barrel distortion at the edges of the chessboard, and the straight line markers at the edge of the table are deformed; Figure 5 It can be seen from the visual that the distortion correction by using the custom method is better than that by using the original method. Figure 6 The custom method uses mixed initial values ​​and performs normalization to prevent excessive convergence or divergence during the correction process. It also uses a two-stage optimization method, performing global optimization followed by accurate local optimization. While the custom method's vertical correction performance is slightly lower than that of the OpenCV method in some cases, overall, the custom method achieves its intended purpose and outperforms the OpenCV method.

[0127] In summary, this embodiment combines the traditional Zhang method and the analytical method to calculate the initial parameters, and improves the accuracy of the initial values ​​by setting different weights; reasonably restricts the parameter range during the optimization process to avoid unrealistic solutions, and develops a two-stage optimization algorithm. First, the trust region reflection algorithm is used for global coarse optimization, and then the LM algorithm is used to achieve local fine adjustment, breaking through the convergence limitations of traditional optimization methods and effectively improving the accuracy and stability of the calibration results.

[0128] Experiments show that the method of this embodiment can better cope with the lens distortion problem in complex scenes while maintaining the advantages of simple operation and low cost. It is suitable for practical applications such as photogrammetry and three-dimensional reconstruction that require high-precision lens correction.

[0129] Figure 7 This is a schematic structural diagram of a camera lens calibration device provided in this embodiment. The camera lens calibration device may include: a first acquisition module 710, a first determination module 720, a second determination module 730, a second acquisition module 740 and an optimization module 750.

[0130] The first acquisition module 710 is used to acquire the calibration plate image data collected by the distortion camera.

[0131] The first determination module 720 is used to determine the first calibration intrinsic parameter of the distorted camera based on the calibration plate image data using Zhang's method, and to determine the second calibration intrinsic parameter of the distorted camera based on the calibration plate image data using an analytical method. Zhang's method is implemented using the calibrateCamera function of OpenCV, and the analytical method is implemented through homography matrix decomposition.

[0132] The second determination module 730 is configured to weight the first calibration internal parameter and the second calibration internal parameter, respectively, and determine a hybrid calibration internal parameter of the distorted camera based on the first calibration internal parameter, the calibration weight corresponding to the first calibration internal parameter, the second calibration internal parameter, and the calibration weight corresponding to the second calibration internal parameter.

[0133] The second acquisition module 740 is used to obtain the initial calibration extrinsic parameters of the distortion camera.

[0134] The optimization module 750 is used to perform phased nonlinear optimization on the hybrid calibration intrinsic parameters and the initial calibration extrinsic parameters of the distorted camera to obtain the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera, so as to perform lens calibration on the distorted camera.

[0135] In this embodiment, optionally, the first determining module 720 is specifically configured to:

[0136] Obtain a homography matrix representation corresponding to the calibration plate image data; and determine a second calibration intrinsic parameter of the distortion camera based on the homography matrix representation corresponding to the calibration plate image data.

[0137] In this embodiment, optionally, the first determining module 720 is specifically configured to:

[0138] Based on the two-dimensional coordinates of the points corresponding to the calibration plate image data, a first linear constraint equation and a second linear constraint equation are constructed; based on the first linear constraint equation and the second linear constraint equation, a linear constraint matrix is ​​determined; the linear constraint matrix is ​​decomposed by SVD to obtain multiple target singular vectors, and the multiple target singular vectors are arranged to obtain a homography matrix representation corresponding to the calibration plate image data.

[0139] In this embodiment, optionally, the first determining module 720 is specifically configured to:

[0140] Orthogonal constraints are imposed on the homography matrix representation corresponding to the calibration plate image data, and linear constraint parameters are obtained through Cholesky decomposition. A recombined constraint matrix is ​​obtained based on the linear constraint parameters, and the recombined constraint matrix is ​​decomposed to obtain the second calibration intrinsic parameters of the distortion camera.

[0141] In this embodiment, the optimization module 750 is optionally configured to:

[0142] The trust region reflection algorithm is used to perform initial optimization on the hybrid calibration intrinsic parameters of the distorted camera. The Levenberg-Marquardt algorithm is used to fine-tune the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera after initial optimization to obtain the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera.

[0143] In this embodiment, the optimization module 750 is optionally configured to:

[0144] Define the initialization parameter range of the objective function to be optimized and set the objective function. The objective function is calculated through the reprojection error of the calibration plate image; define the trust region around the current parameter value to limit the range of parameter change in each step; dynamically adjust according to the optimization effect during the iteration process, and solve the optimal parameter adjustment direction of the approximate model under the trust region constraint; if the optimal parameter adjustment direction exceeds the physical boundary of the parameter, use the reflection strategy to adjust the optimal parameter adjustment direction to ensure that the parameter always changes within a reasonable range; compare the actual error reduction with the model prediction value. When the parameter approaches the preset physical limit, correct the optimal parameter adjustment direction through reflection or truncation operation to ensure that the iteration is always carried out within the feasible domain; when the objective function value changes very little, the parameter adjustment range is lower than the threshold or the maximum number of iterations is reached, terminate the optimization and output the current parameters as the result of the initial optimization.

[0145] In this embodiment, the optimization module 750 is optionally configured to:

[0146] For the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera after initial optimization, the corresponding Jacobian matrix and residual vector are calculated by the finite difference method; the linear equation is solved based on the Jacobian matrix and residual vector to obtain the iteration step size; the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera are updated based on the iteration step size, and the objective function values ​​corresponding to the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera are evaluated; based on the objective function values ​​corresponding to the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera, the damping factor is adjusted to control the iteration direction of the iteration step size until the convergence condition is reached, and the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera are obtained.

[0147] The camera lens calibration device provided in the present disclosure can execute the above method embodiments. Its specific implementation principles and technical effects can be found in the above method embodiments, and the present disclosure will not repeat them here.

[0148] The present application also provides a computer device. Figure 8 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.

[0149] The computer device includes a memory 810 and a processor 820 that are interconnected and communicate with each other via a system bus. It should be noted that the figure only shows a computer device with a memory 810 and a processor 820, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0150] Computer devices can be desktop computers, laptops, PDAs, cloud servers, etc. Computer devices can interact with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0151] The memory 810 includes at least one type of readable storage medium, including non-volatile memory or volatile memory, such as flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disk, optical disk, etc. RAM may include static RAM or dynamic RAM. In some embodiments, the memory 810 may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory 810 may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, or a Flash Card equipped on the computer device. Of course, the memory 810 may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory 810 is generally used to store the operating system and various application software installed on the computer device, such as the program code of the above-mentioned method. In addition, the memory 810 may also be used to temporarily store various types of data that have been output or are about to be output.

[0152] The processor 820 is generally used to perform the overall operation of the computer device. In this embodiment, the memory 810 is used to store program code or instructions, which include computer operating instructions. The processor 820 is used to execute the program code or instructions stored in the memory 810 or process data, such as the program code for running the above method.

[0153] In this document, a bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. This bus system can be divided into address buses, data buses, and control buses. For ease of illustration, the figure uses only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0154] Another embodiment of the present application further provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads the computer-readable program code stored in the computer-readable medium, enabling the processor to execute the functional actions specified in each step or combination of steps in the above method, and to generate a device that implements the functional actions specified in each block or combination of blocks in the block diagram.

[0155] Computer-readable media include but are not limited to electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any appropriate combination of the foregoing, the memory is used to store program codes or instructions, the program codes include computer operating instructions, and the processor is used to execute the program codes or instructions of the above-mentioned methods stored in the memory.

[0156] For the definitions of memory and processor, please refer to the description of the aforementioned computer device embodiment and will not be repeated here.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0158] Each functional unit or module in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0160] In the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" described in the present application does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented with the aid of hardware comprising several different elements and with the aid of a suitably programmed computer. In a unit claim that lists several means, several units of these means may be embodied by the same hardware item. The use of first, second, and third etc. does not indicate any order and these words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

[0161] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A camera lens calibration method, characterized in that: include: Obtain the calibration plate image data collected by the distortion camera; Based on the calibration plate image data, a first calibration intrinsic parameter of the distorted camera is determined using Zhang's method, and based on the calibration plate image data, a second calibration intrinsic parameter of the distorted camera is determined using an analytical method, wherein the Zhang's method is implemented using the calibrateCamera function of OpenCV, and the analytical method is implemented through homography matrix decomposition; performing weighted ratio matching on the first calibration internal parameter and the second calibration internal parameter respectively, and determining a hybrid calibration internal parameter of the anamorphic camera based on the first calibration internal parameter, the calibration weight corresponding to the first calibration internal parameter, the second calibration internal parameter, and the calibration weight corresponding to the second calibration internal parameter; Obtaining initial calibration extrinsic parameters of the distortion camera; The hybrid calibration intrinsic parameters of the distorted camera and the initial calibration extrinsic parameters of the distorted camera are nonlinearly optimized in stages to obtain target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera, so as to perform lens calibration on the distorted camera.

2. The method according to claim 1, characterized in that The method of determining a second calibration internal parameter of the distortion camera based on the calibration plate image data by an analytical method includes: Obtaining a homography matrix representation corresponding to the calibration plate image data; Based on the homography matrix representation corresponding to the calibration plate image data, a second calibration intrinsic parameter of the distortion camera is determined.

3. The method according to claim 2, characterized in that The obtaining of a homography matrix representation corresponding to the calibration plate image data includes: Constructing a first linear constraint equation and a second linear constraint equation based on the two-dimensional coordinates of the points corresponding to the calibration plate image data; Determining a linear constraint matrix based on the first linear constraint equation and the second linear constraint equation; Performing SVD decomposition on the linear constraint matrix to obtain a plurality of target singular vectors, and arranging the plurality of target singular vectors to obtain the homography matrix representation corresponding to the calibration plate image data.

4. The method according to claim 3, characterized in that The determining of a second calibration intrinsic parameter of the distortion camera based on the homography matrix representation corresponding to the calibration plate image data includes: Performing orthogonal constraints on the homography matrix representation corresponding to the calibration plate image data, and obtaining linear constraint parameters through Cholesky decomposition; A reorganized constraint matrix is ​​obtained based on the linear constraint parameter reorganization, and the reorganized constraint matrix is ​​decomposed to obtain a second calibration intrinsic parameter of the distortion camera.

5. The method according to claim 1, wherein The step of performing phased nonlinear optimization on the hybrid calibration intrinsic parameters of the distorted camera and the initial calibration extrinsic parameters of the distorted camera to obtain the target calibration intrinsic parameters and the target calibration extrinsic parameters of the distorted camera includes: Using a trust region reflective algorithm to initially optimize the hybrid calibration internal parameters of the distortion camera; The Levenberg-Marquardt algorithm is used to fine-tune the hybrid calibration intrinsic parameters and the initial calibration extrinsic parameters of the distorted camera after initial optimization to obtain the target calibration intrinsic parameters and the target calibration extrinsic parameters of the distorted camera.

6. The method according to claim 5, characterized in that The initial optimization of the hybrid calibration internal parameters of the distortion camera using the trust region reflection algorithm includes: Define the initialization parameter range of the objective function to be optimized and set the objective function, which is calculated by the reprojection error of the calibration plate image; define a trust region around the current parameter value to limit the amplitude of the parameter change at each step; dynamically adjust according to the optimization effect during the iteration process, and solve the optimal parameter adjustment direction of the approximate model under the trust region constraint; if the optimal parameter adjustment direction exceeds the physical boundary of the parameter, use a reflection strategy to adjust the optimal parameter adjustment direction to ensure that the parameter always changes within a reasonable range; Compare the actual error reduction with the model prediction value. When the parameter approaches the preset physical limit, the optimal parameter adjustment direction is corrected through reflection or truncation operations to ensure that the iteration is always within the feasible domain. When the objective function value changes very little, the parameter adjustment amplitude is lower than the threshold, or the maximum number of iterations is reached, the optimization is terminated and the current parameters are output as the result of the initial optimization.

7. The method according to claim 5, characterized in that The Levenberg-Marquardt algorithm is used to fine-tune the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distorted camera after initial optimization to obtain the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera, including: For the hybrid calibration intrinsic parameters and initial calibration extrinsic parameters of the distortion camera after initial optimization, the corresponding Jacobian matrix and residual vector are calculated by finite difference method; Solve the linear equation based on the Jacobian matrix and the residual vector to obtain an iterative step size; updating the hybrid calibration intrinsic parameters and the initial calibration extrinsic parameters of the distorted camera based on the iteration step size, and evaluating the objective function values ​​corresponding to the hybrid calibration intrinsic parameters and the initial calibration extrinsic parameters of the distorted camera; Based on the objective function values ​​corresponding to the hybrid calibration intrinsic parameters and the initial calibration extrinsic parameters of the distorted camera, the damping factor is adjusted to control the iterative direction of the iterative step size until a convergence condition is reached, thereby obtaining the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera.

8. A camera lens calibration device, characterized in that: include: A first acquisition module is used to acquire calibration plate image data collected by the distortion camera; a first determination module, configured to determine a first calibration intrinsic parameter of the anamorphic camera using a Zhang method based on the calibration plate image data, and to determine a second calibration intrinsic parameter of the anamorphic camera using an analytical method based on the calibration plate image data; The Zhang method is implemented by the calibrateCamera function of OpenCV, and the analytical method is implemented by homography matrix decomposition; a second determination module, configured to weight the first calibration internal parameter and the second calibration internal parameter, respectively, and determine a hybrid calibration internal parameter of the anamorphic camera based on the first calibration internal parameter, the calibration weight corresponding to the first calibration internal parameter, the second calibration internal parameter, and the calibration weight corresponding to the second calibration internal parameter; A second acquisition module is used to obtain the initial calibration extrinsic parameters of the distortion camera; The optimization module is used to perform phased nonlinear optimization on the hybrid calibration intrinsic parameters of the distorted camera and the initial calibration extrinsic parameters of the distorted camera to obtain the target calibration intrinsic parameters and target calibration extrinsic parameters of the distorted camera, so as to perform lens calibration on the distorted camera.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the camera lens calibration method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the camera lens calibration method according to any one of claims 1 to 7 is implemented.

Images