Asymmetric lens distortion compensation method for linear reconstruction guided by feature point projection transformation

Abstract

The invention discloses an asymmetric lens distortion compensation method for linear reconstruction guided by feature point projection transformation. The method comprises the following steps: traversing and calibrating a coordinate matrix of world coordinates of image feature points after projection transformation through minimum reference grids, calculating a projection transformation comprehensive error of all the minimum reference grids, and obtaining the projection transformation comprehensive error through weighted calculation of straight line constraint, cross ratio constraint and parallel line constraint; and carrying out secondary screening to obtain a reference grid region with the minimum projection transformation comprehensive error. Thirdly, optimizing the optimal reference grid by taking the minimum comprehensive error as a target; and finally, solving the optimized feature point coordinates and the corresponding world coordinates to obtain a homography transformation matrix, performing linear reconstruction on the coordinates of projection transformation of the feature points of the whole calibration image, and performing parameter optimization of the lens asymmetric distortion model with the coordinates before reconstruction, thereby obtaining a high-precision lens asymmetric distortion model. And the asymmetric distortion of the lens is compensated and corrected.

Description

Technical Field

[0001] The invention relates to a lens distortion compensation method in the field of camera calibration, and in particular to an asymmetric lens distortion compensation method of linear reconstruction guided by feature point projection transformation. Background Art

[0002] Camera calibration technology is a core link in the field of computer vision and optoelectronic imaging, and is widely used in many fields such as robot navigation, autonomous driving, 3D reconstruction, and virtual reality. The main goal of camera calibration is to determine the internal and external parameters of the camera, which are essential for accurately mapping points in the three-dimensional world to the two-dimensional image plane. However, due to the inevitable process defects and errors in the manufacturing and installation process of camera lenses, various types of lens distortion occur, which significantly affects the imaging quality and measurement accuracy.

[0003] In the process of camera calibration, the acquisition of internal and external parameters is a linear solution process, while the solution of distortion coefficients is a nonlinear process. The error of camera calibration mainly comes from the fitting and solution process of camera distortion parameters. Therefore, establishing an accurate distortion model and performing high-precision parameter fitting are the keys to improving the accuracy of camera calibration. Common lens distortions include radial distortion and tangential distortion. Radial distortion refers to the radial bending of light when it passes through the lens, resulting in inconsistent magnification ratios between the center and edge areas of the image; tangential distortion is the light deviation caused by imperfect lens assembly. For these distortions, mathematical models such as polynomial fitting are usually used for correction to restore the true geometric shape of the image.

[0004] As the application scenarios of camera lenses continue to expand, the requirements for the applicability of camera calibration are becoming higher and higher. In some complex camera distortion situations, such as telephoto lenses, wide-angle imaging, lens optical axis offset and other scenarios, asymmetric distortion often occurs. For these situations, the current mainstream distortion models are difficult to achieve a good fitting effect, resulting in a decrease in camera calibration accuracy. In order to establish a high-precision and wider-range distortion model, more parameters need to be introduced, which increases the difficulty of distortion model fitting. The current calibration process mainly uniformly fits the camera's internal and external parameters and the distortion model parameters. While the optimization method converges the camera's internal and external parameters, it also converges the distortion model parameters. In order to improve the fitting accuracy of the distortion model, a large amount of feature point data of the calibration image is required, but this also introduces more camera external parameter fitting, making it difficult to obtain a good optimization effect for the distortion model parameters. Summary of the invention

[0005] In order to solve the problem of insufficient optimization accuracy of distortion model parameters in the case of asymmetric offset distortion caused by unified optimization and solution of the internal and external parameters of the camera and the parameters of the distortion model in the current mainstream camera calibration process, the present invention proposes a projection transformation-guided asymmetric lens distortion optimization method based on linear reconstruction of feature points. The method traverses the feature point coordinate matrix of the calibration image, calculates the comprehensive error of the linear projection, and obtains the area with the minimum linear projection error after secondary screening. Then, the feature points in the area are linearly corrected using straight line constraints, cross ratio constraints and parallel line constraints. Next, the point coordinates after linear correction are solved with the corresponding world coordinates to obtain a homography transformation matrix. Thus, according to the obtained homography transformation matrix, the distortion feature points of the calibration image can be linearly reconstructed based on the world coordinates of the feature points to obtain the corresponding non-distorted feature point coordinates, and the distortion model parameters are optimized with the distorted feature point coordinates, so that a high-precision asymmetric distortion model can be obtained and the asymmetric distortion of the lens can be compensated.

[0006] The technical solution of the present invention is as follows:

[0007] 1. A method for compensating asymmetric lens distortion by guiding linear reconstruction with feature point projection transformation

[0008] s1: Construct the distorted image feature point coordinate matrix based on the feature points of the checkerboard calibration image;

[0009] s2: After sliding grid extraction of the distorted image coordinate matrix using the reference grid R, a reference grid set U is obtained;

[0010] s3: Use the asymmetric distortion model to find the reference grid with the smallest distortion from the reference grid set U and use it as the optimal reference grid R t ; Then the optimal reference grid R t The coordinates of the feature points in are optimized to obtain the optimized reference grid R t ;

[0011] s4: According to the optimized optimal reference grid R t Reconstruct the linear image feature point coordinate matrix;

[0012] s5: After optimizing the parameters in the asymmetric distortion model according to the distorted image feature point coordinate matrix and the linear image feature point coordinate matrix, an optimized asymmetric distortion model is obtained.

[0013] The s1 is specifically:

[0014] s11: Use the sub-pixel feature point detection method based on the Harris operator to detect all feature points of the checkerboard calibration image to obtain the coordinates of each feature point and the size of the checkerboard;

[0015] s12: Initialize the grid point coordinate matrix according to the chessboard size, store the coordinates of each feature point detected in the form of a two-dimensional grid into the grid point coordinate matrix and record it as the distorted image coordinate matrix.

[0016] The s3 is specifically:

[0017] s31: Construct an asymmetric distortion model. Input the non-distorted checkerboard image into the asymmetric distortion model to generate a reference distorted image. The coordinates of the distortion center point of the reference distorted image are used as the standard distortion center. The formula of the asymmetric distortion model is as follows:

[0018]

[0019]

[0020] Among them, λ is the distance between the feature point coordinates in the distorted image coordinate matrix and the distortion center point (c1, c2), k1, k2, k3 are three radial distortion coefficients, p1, p2 are two tangential distortion coefficients, c3, c4 are two edge strong distortion coefficients, is the first The coordinates of feature points, is the predicted distorted feature point coordinates;

[0021] s32: Calculate the linear error E corresponding to each reference grid in the reference grid set U st , cross ratio error E cr and parallel line error E pa , delete the straight line error E in the reference grid set U st , cross ratio error E cr Or parallel line error E pa After the reference grids with error greater than the preset threshold are found, the reference grid set U′ after preliminary screening is obtained;

[0022] s33: Generate straight line error E st , cross ratio error E cr and parallel line error E pa The corresponding weight coefficients are recorded as a weight coefficient combination, and the comprehensive errors corresponding to each reference grid in the reference grid set U′ after the initial screening are calculated. The reference grid with the smallest comprehensive error in the reference grid set U′ after the initial screening is taken as the optimal reference grid R corresponding to the current weight coefficient combination t ;

[0023] s34: Change the value of the weight coefficient and repeat s33 to obtain the optimal reference grid R corresponding to different weight coefficient combinations t , calculate the optimal reference grid R for different weight coefficient combinations t The deviation from the standard distortion center is Ep , the offset error E p The smallest optimal reference grid R t As the final optimal reference grid R t ;

[0024] s35: Take the final optimal reference grid R t The corresponding comprehensive error function E t As the objective function, a nonlinear optimization method is used to find the optimal reference grid R t The coordinates of each feature point in are optimized to obtain the optimized optimal reference grid R t .

[0025] In s32, the straight line error E of each reference grid st The calculation formula is as follows:

[0026]

[0027] Among them, v m,n represents the mth straight line l m The ordinate of the nth feature point on the represents the mth straight line l m The fitted ordinate of the nth feature point on the grid, r is the number of lines in each reference grid, s is the number of feature points in each reference grid, is the fitting straight line l m Substitute the ordinate value calculated by the feature point's horizontal coordinate u, and They are the fitting straight lines l m The slope and intercept of and The straight lines l m The horizontal and vertical coordinates of the nth point on and Respectively represent the straight line l m The average value of the horizontal and vertical coordinates of s points on the graph.

[0028] In s32, the cross-ratio error function E of each reference grid cr The calculation formula is as follows:

[0029] E cr =(E crh +E crv ) / 8

[0030]

[0031] Among them, E crh is the sum of the cross-ratio errors calculated for each row in the reference grid, E crvThe sum of the cross ratio errors of each column in the reference grid is calculated, r is the number of lines in each reference grid, s is the number of feature points in each reference grid, and P1, P2, P3, and P4 are expressed as The four points correspond to the non-distorted points in the world coordinate system, F cr () represents the cross ratio function, and The straight lines l m The horizontal and vertical coordinates of the nth point on d ij Indicates the midpoint of the mth row and The distance between them is i=1, 2, j=3, 4.

[0032] In s32, the parallel line error E of each reference grid pa The calculation formula is as follows:

[0033]

[0034]

[0035] Where r is the number of lines in each reference grid, and are the mth fitting straight line l m The slope and intercept of , m = 1, 2, ..., r, A is the coefficient matrix, is the assumed vanishing point where the 1st to rth parallel lines intersect, and Vanishing Point The horizontal and vertical coordinates of , b is a constant matrix, and T represents transpose.

[0036] In s33, the comprehensive error E of the δth reference grid δ The formula is as follows:

[0037] E δ =λ cr E cr +λ st E st +λ pa E pa

[0038] Among them, λ cr , st , pa are the first to third weight coefficients respectively.

[0039] The s4 is specifically:

[0040] s41: construct a linear equation system for solving the homography matrix according to the coordinates of the feature points of the optimized optimal reference grid and their corresponding world coordinates;

[0041] s42: Use the singular value decomposition method to solve the linear equations of the homography matrix, solve the eigenvector with the minimum singular value as the solution vector h, and reshape the homography matrix H according to the solution vector h;

[0042] s43: Expand the feature point world coordinate vector into homogeneous coordinates to obtain the expanded homogeneous coordinate matrix W h ; Then use the homography matrix H to expand the homogeneous coordinate matrix W h Perform perspective transformation to obtain the homogeneous coordinates T after perspective transformation h , and then the homogeneous coordinates after perspective transformation T h Convert to non-homogeneous coordinates, obtain the non-homogeneous coordinate vector T and use it as the undistorted image coordinate matrix.

[0043] The s5 is specifically:

[0044] After the feature point coordinates in the distorted image coordinate matrix are brought into the asymmetric distortion model, the predicted distorted feature point coordinates are obtained, and then the error function E of the asymmetric distortion model is used dis The error between the predicted distorted feature point coordinates and the angle coordinates in the distorted image coordinate matrix is ​​calculated, and the parameters in the asymmetric distortion model are continuously adjusted until the error is minimized to obtain the optimized asymmetric distortion model.

[0045] The error function E of the asymmetric distortion model dis The formula is as follows:

[0046]

[0047] in, is the first feature point coordinates, n represents the number of feature points, are the predicted distorted feature point coordinates.

[0048] 2. A computer device

[0049] The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the asymmetric lens distortion compensation method of linear reconstruction guided by feature point projection transformation when executing the computer program.

[0050] 3. A computer-readable storage medium

[0051] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the asymmetric lens distortion compensation method for linear reconstruction guided by feature point projection transformation are implemented.

[0052] IV. A COMPUTER PROGRAM PRODUCT

[0053] The computer program product comprises a computer program / instruction, which, when executed by a processor, implements the steps of the asymmetric lens distortion compensation method for linear reconstruction guided by feature point projection transformation.

[0054] The beneficial effects of the present invention are:

[0055] According to the current situation that various types of lens distortion are complicated and the requirements for camera calibration accuracy are getting higher and higher, the present invention proposes an asymmetric lens distortion compensation method for linear reconstruction guided by feature point projection transformation. The present invention defines a minimum reference grid and performs traversal sliding on the detected feature point coordinate matrix to calculate the corresponding straight line error, cross ratio error and parallel line error, and uses the comprehensive error calculated by weighting the three errors as the evaluation index of the reference grid set after the initial screening, so as to find the optimal reference grid and calculate the corresponding homography matrix H, so as to obtain the non-distorted feature point coordinates corresponding to the distorted feature point coordinates of the checkerboard image, and finally perform asymmetric distortion model fitting.

[0056] Compared with traditional camera calibration methods, the method proposed in the present invention can solve the problem that traditional camera distortion models are difficult to fit with high precision in complex lens distortion scenes, while reducing the reprojection error of camera calibration and improving the applicability of camera calibration technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is the best reference grid detection and feature point expansion for 18*24 distorted checkerboard images.

[0058] Figure 2 It is a comparison chart of the distorted feature point coordinates and the non-distorted feature point coordinates detected in the checkerboard image.

[0059] Figure 3 It is a comparison chart of the non-distorted feature point coordinates and the fitted feature point coordinates.

[0060] Figure 4 It is the distortion compensation error map in the current mainstream calibration method.

[0061] Figure 5 It is a distortion compensation error diagram in the calibration method used in the present invention.

[0062] Figure 6 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0064] like Figure 6 As shown, the present invention proposes an asymmetric lens distortion compensation method for linear reconstruction guided by feature point projection transformation, which includes the following steps:

[0065] s1: Construct the distorted image coordinate matrix based on the feature points of the checkerboard calibration image;

[0066] s1 is specifically:

[0067] s11: Use the sub-pixel feature point detection method based on the Harris operator to detect all feature points of the checkerboard calibration image, and obtain the sub-pixel coordinates (u, v) of each feature point and the checkerboard size N×M;

[0068] s12: Initialize the grid point coordinate matrix according to the chessboard size, the size of which satisfies N×M×2, and store the coordinates of each feature point (N×M) obtained by the feature point detection method in the grid point coordinate matrix in the form of a two-dimensional grid and record it as the distorted image coordinate matrix.

[0069] s2: After sliding grid extraction of the distorted image coordinate matrix using the reference grid R, a reference grid set U is obtained; the size of the reference grid R is 4×4×2, consisting of 4×4 image feature point coordinates (u, v).

[0070] s3: Find the reference grid with the smallest distortion from the reference grid set U and use it as the optimal reference grid R t ; Then the optimal reference grid R t The coordinates of the feature points in are optimized to obtain the optimized reference grid R t ;

[0071] s3 specifically:

[0072] The current mainstream distortion model mainly considers radial distortion and tangential distortion. The radial distortion polynomial is as follows:

[0073] x distorted =x(1+k1r 2 +k2r 4 +k3r 6 )

[0074] y distorted =y(1+k1r 2 +k2r 4 +k3r 6 )

[0075] Among them, (x, y) is the undistorted normalized coordinate, (x distorted ,y distorted ) is the distorted coordinate, r is the distance from the point to the center of the image, and k1, k2, k3 are the radial distortion coefficients.

[0076] The tangential distortion expression is as follows:

[0077] x distorted =x+(2p1y+p2(r 2 +2x 2 ))

[0078] y distorted =y+(p1(r 2 +2y 2 )+2p2x)

[0079] Among them, p1 and p2 are tangential distortion coefficients.

[0080] For complex lens distortion situations, such as the distortion of periscope telephoto lenses and fisheye lenses, in addition to considering radial distortion and tangential distortion, it is also necessary to introduce asymmetric distortion factors and edge over-distortion factors.

[0081] s31: Construct an asymmetric distortion model. The initial values ​​of the parameters in the asymmetric distortion model are set according to the distortion parameters corresponding to the current mainstream telephoto lenses. The standard non-distorted checkerboard image is input into the asymmetric distortion model to generate a reference distorted image. The coordinates of the distortion center point of the reference distorted image are used as the standard distortion center. The formula of the asymmetric distortion model is as follows:

[0082]

[0083] Among them, λ is the distance between the feature point coordinates in the distorted image coordinate matrix and the distortion center point (c1, c2), k1, k2, k3 are three radial distortion coefficients, p1, p2 are two tangential distortion coefficients, c3, c4 are two edge strong distortion coefficients, is the first feature point coordinates;

[0084] s32: In the perspective projection transformation process, if the points in the scene are structured points, the collinearity and correlation in the scene will be preserved. Therefore, the world coordinates of the feature points should satisfy the invariance of the cross ratio, the invariance of the straight line, and the intersection of parallel straight lines at the common point after the projection transformation. Calculate the linear error E corresponding to each reference grid in the reference grid set U st , cross ratio error E cr and parallel line error E pa , delete the straight line error E in the reference grid set U st , cross ratio error Ecr Or parallel line error E pa After the reference grids with error greater than the preset threshold are found, the reference grid set U′ after preliminary screening is obtained;

[0085] The straight line error E of each reference grid st The calculation formula is as follows:

[0086]

[0087] Among them, v m,n represents the mth straight line l m The ordinate of the nth feature point on the represents the mth straight line l m The fitted ordinate of the nth feature point on the grid, r is the number of lines in each reference grid, s is the number of feature points in each reference grid, is the fitting straight line l m Substitute the ordinate value calculated by the feature point's horizontal coordinate u, and They are the fitting straight lines l m The slope and intercept of and The straight lines l m The horizontal and vertical coordinates of the nth point on and Respectively represent the straight line l m For each row and column of feature points on the minimum reference network R, the least squares method can be used to calculate the average value of the horizontal and vertical coordinates of the s points on the line l. m Perform straight line fitting to obtain straight line l m The regression linear equation.

[0088] The cross-ratio error function E of each reference grid cr The calculation formula is as follows:

[0089] E cr =(E crh +E crv ) / 8

[0090]

[0091] Among them, E crh is the sum of the cross-ratio errors calculated for each row in the reference grid, E crv The sum of the cross ratio errors of each column in the reference grid is calculated, r is the number of lines in each reference grid, s is the number of feature points in each reference grid, and P1, P2, P3, and P4 are expressed as The four points correspond to the non-distorted points in the world coordinate system, F cr () represents the cross ratio function, and The straight lines l m The horizontal and vertical coordinates of the nth point on d ij Indicates the midpoint of the mth row and The distance between them is i = 1, 2, j = 3, 4. Since each feature point is the intersection of a vertical line and a horizontal line, the cross-ratio error calculation of the minimum reference network needs to be performed simultaneously in the horizontal and vertical directions.

[0092] After fitting the equations of each r rows and each s columns of the minimum reference grid, the two sets of parallel lines, each row and each column, will intersect at two different vanishing points according to the intersection of parallel lines. p[ The calculation formula is as follows:

[0093]

[0094] Where r is the number of lines in each reference grid, and are the mth fitting straight line l m The slope and intercept of , m = 1, 2, ..., r, A is the coefficient matrix, is the assumed vanishing point where the 1st to rth parallel lines intersect, and Vanishing Point The horizontal and vertical coordinates of , b is a constant matrix, and T represents transpose.

[0095] s33: Generate straight line error E st , cross ratio error E cr and parallel line error E pa The corresponding weight coefficients are recorded as a weight coefficient combination, and the comprehensive errors corresponding to each reference grid in the reference grid set U′ after the initial screening are calculated. The reference grid with the smallest comprehensive error in the reference grid set U′ after the initial screening is taken as the optimal reference grid R corresponding to the current weight coefficient combination t , the optimal reference grid R t The corresponding lens area is considered to be the optimal linear area;

[0096] The comprehensive error E of the δth reference grid δ The formula is as follows:

[0097] E δ =λ cr E cr +λ st E st +λ pa E pa

[0098] Among them, λ cr , st , pa are the first to third weight coefficients respectively.

[0099] s34: weight coefficient λ cr , st , pa The range is [0,1], and the value of the weight coefficient is changed in steps of 0.1. S33 is repeated to obtain the optimal reference grid R corresponding to different weight coefficient combinations. t , calculate the optimal reference grid R for different weight coefficient combinations t The deviation from the standard distortion center is E p , the offset error E p The smallest optimal reference grid R t As the final optimal reference grid R t That is, select the offset error E p The smallest optimal reference grid R t The corresponding weight coefficients are combined to calculate the comprehensive error.

[0100] The coordinates of each grid point are expressed as (x αβ ,y αβ ), where i and j represent the rows and columns of the grid respectively (α, β = 1, 2, 3, 4), and the offset error E p The calculation formula is as follows:

[0101]

[0102] s35: Take the final optimal reference grid R t The comprehensive error function E obtained by combining the corresponding weight coefficients t As the objective function, the quasi-newton nonlinear optimization method is used to find the final optimal reference grid R t The coordinates of each feature point in are optimized so that the optimized feature point coordinates satisfy the linear constraints of the perspective projection transformation as much as possible, and the optimized optimal reference grid R is obtained. t (like Figure 1 As shown in the figure, the * is the optimal reference grid R after linear reconstruction. t 4×4 feature points).

[0103] s4: According to the optimized optimal reference grid R t Construct the undistorted feature point image coordinate matrix, such as Figure 1 As shown in the figure, the blue dots are the non-distorted feature points corresponding to the checkerboard distorted feature points after linear reconstruction;

[0104] S4 is specifically:

[0105] s41: Construct a linear equation system for solving the homography matrix based on the coordinates of the feature points of the optimized optimal reference grid and their corresponding world coordinates; assuming there are n point pairs, a matrix P with 2n rows needs to be constructed. Let the homography matrix solution vector h = [h 11 ,h 12 ,h 13 ,g 21 ,g 22 ,g 23 ,g 31 ,g 32 ,h 33 ] T , for each feature point's world coordinate (X w ,Y w ) and feature point coordinates (u, v), the linear equations for solving the homography matrix are as follows:

[0106]

[0107] s42: Use the singular value decomposition (SVD) method to solve the linear equations of the homography matrix, solve the eigenvector with the minimum singular value as the solution vector h, and reshape the homography matrix H according to the solution vector h;

[0108] s43: Expand the feature point world coordinate vector into homogeneous coordinates to obtain the expanded homogeneous coordinate matrix W h It is expressed as:

[0109]

[0110] Then use the homography matrix H to expand the homogeneous coordinate matrix W h Perform perspective transformation to obtain the homogeneous coordinates T after perspective transformation h :

[0111] T h =HW h T

[0112]

[0113] Then the homogeneous coordinates T after perspective transformation are h Convert to non-homogeneous coordinates, obtain the non-homogeneous coordinate vector T and use it as the non-distorted image coordinate matrix. The formula is as follows:

[0114]

[0115] The non-homogeneous coordinate vector T Coordinates The coordinates of each image feature point (u, v) are the non-distorted image feature point coordinates corresponding to each image feature point coordinate.

[0116] s5: According to the distorted image feature point coordinate matrix (such as Figure 2 The red markers in the image) and the coordinate matrix of the undistorted image feature points (such as Figure 2 After optimizing the parameters in the asymmetric distortion model (the blue mark points in the figure), the optimized asymmetric distortion model is obtained.

[0117] S5 is specifically:

[0118] After the feature point coordinates in the non-distorted image feature point coordinate matrix are brought into the asymmetric distortion model, the predicted distorted feature point coordinates are obtained (e.g. Figure 3 The blue mark points shown), using the error function E of the distortion model dis The error between the predicted distorted feature point coordinates and the angular coordinates in the distorted image feature point coordinate matrix is ​​calculated, and the parameters (k1, k2, k3, p1, p2, c1, c2, c3, c4) in the asymmetric distortion model are continuously adjusted using the interior-point nonlinear optimization method until the error is minimized. The optimized asymmetric distortion model is obtained, which can perform high-precision compensation correction for complex distortion situations, such as asymmetric distorted lenses. Figure 3 Figure 1 is a schematic diagram of the compensated image feature point matrix and the original image feature point matrix when the error is minimized after parameter optimization. In order to illustrate the compensation effect of the method of the present invention, four calibrated chessboard images at different angles were tested. Figure 5 The average compensation error corresponding to each image is shown in Figure 1, and the compensation error of the four images under the current mainstream method of Matlab is compared. The results are as follows: Figure 4 shown.

[0119] Among them, the error function E of the distortion model dis The formula is as follows:

[0120]

[0121] in, is the first feature point coordinates, n represents the number of feature points, is the coordinates of the distorted feature points to be compensated.

[0122] Finally, it should be noted that the above embodiments and explanations are only used to illustrate the technical solution of the present invention rather than to limit it. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope disclosed in the technical solution of the present invention, which should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for compensating asymmetric lens distortion by linear reconstruction guided by feature point projection transformation, characterized in that: The following steps are involved: s1: Construct the distorted image feature point coordinate matrix based on the feature points of the checkerboard calibration image; s2: After sliding grid extraction of the distorted image coordinate matrix using the reference grid R, a reference grid set U is obtained; s3: Use the asymmetric distortion model to find the reference grid with the smallest distortion from the reference grid set U and use it as the optimal reference grid R t ; Then the optimal reference grid R t The coordinates of the feature points in are optimized to obtain the optimized reference grid R t ; s4: According to the optimized optimal reference grid R t Reconstruct the linear image feature point coordinate matrix; s5: After optimizing the parameters in the asymmetric distortion model according to the distorted image feature point coordinate matrix and the linear image feature point coordinate matrix, an optimized asymmetric distortion model is obtained.

2. The asymmetric lens distortion compensation method of claim 1 wherein the feature point projection transformation guides linear reconstruction, characterized in that: The s1 is specifically: s11: Use the sub-pixel feature point detection method based on the Harris operator to detect all feature points of the checkerboard calibration image to obtain the coordinates of each feature point and the size of the checkerboard; s12: Initialize the grid point coordinate matrix according to the chessboard size, store the coordinates of each feature point detected in the form of a two-dimensional grid into the grid point coordinate matrix and record it as the distorted image coordinate matrix.

3. The asymmetric lens distortion compensation method of claim 1 wherein the linear reconstruction is guided by feature point projection transformation, characterized in that: The s3 is specifically: s31: Construct an asymmetric distortion model. Input the non-distorted checkerboard image into the asymmetric distortion model to generate a reference distorted image. The coordinates of the distortion center point of the reference distorted image are used as the standard distortion center. The formula of the asymmetric distortion model is as follows: Among them, λ is the distance between the feature point coordinates in the distorted image coordinate matrix and the distortion center point (c1, c2), k1, k2, k3 are three radial distortion coefficients, p1, p2 are two tangential distortion coefficients, c3, c4 are two edge strong distortion coefficients, is the first The coordinates of feature points, is the predicted distorted feature point coordinates; s32: Calculate the linear error E corresponding to each reference grid in the reference grid set U st , cross ratio error E cr and parallel line error E pa , delete the straight line error E in the reference grid set U st , cross ratio error E cr Or parallel line error E pa After the reference grids with error greater than the preset threshold are found, the reference grid set U′ after preliminary screening is obtained; s33: Generate straight line error E st , cross ratio error E cr and parallel line error E pa The corresponding weight coefficients are recorded as a weight coefficient combination, and the comprehensive errors corresponding to each reference grid in the reference grid set U′ after the initial screening are calculated. The reference grid with the smallest comprehensive error in the reference grid set U′ after the initial screening is taken as the optimal reference grid R corresponding to the current weight coefficient combination t ; s34: Change the value of the weight coefficient and repeat s33 to obtain the optimal reference grid R corresponding to different weight coefficient combinations t , calculate the optimal reference grid R for different weight coefficient combinations t The deviation from the standard distortion center is E p , the offset error E p The smallest optimal reference grid R t As the final optimal reference grid R t ; s35: Take the final optimal reference grid R t The corresponding comprehensive error function E t As the objective function, a nonlinear optimization method is used to find the optimal reference grid R t The coordinates of each feature point in are optimized to obtain the optimized optimal reference grid R t .

4. The asymmetric lens distortion compensation method of claim 3 wherein the feature point projection transformation guides linear reconstruction, In s32, the straight line error E of each reference grid st The calculation formula is as follows: Among them, c m,n represents the mth straight line l m The ordinate of the nth feature point on the represents the mth straight line l m The fitted ordinate of the nth feature point on the grid, r is the number of lines in each reference grid, s is the number of feature points in each reference grid, is the fitting straight line l m Substitute the ordinate value calculated by the feature point's horizontal coordinate u, and They are the fitting straight lines l m The slope and intercept of and The straight lines l m The horizontal and vertical coordinates of the nth point on and Respectively represent the straight line l m The average value of the horizontal and vertical coordinates of s points on the graph.

5. The asymmetric lens distortion compensation method of claim 3 wherein the feature point projection transformation guides linear reconstruction, In s32, the cross-ratio error function E of each reference grid cr The calculation formula is as follows: AND cr =(And crh +E crv ) / 8 Among them, E crh is the sum of the cross-ratio errors for each row in the reference grid, E crv The sum of the cross ratio errors of each column in the reference grid is calculated, r is the number of lines in each reference grid, s is the number of feature points in each reference grid, and P1, P2, P3, and P4 are expressed as The four points correspond to the non-distorted points in the world coordinate system, F cr () represents the cross ratio function, and The straight lines l m The horizontal and vertical coordinates of the nth point on d ij Indicates the midpoint of the mth row and The distance between them is i=1, 2, j=3, 4.

6. The asymmetric lens distortion compensation method of claim 3 wherein the linear reconstruction guided by feature point projection transformation is characterized in that: In s32, the parallel line error E of each reference grid pa The calculation formula is as follows: Where r is the number of lines in each reference grid, and are the mth fitting straight line l m The slope and intercept of , m = 1, 2, ..., r, A is the coefficient matrix, is the assumed vanishing point where the 1st to rth parallel lines intersect, and Vanishing Point The horizontal and vertical coordinates of , b is a constant matrix, and T represents transpose.

7. The asymmetric lens distortion compensation method of claim 3 wherein the linear reconstruction guided by feature point projection transformation is characterized in that: In s33, the comprehensive error E of the δth reference grid δ The formula is as follows: E δ =λ cr E cr +λ st E st +λ pa E pa Among them, λ cr , st , pa are the first to third weight coefficients respectively.

8. The asymmetric lens distortion compensation method of claim 1 wherein the feature point projection transformation guides linear reconstruction, characterized in that: The s4 is specifically: s41: construct a linear equation system for solving the homography matrix according to the coordinates of the feature points of the optimized optimal reference grid and their corresponding world coordinates; s42: Use the singular value decomposition method to solve the linear equations of the homography matrix, solve the eigenvector with the minimum singular value as the solution vector h, and reshape the homography matrix H according to the solution vector h; s43: Expand the feature point world coordinate vector into homogeneous coordinates to obtain the expanded homogeneous coordinate matrix W h ; Then use the homography matrix H to expand the homogeneous coordinate matrix W h Perform perspective transformation to obtain the homogeneous coordinates T after perspective transformation h , and then the homogeneous coordinates after perspective transformation T h Convert to non-homogeneous coordinates, obtain the non-homogeneous coordinate vector T and use it as the undistorted image coordinate matrix.

9. The asymmetric lens distortion compensation method of claim 1 wherein the feature point projection transformation guides linear reconstruction, characterized in that: The s5 is specifically: After the feature point coordinates in the distorted image coordinate matrix are brought into the asymmetric distortion model, the predicted distorted feature point coordinates are obtained, and then the error function E of the asymmetric distortion model is used dis The error between the predicted distorted feature point coordinates and the angle coordinates in the distorted image coordinate matrix is ​​calculated, and the parameters in the asymmetric distortion model are continuously adjusted until the error is minimized to obtain the optimized asymmetric distortion model.

10. The asymmetric lens distortion compensation method of claim 9 wherein the feature point projection transformation guides linear reconstruction, characterized in that: The error function E of the asymmetric distortion model dis The formula is as follows: in, is the first feature point coordinates, n represents the number of feature points, are the predicted distorted feature point coordinates.

Images