Asymmetric lens distortion compensation method using feature point projection transformation to guide linear reconstruction

Abstract

The application discloses a kind of feature point projection transformation guide linear reconstruction asymmetric lens distortion compensation method.The application is by minimum reference grid traversal the coordinate matrix of the world coordinate of the feature point of calibration image after projection transformation, the projection transformation comprehensive error of all minimum reference grid is calculated, and projection transformation comprehensive error is obtained by the weighted calculation of straight line constraint, cross ratio constraint and parallel line constraint;It is obtained after the secondary screening that the reference grid area of projection transformation comprehensive error minimum.Then, with the minimum comprehensive error as the target, the best reference grid is optimized;Finally, the optimized feature point coordinates and corresponding world coordinates are solved, to obtain homography transformation matrix, and then the coordinates of the entire calibration image feature point projection transformation are linearly reconstructed, and the parameter optimization of lens asymmetric distortion model is carried out with the coordinates before reconstruction, so that high-precision lens asymmetric distortion model can be obtained, and lens asymmetric distortion is compensated and corrected.

Description

Technical Field

[0001] This invention relates to a lens distortion compensation method in the field of camera calibration, specifically to an asymmetric lens distortion compensation method guided by feature point projection transformation for linear reconstruction. Background Technology

[0002] Camera calibration is a core component of computer vision and optoelectronic imaging, widely applied in fields such as robot navigation, autonomous driving, 3D reconstruction, and virtual reality. The primary goal of camera calibration is to determine the camera's intrinsic and extrinsic parameters, which are crucial for accurately mapping points from the three-dimensional world onto the two-dimensional image plane. However, unavoidable manufacturing defects and errors in camera lenses during production and assembly lead to various types of distortion, significantly impacting image quality and measurement accuracy.

[0003] In camera calibration, obtaining intrinsic and extrinsic parameters is a linear process, while solving for distortion coefficients is a nonlinear process. The main error in camera calibration stems from the fitting and solving of camera distortion parameters. Therefore, establishing an accurate distortion model and performing high-precision parameter fitting is crucial for improving camera calibration accuracy. Common lens distortions include radial distortion and tangential distortion. Radial distortion refers to the radial bending of light as it passes through the lens, resulting in inconsistent magnification ratios between the image's center and edge regions; tangential distortion is caused by light shift due to imperfect lens assembly. For these distortions, mathematical models such as polynomial fitting are typically used for correction to restore the image's true geometry.

[0004] As the application scenarios of camera lenses continue to expand, the requirements for the applicability of camera calibration are also increasing. In some complex camera distortion situations, such as telephoto lenses, wide-angle imaging, and lens optical axis shift, asymmetric distortion often occurs. For these situations, current mainstream distortion models struggle to achieve good fitting results, leading to a decrease in camera calibration accuracy. To establish a high-precision distortion model with a wider range of applications, more parameters need to be introduced, which increases the difficulty of fitting the distortion model. Current calibration processes mainly involve uniformly fitting the camera's intrinsic and extrinsic parameters with the distortion model parameters, converging the distortion model parameters while optimizing the convergence of the camera's intrinsic and extrinsic parameters. To improve the fitting accuracy of the distortion model, a large amount of feature point data from the calibration images is required, but this also introduces more camera extrinsic parameter fitting, making it difficult to achieve good optimization results for the distortion model parameters. Summary of the Invention

[0005] To address the issue of insufficient accuracy in optimizing distortion model parameters under asymmetric offset distortion conditions caused by the unified optimization of camera intrinsic and extrinsic parameters and distortion model parameters in current mainstream camera calibration processes, this invention proposes an asymmetric lens distortion optimization method based on linear reconstruction of feature points guided by projection transformation. This method calculates the comprehensive error of linear projection by traversing the feature point coordinate matrix of the calibration image and, after secondary screening, obtains the region with the minimum linear projection error. Then, linear correction is performed on the feature points in this region using line constraints, cross-ratio constraints, and parallel line constraints. Next, the homography transformation matrix is ​​obtained by solving the linearly corrected point coordinates with the corresponding world coordinates. Based on the obtained homography transformation matrix, the distorted feature points of the calibration image can be linearly reconstructed based on the world coordinates of the feature points to obtain the corresponding undistorted feature point coordinates. These coordinates are then used to optimize the distortion model parameters, resulting in a high-precision asymmetric distortion model that compensates for lens asymmetric distortion.

[0006] The technical solution of the present invention is as follows:

[0007] I. An Asymmetric Lens Distortion Compensation Method Guided by Feature Point Projection Transformation for Linear Reconstruction

[0008] s1: Construct a coordinate matrix of distorted image feature points based on the feature points of the checkerboard-calibrated image;

[0009] s2: After using the reference grid R to extract the coordinate matrix of the distorted image by sliding grid, the reference grid set U is obtained;

[0010] s3: Using an asymmetric distortion model, find the reference mesh with the minimum distortion from the reference mesh set U and use it as the optimal reference mesh R. t Next, the optimal reference grid R is... t The feature point coordinates in the mesh are optimized to obtain the optimized reference mesh R. t ;

[0011] s4: Based on the optimized optimal reference grid R t Reconstruct a linear image feature point coordinate matrix;

[0012] s5: After optimizing the parameters in the asymmetric distortion model based on the distorted image feature point coordinate matrix and the linear image feature point coordinate matrix, the optimized asymmetric distortion model is obtained.

[0013] Specifically, s1 refers to:

[0014] s11: A sub-pixel feature point detection method based on the Harris operator is used to detect all feature points in the checkerboard calibration image, and the coordinates of each feature point and the size of the checkerboard are obtained.

[0015] s12: Initialize the grid point coordinate matrix according to the checkerboard size, store the coordinates of each detected feature point in the grid point coordinate matrix in the form of a two-dimensional grid and record it as the distorted image coordinate matrix.

[0016] Specifically, s3 refers to:

[0017] s31: Construct an asymmetric distortion model. Input the undistorted checkerboard image into the asymmetric distortion model to generate a reference distorted image. Use the coordinates of the distortion center point of the reference distorted image as the standard distortion center. The formula for the asymmetric distortion model is as follows:

[0018]

[0019]

[0020] Where λ is the distance between the feature point coordinates in the distorted image coordinate matrix and the distortion center point (c1, c2), k1, k2, k3 are three radial distortion coefficients, p1, p2 are two tangential distortion coefficients, and c3, c4 are two edge strong distortion coefficients. The distorted image coordinate matrix is ​​the first... Coordinates of feature points These are the coordinates of the predicted distortion feature points;

[0021] s32: Calculate the linear error E corresponding to each reference grid in the reference grid set U. st Cross-ratio error E cr Parallelism error E pa Remove the straightness error E from the reference grid set U. st Cross-ratio error E cr Or parallel line error E pa After identifying reference grids that exceed a preset error threshold, a preliminary set of reference grids, U′, is obtained.

[0022] s33: Generates linear error E st Cross-ratio error E cr Parallelism error E pa The corresponding weight coefficients are recorded as a weight coefficient combination. The comprehensive error of each reference grid in the initially screened reference grid set U′ is calculated. The reference grid with the smallest comprehensive error in the initially screened reference grid set U′ is taken as the optimal reference grid R corresponding to the current weight coefficient combination. t ;

[0023] s34: Change the values ​​of the weight coefficients and repeat s33 to obtain the optimal reference grid R corresponding to different combinations of weight coefficients. t Calculate the optimal reference grid R for different combinations of weighting coefficients. t Offset error E from the standard distortion centerp The offset error E p Minimum optimal reference grid R t As the final optimal reference grid R t ;

[0024] s35: With the final optimal reference grid R t The corresponding comprehensive error function E t Using the objective function, a nonlinear optimization method is used to optimize the final optimal reference grid R. t The coordinates of each feature point in the grid are optimized to obtain the optimal reference grid R. t .

[0025] In s32, the straightness error E of each reference grid is... st The calculation formula is as follows:

[0026]

[0027] Among them, v m,n Represents the m-th straight line l m The ordinate of the nth feature point Represents the m-th straight line l m The fitted ordinate of the nth feature point on the grid, r is the number of lines in each reference grid, and s is the number of feature points in each reference grid. To fit the straight line l m Substitute the x-coordinate u of the feature point into the calculated y-coordinate value. and The fitted straight line l m The slope and intercept, and They are lines l m The x and y coordinates of the nth point on the graph. and They represent the lines l and l respectively. m The average of the x and y coordinates of s points on the graph.

[0028] In s32, the cross ratio error function E of each reference grid cr The calculation formula is as follows:

[0029] E cr =(E crh +E crv ) / 8

[0030]

[0031] Among them, E crh To calculate the sum of the cross ratio errors for each row in the reference grid, E crvTo calculate the sum of the cross-ratio errors for each column in the reference grid, where r is the number of lines in each reference grid, s is the number of feature points in each reference grid, and P1, P2, P3, and P4 are denoted as... The four points correspond to the undistorted points in the world coordinate system, F cr () represents the cross-ratio function. and They are lines l m The x and y coordinates of the nth point on the graph, d ij Represents the midpoint of the m-th row. and The distances between them, i = 1, 2, j = 3, 4.

[0032] In s32, the parallelism error E of each reference grid is... pa The calculation formula is as follows:

[0033]

[0034]

[0035] Where r is the number of lines in each reference grid. and The m-th fitted line l m The slope and intercept, m = 1, 2, ..., r, and A is the coefficient matrix. Let r be the assumed vanishing point where the first to rth parallel lines intersect. and Vanishing points The x and y coordinates are given, b is a constant matrix, and T represents the transpose.

[0036] In s33, the comprehensive error E of the δ-th reference grid is... δ The formula is as follows:

[0037] E δ =λ cr E cr +λ st E st +λ pa E pa

[0038] Where, λ cr , λ st , λ pa These are the first to third weighting coefficients, respectively.

[0039] Specifically, s4 refers to:

[0040] s41: Construct a system of linear equations to solve the homography matrix based on the feature point coordinates of the optimized optimal reference mesh and their corresponding world coordinates;

[0041] s42: Solve the linear equation system of the homography matrix using the singular value decomposition method, obtain the eigenvector with the smallest singular value as the solution vector h, and reshape the homography matrix H based on the solution vector h;

[0042] s43: Extend the world coordinate vectors of the feature points to homogeneous coordinates to obtain the extended homogeneous coordinate matrix W. h Then, the homography matrix H is used to compare the expanded homogeneous coordinate matrix W. h Perform a perspective transformation to obtain the homogeneous coordinates T after the perspective transformation. h Then, the homogeneous coordinates T after perspective transformation h Convert to non-homogeneous coordinates to obtain a non-homogeneous coordinate vector T, which is then used as the undistorted image coordinate matrix.

[0043] Specifically, s5 refers to:

[0044] After substituting the coordinates of feature points in the distorted image coordinate matrix into the asymmetric distortion model, the predicted coordinates of the distorted feature points are obtained. Then, the error function E of the asymmetric distortion model is used. dis The error between the predicted coordinates of the distorted feature points and the angular coordinates in the distorted image coordinate matrix is ​​calculated. The parameters in the asymmetric distortion model are continuously adjusted until the error is minimized, thus obtaining the optimized asymmetric distortion model.

[0045] The error function E of the asymmetric distortion model dis The formula is as follows:

[0046]

[0047] in, The first element in the distorted image coordinate matrix The coordinates of feature points are given by n, where n represents the number of feature points. These are the coordinates of the predicted distortion feature points.

[0048] II. A computer device

[0049] The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the asymmetric lens distortion compensation method for feature point projection transformation-guided linear reconstruction.

[0050] III. A computer-readable storage medium

[0051] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the asymmetric lens distortion compensation method for feature point projection transformation-guided linear reconstruction.

[0052] IV. A computer program product

[0053] The computer program product includes a computer program / instruction that, when executed by a processor, implements the steps of the asymmetric lens distortion compensation method for feature point projection transformation-guided linear reconstruction.

[0054] The beneficial effects of this invention are:

[0055] This invention addresses the complexities of various lens distortions and the increasingly stringent requirements for camera calibration accuracy. It proposes an asymmetric lens distortion compensation method guided by feature point projection transformation for linear reconstruction. The method involves defining a minimum reference grid and sliding it across the detected feature point coordinate matrix to calculate the corresponding straight-line error, cross-ratio error, and parallelism error. The weighted average error of these three errors is used as an evaluation metric for the initially selected reference grid set, thereby identifying the optimal reference grid and calculating the corresponding homography matrix H. This allows for the determination of the coordinates of the non-distorted feature points corresponding to the distorted feature points in the checkerboard image, ultimately leading to the fitting of an asymmetric distortion model.

[0056] Compared to traditional camera calibration methods, the method proposed in this invention can solve the problem that traditional camera distortion models are difficult to fit with high accuracy in complex lens distortion scenarios, while reducing the reprojection error of camera calibration and improving the applicability of camera calibration technology. Attached Figure Description

[0057] Figure 1 It is the best reference grid for detecting and expanding feature points in 18*24 distorted checkerboard images.

[0058] Figure 2 This is a comparison chart of the coordinates of distorted and undistorted feature points in a checkerboard image.

[0059] Figure 3 This is a comparison chart of the coordinates of the undistorted feature points and the coordinates of the fitted feature points.

[0060] Figure 4 It is the distortion compensation error diagram in the current mainstream calibration method.

[0061] Figure 5 This is the distortion compensation error diagram in the calibration method used in this invention.

[0062] Figure 6 This is a flowchart of the method of the present invention. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the invention.

[0064] like Figure 6 As shown, the asymmetric lens distortion compensation method proposed in this invention, which uses feature point projection transformation to guide linear reconstruction, includes the following steps:

[0065] s1: Construct the distorted image coordinate matrix based on the feature points of the checkerboard-calibrated image;

[0066] s1 specifically refers to:

[0067] s11: A subpixel feature point detection method based on the Harris operator is used to detect all feature points in the checkerboard calibration image, and the subpixel coordinates (u,v) of each feature point and the checkerboard size N×M are obtained.

[0068] s12: Initialize the grid point coordinate matrix according to the chessboard size, with a size of N×M×2. Store the coordinates of each feature point (N×M) obtained by the feature point detection method in the grid point coordinate matrix in the form of a two-dimensional grid and record it as the distorted image coordinate matrix.

[0069] s2: After using the reference grid R to extract the coordinate matrix of the distorted image by sliding grid, the reference grid set U is obtained; the size of the reference grid R is 4×4×2, which consists of 4×4 image feature point coordinates (u,v).

[0070] s3: Find the reference grid with the smallest distortion from the reference grid set U and use it as the optimal reference grid R. t Next, the optimal reference grid R is... t The feature point coordinates in the mesh are optimized to obtain the optimized reference mesh R. t ;

[0071] s3 specifically refers to:

[0072] Currently, the mainstream distortion models mainly consider radial distortion and tangential distortion. The radial distortion polynomial is as follows:

[0073] x distorted =x(1+k1r) 2 +k2r 4 +k3r 6 )

[0074] y distorted =y(1+k1r 2 +k2r 4 +k3r 6 )

[0075] Where (x,y) are the undistorted normalized coordinates, (x... distorted ,y distorted ) is the distorted coordinate, r is the distance from the point to the center of the image, and k1, k2, k3 are the radial distortion coefficients.

[0076] The expression for tangential distortion is as follows:

[0077] x distorted =x+(2p1y+p2(r) 2 +2x 2 ))

[0078] y distorted =y+(p1(r) 2 +2y 2 )+2p2x)

[0079] Where p1 and p2 are tangential distortion coefficients.

[0080] For complex lens distortion situations, such as the distortion of periscope telephoto lenses and fisheye lenses, in addition to considering radial distortion and tangential distortion, it is also necessary to introduce asymmetric distortion factors and edge over-distortion factors.

[0081] s31: Construct an asymmetric distortion model. The initial values ​​of each parameter in the asymmetric distortion model are set according to the distortion parameters corresponding to the current mainstream telephoto lenses. After inputting a standard undistorted checkerboard image into the asymmetric distortion model, a reference distorted image is generated. The coordinates of the distortion center point of the reference distorted image are used as the standard distortion center. The formula of the asymmetric distortion model is as follows:

[0082]

[0083] Where λ is the distance between the feature point coordinates in the distorted image coordinate matrix and the distortion center point (c1, c2), k1, k2, k3 are three radial distortion coefficients, p1, p2 are two tangential distortion coefficients, and c3, c4 are two edge strong distortion coefficients. The distorted image coordinate matrix is ​​the first... Coordinates of one feature point;

[0084] S32: During perspective projection transformation, if the points in the scene are structured, collinearity and correlation within the scene will be preserved. Therefore, the world coordinates of feature points, after projection transformation, should satisfy cross-ratio invariance, line invariance, and the condition that parallel lines intersect at a common point. Calculate the line error E corresponding to each reference mesh in the reference mesh set U. st Cross-ratio error E cr Parallelism error E pa Remove the straightness error E from the reference grid set U. st Cross-ratio error Ecr Or parallel line error E pa After identifying reference grids that exceed a preset error threshold, a preliminary set of reference grids, U′, is obtained.

[0085] The straightness error E of each reference grid st The calculation formula is as follows:

[0086]

[0087] Among them, v m,n Represents the m-th straight line l m The ordinate of the nth feature point Represents the m-th straight line l m The fitted ordinate of the nth feature point on the grid, r is the number of lines in each reference grid, and s is the number of feature points in each reference grid. To fit the straight line l m Substitute the x-coordinate u of the feature point into the calculated y-coordinate value. and The fitted straight line l m The slope and intercept, and They are lines l m The x and y coordinates of the nth point on the graph. and They represent the lines l and l respectively. m The average of the x and y coordinates of s points on the network. For each row and column of feature points on the minimum reference network R, the least squares method can be used to calculate the average of the x and y coordinates of line l. m Perform line fitting to obtain the line l m The regression linear equation.

[0088] Cross-ratio error function E for each reference grid cr The calculation formula is as follows:

[0089] E cr =(E crh +E crv ) / 8

[0090]

[0091] Among them, E crh To calculate the sum of the cross ratio errors for each row in the reference grid, E crv To calculate the sum of the cross-ratio errors for each column in the reference grid, where r is the number of lines in each reference grid, s is the number of feature points in each reference grid, and P1, P2, P3, and P4 are denoted as... The four points correspond to the undistorted points in the world coordinate system, F cr () represents the cross-ratio function. and They are lines l m The x and y coordinates of the nth point on the graph, d ij Represents the midpoint of the m-th row. and The distances between them are i = 1, 2, j = 3, 4. Since each feature point is the intersection of a vertical line and a horizontal line, the cross-ratio error calculation for the minimum reference network must be performed simultaneously in both the horizontal and vertical directions.

[0092] After fitting the equations of the lines in each r rows and s columns of the minimum reference network, based on the parallelism of parallel lines, these two sets of parallel lines in each row and column will intersect at two different vanishing points. The parallelism error E of each reference grid is... p[ The calculation formula is as follows:

[0093]

[0094] Where r is the number of lines in each reference grid. and The m-th fitted line l m The slope and intercept, m = 1, 2, ..., r, and A is the coefficient matrix. Let r be the assumed vanishing point where the first to rth parallel lines intersect. and Vanishing points The x and y coordinates are given, b is a constant matrix, and T represents the transpose.

[0095] s33: Generates linear error E st Cross-ratio error E cr Parallelism error E pa The corresponding weight coefficients are recorded as a weight coefficient combination. The comprehensive error of each reference grid in the initially screened reference grid set U′ is calculated. The reference grid with the smallest comprehensive error in the initially screened reference grid set U′ is taken as the optimal reference grid R corresponding to the current weight coefficient combination. t Optimal reference grid R t The corresponding lens area is considered to be the optimal linear region;

[0096] The combined error E of the δ-th reference grid δ The formula is as follows:

[0097] E δ =λ cr E cr +λ st E st +λ pa E pa

[0098] Where, λ cr , λ st , λ pa These are the first to third weighting coefficients, respectively.

[0099] s34: Weighting coefficient λ cr , λ st , λ pa The range is [0,1]. The weight coefficients are changed in steps of 0.1. Repeat step s33 to obtain the optimal reference grid R corresponding to different combinations of weight coefficients. t Calculate the optimal reference grid R for different combinations of weighting coefficients. t Offset error E from the standard distortion center p The offset error E p Minimum optimal reference grid R t As the final optimal reference grid R t That is, choosing the offset error E. p Minimum optimal reference grid R t The combined error is calculated using the corresponding weighting coefficients.

[0100] The coordinates of each grid point are represented as (x... αβ ,y αβ ), where i and j represent the row and column of the grid, respectively (α,β=1,2,3,4), and the offset error E p The calculation formula is as follows:

[0101]

[0102] s35: With the final optimal reference grid R t The comprehensive error function E obtained by calculating the corresponding weight coefficient combination t Using the quasi-newton nonlinear optimization method as the objective function, the final optimal reference grid R is optimized. t The coordinates of each feature point in the reference mesh are optimized to ensure that the optimized feature point coordinates satisfy the linear constraints of the perspective projection transformation as much as possible, thus obtaining the optimized optimal reference mesh R. t (like Figure 1 As shown in the figure, the grid marked with an asterisk (*) is the optimal reference grid R after linear reconstruction. t (4×4 feature points).

[0103] s4: Based on the optimized optimal reference grid R t Construct an undistorted feature point image coordinate matrix, such as... Figure 1 As shown, the blue dots mark the non-distorted feature points corresponding to the chessboard distortion feature points after linear reconstruction.

[0104] s4 specifically refers to:

[0105] s41: Construct a system of linear equations to solve the homography matrix based on the feature point coordinates of the optimized reference mesh and their corresponding world coordinates; assuming there are n point pairs, a 2n-row matrix P needs to be constructed. Let the homography matrix solution vector h = [h 11 ,h 12 ,h 13 ,g 21 ,g 22 ,g 23 ,g 31 ,g 32 ,h 33 ] T For each feature point, the world coordinates (X) w ,Y w Given the coordinates of the feature point (u, v) and the homography matrix, the following system of linear equations is used to solve the system:

[0106]

[0107] s42: Solve the linear equation system of the homography matrix using the singular value decomposition (SVD) method, obtain the eigenvector with the smallest singular value as the solution vector h, and reshape the homography matrix H based on the solution vector h;

[0108] s43: Extend the world coordinate vectors of the feature points to homogeneous coordinates to obtain the extended homogeneous coordinate matrix W. h Represented as:

[0109]

[0110] Then use the homography matrix H to compare the expanded homogeneous coordinate matrix W h Perform a perspective transformation to obtain the homogeneous coordinates T after the perspective transformation. h :

[0111] T h =HW h T

[0112]

[0113] Then the homogeneous coordinates T after perspective transformation h Convert to non-homogeneous coordinates to obtain a non-homogeneous coordinate vector T, which is then used as the undistorted image coordinate matrix, as shown in the following formula:

[0114]

[0115] The first nonhomogeneous coordinate vector T coordinates This gives the non-distorted image feature point coordinates corresponding to each image feature point coordinate (u,v).

[0116] s5: Based on the coordinate matrix of the distorted image feature points (e.g., Figure 2 (red markers in the image) and the coordinate matrix of undistorted image feature points (e.g., red markers) Figure 2 After optimizing the parameters in the asymmetric distortion model (marked by the blue markers in the diagram), the optimized asymmetric distortion model is obtained.

[0117] S5 specifically refers to:

[0118] After substituting the coordinates of the feature points in the undistorted image feature point coordinate matrix into the asymmetric distortion model, the predicted coordinates of the distorted feature points are obtained (e.g., ...). Figure 3 (As shown by the blue marker), using the error function E of the distortion model. dis The error between the predicted coordinates of the distortion feature points and the angular coordinates in the matrix of the distorted image feature points is calculated. The parameters (k1,k2,k3,p1,p2,c1,c2,c3,c4) in the asymmetric distortion model are continuously adjusted using the interior-point nonlinear optimization method until the error is minimized, thus obtaining the optimized asymmetric distortion model. This model can perform high-precision compensation and correction for complex distortion situations, such as lenses with asymmetric distortion. Figure 3 This diagram illustrates the compensated image feature point matrix and the original image feature point matrix under the condition of minimum error after parameter optimization. To illustrate the compensation effect of the method of this invention, tests were conducted on four calibration checkerboard images at different angles. Figure 5 The table shows the average compensation error for each image and compares it with the compensation error of the four images using the current mainstream MATLAB methods. The results are as follows: Figure 4 As shown.

[0119] Among them, the error function E of the distortion model dis The formula is as follows:

[0120]

[0121] in, The first element in the distorted image coordinate matrix The coordinates of feature points are given by n, where n represents the number of feature points. The coordinates of the distortion feature points are used for compensation.

[0122] Finally, it should be noted that the above embodiments and descriptions are only used to illustrate the technical solutions of the present invention and not to limit it. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the disclosure of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the protection scope of the claims of the present invention.

Claims

1. An asymmetric lens distortion compensation method for feature point projection transformation guided linear reconstruction, characterized in that, The method comprises the following steps: s1: constructing a distorted image feature point coordinate matrix according to feature points of a checkerboard calibration image; s2: obtaining a reference grid set U by slidingly grid extracting the distorted image coordinate matrix using a reference grid R; s3: finding the reference grid with the least distortion from the reference grid set U by using the asymmetric distortion model and taking it as the optimal reference grid R t ; then optimizing the feature point coordinates in the optimal reference grid R t , and obtaining the optimized reference grid R t ; s4: reconstructing the image feature point coordinate matrix according to the optimized optimal reference grid R t reconstructing the linear image feature point coordinate matrix; s5: obtaining an optimized asymmetric distortion model by optimizing parameters in the asymmetric distortion model according to the distorted image feature point coordinate matrix and the linear image feature point coordinate matrix; The s3 is specifically: s31: constructing an asymmetric distortion model, inputting a non-distorted checkerboard image into the asymmetric distortion model to generate a reference distorted image, taking a distortion center point coordinate of the reference distorted image as a standard distortion center, and the asymmetric distortion model has the following formula: wherein, is the distance between the feature point coordinate in the distorted image coordinate matrix and the distortion center point , , k1, k2, k3 are three radial distortion coefficients, p1, p2 are two tangential distortion coefficients, , are two edge strong distortion coefficients, is the coordinate of the mth feature point in the distorted image coordinate matrix, is the coordinate of the mth feature point in the distorted image coordinate matrix, is the predicted distorted feature point coordinate; s32: calculating the straight line error corresponding to each reference grid in the reference grid set U , the cross ratio error and the parallel line error , deleting the reference grid with the straight line error , the cross ratio error or the parallel line error greater than the preset error threshold value in the reference grid set U to obtain the preliminary screened reference grid set ; s33: generating straight line error , cross ratio error and parallel line error corresponding weight coefficients and recorded as a weight coefficient combination, calculate the comprehensive error of each reference grid in the preliminary screened reference grid set , the preliminary screened reference grid set The reference grid with the smallest comprehensive error in the preliminary screened reference grid set is taken as the optimal reference grid R corresponding to the current weight coefficient combination t ; s34: change the value of the weight coefficient, repeat s33, and obtain the optimal reference grid R corresponding to different combinations of weight coefficients t , calculate the optimal reference grid R of different combinations of weight coefficients t The offset error of the standard distortion center , the offset error The smallest optimal reference grid R t As the final optimal reference grid R t ; s35: with the final optimal reference mesh R t corresponding integrated error function as the objective function, using a nonlinear optimization method to optimize each feature point coordinate in the final optimal reference mesh R t , and obtaining the optimized optimal reference mesh R t .

2. The method according to claim 1, wherein, The s1 is specifically: s11: detecting all feature points of the checkerboard calibration image by adopting a sub-pixel feature point detection method based on a Harris operator, and obtaining coordinates of the feature points and a checkerboard size; s12: initializing a grid point coordinate matrix according to the checkerboard size, storing the obtained coordinates of the feature points in the grid point coordinate matrix in a two-dimensional grid form, and recording the grid point coordinate matrix as a distorted image coordinate matrix.

3. The method of claim 1, wherein, In the s32, the straight line error of each reference grid The calculation formula is as follows: wherein, represents the mth straight line represents the longitudinal coordinate of the nth feature point on the mth straight line represents the mth straight line represents the fitted longitudinal coordinate of the nth feature point on the mth straight line, r is the number of straight lines in each reference grid, and s is the number of feature points in each reference grid, is the fitted straight line is substituted into the horizontal coordinate of the feature point is the calculated longitudinal coordinate value, and are the slope and intercept of the fitted straight line respectively, and are the horizontal coordinate and longitudinal coordinate of the nth point on the straight line respectively, and represent the average value of the horizontal coordinate and longitudinal coordinate of the s points on the straight line respectively.

4. The method of claim 1, wherein, In the s32, the cross ratio error function of each reference grid The calculation formula is as follows: wherein, is the sum of the cross-ratio errors of each row in the reference grid, is the sum of the cross-ratio errors of each column in the reference grid, r is the number of straight lines in each reference grid, and s is the number of feature points in each reference grid, is represented as the non-distorted points in the world coordinate system corresponding to the four points, represents the cross-ratio function, and are the horizontal coordinate and the vertical coordinate of the nth point on the straight line , respectively, represents the distance between the points and in the mth row, .

5. The method of claim 1, wherein, In the s32, the parallel line error of each reference grid The calculation formula is as follows: ， ， where r is the number of straight lines in each reference grid, and are the slope and intercept of the m-th fitted straight line respectively, m = 1, 2,..., r, is the coefficient matrix, is the assumed vanishing point of intersection of the 1st to r-th parallel lines, and are the horizontal and vertical coordinates of the vanishing point respectively, is the constant matrix, denotes the transpose.

6. The method of claim 1, wherein, In the s33, the formula of the comprehensive error of the first reference grid is as follows: ​ wherein are first-third weight coefficients, respectively.

7. The method of claim 1, wherein, The s4 is specifically: s41: constructing a linear equation set for solving a homography matrix according to feature point coordinates of an optimal reference grid after optimization and corresponding world coordinates; s42: solving the linear equation set of the homography matrix by using a singular value decomposition method, obtaining a feature vector with a minimum singular value as a solution vector h by solving, and remodeling the homography matrix H according to the solution vector h; s43: extend the feature point world coordinate vector to homogeneous coordinates to obtain an extended homogeneous coordinate matrix ; use the homography matrix H on the extended homogeneous coordinate matrix to perform perspective transformation to obtain a perspective-transformed homogeneous coordinate , and then convert the perspective-transformed homogeneous coordinate to non-homogeneous coordinates to obtain a non-homogeneous coordinate vector T and as a non-distorted image coordinate matrix.

8. The method of claim 1, wherein, The s5 is specifically: After the feature point coordinates in the distorted image coordinate matrix are brought into the asymmetric distortion model, predicted distorted feature point coordinates are obtained, and an error function of the asymmetric distortion model is used to The error between the predicted distorted feature point coordinates and the angle coordinates in the distorted image coordinate matrix is calculated, the parameters in the asymmetric distortion model are continuously adjusted until the error is minimized, and the optimized asymmetric distortion model is obtained.

9. The method according to claim 8, wherein, error function of the asymmetric distortion model The formula is as follows: wherein, is the i-th feature point coordinate in the distorted image coordinate matrix, is the i-th feature point coordinate in the distorted image coordinate matrix, denotes the number of feature points, is the predicted distorted feature point coordinate.

Images