A method for calibrating the distortion center based on a single image in the AA process of image sensors

By using an image sensor calibration method based on a single image, the distortion center is calculated directly in a two-dimensional image coordinate system, which solves the problem of accuracy and efficiency in determining the location of the distortion center in the AA process, and achieves fast and accurate distortion center calibration.

CN118015094BActive Publication Date: 2024-11-01ZHUHAI CITY GUANGHAOJIE PRECISION MACHINERY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410136891.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-31
Publication Date
2024-11-01
Estimated Expiration
2044-01-31

AI Technical Summary

Technical Problem

In the AA process, existing technologies struggle to quickly and accurately determine the location of the distortion center under limited hardware conditions, and the calculation process is time-consuming and lacks robustness.

Method used

A single-image-based image sensor calibration method is adopted. Collinear point sets are collected by a positioning plate with a straight line pattern. Image coordinates are normalized and iteratively optimized to establish a radial distortion model. The inverse distortion error is calculated and the total error function is optimized. The distortion center point is directly solved in the two-dimensional image coordinate system.

Benefits of technology

The calibration requirements have been simplified, requiring only one image to complete the calibration. This improves the accuracy and computational efficiency of the distortion center calibration, reduces the number of computational parameters, and enhances system stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118015094B_ABST
    Figure CN118015094B_ABST
Patent Text Reader

Abstract

The present invention discloses a camera calibration method based on an improved distortion model. A calibration plate is used to calibrate the coordinates of an image, and a plurality of collinear point sets are collected to detect corner points and collinear points, and the coordinates of the image are normalized; a distortion model is established for the point set of the image coordinates, and then the point set is subjected to an anti-distortion operation, and the errors before and after the anti-distortion are calculated, and a total error optimization objective function for all points is established; the total error optimization objective function is solved, and an inverse normalization operation is performed on the optimal point to obtain the distortion center point. The present invention realizes accurate positioning of the coordinates of a single photo. The inverse distortion model is used to perform calculations under the coordinates of a two-dimensional image to improve the efficiency of calculations and the stability of the system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of camera module assembly, and in particular to a method for calibrating a distortion center based on a single picture in an AA process of an image sensor. Background Art

[0002] The AA process, short for Active Alignment, is an assembly technology used in the camera assembly process. It obtains multiple images by scanning, calculates the resolution of the specified position in the scanned image, obtains the positional relationship between the image sensor and the lens, and then adjusts the image sensor plane to be perpendicular to the lens optical axis through a 6-axis platform to ensure the production yield. At present, the principles and methods for calibrating the lens distortion center are mostly based on the "Zhang Zhengyou calibration method". This method requires adjusting the relative posture of the camera module and the calibration plate, collecting dozens of calibration pictures, and calculating external parameter information such as relative rotation and translation, camera internal parameters, distortion coefficients, etc. based on the world coordinates and image coordinates of the calibration plate.

[0003] In the AA process, the spatial freedom of the calibration plate is limited by the size of the AA process equipment. Since the hardware environment is difficult to meet the image acquisition requirements of the "Zhang Zhengyou calibration method", the calibration process has problems such as insufficient camera internal parameter constraints, poor robustness and accuracy of the calculation results, and the location of the distortion center COD cannot be accurately determined. In addition, the "Zhang calibration method" has the problem of calculating multiple pictures, resulting in large data volumes, many parameters, and serious time consumption. Summary of the invention

[0004] The purpose of the present invention is to design a method for calibrating the distortion center based on a single image in the AA process of an image sensor, so as to realize rapid and accurate confirmation of the distortion center point.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for calibrating a distortion center based on a single image in an AA process of an image sensor comprises the following steps:

[0007] Step 1: Use a positioning plate with a straight line pattern as a calibration plate to accurately locate the image coordinates, collect several sets of collinear point sets, and detect corner points and collinear points;

[0008] Step 2: Normalize the image coordinate points and perform iterative optimization;

[0009] Step 3: According to Zhang Zhengyou calibration method, a radial distortion forward model is established for the image coordinate point set;

[0010] Step 4: Perform an anti-distortion operation on the image coordinate point set to obtain the anti-distorted point set;

[0011] Step 5: Calculate the error of the image coordinate points before and after anti-distortion;

[0012] Step 6: Establish the total error optimization objective function of all points;

[0013] Step 7: Solve the total error optimization objective function, perform inverse normalization on the optimal point, and find the center of distortion.

[0014] Furthermore, the calibration plate in step 1 uses a chessboard as a positioning plate, uses a camera module to capture a clear image as a calibration image, and obtains multiple sets of collinear point sets on the calibrated image.

[0015] Furthermore, in step 1, opencv is called to perform sub-pixel corner detection and collinear point detection.

[0016] Furthermore, in step 1, corner point detection is performed using the Harris method, and then collinear detection of the corner points is performed along the edge direction.

[0017] Furthermore, the image coordinate points in step 2 are normalized and iteratively optimized, as follows:

[0018] x n =sx,y n =sy (1)

[0019]

[0020] where P(x n ,y n ) represents the point after the normalization operation of the image focus, denoted as P n ; s is a constant, representing the normalized scaling factor; w, h are constants, representing the length and width of the image, respectively.

[0021] Furthermore, in step 3, the radial distortion forward model is established for the image coordinate point set as follows:

[0022]

[0023] Among them, (x u ,y u ) anti-distortion point, denoted as P u , (x0, y0) represents the center of distortion, denoted as Represents the Euclidean distance from the distortion point to the distortion center.

[0024] Step 4 is as follows:

[0025]

[0026] Perform an anti-distortion operation on the i-th group of collinear point sets on the image to obtain a set of anti-distorted point sets. According to the physical properties of the calibration plate pattern, the group of anti-distorted point sets are collinear, and the collinear general equation is set as:

[0027] A i x+B i y+C i =0 (5)

[0028] Among them, A i , B i , C i is the constant in the straight line equation corresponding to the i-th straight line.

[0029] Furthermore, step 5 is as follows:

[0030] according to Figure 1 , from the similar triangle relationship and vector relationship, we can know that:

[0031]

[0032]

[0033] in, Represents point P′ u To point P d The vector of Represents point P′ u The vector to point P0, d0, d d Represent P0, P d Projection points P′0 and P′ to line l d Distance, rearrange the above formula to get:

[0034]

[0035] For the jth point on the i-th line, the error of the point after anti-distortion is:

[0036]

[0037] Furthermore, step 6 is as follows:

[0038] For the m straight lines (m>0) extracted from the actual calibration plate and the n points (n>2) on the straight lines, the total error optimization objective function of all points can be obtained as follows:

[0039]

[0040] Furthermore, step 7 is as follows:

[0041] Use the nonlinear optimization method to solve the total error optimization objective function through the Gauss-Newton method or the LM method. Perform an inverse normalization operation on the optimal COD point obtained by the nonlinear solution to obtain the current image distortion center point:

[0042]

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The requirements for the calibration plate are simplified. No specific chessboard is required, only a straight line pattern is required; no large number of images are required, only one image is required; the requirements for image acquisition are simplified. There is no need to change the relative posture of the camera and the calibration plate to acquire a large number of images. Only one image needs to be collected to complete the calibration, which makes it more convenient to collect images in the compact and closed environment of the AA process; the accuracy of the COD calibration results is improved. The traditional calibration method requires the three-dimensional point information of the calibration plate, and then transforms it to the camera coordinate system through the projection of external parameters. This process will also introduce calculation errors. The present invention adopts the "inverse distortion model", which directly performs a series of calculations in the two-dimensional image coordinate system, omitting the three-dimensional projection process, improving the calculation accuracy, optimizing from the data volume, not relying on multiple image data, and improving the calculation speed. Optimization in terms of calculation amount: the model omits the projection process from three-dimensional to two-dimensional, and directly models on the 2-bit plane of the image, thereby improving the calculation efficiency; reducing model parameters, introducing normalization and other operations, and improving the stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the corresponding geometric relationship between the front and back points of anti-distortion;

[0046] Figure 2 is the original collection picture;

[0047] Figure 3 To find the corner point effect diagram;

[0048] Figure 4 This is the anti-distortion effect diagram. DETAILED DESCRIPTION

[0049] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0050] A method for calibrating a distortion center based on a single image in an AA process of an image sensor comprises the following steps:

[0051] Step 1: Figure 2As shown, a positioning plate with a straight line pattern is used as a calibration plate to accurately locate the image coordinates. A chessboard can be used as a calibration plate. A relatively clear picture is taken to accurately locate the image coordinates and collect several groups of collinear point sets, and the corner points and collinear points are detected.

[0052] Step 2: Normalize the image coordinate points, call OpenCV to perform sub-pixel corner point detection and collinear point detection or use the Harris method to perform corner point detection, and then perform collinear detection of corner points along the edge direction, and iteratively optimize the collected point set, as follows:

[0053] x n =sx,y n =sy (1)

[0054]

[0055] where P(x n ,y n ) represents the point after the normalization operation of the image focus, denoted as P n ; s is a constant, representing the normalized scaling factor; w, h are constants, representing the length and width of the image, respectively.

[0056] Step 3: Establish a radial distortion forward model for the image coordinate point set as follows:

[0057]

[0058] Among them, (x u ,y u ) anti-distortion point, denoted as P u , (x0, y0) represents the center of distortion, denoted as Represents the Euclidean distance from the distortion point to the distortion center.

[0059] Step 4: Perform an anti-distortion operation on the coordinate point set to obtain the anti-distorted point set, as follows:

[0060]

[0061] Perform an anti-distortion operation on the i-th group of collinear point sets on the image to obtain a set of anti-distorted point sets. According to the physical properties of the calibration plate pattern, the group of anti-distorted point sets are collinear, and the collinear general equation is set as:

[0062] A i x+B i y+C i =0 (5)

[0063] Among them, A i , B i , Ci is the constant in the straight line equation corresponding to the i-th straight line.

[0064] Step 5: Calculate the error of the image coordinate points before and after anti-distortion. Figure 1 , from the similar triangle relationship and vector relationship, we can know that:

[0065]

[0066] in, Represents point P′ u The vector to point Pd, Represents point P′ u The vector to point P0, d0, d d Represent P0, P d Projection points P′0 and P′ to line l d Distance, rearrange the above formula to get:

[0067]

[0068] For the jth point on the i-th line, the error of the point after anti-distortion is:

[0069]

[0070] Step 6: Establish the total error optimization objective function of all points. For the m straight lines (m>0) extracted from the actual calibration plate, and the n points (n>2) on the straight lines, the total error optimization objective function of all points can be obtained as:

[0071]

[0072] Step 7: Solve the total error optimization objective function, perform inverse normalization on the optimal point, and find the center of distortion. Use the nonlinear optimization method to solve the total error optimization objective function through the Gauss-Newton method or the LM method. Perform an inverse normalization operation on the optimal COD point obtained by the nonlinear solution to obtain the center of distortion of the current image:

[0073]

[0074] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for calibrating the center of distortion based on a single image in an image sensor AA process, characterized in that: The following steps are involved: Step 1: Use a positioning plate with a straight line pattern as a calibration plate to accurately locate the image coordinates of a picture with clear pixel contrast, collect several sets of collinear image coordinate point sets, and detect corner points and collinear points; Step 2: Normalization of image coordinate points; Step 3: According to Zhang Zhengyou's calibration method, establish the radial distortion forward model: Among them, (x u ,y u ) anti-distortion point, denoted as P u , (x0, y0) represents the center point of distortion, denoted as P0; represents the Euclidean distance from the distortion point to the distortion center, (x n ,y n ) represents the point after the image focus is normalized in step 2; Step 4: Establish an inverse solution anti-distortion model, and perform anti-distortion operations on the image coordinate point set to obtain the anti-distorted point set: The i-th group of collinear point sets on the image is subjected to an anti-distortion operation to obtain a group of anti-distorted point sets. According to the physical properties of the calibration plate pattern, the group of anti-distorted point sets are collinear, and the collinear general equation is set as: A i x+B i y+C i =0 (5) Among them, A i , B i , C i is the constant in the linear equation corresponding to the i-th straight line; Step 5: Calculate the error of the image coordinate points before and after anti-distortion; Step 6: Establish the total error optimization objective function of all points; Step 7: Solve the total error optimization objective function, perform inverse normalization on the optimal COD point, and obtain the distortion center point.

2. The method for calibrating the distortion center based on a single image in the AA process of the image sensor according to claim 1, characterized in that: In step 1, a chessboard or a positioning plate with a collinear point pattern is used as a positioning plate, and then a camera module is used to capture a clear image as a calibration image, and multiple sets of collinear image coordinate point sets are obtained on the calibration image.

3. The method for calibrating the distortion center based on a single image in the AA process of the image sensor according to claim 1, characterized in that: In step 1, opencv is called to perform sub-pixel corner detection and collinear point detection.

4. The method for calibrating the distortion center based on a single image in the AA process of an image sensor according to claim 1, characterized in that: In step 1, the Harris method is used to detect corner points, and then the collinearity detection of the corner points is performed along the edge direction.

5. The method for calibrating the distortion center based on a single image in the AA process of an image sensor according to claim 3 or 4, characterized in that: Step 2 normalizes the image coordinate points and performs iterative optimization, as follows: x n =sx,y n =and (1) where P represents the image corner point; (x n ,y n ) represents the point after the normalization operation of the image focus, denoted as P n ; s is a constant, representing the normalized scaling factor; w, h are constants, representing the length and width of the image respectively.

6. The method for calibrating the distortion center based on a single image in the AA process of the image sensor according to claim 1, characterized in that: Step 5 is as follows: According to Figure 1, from the similar triangle relationship and vector relationship, we can know that: in, Represents point P′ u The vector to point Pd, Represents point P′ u The vector to point P0, d0, d d Represent P0, P d Projection points P′0 and P′ to line I d Distance, rearrange the above formula to get: For the jth point on the i-th line, the error of the point after anti-distortion is:

7. The method for calibrating the distortion center based on a single image in the AA process of the image sensor according to claim 6, characterized in that: Step 6 is as follows: For the m straight lines extracted from the actual calibration plate and the n points on the straight lines, m>0, n>2, the total error optimization objective function of all points is obtained as follows:

8. The method for calibrating the distortion center based on a single image in the AA process of the image sensor according to claim 5, characterized in that: Step 7 is as follows: Use the nonlinear optimization solution method to solve the total error optimization objective function through the Gauss-Newton method or the LM method. For the optimal COD point (x0, y0) obtained by the nonlinear solution, perform an inverse normalization operation to obtain the current image distortion center point (x′0, y′0).