Detection method for optical center of camera

Abstract

The invention relates to a method for detecting the optical center of a camera. Comprising the following steps: acquiring SFR mark card images at a plurality of positions; extracting each angular point in the images, and calculating the coordinates of each angular point in each image according to the sequence; establishing a three-dimensional coordinate of each angular point, and performing iterative convergence to obtain an internal parameter matrix and a distortion coefficient; the optical center is corrected according to the internal parameter matrix and the distortion coefficient which are obtained through iteration, high-precision optical center detection can be achieved by shooting SFR mark card images of multiple angles and combining the least square method and singular value decomposition (SVD), measurement errors are reduced through multi-angle data fusion, and the detection precision is improved. And a more accurate internal reference matrix and a more accurate distortion coefficient are obtained through an iterative optimization mode, so that the accuracy of optical center measurement is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of camera testing, and particularly to a method for detecting the optical center of a camera. Background Art

[0002] With the continuous development of camera technology, cameras have been widely used in multiple fields (such as mobile phones, vehicles, medical, and drones, etc.). For applications with high precision requirements, especially multi-camera systems, the accurate detection of the optical center is particularly important. If there is a large deviation in the optical center, it may lead to the loss of details during binocular image synthesis, affecting the visual effect. Therefore, accurately detecting the deviation of the camera optical center, especially for complex camera modules (such as periscope cameras), is of great significance.

[0003] Traditional optical center detection methods mostly rely on testing under a white field. However, in some cases, especially for camera modules with prism screen printing, the accuracy of the test results is poor. This is because traditional methods cannot effectively handle the distortion of the camera. Especially for camera modules with small distortion, the applicability of the test algorithm is limited, and there are still certain challenges in the prior art on how to more accurately obtain the optical center.

[0004] Therefore, the present invention proposes a method for detecting the optical center of a camera. Summary of the Invention

[0005] In view of the technical problems existing in the prior art, the present invention provides a method for detecting the optical center of a camera. Through accurate corner extraction and a three-dimensional to two-dimensional projection model, using an iterative optimization algorithm (such as the least squares method), the optical center and internal parameters of the camera are finally obtained. By optimizing the internal parameters, distortion coefficients, rotation matrix, and translation vector, the imaging of the camera is made more accurate and the distortion is reduced.

[0006] The technical solution for the present invention to solve the above technical problems is as follows: A method for detecting the optical center of a camera; comprising the following steps:

[0007] S1: Obtain SFR target card images at multiple positions;

[0008] S2: Extract each corner point in the image, and calculate the coordinates of each corner point in each image according to the sorting;

[0009] S3: Establish the three-dimensional coordinates of each corner point, and perform iterative convergence to obtain the internal parameter matrix and distortion coefficients;

[0010] S31: Set the initial internal parameter adoption matrix and distortion coefficients, and set the rotation matrix and translation vector to 0;

[0011] S32: Set the preset reprojection error to infinity;

[0012] S33: According to the three-dimensional coordinates of the corner points, the current internal parameter matrix, distortion coefficients, rotation matrix, and translation vector, complete the 3D to 2D projection according to the distortion model to obtain the two-dimensional coordinates, and obtain the average reprojection error of all images by comparing the two-dimensional coordinates with the corner point coordinates in S2;

[0013] S34: Judge the convergence of the average reprojection error and the preset reprojection error. If the condition is met, the iteration ends; otherwise, proceed to the next step;

[0014] S35: Use the average reprojection error as the preset reprojection error;

[0015] S36: Fit the distortion coefficients;

[0016] S37: Optimize the internal parameter matrix;

[0017] S38: Update the internal parameters and obtain the image matrix and translation vector at each image position, and iterate to S33;

[0018] S4: Calibrate the optical center according to the internal parameter matrix and distortion coefficients obtained by iteration.

[0019] Furthermore, in S1, it includes:

[0020] Set up the shooting environment. Place the SFR calibration card within the depth of field in front of the module, ensure its flatness and the clarity of the module shooting, and add a light source at the same time to ensure that the appropriate color temperature and illuminance are set through the light source board;

[0021] Adjust the position parameters of the module at various positions. At least four different positions are required.

[0022] Furthermore, in S2, it includes: Use the cv::goodFeaturesToTrack function provided by opencv to detect corner points, and at the same time perform sub-pixel level corner refinement through the cv::cornerSubPix function to improve the corner positioning accuracy. Then, rely on the characteristics of the inner corner points of the checkerboard to sort the inner corner points and calculate the coordinates of each corner point in each image: (x ij , y ij ), where i represents the i-th image, j represents the j-th corner point in the i-th image, where i ∈ [1, n] and j ∈ [1, m].

[0023] Furthermore, in S3, it includes: the internal parameter matrix K and the distortion coefficients Coef;

[0024]

[0025] Coef = [k1 k2 k3 k4].

[0026] Further, in S31, it includes:

[0027]

[0028] Coef = [k1 k2 k3 k4] = [0 0 0 0];

[0029] where efl is the effective focal length, pixelsize is the pixel size of the sensor, and width and height are the width and height of the image respectively.

[0030] Further, in S33, it includes:

[0031] where i ∈ [1, n] and j ∈ [1, m];

[0032] where is the 3D to 2D projection and obtains the two-dimensional coordinates.

[0033] Further, in S34, it includes:

[0034] The conditions are:

[0035] where RMS mean is the preset reprojection error.

[0036] Further, in S36, it includes:

[0037] Calculate the undistorted radius and the distorted radius

[0038]

[0039] Solve using the least squares method, construct the coefficient matrix A and the result vector b according to the 5th-order polynomial distortion model and solve the linear equation system Ax = b using singular value decomposition;

[0040] Coefficient matrix A:

[0041]

[0042] Result vector b:

[0043]

[0044] Obtain

[0045] Further, in S37, it includes:

[0046] Use the least squares method, set the coefficient matrix Q and the right-side vector E and solve the linear equation system Qd = E using singular value decomposition;

[0047]

[0048] Among them

[0049]

[0050]

[0051] The beneficial effects of the present invention are as follows:

[0052] By taking SFR target card images from multiple angles and combining the least squares method and singular value decomposition (SVD), the present invention can achieve high-precision optical center detection. Through multi-angle data fusion, measurement errors are reduced, and a more accurate internal parameter matrix and distortion coefficient are obtained through iterative optimization, significantly improving the accuracy of optical center measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a schematic flowchart of a method for detecting the optical center of a camera.

[0054] Figure 2 is a schematic diagram of the shooting positions of a method for detecting the optical center of a camera;

[0055] Figure 3 is a schematic diagram of the corner point sorting of a method for detecting the optical center of a camera;

[0056] Figure 4 is a schematic diagram of the locally enlarged two-dimensional coordinates in a method for detecting the optical center of a camera;

[0057] Figure 5 is a shooting schematic diagram of an FF (fixed focal length) camera in an embodiment;

[0058] Figure 6 is a shooting schematic diagram of a periscope autofocus (AF) camera in an embodiment;

[0059] Figure 7 is a schematic flowchart of S3 in a method for detecting the optical center of a camera. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0061] In the description of the present application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present application, "a plurality of" means two or more unless otherwise specifically defined.

[0062] In the description of the present application, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present application is not necessarily construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to implement and use the present invention. In the following description, details are set forth for purposes of explanation. It should be understood that those of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed in the present application.

[0063] In one embodiment, a method for detecting the optical center of a camera; includes the following steps:

[0064] S1: Obtain SFR target card images at multiple positions;

[0065] S2: Extract each corner point in the image, and calculate the coordinates of each corner point in each image according to the sorting;

[0066] S3: Establish the three-dimensional coordinates of each corner point, and perform iterative convergence to obtain the internal parameter matrix and distortion coefficients;

[0067] S31: Set the initial internal parameter adoption matrix and distortion coefficients, and set the rotation matrix and translation vector to 0;

[0068] S32: Set the preset reprojection error to infinity;

[0069] S33: According to the three-dimensional coordinates of the corner points and the current internal parameter matrix, distortion coefficients, rotation matrix and translation vector, complete the 3D to 2D projection according to the distortion model and obtain the two-dimensional coordinates, and obtain the average reprojection error of all images by comparing the two-dimensional coordinates with the corner point coordinates in S2;

[0070] S34: Judge the convergence situation of the average reprojection error and the preset reprojection error. If the condition is satisfied, the iteration ends; otherwise, proceed to the next step;

[0071] S35: Take the average reprojection error as the preset reprojection error;

[0072] S36: Fitting distortion coefficient;

[0073] S37: Optimizing the intrinsic parameter matrix;

[0074] S38: Updating the intrinsic parameters and obtaining the image matrix and translation vector at each image position, and iterating to S33;

[0075] S4: Calibrating the optical center according to the intrinsic parameter matrix and distortion coefficient obtained by iteration.

[0076] In this embodiment, the detection of the optical center is realized by means of different relative positions. Originally, it was done together with other tests under the white field, but now it is changed to be done at the SFR test station. No additional station needs to be added, and only the module orientation needs to be adjusted;

[0077] Specifically: Set up the shooting environment, place the SFR target card within the depth of field in front of the module, ensure its flatness and the clarity of the module shooting, and add a light source at the same time to ensure that the appropriate color temperature and illuminance are set through the light source board.

[0078] Adjust the parameters of each position of the module placement. The jig equipped with the camera can adjust the three-axis position. For example, the camera is placed in the center (x = 0, y = 0, z = 0), the center position when placed upward is (x = 0, y = 10, z = 0), the center position when placed downward is (x = 0, y = -10, z = 0), the center position when placed to the left is (x = -10, y = 0, z = 0), and the center position when placed to the right is (x = -10, y = 0, z = 0). Of course, the z-axis movement can also be controlled. Here, at least four different positions are required, as Figure 2 shown in the schematic diagram of five positions.

[0079] Place the module well, take and save pictures at multiple positions according to Step 2. The pictures can be in formats such as raw, bmp, jpg, png, jpeg, etc.

[0080] Perform optical operations: Establish a distortion model. Here, a fifth-order polynomial distortion model is used

[0081] r u = r d + k1r d 2 + k2r d 3 + k3r d 4 + k4r d 5 ;

[0082] r d is the distortion radius normalized to the distance from the center to the corner point, r uIs the radius for distortion removal.

[0083]

[0084] Pixel coordinates:

[0085]

[0086] Extract the corner points of the images at multiple saved positions. Here, since the chart is shared with SFR, the cv::goodFeaturesToTrack function provided by OpenCV is generally used to detect the corner points. At the same time, the cv::cornerSubPix function is used for sub-pixel level corner refinement to improve the corner positioning accuracy. Then, the interior corner points are sorted based on the characteristics of the interior corner points of the checkerboard (not elaborated here); calculate the coordinates (x ij , y ij ) of each corner point in each image, where i represents the i-th image and j represents the j-th corner point in the i-th image, where i ∈ [1, n] and j ∈ [1, m].

[0087] Establish a three-dimensional coordinate system for each corner point, (X ij , Y ij , Z ij ), as shown in Figure 3-4 . By default, Z ij is 0. In the figure, (X ij , Y ij ) represents the relative two-dimensional coordinate system, and (X ij , Y ij , 0) is the established three-dimensional coordinate system.

[0088] Perform iterative convergence to obtain the internal parameter matrix K and the distortion coefficient Coef;

[0089]

[0090] Coef = [k1 k2 k3 k4].

[0091] Among them, f x and f y are the focal lengths in the horizontal and vertical directions respectively, and the unit is usually pixels; c x and c y are the principal point coordinates (optical center), usually the center point of the image.

[0092]

[0093] Set the initial internal parameter matrix K and the distortion coefficient Coef. The rotation matrix R and the translation vector T for each image are both 0.

[0094]

[0095] Coef = [k1 k2 k3 k4] = [0 0 0 0];

[0096] Where efl is the effective focal length, pixelsize is the pixel size of the sensor, and width and height are the width and height of the image respectively.

[0097] Set the preset reprojection error to infinity.

[0098] According to the established three-dimensional coordinate system and the current internal parameter matrix K, distortion coefficient Coef, rotation matrix R, and translation vector T, and complete the 3D to 2D projection according to the distortion model to obtain the two-dimensional coordinates And the two-dimensional coordinates (x ij , y ij ) of the corner points to obtain the average reprojection error of all the images

[0099] Where i ∈ [1, n], j ∈ [1, m];

[0100] Where is the 3D to 2D projection and obtains the two-dimensional coordinates.

[0101] Judge the convergence situation of the reprojection error, that is, compare RMS mean and to see if the convergence condition is met. If it is met, it means the iteration ends, otherwise continue to execute.

[0102]

[0103] Where RMS mean is the preset reprojection error.

[0104] After the first iteration, set the average reprojection error as the preset reprojection error for the next iteration.

[0105] Perform fitting of the distortion coefficient:

[0106] Calculate the undistorted radius of each corner point and the distorted radius

[0107]

[0108] Solve using the least squares method. Construct the coefficient matrix A and the result vector b according to the 5th-order polynomial distortion model, and use singular value decomposition to solve the linear equation system Ax = b. Here, you can call the function A.jacobiSvd(Eigen::ComputeThinU|Eigen::ComputeThinV).solve(b) to obtain the distortion coefficients k1, k2, k3, and k4.

[0109] Coefficient matrix A:

[0110]

[0111] Result vector b:

[0112]

[0113] Obtain

[0114] Optimize the intrinsic matrix: Use the least squares method, set the coefficient matrix Q and the right-side vector E, and use singular value decomposition to solve the linear equation system Qd = E;

[0115]

[0116] Among them

[0117]

[0118]

[0119] Update the intrinsic parameters, and obtain the rotation matrix R and the translation vector T for each image. Here, you can directly use the cv::solvePnP function in OpenCV to obtain them, and then perform iteration to calculate the reprojection error.

[0120] Example: Optical center detection and deviation analysis of an FF (fixed focal length) camera;

[0121] Equipment and parameters

[0122] Camera type: FF fixed focal length camera

[0123] Image resolution: 3264×2448

[0124] Equivalent focal length (EFL): 2.785mm

[0125] Pixel size (PixelSize): 1.12μm

[0126] Number of test samples: 3 cameras.

[0127] Implementation steps: Calibration environment preparation

[0128] In the laboratory environment, adjust the lighting conditions, use a light source board with stable color temperature and brightness, and avoid ambient light interference.

[0129] Place an SFR card (Spatial Frequency Response Chart) within the depth of field directly in front of the camera module, ensuring that the card is flat and free of deformation.

[0130] Set four different orientations (up, down, left, right) for the camera to shoot, ensuring sufficient view angle offset to calculate the optical center, as Figure 5 shown.

[0131] Image acquisition: Have each camera capture SFR card images (in bmp format) in four directions.

[0132] After collecting the data, use cv::goodFeaturesToTrack in OpenCV to detect corner points, and use cv::cornerSubPix for sub-pixel refinement to improve the corner point localization accuracy.

[0133] Optical center calculation: Based on the coordinates of the feature points of the SFR card, establish a three-dimensional coordinate model of the camera, and calculate the internal parameter matrix K and distortion coefficients through the least squares method and singular value decomposition (SVD).

[0134] Perform a 3D to 2D projection transformation, calculate the optical center coordinates (COD_x, COD_y), and conduct a deviation analysis with the physical center (1632, 1224). The optical center and its deviation from the physical center are shown in Table 1 below:

[0135] Number COD_x COD_y delta_x delta_y 1#FF 1623.750098 1226.438229 -8.249902264 2.43822865 2#FF 1623.307998 1220.481176 -8.692002172 -3.518824466 3#FF 1630.032178 1214.929282 -1.967821608 -9.070718219

[0136] Table 1

[0137] Observation results: The optical center of the first camera is slightly shifted 8.25 pixels to the left and 2.44 pixels up compared to the physical center. The optical center of the second camera is slightly shifted 8.69 pixels to the left and 3.52 pixels down. The optical center of the third camera is basically close to the physical center, but there are still deviations of -1.97 and -9.07.

[0138] Conclusion: Through this method, the offset of the optical center can be quantified, and then the installation of the camera module can be adjusted to improve the consistency of camera imaging.

[0139] Example: Optical center detection and deviation analysis of a periscope autofocus (AF) camera;

[0140] Equipment and parameters:

[0141] Camera Type: Periscope autofocus (AF) camera

[0142] Image Resolution: 4096×3072;

[0143] Equivalent Focal Length (EFL): 10.68mm;

[0144] Pixel Size: 1.28μm;

[0145] Number of Test Samples: 3 cameras;

[0146] Implementation Steps:

[0147] Test Environment Preparation:

[0148] Since periscope cameras usually have a longer focal length and a narrower depth of field, the shooting environment needs to be adjusted so that the SFR target card is within the optimal focus range.

[0149] Place a high-precision light source board in front of the camera to ensure uniform illumination and avoid reflections affecting corner detection.

[0150] Data Acquisition and Processing:

[0151] Let the camera capture SFR target card images in four directions (up, down, left, right) to ensure full coverage of the viewing angle change, as Figure 6 shown.

[0152] Extract the corner coordinates through OpenCV and perform sub-pixel optimization.

[0153] Establish a three-dimensional coordinate system and use the least squares method to iteratively optimize the internal parameter matrix and distortion coefficients.

[0154] Optical Center Calculation: The optical center obtained according to the above formula and the deviation from the physical center are shown in Table 2 below:

[0155] Number COD_x COD_y delta_x delta_y 1#Periscope 2028.397166 1513.724419 -19.60283432 -22.27558065 2#Periscope 2049.55569 1538.061845 1.555689833 2.061845388 3#Periscope 2035.933716 1535.767346 -12.06628441 -0.232653649

[0156] Table 2

[0157] Observation Results: For the first camera, the optical center is deviated 19.60 pixels to the left and 22.28 pixels downwards from the physical center, with a relatively large deviation; for the second camera, the optical center is basically aligned with the physical center, with only minor deviations of 1.56 and 2.06 pixels; for the third camera, the optical center is deviated 12.07 pixels to the left and 0.23 pixels downwards.

[0158] Conclusion: Due to the complex optical structure and long optical path of periscope cameras, the installation error has a greater impact. It is necessary to strictly control the installation accuracy of the module to reduce the deviation of the optical center; this method can accurately measure the optical center and quantify the deviation, providing data support for camera debugging and optimization.

[0159] This method is applicable to different types of cameras (FF / Periscope AF). By capturing SFR target card images in four directions and combining the least squares method and SVD optimization, the optical center coordinates can be calculated and the deviation can be quantified. The optical center deviation of the FF camera is relatively small, and the main impact is the module assembly error. The deviation of the periscope camera is relatively large and may be greatly affected by optical design and lens component assembly error, requiring more stringent assembly tolerance control. Through this technology, the optical performance of the camera can be improved, the module production process can be optimized, and the imaging consistency can be enhanced.

[0160] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they know the basic inventive concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0161] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A method for detecting the optical center of a camera, characterized in that, It includes the following steps: S1: Obtain SFR calibration card images at multiple positions; S2: Extract each corner point in the image, and calculate the coordinates of each corner point of each image according to the sorting; S3: Establish the three-dimensional coordinates of each corner point, and perform iterative convergence to obtain the internal parameter matrix and distortion coefficients; S31: Set the initial internal parameter adoption matrix and distortion coefficients, and set the rotation matrix and translation vector to 0; S32: Set the preset reprojection error to infinity; S33: According to the three-dimensional coordinates of the corner points and the current internal parameter matrix, distortion coefficients, rotation matrix and translation vector, complete the 3D to 2D projection according to the distortion model and obtain the two-dimensional coordinates, and obtain the average reprojection error of all images by comparing the two-dimensional coordinates with the corner point coordinates in S2; S34: Judge the convergence of the average reprojection error and the preset reprojection error. If the condition is met, the iteration ends; otherwise, proceed to the next step; S35: Take the average reprojection error as the preset reprojection error; S36: Fit the distortion coefficients; S37: Optimize the internal parameter matrix; S38: Update the internal parameters and obtain the image matrix and translation vector at each image position, and iterate to S33; S4: Calibrate the optical center according to the internal parameter matrix and distortion coefficients obtained by iteration.

2. The detection method of the optical center of a camera according to claim 1, characterized in that, In S1, it includes: Set up the shooting environment, place the SFR calibration card within the depth of field directly in front of the module, ensure its flatness and the clarity of the module shooting, and add a light source at the same time to ensure that the appropriate color temperature and illuminance are set through the light source board; Adjust the position parameters of the module at each position. At least four different positions are required.

3. The detection method of the optical center of a camera according to claim 1, characterized in that, In S2, it includes: detecting corner points using the cv::goodFeaturesToTrack function provided by opencv, and at the same time refining the corner points at the sub-pixel level through the cv::cornerSubPix function to improve the corner point positioning accuracy. Then, relying on the characteristics of the inner corner points of the checkerboard, the inner corner points are sorted, and the coordinates of each corner point in each image are calculated: (x ij , y ij ), where i represents the i-th image, j represents the j-th corner point in the i-th image, where i ∈ [1, n], j ∈ [1, m].

4. The detection method of the optical center of a camera according to claim 3, characterized in that In S3, it includes: the internal parameter matrix K and the distortion coefficients Coef; Coef = [k1 k2 k3 k4].

5. The detection method of the optical center of a camera according to claim 4, wherein In S31, it includes: Coef = [k1 k2 k3 k4] = [0 0 0 0]; Where efl is the effective focal length, pixelsize is the pixel size of the sensor, and width and height are the width and height of the image respectively.

6. The detection method of the optical center of a camera according to claim 5, characterized in that, In S33, it includes: where \(i\in[1,n]\) and \(j\in[1,m]\); Among them is a 3D to 2D projection and obtains two-dimensional coordinates.

7. The detection method of the optical center of a camera according to claim 6, characterized in that In S34, it includes: The conditions are as follows: Among them, RMS mean is the preset reprojection error.

8. The detection method of the optical center of a camera according to claim 7, wherein In S36, it includes: Calculate the undistorted radius of each corner point and the distorted radius Solve using the least squares method, construct the coefficient matrix A and the result vector b according to the 5th-order polynomial distortion model, and use singular value decomposition to solve the linear equation system Ax = b; Coefficient matrix A: Result vector b: Obtain 9. The detection method of the optical center of a camera according to claim 8, wherein, In S37, it includes: Use the least squares method, set the coefficient matrix Q and the right-side vector E, and use singular value decomposition to solve the linear equation system Qd = E; Among them

Images