Camera calibration method for low-precision plane target

Abstract

The invention discloses a camera calibration method for a low-precision plane target, and relates to the field of camera parameter calibration, and the method comprises the following steps: carrying out the multi-view shooting through a camera, and collecting data; checkerboard angular points are detected, and pixel coordinates of each angular point are determined; establishing a corresponding relation between the coordinates of the angular points and the coordinates of the actual world, adding a three-dimensional offset factor on the angular points of the target, and solving the parameters of the camera when the sum of the offset between the feature point pairs on the ideal target and the actual target is zero; global constraint is applied to the offset factor, a cost function is rewritten, and optimization is carried out through a nonlinear optimization algorithm to minimize a re-projection error. Under the condition that a low-precision target is used, high-precision camera calibration is realized by correcting feature points on the target. Simulation and real experiment results show that under the condition that a low-precision target is used, the method can still obtain a high-precision calibration result and measurement precision which are comparable with those of a traditional calibration method adopting a high-precision target.

Description

Technical Field

[0001] The invention relates to a camera calibration method for low-precision planar targets. Background Art

[0002] 3D reconstruction technology has been widely used in many fields due to its unique advantages of non-contact measurement and fast data acquisition. This technology can efficiently and accurately capture the three-dimensional information of objects from two-dimensional images, which has greatly promoted the technological progress and application expansion in these fields. At present, camera calibration, as the core step in establishing the relationship between two-dimensional images and three-dimensional space, is crucial to ensure the accuracy and reliability of three-dimensional reconstruction technology. The process of camera calibration involves determining the internal and external parameters of the camera, so that points in the two-dimensional image can be accurately mapped to corresponding points in three-dimensional space. At present, there are several commonly used camera calibration methods.

[0003] 1. Calibration method based on calibration objects. This method uses multiple plane grid templates in different positions and directions to determine the internal parameters of the camera, and then calculates its external parameters to calibrate the camera. However, in the process of estimating linear internal and external parameters, this method assumes that the straight lines on the template image remain straight after perspective projection. This assumption is used to obtain sub-pixel point coordinates in image processing, but it also introduces errors. The calibration accuracy of the camera calibration method based on calibration objects has high requirements on the calibration objects, and some work environments are not suitable for their placement, which also limits the scope of application.

[0004] 2. Calibration method based on active vision. This method uses an active system to control the camera to make specific movements, capture multiple sets of images, and solve the internal and external parameters of the camera based on image information and known displacement changes. The calibration method based on active vision can simplify the calculation process and obtain linear results. The disadvantages of this method are that it requires a precise control platform, the system cost is high, and it is not applicable to some occasions where the motion parameters are unknown or cannot be controlled.

[0005] 3. Camera self-calibration method. This method does not need to know the three-dimensional coordinates of the image points. It determines the camera calibration parameters by calculating the relative relationship of a certain point in the scene graph at different shooting angles. This feature of the self-calibration method enables it to calibrate some unknown camera parameters. However, this method solves the parameters through the constraint relationship of the parameters inside the camera, completely ignoring the environment outside the camera system, so this calibration method is more flexible than other methods, but the stability is not high.

[0006] The key factor for 3D reconstruction accuracy is the accuracy of camera calibration. With the development of technology and the growth of application demand, camera calibration technology is also constantly improving. However, due to the limitation of processing accuracy, the target is not an ideal plane. In practical applications, even if it is a 2D target, when using a point set as an ideal plane for input, the inherent coordinate error of the target will inevitably affect other optimization parameters, thereby affecting the accuracy of camera calibration. Therefore, under limited budget, exploring more efficient and convenient algorithms is crucial to achieve high-precision camera calibration. Summary of the invention

[0007] In order to solve the above technical problems, the present invention proposes a camera calibration method combined with additional target geometry optimization. The method can achieve high-precision camera calibration by correcting feature points on the target in the case of non-ideal (low-precision) targets.

[0008] The technical solution of the present invention to solve the above problem is: a camera calibration method for low-precision planar targets, comprising the following steps:

[0009] Step 1: Use the camera to shoot from multiple angles and collect data;

[0010] Step 2: Detect feature points on the collected photos;

[0011] Step 3: Use the correspondence between the detected corner point coordinates and the actual world coordinates to establish a model, use the least squares method to optimize the camera's internal and external parameters, and on this basis add a three-dimensional offset factor to the corner points on the target to ensure that the sum of the offsets between the feature point pairs on the ideal target and the actual target is zero;

[0012] Step 4: Apply global constraints on the offset factor, rewrite the cost function, and optimize it through a nonlinear parameter optimization algorithm to minimize the reprojection error;

[0013] The specific process of the above-mentioned camera calibration method for low-precision planar targets is as follows:

[0014] 1-1) Photograph the target from different angles (looking down, looking up, and looking from the side) to ensure that the target pattern is evenly distributed across the entire field of view of the camera.

[0015] 1-2) During the calibration of a monocular camera, the target is placed in the field of view any number of times. Consider that in the jth pose, the world coordinate system and the camera coordinate system O C -X C Y C Z C The rigid transformation between them can be done with a rotation matrix to represent and a translation vector To indicate that P W =(x w ,y w ,z w ) is the feature point on the target in the world coordinate system. The relationship between the feature point transformation from the world coordinate system to the camera coordinate system can be expressed as:

[0016]

[0017] Where P c is the position of the point in the camera coordinate system, It is the rotation matrix from the world coordinate system to the camera coordinate system, which is used to describe the directional relationship between the two coordinate systems. is the translation vector from the world coordinate system to the camera coordinate system, describing the positional relationship between the origins of the two coordinate systems. w is the position of the point in the world coordinate system.

[0018] The specific process of the above-mentioned camera calibration method for low-precision planar targets is as follows:

[0019] 2-1) Image preprocessing: The image is usually converted to grayscale first, because corner detection does not require color information and grayscale image processing is faster.

[0020] 2-2) Edge Detection: Use edge detection techniques to find edge information in an image. Edge detection can help highlight lines and corners in an image.

[0021] 2-3) Corner point positioning: The intersection points of the chessboard lines are identified through edge information. These intersection points are the inner corners of the chessboard.

[0022] The specific process of step 3 of the above-mentioned camera calibration method for low-precision planar targets is as follows:

[0023] 3-1) Add a 3D offset factor to each corner point on the target and modify the corner point coordinates as follows:

[0024]

[0025] where e x , e y and e z Indicates the offset factor in three directions.

[0026] 3-2) Constraints are imposed on the offset factor, using the fact that for any surface of arbitrary shape, we can always find an ideal target whose position satisfies the condition that the sum of the offsets between the feature point pairs on the ideal target and the actual target is zero. It can be summarized as a mathematical formula as follows:

[0027]

[0028] The specific process of the above-mentioned camera calibration method for low-precision planar targets is as follows:

[0029] 4-1) In the case of single-objective localization, the cost function will be modified to add a three-dimensional offset factor to each corner point and impose a global constraint on the offset factor, thereby constraining the scale factor in the nonlinear optimization algorithm, namely:

[0030]

[0031] 4-2) In the case of stereo calibration, the additional cost function is modified as:

[0032]

[0033] It should be noted that the optimized world coordinate system O' W -X' W Y' W Z' W and the original world coordinate system O W -X W Y W Z W There is a difference. Using the rotation matrix R t and the translation vector T t Represents the rigid body transformation between the optimized world coordinate system and the original world coordinate system.

[0034] The beneficial effects of the present invention are:

[0035] 1. Traditional camera calibration methods usually require the use of a calibration plate with a known geometric shape, such as a chessboard, to achieve accurate camera calibration. However, this method achieves accurate camera calibration without the need for additional measurements of the target. This is because the method relies on feature points on the target, rather than the specific geometry of the target. By detecting the feature points on the target and using their positional relationship, the camera can be accurately calibrated, thereby achieving precise control of the camera.

[0036] 2. Compared with traditional methods, this method has more explanatory constraints, which makes the calibration process more reliable. These constraints may involve the positional relationship of feature points on the target, the constraints of the camera's internal parameters, etc. By introducing these constraints, the calibration process becomes more stable and can better handle various noises and errors.

[0037] In addition, the reprojection error and measurement error of this method are better than those of traditional methods. The reprojection error refers to the error when the points in the world coordinate system are reprojected onto the image plane using the calibrated camera parameters, while the measurement error refers to the error between the image points actually observed and the image points calculated using the calibrated parameters. By optimizing these errors, this method can obtain more accurate camera calibration results.

[0038] 3. Compared with the existing target geometry optimization calibration method, a significant advantage of this method is that it has fewer iterations and faster convergence. Traditional target geometry optimization methods usually require multiple iterations to optimize camera parameters and target geometry, which may take a long time to converge. This method reduces the number of iterations by introducing more explanatory constraints, thereby speeding up the convergence of the calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a camera calibration flow chart of the present invention.

[0040] Figure 2 Schematic diagram of the binocular camera model of the present invention.

[0041] Figure 3 This is a flow chart of the present invention for imposing constraints on detected corner points.

[0042] Figure 4 There are targets with different degrees of deviation in the present invention.

[0043] Figure 5 Flow chart of parameter optimization for the present invention.

[0044] Figure 6 Schematic diagram of a binocular camera for implementing the present invention.

[0045] Figure 7 Schematic diagram of two types of targets used in the present invention. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0047] The main optical structure of the camera calibration method for low-precision planar targets of the present invention is as follows: Figure 6 As shown, its structure consists of a binocular camera and targets of different precisions.

[0048] In the simulation experiment, a dual-target positioning process was simulated to verify the practicability of the proposed method. In order to verify the adaptability of the algorithm, three different types of targets were produced. Figure 5As shown in the figure, by gradually increasing the control parameters of these targets, the deformation degree of the targets is increased accordingly. Under each parameter, 20 sets of data are generated by randomly changing the position of the targets in the field of view. During the simulation process, the corners of the chessboard are directly obtained through numerical calculation, avoiding corner extraction errors.

[0049] In the actual experiment, Figure 6 As shown in the figure, the binocular camera uses two industrial cameras with a resolution of 1920×1080 pixels and two lenses with a focal length of 16 mm. Figure 7 As shown, the target uses a 12×9 grid and a chessboard with a spacing of 10mm. Two different materials are selected to make two targets respectively. The first target is a precision calibration plate using a float glass substrate with a calibration accuracy of 10μm. The second target is printed and glued to an acrylic plate. During the bonding process, deformation of the paper-printed calibration plate is inevitable. The traditional calibration method and the improved optimization algorithm of the present invention are used to conduct comparative tests on high-precision calibration plates and paper calibration plates respectively. The method proposed in the present invention has a smaller reprojection error due to its more appropriate constraints. And without increasing additional workload, the interference of undesirable targets on calibration accuracy is reduced.

[0050] like Figure 1 As shown, a camera calibration method for a low-precision planar target includes the following steps:

[0051] Step 1: Use the camera to shoot from multiple angles and collect data;

[0052] The specific process of step one is:

[0053] During the imaging process, the random placement of the targets was designed. The targets were randomly placed to cover the entire measurement area, thereby achieving full coverage of the imaging area. Specifically, the targets were randomly placed 20 times within the camera field of view to ensure that as many target images as possible at different positions and angles were captured.

[0054] Step 2: Detect feature points on the collected photos;

[0055] The specific steps are:

[0056] 2-1) Image preprocessing: convert the color image into a grayscale image. Use the imported binocular camera to shoot, determine the number of channels of the image, and convert the image into grayscale. gray , the specific judgment is as follows:

[0057]

[0058] 2-2) Use the Sobel operator to perform edge detection on the grayscale image, two matrices, G x =AX P, G y =A y P. The P matrix is ​​a 3*3 matrix of the image, and each pixel of the image matrix is ​​multiplied by A. X and A y Each pixel of A X is the row edge detection module, G x What it means is that the right row of the P matrix is ​​subtracted from the left row to get a threshold, G y The same is true. x and G y After that, it is equivalent to obtaining the row edge threshold and the column edge threshold, and calculating the sum of the squares G of the two. If G is greater than the set threshold, it means that this is an edge, otherwise it is not.

[0059] 2-3) Based on the edge detection results, the corner points are located by using a corner detector. Corner points are obvious points where two edges intersect in the image, usually corresponding to the inner corners of the chessboard. Using the edge detection results, the corner points are accurately identified by finding the concentrated areas where the edges intersect.

[0060] Step 3: Use the correspondence between the detected corner point coordinates and the actual world coordinates to establish a model, use the least squares method to optimize the camera's internal and external parameters, and on this basis add a three-dimensional offset factor to the corner points on the target to ensure that the sum of the offsets between the feature point pairs on the ideal target and the actual target is zero;

[0061] The specific steps are:

[0062] 3-1) Add a 3D offset factor to each corner point. The coordinates P of each corner point on the target W =(x W ,y W ,z W ) is adjusted by adding a three-dimensional offset vector e. Here e x 、e y 、e z Represents the offset on the coordinate axis x, y, and z, respectively, to describe the degree to which the corner point deviates from its ideal position. Get the new corner point The formula is as follows:

[0063]

[0064] 3-2) Add constraints to the new corner points, expressed as follows:

[0065]

[0066] The formula represents three independent constraints, each of which requires that the sum of the offset factors of all feature points in the corresponding coordinate direction must be zero. Specifically:

[0067] Indicates that the sum of the offsets of all feature points in the x direction is zero.

[0068] Indicates that the sum of the offsets of all feature points in the y direction is zero.

[0069] Indicates that the sum of the offsets of all feature points in the z direction is zero.

[0070] 3-3) For the image coordinates and world coordinates of each corner point on the target, establish a set of equations and use the direct least squares algorithm to obtain the homography matrix of each view.

[0071] Step 4: Apply global constraints on the offset factor, rewrite the cost function, and optimize it through a nonlinear parameter optimization algorithm to minimize the reprojection error;

[0072] The specific steps are:

[0073] 4-1) A global constraint is imposed on the set offset factor, thereby constraining the scale factor in the nonlinear optimization algorithm, namely:

[0074]

[0075] 4-2) Use nonlinear least squares method to calculate the internal parameter K and external parameter R W , T W And the offset factor e of each corner point is optimized.

[0076] In each iteration, the predicted position of each corner point on the image is calculated using the camera model function f according to the current parameter estimate.

[0077] Calculate the difference between the predicted position and the actual observed position, i.e. the reprojection error. Adjust the internal and external parameters and the offset factor to minimize the overall reprojection error.

[0078] 4-3) For each corner point of each image, calculate its reprojection error and sum them up.

[0079] 4-4) Perform the above optimization until convergence, that is, when further iterations no longer significantly reduce the total reprojection error or the maximum number of iterations is reached. After the optimization is completed, K, R W , T W , and the offset factor e x , e y , e z .

Claims

1. A camera calibration method for low-precision planar targets, characterized in that: The following steps are involved: Step 1: Use the camera to shoot from multiple angles and collect data; Step 2: Detect feature points on the collected photos; Step 3: Use the correspondence between the detected corner point coordinates and the actual world coordinates to establish a model, use the least squares method to optimize the camera's internal and external parameters, and on this basis add a three-dimensional offset factor to the corner points on the target to ensure that the sum of the offsets between the feature point pairs on the ideal target and the actual target is zero; Step 4: Apply global constraints on the offset factor, rewrite the cost function, and optimize it through a nonlinear parameter optimization algorithm to minimize the reprojection error.

2. The camera calibration method for low-precision planar targets according to claim 1, characterized in that: The specific process of step 2 is: 2-1) Image preprocessing: converting color images into grayscale images. Import the images taken by the binocular camera, determine the number of channels of the image, and convert the image into grayscale. gray , the specific judgment is as follows: 2-2) Perform edge detection on the grayscale image using the Sobel operator. Two matrices, G x =A X P, G y =A y P. The P matrix is ​​a 3*3 matrix of the image, and each pixel of the image matrix is ​​multiplied by A. X and A y Each pixel of A X is the row edge detection module, G x What it means is that the right row of the P matrix is ​​subtracted from the left row to get a threshold, G y The same is true. x and G y After that, it is equivalent to obtaining the row edge threshold and the column edge threshold, and calculating the sum of the squares G of the two. If G is greater than the set threshold, it means that this is an edge, otherwise it is not. 2-3) Based on the edge detection results, the corner points are located by using a corner detector. Corner points are obvious points where two edges intersect in the image, usually corresponding to the inner corners of the chessboard. Using the edge detection results, the corner points are accurately identified by finding the concentrated areas where the edges intersect.

3. The camera calibration method for low-precision planar targets according to claim 2, characterized in that: The specific process of step three is: 3-1) Add a 3D offset factor to each corner point. The coordinates P of each corner point on the target W =(x W ,y W ,z W ) is adjusted by adding a three-dimensional offset vector e. Here e x 、e y 、e z Represents the offset on the coordinate axis x, y, and z, respectively, to describe the degree to which the corner point deviates from its ideal position. Get the new corner point The formula is as follows: 3-2) Add constraints to the new corner points, expressed as follows: The formula represents three independent constraints, each of which requires that the sum of the offset factors of all feature points in the corresponding coordinate direction must be zero. Specifically: Indicates that the sum of the offsets of all feature points in the x direction is zero. Indicates that the sum of the offsets of all feature points in the y direction is zero. Indicates that the sum of the offsets of all feature points in the z direction is zero. 3-3) For the image coordinates and world coordinates of each corner point on the target, establish a set of equations and use the least squares algorithm to obtain the homography matrix of each view.

4. The camera calibration method for low-precision planar targets according to claim 3, characterized in that: The specific process of step 4 is as follows: 4-1) A global constraint is imposed on the set offset factor, thereby constraining the scale factor in the nonlinear optimization algorithm, namely: 4-2) Use nonlinear least squares method to calculate the internal parameter K and external parameter R W , T W And the offset factor e of each corner point is optimized. In each iteration, the predicted position of each corner point on the image is calculated using the camera model function f according to the current parameter estimate. Calculate the difference between the predicted position and the actual observed position, i.e. the reprojection error. Adjust the internal and external parameters and the offset factor to minimize the overall reprojection error. 4-3) For each corner point of each image, calculate its reprojection error and sum them up. 4-4) Perform the above optimization until convergence, that is, when further iterations no longer significantly reduce the total reprojection error or the maximum number of iterations is reached. After the optimization is completed, K, R W , T W , and the offset factor e x , e y , e z .

Images