A method for fitting camera internal parameters

Abstract

The invention belongs to the technical field of image measurement and discloses a method for fitting camera internal parameters, comprising the following steps: S1, setting an initial order t, a maximum order threshold T and a target mean square error threshold θ, and obtaining initial calibration data; S2, constructing coefficients to perform high-order mapping on image coordinates; S3, fitting a coefficient w=(w0, w1, …, w p ), S4, obtain the coefficient value by calculating the error; S5, calculate the mean square error, and determine whether it is less than the set target mean square error threshold θ, if it is already less than, record it as the optimal parameter table W; if it does not reach the set threshold θ, determine whether it reaches the maximum order threshold T, if it has reached, record the parameter table W; if it has not reached, add 1 to the order, and repeat steps S2 to S5. The present invention can further reduce the fitting error and improve the accuracy of the calculated camera internal parameters.

Description

Technical Field

[0001] The invention belongs to the technical field of image measurement, and in particular relates to a method for fitting internal parameters of a camera. Background Art

[0002] In the image measurement process and machine vision applications, the relationship between the 3D geometric position of a point on the surface of a spatial object and its corresponding point in the image is not determined, and a geometric model of imaging must be established. The accuracy of the calibration of the camera's internal parameters will directly affect the subsequent robot hand-eye calibration, so it is very important to do a good job of the camera's internal parameter calibration.

[0003] The existing camera calibration methods include: active vision camera calibration method, camera self-calibration method, and traditional camera calibration method. The camera calibration method based on active vision refers to calibrating the camera when some motion information of the camera is known. This method does not require a calibration object, but needs to control the camera to do some special movements. The camera internal parameters can be calculated using the particularity of this movement. The advantages of this method are simple algorithm, linear solution, and high robustness. The disadvantages are high system cost, high experimental requirements, and it is not suitable for scenes where the motion parameters are unknown or uncontrollable. The camera self-calibration method mainly uses some parallel or orthogonal information in the camera motion scene. The self-calibration method is more flexible, but because it is based on quadratic curves or surfaces, the robustness is poor. At present, the traditional camera calibration method is still mainly used in the industry. By using a calibration object with known size, establishing the correspondence between the points on the calibration object with known coordinates and its image points, a certain algorithm is used to obtain the internal and external parameters of the camera model. The calibration object is generally a three-dimensional calibration object or a flat calibration object. The three-dimensional calibration object can be calibrated by a single image, and the calibration accuracy is high, but the processing and maintenance of high-precision three-dimensional calibration objects are more difficult. The production of flat calibration objects is relatively simple, and the accuracy is easy to ensure, but multiple images must be used during calibration. Traditional camera calibration methods always require calibration objects during the calibration process, and the production accuracy of the calibration object will affect the calibration results. Therefore, a more adaptable and robust calibration algorithm is needed. Summary of the invention

[0004] In order to solve the above problems in the prior art, the present invention proposes a camera internal parameter fitting method to improve the calculation accuracy of the camera internal parameters.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a method for fitting internal parameters of a camera, comprising the following steps:

[0006] S1, set the initial order t, the maximum order threshold T and the target mean square error threshold θ, and obtain the initial calibration data;

[0007] S2, construction coefficient, high-order mapping of image coordinates;

[0008] S3, fitting a least square method with coefficients w=(w0, w1, ..., w p ), the model expression is:

[0009] f(w)=w0+w1x+w2y+w3xy+w4x 2 +w5y 2 +…+w p-1 x t +w p y t ;

[0010] S4. Obtain the coefficient value by calculating the error. The calculation formula is:

[0011]

[0012] Among them, the first part indicates that the error after fitting is minimized, and the second part indicates that the root mean square of the coefficient is minimized, that is, the penalty term is introduced; f(w) represents the predicted value, h(x, y) is the true value, α≥0, represents the complexity parameter of the control coefficient shrinkage, is the regularization parameter.

[0013] S5, calculate the mean square error, and determine whether it is less than the set target mean square error threshold θ. If it is less than, it is recorded as the optimal parameter table w, and the process goes to step S6; if it does not reach the set threshold θ, determine whether it reaches the maximum order threshold T. If it has reached, it is recorded as the optimal parameter table w, and the process goes to step S6; if it has not reached, add 1 to the order, and repeat steps S2 to S5;

[0014] S6. Save the optimal parameter table as the final internal reference calibration data.

[0015] In step S1, the initial calibration data includes the image coordinates (x, y), the distance y0 between each marking block on the calibration plate and the central largest marking block, and the distance z0 between the calibration plate and the camera.

[0016] The step S1 also includes a step of performing Z-Score standardization on the coordinate data, and the specific formula is:

[0017]

[0018] Among them, x i represents the x-coordinate of the ith point, y i Represents the y coordinate of the i-th point; Respectively represent the means of the corresponding coordinates; δ x and δ y Represent the standard deviation of the corresponding coordinates; x′i , y′ i Represents the standardized results.

[0019] The method for iso-densification processing of a two-dimensional laser point cloud also includes the steps of generating image data corresponding to a calibration table, performing high-order mapping using an optimal order t' image, and calculating the deviation between the mapping result and the true value.

[0020] When generating the calibration table corresponding to the image data, the x direction starts from pixel 1 to pixel 1920 with a step length of 1 pixel, and the y direction starts from pixel 0.5 to pixel 1080 with a step length of 0.5 pixels.

[0021] The present invention proposes a method for fitting camera internal parameters, which has the following beneficial effects compared with the prior art:

[0022] 1. Improved the traditional camera internal parameter fitting algorithm and reduced the accuracy requirements for calibration objects.

[0023] 2. By adding high-order terms to capture nonlinear changes in the data, the fitting error of the camera intrinsic parameters is reduced and the fitting accuracy is improved.

[0024] 3. The algorithm automatically selects a reasonable fitting order, which reduces the time for internal parameter calibration and improves efficiency.

[0025] In summary, the polynomial regression method based on machine learning is used to fit the camera internal parameters. By mapping the image coordinates to high powers to add high-order terms, the degree of freedom of the model is increased, which can be used to capture nonlinear changes in the data and solve the situation where the linear equation cannot fit the data well. When adding high-order terms, the complexity of the model is also increased, making it more robust, increasing the capacity of the model and the ability to fit data, which can further reduce the fitting error and improve the accuracy of the calculated camera internal parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic flow chart of a method for fitting camera internal parameters provided by an embodiment of the present invention;

[0027] Figure 2 Schematic diagram of absolute error of y0 training set in an embodiment of the present invention;

[0028] Figure 3 Schematic diagram of the absolute error of the y0 test set in an embodiment of the present invention;

[0029] Figure 4 Schematic diagram of y0 accuracy distribution in an embodiment of the present invention;

[0030] Figure 5Schematic diagram of absolute error of z0 training set in an embodiment of the present invention;

[0031] Figure 6 Schematic diagram of the absolute error of the z0 test set in an embodiment of the present invention;

[0032] Figure 7 Schematic diagram of z0 accuracy distribution in an embodiment of the present invention.

[0033] Specific embodiment

[0034] In order to make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in combination with specific embodiments and drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0035] like Figure 1 As shown, an embodiment of the present invention provides a method for fitting camera internal parameters, which includes the following steps:

[0036] S1. Obtain the initial calibration data, including the image coordinates x, y, the distance y0 between each marker block on the calibration plate and the center largest marker block, and the distance z0 between the calibration plate and the camera. At the same time, set the initial order t, the maximum order threshold T, and the target mean square error θ.

[0037] It also includes the step of Z-Score normalization of coordinate data x, y, the formula is as follows:

[0038]

[0039] Among them, x i represents the x-coordinate of the ith point, y i Represents the y coordinate of the i-th point; Respectively represent the means of the corresponding coordinates; δ x and δ y Represent the standard deviation of the corresponding coordinates; x′ i , y′ i Represents the result after coordinate standardization.

[0040] S2, construction coefficients, perform high-order mapping on image coordinates.

[0041] Perform a high-order mapping on the image coordinates and add high-order terms based on the coordinates x and y, as shown below:

[0042] z = [1, x, y, x 2 ,y 2 ,xy,x 3 ,y3 , x 2 y,xy 2 ......x t ,y t ......]; (2)

[0043] S3, fitting a least square method with coefficients w=(w0, w1, ..., w p ), the model expression is:

[0044] f(w)=w0+w1x+w2y+w3xy+w4x 2 +w5y 2 +…+w p-1 x t +w p y t ; (3)

[0045] S4, calculate the error and minimize the sum of squares of the residuals between the actual data and the predicted estimate.

[0046]

[0047] Ridge regression is used to solve the overfitting problem that may be caused by ordinary least squares by imposing a penalty term on the size of the coefficient. The specific formula is:

[0048]

[0049] Among them, the first part indicates that the error after fitting is minimized, and the second part indicates that the root mean square of the coefficient is minimized, that is, the penalty term is introduced; f(w) represents the predicted value, h(x, y) is the true value, α≥0, represents the complexity parameter of the control coefficient shrinkage, is the regularization parameter.

[0050] S5, calculate the mean square error, and determine whether it is less than the set target mean square error threshold θ. If it is less than, it is recorded as the optimal parameter table w, and the process goes to step S6; if it does not reach the set threshold θ, determine whether it reaches the maximum order threshold T. If it has reached, it is recorded as the optimal parameter table w, and the process goes to step S6; if it has not reached, add 1 to the order, and repeat steps S2 to S5;

[0051] S6. Save the obtained optimal parameter table as the final internal reference calibration data.

[0052] In addition, in this embodiment, the following steps may also be included:

[0053] S7, generate the image data corresponding to the calibration table, where the x direction starts from pixel 1 to pixel 1920, with a step length of 1 pixel. The y direction starts from pixel 0.5 to pixel 1080, with a step length of 0.5 pixels. Use the optimal order t' image for high-order mapping, and calculate the deviation between the mapping result and the true value. The calculated deviation can be used to evaluate the accuracy of the parameters obtained by the fitting method of the present application.

[0054] The parameter fitting method of the present invention is used to calculate the internal calibration data of a certain industrial camera of Dahua. After algorithm calculation, it is found that the optimal order of the camera is 10, and its y0 constant term is w0=99.15671277540206, and the coefficient is:

[0055] [w1, ..., w p<h2 style=";text-align:left;direction:ltr">]=[-6.02827607e-11,1.93884581e+01,1.82882363e-01,7.15224879e-0 2,-3.89402061e+00,-2.89167668e-02,-6.36062692e-03,-5.84488741e -02,7.68035392e-01,2.06705121e-03,-1.00232455e-02,1.69511812e- 02,-5.11105016e-03,-1.56341069e-01,-2.55879368e-03,-2.85788211e -02,6.38069921e-02,-1.44727161e-02,2.07093846e-02,3.00832662e-02,2.73742650e-03,2.15909086e-03,7.73282129e-04,-5.19410970e-03 ,8.88650527e-03,3.08850474e-03,-1.51080527e-03,-5.81127702e-04 ,6.95673150e-03,-3.40657684e-02,7.52878338e-03,-2.19041488e-02, -5.25429095e-03,-4.77218530e-03,-2.15749559e-04,-5.47907489e-04,1.40420519e-03,-1.47073575e-04,9.77564124e-03,-2.04683630e-03 ,2.35397978e-03,3.31722605e-05,-6.15778995e-05,-5.99698721e-04,3.96662349e-04,-3.71184052e-04,5.47928128e-03,-1.91133124e-03, 5.16677895e-03,5.04859176e-04,2.42469352e-03,9.41728221e-04,4.34486082e-04,1.43679895e-04,-1.09293684e-04,-4.00762369e-04,3.3 0532044e-05,-2.28903655e-03,4.88290317e-04,-1.43045003e-03,-1. 55044245e-04,-4.51053215e-04,-1.97272546e-04,-5.80007188e-05,8.64169565e-06, 1.29242990e-05].

[0056] The constant term z0 is w0=176.06482243080296, and the coefficient is:

[0057] [w1, ..., w p<h2 style=";text-align:left;direction:ltr">]=[8.30704329e-11,3.19733876e-01,5.06675621e+01,1.18524581e-01 ,2.90192604e-02,-1.01688562e+01,-1.03855518e+00,-9.92457537e-0 2,-1.44686333e-01,1.93935987e+00,-3.15780897e-01,3.78195548e-01,1.29736304e-01,2.88052617e-02,-2.81147035e-01,1.07323954e+00, -1.38520434e-01,4.31055310e-01,-2.20823756e-02,-1.35531257e-02,9.57904869e-02,4.00529105e-01,-5.36111023e-01,-1.44666687e-02, -8.99948492e-02,1.05201425e-01,-6.65184343e-03,-1.16393718e-01,-4.32828200e-01,6.93325860e-02,-3.14597865e-01,1.75330070e-02, -3.66263296e-02,-9.03976070e-02,1.58588661e-02,3.05420213e-02,-1.45847565e-01,2.66147877e-01,-2.30643087e-02,9.17340675e-02, -2.11632761e-02,6.79193878e-03,-2.43999904e-03,-1.71922750e-03,2.95051255e-02,5.94147546e-02,-1.03657266e-02,6.12195937e-02,- 1.07462911e-02,1.80090032e-02,2.51497768e-02,-4.30551139e-03,1.58733503e-02,-1.85074392e-03,-1.85170048e-02,1.64340478e-02,-4 .18787630e-02,8.65163360e-03,-2.25105958e-02,1.10009712e-03,-3.90877373e-03,-6.09517675e-03,1.49672034e-03,-2.99607281e-03,3.51091474e-04, 2.92118503e-03].

[0058] like Figures 2 to 7 The experimental data of this embodiment is shown in FIG. Figure 2 It represents the error between y′0 and y0 in the training set, which mainly shows the fitting ability of the method of the present invention. y′0 is the calculated value of y0 obtained by substituting the camera parameters fitted by the method of the present invention into the image coordinates in the training set. y0 represents the distance between each marker block and the central largest marker block on the calibration plate in the training set. Figure 3 It represents the error between y′0 and y0 in the test set, which mainly shows the generalization ability of the present invention. y′0 is the calculated value of y0 obtained by substituting the camera parameters fitted by the method of the present invention into the image coordinates of the test set. y0 represents the distance between each marker block and the central largest marker block on the calibration plate in the test set. Figure 4 It represents the accuracy distribution range of the y0 parameter. Figure 5 It represents the error between z′0 and z0 in the training set, which mainly shows the fitting ability of the method of the present invention. z′0 is the calculated value of z0 obtained by substituting the camera parameters fitted by the method of the present invention into the image coordinates in the training set. z0 represents the distance between the calibration plate and the camera in the training set. Figure 6 It represents the error between z′0 and z0 in the test set, which mainly shows the generalization ability of the present invention. z′0 is the calculated value of z0 obtained by substituting the camera parameters fitted by the method of the present invention into the image coordinates of the test set. z0 represents the distance between the calibration plate and the camera in the training set. Figure 7 It represents the accuracy distribution range of the z0 parameter.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fitting camera internal parameters, characterized in that: It includes the following steps: S1, set the initial order t, the maximum order threshold T and the target mean square error threshold θ, and obtain the initial calibration data; S2, construction coefficient, high-order mapping of image coordinates; S3, use the least squares method to fit a coefficient w=(w0,w1,…,w p ), the model expression is: f(w)=w0+w1x+w2y+w3xy+w4x 2 +w5y 2 +…+w p-1 x t +w p y t ; w0, w1, ... w p They represent the coefficients corresponding to the 1st, 2nd, ...p+1th items of the model respectively; x, y represent the image coordinates; S4. Obtain the coefficient value by calculating the error. The calculation formula is: min(||f(w)-h(x,y)|| 2 +a||w|| 2 ); Among them, f(w) represents the predicted value, h(x,y) is the true value, α≥0, represents the complexity parameter of the control coefficient shrinkage, ||w|| 2 is the regularization parameter; S5, calculate the mean square error, and determine whether it is less than the set target mean square error threshold θ. If it is less than, it is recorded as the optimal parameter table w, and the process goes to step S6; if it does not reach the set threshold θ, determine whether it reaches the maximum order threshold T. If it has reached, it is recorded as the optimal parameter table w, and the process goes to step S6; if it has not reached, add 1 to the order, and repeat steps S2 to S5; S6. Save the optimal parameter table as the final internal reference calibration data.

2. The camera internal parameter fitting method according to claim 1, characterized in that: In step S1, the initial calibration data includes the image coordinates (x, y), the distance y0 between each marking block on the calibration plate and the central largest marking block, and the distance z0 between the calibration plate and the camera.

3. The camera internal parameter fitting method according to claim 2, characterized in that: The step S1 also includes a step of performing Z-Score standardization on the coordinate data, and the specific formula is: Among them, x i represents the x-coordinate of the ith point, y i Represents the y coordinate of the i-th point; Represent the mean of the corresponding coordinates; δ x and δ y Represent the standard deviation of the corresponding coordinates; x′ i ,y′ i Represents the standardized results.

4. The method for fitting camera internal parameters according to claim 1, characterized in that: The method also includes the steps of generating image data corresponding to a calibration table, performing high-order mapping using an optimal order t' image, and calculating a deviation between a mapping result and a true value.

5. The method for fitting camera internal parameters according to claim 4, characterized in that: When the calibration table is generated corresponding to the image data, the x direction starts from pixel 1 to pixel 1920 with a step length of 1 pixel, and the y direction starts from pixel 0.5 to pixel 1080 with a step length of 0.5 pixels.

Images