A camera calibration optimization method based on sample space

Abstract

The present invention relates to a camera calibration optimization method based on sample space, and relates to the technical field of camera parameter calibration, comprising the following steps: collecting calibration pictures to form a sample space; global sample optimization to achieve sample screening; local parameter fitting to improve calibration accuracy; obtaining calibration results; and result verification. The advantages are: the camera calibration optimization method based on sample space provided by the present invention solves the problem that the accuracy of camera calibration parameters is greatly affected by calibration pictures, can select samples that are beneficial to calibration, and obtains a higher calibration accuracy than general calibration algorithms.

Description

Technical Field

[0001] The present invention relates to the technical field of camera parameter calibration, and more specifically, to a camera calibration optimization method based on sample space. Background Art

[0002] Camera calibration refers to establishing the relationship between the camera image pixel position and the scene point position. According to the camera imaging model, the parameters of the camera model are solved by the correspondence between the coordinates of the feature points in the image and the world coordinates.

[0003] Zhang Zhengyou's camera calibration algorithm is a single-plane chessboard camera calibration method proposed by Professor Zhang Zhengyou in 1998. The calibration plate of the traditional calibration method needs to be three-dimensional and very precise, which is difficult to make. The method proposed by Professor Zhang Zhengyou is between the traditional calibration method and the self-calibration method, but it overcomes the disadvantage of the high-precision calibration object required by the traditional calibration method, and only a printed chessboard is needed. At the same time, compared with self-calibration, it also improves accuracy and is easy to operate. Therefore, Zhang's calibration method is widely used in computer vision.

[0004] The current camera calibration algorithm can already estimate camera parameters well, but the accuracy of camera calibration parameters is greatly affected by the calibration images. It is difficult to select samples that are beneficial to calibration, and the obtained calibration accuracy is low.

[0005] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0006] The purpose of the present invention is to provide a camera calibration optimization method based on sample space, which solves the problem that the accuracy of camera calibration parameters is greatly affected by the calibration pictures, can select samples that are beneficial to calibration, and obtain higher calibration accuracy than general calibration algorithms.

[0007] The present invention provides a camera calibration optimization method based on sample space, comprising the following steps:

[0008] S1: Collect calibration images to form a sample space;

[0009] S2: Global sample optimization to achieve sample screening;

[0010] S3: local parameter fitting;

[0011] S4: Get the calibration result;

[0012] S5: Result verification.

[0013] Furthermore, the step S1 comprises: constructing a constrained discrete nonlinear equation between the sample and the reprojection error in the sample space, wherein the discrete nonlinear equation is:

[0014] y error =f(x1,x2)

[0015]

[0016] The number of calibration images obtained is N, the image x1 required for one calibration is x1, the combination of x1 calibration images selected from the total sample N is recorded as x2, and the calibration reprojection error y is established in the space [x1x2] error The relationship with the sample; C expresses the number of combinations, and the function value of f is given by Zhang's calibration algorithm.

[0017] Furthermore, the purpose of the discrete nonlinear equation optimization is to find the minimum value of the discrete equation. The smaller the extreme point is, the smaller the reprojection error is, and the better the calibration result is.

[0018] Furthermore, step S2 includes: optimizing the samples using a general optimization algorithm with probabilistic global optimization performance, further improving the calibration accuracy at the extreme points of the sample space, selecting batch samples near the extreme points and obtaining a parameter set for camera calibration.

[0019] Furthermore, the general optimization algorithm with probabilistic global optimization performance is a simulated annealing algorithm.

[0020] Furthermore, the step S3 includes: performing clustering and fitting on the camera calibration parameter set to obtain parameter regions and parameter values ​​respectively.

[0021] Furthermore, the step S3 further includes: obtaining local samples, wherein the local sample obtaining includes obtaining the camera internal parameters in miny error To fit, a certain number of samples need to be obtained, and the samples are selected according to the following formula:

[0022] {(x1x2) / f(x1x2)≤miny error +Δy}

[0023] The samples E selected by the above formula all have small reprojection errors.

[0024] Furthermore, the step S3 also includes: in the calibration result set e={e1 e2 ... e9} in the sample E, e includes four intrinsic parameters of the camera, two radial distortion parameters, two tangential distortion parameters and a reprojection error, the frequency of the eight parameters in the sample space is counted, and the area with the highest density is obtained as the area where the camera parameters are located through the clustering DBSCAN clustering algorithm, and the eight areas are counted to form eight camera parameter distribution maps, each of which represents the distribution of a camera parameter in the parameter space, the abscissa in the figure represents the size of the parameter value, and the ordinate represents the frequency of occurrence of the parameter value; in the camera parameter distribution map, a certain function distribution is satisfied and there is a peak value; the point at the peak of the distribution is the true value point of the camera parameter.

[0025] Furthermore, the step S3 further includes: fitting the distribution function of the parameter again by a fourth-order polynomial, and for the fitted function, obtaining the parameter value at the frequency maximum value based on the accurate value of the camera parameter.

[0026] Furthermore, the step S5 comprises:

[0027] Randomly select points in the entire sample space to obtain batch calibration results, and arrange the parameters of the calibration results from small to large according to the reprojection error; select the calibration parameter of the sample with the smallest reprojection error in this sample set, and regard this parameter as the approximate accurate value of the camera parameter e real ; Then divide the samples into n sample sets according to the reprojection error from small to large, E=[E1 E2 … E n ], in which each sample in the latter sample set has a larger reprojection error value than each sample in the former sample set, E i This reflects the quality of the sample;

[0028] Calculate the sample calibration result with the minimum reprojection error in each sub-sample in the sample set E and e real The mean square error err1 = [r 11 r 12 … r 1n ]; and the result of the cluster fitting algorithm err2 = [r 21 r 22 … r 2n ]; err1 and err2 respectively reflect the accuracy of the minimum reprojection error sample of each group of samples and the accuracy of the DBSCAN clustering algorithm

[0029] Furthermore, the DBSCAN clustering is to determine the sample distribution area, select the area with the largest density from multiple areas, and the area with the largest density has the largest confidence; then perform parameter fitting, and the fitting result is consistent with e real The variance is err2.

[0030] The camera calibration optimization method based on sample space provided by the present invention solves the problem that the accuracy of camera calibration parameters is greatly affected by the calibration pictures, can select samples that are beneficial to calibration, and obtain a higher calibration accuracy than general calibration algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic flow chart of a camera calibration optimization method based on sample space provided in an embodiment of the present invention.

[0032] Figure 2 (a)- Figure 2 (f) is a schematic diagram of camera parameter distribution.

[0033] Figure 3 (a)- Figure 3 (f) is a schematic diagram of camera parameter fitting.

[0034] Figure 4 A schematic diagram for comparing algorithm results provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0036] The terms "first", "second", "third", "fourth" and the like in the description and claims of the present invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence.

[0037] Example 1

[0038] Figure 1 A schematic diagram of a flow chart of a camera calibration optimization method based on sample space provided by an embodiment of the present invention. Figure 1 The camera calibration optimization method based on sample space provided by the embodiment of the present invention comprises the following steps:

[0039] S1: Collect calibration images to form a sample space;

[0040] S3: Global sample optimization to achieve sample screening;

[0041] S4: local parameter fitting;

[0042] S5: Get the calibration result;

[0043] S6: Result verification.

[0044] It should be noted that the camera calibration optimization method based on sample space provided by the embodiment of the present invention realizes optimization in the entire sample space. First, a constrained discrete nonlinear equation between the sample and the reprojection error is constructed in the sample space, and the entire sample space is optimized based on this relationship; secondly, based on the traditional calibration method, considering that the cost of traversing the sample space is very high, a general optimization algorithm with probabilistic global optimization performance is used to optimize the sample, and this process is based on the optimization of the reprojection error; finally, the calibration accuracy is further improved at the pole (or the extreme point) of the sample space, and a batch of samples are selected near the pole and a parameter set for camera calibration is obtained. Clustering and fitting are performed on the parameter set to obtain parameter areas and parameter values, respectively. This process is based on the optimization of camera parameters. The algorithm can not only obtain excellent calibration samples, but also obtain camera calibration parameters with higher accuracy. It solves the problem that the accuracy of camera calibration parameters is greatly affected by calibration images, can select samples that are beneficial to calibration, and obtains higher calibration accuracy than general calibration algorithms.

[0045] Furthermore, during the camera calibration process, a certain number of calibration images need to be obtained. Due to the influence of the shooting scene and camera workmanship, the calibration images have a great influence on the final calibration results. In order to reduce the impact of the calibration samples on the calibration results due to uncertain factors in the shooting, it is necessary to optimize the samples in the sample space. First, it is necessary to establish the relationship between the sample and the reprojection error; construct a constrained discrete nonlinear equation between the sample and the reprojection error in the sample space, and optimize the entire sample space based on this equation relationship; the specific discrete nonlinear equation is:

[0046] y error =f(x1,x2)

[0047]

[0048] The number of calibration images is N, the number of images required for one calibration is x1 (the number of samples calibrated using Zhang’s calibration algorithm), the combination of x1 calibration images is selected from the total sample N and recorded as x2, and the calibration reprojection error y is established in the space [x1x2] error The relationship between C and samples; C expresses the number of combinations, and the function value of f is given by Zhang's calibration algorithm. The purpose of discrete nonlinear equation optimization is to find the minimum value of the discrete equation. The smaller the extreme point, the smaller the reprojection error, and the better the calibration result.

[0049] Furthermore, due to the large sample space, when the number of calibration shots is 30, the total sample space size is about 10 10At this time, it is necessary to use a general optimization algorithm with probabilistic global optimization performance to optimize the samples, further improve the calibration accuracy at the extreme point of the sample space, select batch samples near the extreme point and obtain the parameter set of camera calibration.

[0050] Specifically, the general optimization algorithm with probabilistic global optimization performance in the present invention is a simulated annealing algorithm, which is used to optimize the samples. This includes the selection of initial values ​​(although the simulated annealing algorithm has global optimization performance, it is still possible to fall into a local optimal solution. Here, a fixed number of random vectors are established, and each group of numbers in the vector is used as an optimization initial value. More accurate results are obtained through the optimization of multiple initial points). It also includes the iteration step size and direction according to the idea of ​​the simulated annealing algorithm.

[0051] Step S3 of the present invention comprises:

[0052] Step S31: clustering and fitting are performed on the camera calibration parameter set to obtain parameter regions and parameter values ​​respectively.

[0053] It should be noted that the calibration accuracy in camera calibration is not only affected by the number of images, but also by the accuracy of the calibration plate. Although the reprojection error reflects the quality of the camera calibration parameters, in order to obtain more accurate calibration parameters, the camera parameters are optimized at the local sample optimal point based on sample optimization. Specifically, firstly, a batch of samples are obtained in the neighborhood of the local optimal sample, and then the calibration parameters corresponding to the sample are clustered to obtain the actual camera parameter distribution area. Finally, the camera parameter distribution is analyzed, and the camera parameter values ​​are fitted and obtained.

[0054] Step S3 of the present invention also includes:

[0055] Step S32: local sample acquisition, the local sample acquisition includes the camera internal parameters in miny error To fit, a certain number of samples need to be obtained, and the samples are selected according to the following formula:

[0056] {(x1x2) / f(x1x2)≤miny error +Δy}

[0057] Among them, the number of calibration images obtained is N, the number of calibration samples is x1 (the number of samples calibrated using Zhang's calibration algorithm), and x1 calibration images are selected from the total samples N as x2, and the calibration reprojection error y is established in the space [x1x2] error Relationship with sample; miny error is the minimum value of the reprojection error, the function value of f is given by Zhang’s calibration algorithm, and Δy is the extreme value of the reprojection error;

[0058] The samples E selected by the above formula all have small reprojection errors.

[0059] It should be noted that a more refined calibration result can be obtained through this step method.

[0060] Step S3 of the present invention also includes:

[0061] Step S33: In the calibration result set e={e1 e2 … e9} in sample E, e includes four intrinsic parameters of the camera, two radial distortion parameters, two tangential distortion parameters and reprojection error. The frequency of the eight parameters in the sample space is counted, and the area with the highest density is obtained as the area where the camera parameters are located through the clustering DBSCAN clustering algorithm. The eight areas are statistically analyzed to form eight camera parameter distribution maps. The regional statistics are as follows: Figure 2 (a) Figure 2 (b) Figure 2 (c) Figure 2 (d) Figure 2 (e) and Figure 2 (f);

[0062] Each picture shows the distribution of a camera parameter in the parameter space. The horizontal axis shows the value of the parameter, and the vertical axis shows the frequency of occurrence of the parameter value. Since the two tangential distortion parameters of the camera are 0, no statistics are performed.

[0063] As can be seen in the figure, all parameters approximately satisfy a certain function distribution and have a peak value. The point at the peak of the distribution is the true value point of the camera parameter. This distribution is affected by many factors such as the accuracy error of the calibration plate, the shooting environment, and the camera structure.

[0064] Further references Figure 3 (a) Figure 3 (b) Figure 3 (c) Figure 3 (d) Figure 3 (e) and Figure 3 (f); In order to better fit the distribution function of all parameters and consider the generalization performance of the function, the distribution function of the parameters is fitted again by a fourth-order polynomial. For the fitted function, the parameter value at the frequency maximum is obtained based on the accurate value of the camera parameter.

[0065] Step S5 of the present invention includes: randomly selecting points in the entire sample space to obtain batch calibration results. Arrange the parameters of the calibration results from small to large according to the reprojection error. In the verification process, since the reprojection error reflects the quality of the calibration result, the calibration parameters of the sample with the smallest reprojection error are selected from the sample set, and this parameter is regarded as the approximate accurate value of the camera parameter. real ;

[0066] Then, the samples are divided into n sample sets according to the reprojection error from small to large, E = [E1 E2 … E n ], in which each sample in the latter sample set has a larger reprojection error value than each sample in the former sample set, E i This reflects the quality of the sample;

[0067] In the verification, the sample calibration result with the minimum reprojection error in each sub-sample in the sample set E is calculated and compared with e real The mean square error err1 = [r 11 r 12 … r 1n ]; and the result of the cluster fitting algorithm err2 = [r 21 r 22 … r 2n ]; err1 and err2 respectively reflect the accuracy of the minimum reprojection error sample of each group of samples and the accuracy of the DBSCAN clustering algorithm. Specifically, DBSCAN clustering is to determine the sample distribution area, select the area with the largest density from multiple areas, and the area with the largest density has the highest confidence; then perform parameter fitting, and the fitting result is consistent with e real The variance is err2.

[0068] The results are shown in the figure Figure 4 As shown in the figure above, it can be seen that for most samples, DBSCAN clustering and fitting can obtain higher-precision calibration results based on the minimum reprojection error. This verifies the correctness of the algorithm in this paper. After searching for the minimum reprojection error globally, fitting locally on the sample can obtain higher calibration accuracy.

[0069] Figure 4 To a certain extent, it reflects the relationship between the reprojection error and the camera parameters. The reprojection error can estimate the calibration quality of the camera parameters, but it has a large variance and is not very stable. The present invention is more stable for estimating the camera parameters. This method can filter out the random errors introduced by each link of the calibration process.

[0070] The present invention also provides a computer storage medium, comprising a computer program. When the computer program is executed, the above-mentioned camera calibration optimization method based on sample space is executed to optimize the camera parameter calibration.

[0071] Based on the above description, it can be seen that the advantages of the present invention are:

[0072] 1. The camera calibration optimization method based on sample space provided by the present invention solves the problem that the accuracy of camera calibration parameters is greatly affected by the calibration image, can select samples that are beneficial to calibration, and obtain a higher calibration accuracy than the general calibration algorithm.

[0073] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A camera calibration optimization method based on sample space, characterized in that: The following steps are involved: S1: Collect calibration images to form a sample space; S2: Global sample optimization to achieve sample screening; S3: local parameter fitting; S4: Get the calibration result; S5: Result verification; among them, The step S1 comprises: constructing a constrained discrete nonlinear equation between the sample and the reprojection error in the sample space, wherein the discrete nonlinear equation is: y error =f(x1,x2) The number of calibration images obtained is N, the image x1 required for one calibration is x1, the combination of x1 calibration images selected from the total sample N is recorded as x2, and the calibration reprojection error y is established in the space [x1x2] error The relationship with the sample; C expresses the number of combinations, and the function value of f is given by Zhang's calibration algorithm.

2. The camera calibration optimization method based on sample space according to claim 1, characterized in that: The purpose of the discrete nonlinear equation optimization is to find the minimum value of the discrete equation. The smaller the extreme point is, the smaller the reprojection error is, and the better the calibration result is.

3. The camera calibration optimization method based on sample space according to claim 1, characterized in that: The step S2 includes: optimizing the samples by using a general optimization algorithm with probabilistic global optimization performance, further improving the calibration accuracy at the extreme point of the sample space, selecting a batch of samples near the extreme point and obtaining a parameter set for camera calibration.

4. The camera calibration optimization method based on sample space according to claim 3, characterized in that: The general optimization algorithm with probabilistic global optimization performance is a simulated annealing algorithm.

5. The camera calibration optimization method based on sample space according to claim 1, characterized in that: The step S3 includes: performing clustering and fitting on the camera calibration parameter set to obtain parameter regions and parameter values ​​respectively.

6. The camera calibration optimization method based on sample space according to claim 1, characterized in that: The step S3 also includes: obtaining local samples, wherein the local sample acquisition includes obtaining the camera internal parameters in miny error The fitting is performed at , and the samples are selected according to the following formula: {(x1x2) / f(x1x2)≤miny error +Δy}。 7. The camera calibration optimization method based on sample space according to claim 1, characterized in that: The step S3 also includes: in the calibration result set e={e1e2…e9} in the sample E, e includes four intrinsic parameters of the camera, two radial distortion parameters, two tangential distortion parameters and a reprojection error, the frequency of the eight parameters in the sample space is counted, and the area with the highest density is obtained as the area where the camera parameters are located through the clustering DBSCAN clustering algorithm, and the eight areas are counted to form eight camera parameter distribution maps, each of which represents the distribution of a camera parameter in the parameter space, the horizontal axis in the figure represents the size of the parameter value, and the vertical axis represents the frequency of occurrence of the parameter value; in the camera parameter distribution map, the function distribution is satisfied and there is a peak value; the point at the peak of the distribution is the true value point of the camera parameter.

8. The camera calibration optimization method based on sample space according to claim 7, characterized in that: The step S3 further includes: fitting the distribution function of the parameter again by a fourth-order polynomial, and for the fitted function, obtaining the parameter value at the frequency maximum value based on the accurate value of the camera parameter.

9. The camera calibration optimization method based on sample space according to claim 8, characterized in that: The step S5 comprises: Randomly select points in the entire sample space to obtain batch calibration results, and arrange the parameters of the calibration results from small to large according to the reprojection error; select the calibration parameter of the sample with the smallest reprojection error in this sample set, and regard this parameter as the approximate accurate value of the camera parameter e real ; Then divide the samples into n sample sets according to the reprojection error from small to large, E = [E1E2…E n ], in which each sample in the latter sample set has a larger reprojection error value than each sample in the former sample set, E i This reflects the quality of the sample; Calculate the sample calibration result with the minimum reprojection error in each sub-sample in the sample set E and e real The mean square error err1 = [r 11 r 12 …r 1n ]; and the result of the cluster fitting algorithm err2 = [r 21 r 22 …r 2n ]; err1 and err2 respectively reflect the accuracy of the minimum reprojection error sample of each group of samples and the accuracy of the DBSCAN clustering algorithm.

10. The camera calibration optimization method based on sample space according to claim 9, characterized in that: The DBSCAN clustering method is to determine the sample distribution area, select the area with the largest density from multiple areas, and the area with the largest density has the largest confidence; then perform parameter fitting, and the fitting result is consistent with e real The variance is err2.

Images