A camera calibration method and storage medium

Abstract

The application discloses a camera calibration method and a storage medium, and the method comprises the following steps: extracting a QR code with a checkboard in four end points in an input image; establishing a straight line equation according to an end point in each QR code and image coordinates of two adjacent end points of the end point; obtaining a checkboard region according to the straight line equation; selecting an arbitrary point in the checkboard region, and establishing a to-be-detected region according to the point and the end point of the QR code; randomly selecting a to-be-detected point in the to-be-detected region, performing straight line fitting on all the selected to-be-detected points, and obtaining an adjacent side of the end point; calculating an intersection point of the adjacent side of the end point, and obtaining a pixel coordinate of the end point; traversing all the end points of the QR code, and obtaining pixel coordinates of all the end points of the QR code; calculating world coordinates of all the end points of the QR code corresponding to the end point, and obtaining the world coordinates of all the end points of the QR code; establishing a mapping relationship according to the pixel coordinates and the world coordinates of all the end points of the QR code, and obtaining calibration parameters.

Description

Technical Field

[0001] This invention relates to the field of camera calibration in image processing technology, and specifically to a camera calibration method and storage medium. Background Technology

[0002] In image measurement and machine vision applications, to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, it is necessary to establish a geometric model of camera imaging. These geometric model parameters are the camera parameters, and the process of solving for the camera parameters is called camera calibration. Whether in image measurement or machine vision applications, camera parameter calibration is a crucial step. The accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera. Therefore, proper camera calibration is a prerequisite for subsequent work, and improving calibration accuracy is a key focus of scientific research.

[0003] Traditional camera calibration typically involves calculating parameters for the entire image, extracting all pixels of the calibration object in the image, and establishing a unified mapping relationship using the known world coordinates. However, due to camera distortion or other reasons, the different regions of the captured image are not linearly distorted. If a one-to-one correspondence is established for the camera using the entire image, it will seriously affect the accuracy of the calibration results and the accuracy of subsequent results. Summary of the Invention

[0004] The technical problem to be solved by this invention is to improve the accuracy of camera calibration results. The purpose is to provide a camera calibration method and storage medium, which solves the problem that when the regions in the input image are not linearly distorted, the accuracy of the camera calibration results obtained by using a single calibration parameter to establish a mapping relationship between the pixel coordinates and world coordinates of the entire input image is poor.

[0005] This invention is achieved through the following technical solution:

[0006] The first aspect provides a camera calibration method, including the following steps:

[0007] After receiving the input image, extract the QR code from the input image where the checkerboard pattern exists at all four endpoints;

[0008] Establish a straight line equation based on the image coordinates of one endpoint of each of the above QR codes and the two adjacent endpoints of that endpoint;

[0009] Based on the equation of the straight line, determine the orientation of the chessboard grid relative to the endpoint corresponding to the QR code, and obtain the boundary line of the chessboard grid that is collinear with the corresponding straight line in the chessboard grid area;

[0010] Selecting an arbitrary point on the boundary line of the checkerboard, and establishing a detection area according to the selected point and the endpoint of the QR code;

[0011] Selecting an arbitrary detection point in the detection area, and performing straight line fitting on all the selected detection points to obtain the adjacent side of the endpoint;

[0012] Calculating the intersection of the adjacent side of the endpoint to obtain the pixel coordinates of the endpoint;

[0013] Traversing all the endpoints of the QR code to obtain the pixel coordinates of the endpoints of the QR code;

[0014] Calculating the world coordinates of the endpoints of the QR code corresponding to the endpoint to obtain the world coordinates of the endpoints of the QR code;

[0015] Establishing a mapping relationship according to the pixel coordinates and the world coordinates of the endpoints of the QR code to obtain the calibration parameters.

[0016] Using the QR code to divide the input image into multiple independent areas for calibration, and establishing corresponding calibration parameters in each independent area to effectively improve the accuracy of the calibration result.

[0017] Further, extracting the QR code in which the four endpoints of the input image all exist in the checkerboard, including the following steps:

[0018] Reading the image coordinates of all the endpoints of the QR code in the input image;

[0019] Respectively extracting the four endpoints corresponding to the QR code, and generating four regions with the four endpoints as the centers;

[0020] Respectively performing filtering, binarization and morphological processing on the regions to obtain a plurality of connected regions;

[0021] Respectively calculating the areas of the connected regions, and judging whether the areas of the connected regions are greater than the corresponding threshold values;

[0022] If the area of the connected region is greater than the threshold value, it is determined that the endpoint of the region corresponding to the connected region exists in the checkerboard;

[0023] If the area of the connected region is not greater than the threshold value, it is determined that the endpoint of the region corresponding to the connected region does not exist in the checkerboard;

[0024] Judging whether the four endpoints corresponding to the QR code all exist in the checkerboard;

[0025] If the four endpoints corresponding to the QR code all exist in the checkerboard, the QR code is extracted;

[0026] If one or more of the four endpoints corresponding to the QR code does not have a chessboard, the QR code is not extracted.

[0027] The QR code in which the four endpoints in the input image all have a chessboard is extracted, the QR code is taken as an independent region, the calibration parameters are established in the independent region, the input image is divided into multiple independent regions for an input image without linear distortion, the mapping relationship is established according to the independent regions, and the accuracy of the calibration result is effectively improved.

[0028] Further, the side length of the region is 1:2 of the side length of the QR code, the calculation difficulty of the area of the region and the threshold value is simplified, and the algorithm is more stable.

[0029] Further, the threshold value is 1 / 2 of the area of the chessboard region in the region.

[0030] The threshold value compared with the area of the connected region is calculated according to the side length of the QR code, and the calculation method is simple, so that the algorithm is more stable.

[0031] Further, any detection point in the detection region is selected, a straight line fitting is performed on the selected detection point, and the adjacent edge of the endpoint is obtained. The detailed steps are as follows:

[0032] The detection region is divided into multiple equal proportion detection areas, any detection point in each detection area is selected, a set of detection points is obtained, a straight line fitting is performed on all the detection points in the set of detection points by using the least square method, and the adjacent edge of the endpoint is obtained.

[0033] The line obtained by performing the straight line fitting on all the detection points in the set of detection points is the adjacent edge of the endpoint, and the adjacent edge is a side of the chessboard passing through the endpoint.

[0034] Further, the detection point is a point on the boundary line of the chessboard.

[0035] Further, the four endpoints corresponding to the extracted QR code are taken as a group, a mapping relationship is established, and the calibration parameters are obtained.

[0036] The QR code in which the four endpoints all have a chessboard is taken as an independent region, a mapping relationship is established according to the pixel coordinates and the world coordinates of each endpoint of the QR code, the calibration parameters of the independent region are obtained, the input image is divided into multiple independent regions, the mapping is performed on the independent regions respectively, and the calibration parameters of each independent region are obtained; since the range of the calibration parameters in each independent region is only in the independent region, the mapping error is effectively reduced, and the accuracy of the calibration result is improved.

[0037] ​Further, before judging whether the four end points corresponding to the QR code all exist the checkerboard, the method further comprises the following steps:

[0038] initializing a counter;

[0039] receiving a judgment result of one end point corresponding to the QR code;

[0040] if the judgment result is that the checkerboard exists, the counter is added by 1, and the four end points corresponding to the QR code are traversed;

[0041] judging whether the counter is equal to 4;

[0042] if yes, the four end points corresponding to the QR code all exist the checkerboard;

[0043] if no, at least one end point of the four end points corresponding to the QR code does not exist the checkerboard.

[0044] Further, the QR code stores the world coordinates of the center point of the QR code, and the world coordinates of the four end points corresponding to the QR code are calculated according to the side length of the QR code and the world coordinates of the center point of the QR code.

[0045] The second aspect provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program makes the computer execute the camera calibration method.

[0046] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0047] The QR code with the four end points all existing the checkerboard is regarded as an independent region, the input image comprises a plurality of independent regions, two straight line equations are established according to the image coordinates of one end point and two adjacent end points in the QR code, the orientation of the checkerboard relative to the end point corresponding to the QR code is judged according to the two straight line equations, the boundary line of the checkerboard in the checkerboard region which is collinear with the straight line is obtained, the to-be-detected point is selected on the boundary line of the checkerboard to perform straight line fitting, two edges (namely, adjacent edges of the end point) in the checkerboard passing through the end point are obtained, the intersection of the two edges is calculated to obtain the end point (namely, the pixel coordinates of the end point) of the QR code at the sub-pixel level, and the pixel coordinates of the four end points of the QR code are obtained by traversing all the end points of the QR code, so that the precision during mapping is improved.

[0048] The input image is divided into a plurality of independent regions for mapping by using the QR code, and corresponding calibration parameters are established in each independent region, so that the precision of the calibration result is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as a limitation to the scope. Other related drawings can also be obtained by those skilled in the art without any creative effort, based on the drawings. In the drawings:

[0050] Figure 1 Main flowchart provided for Example 1;

[0051] Figure 2 Flowchart provided for extracting QR code for Example 1;

[0052] Figure 3 Schematic diagram for input image;

[0053] Figure 4 Schematic diagram for an independent area from input image.

[0054] Markings in the drawings and corresponding names of parts:

[0055] 1-area, 2-checkered area, 3-detection area. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will further describe the present application in combination with embodiments and drawings. The exemplary embodiments of the present application and their descriptions are only used to explain the present application, and should not be considered as a limitation to the present application.

[0057] Example 1

[0058] This Example 1 provides a camera calibration method, as shown in Figure 1 , Figure 3 and Figure 4 , including the following steps:

[0059] S1, after receiving the input image, extracting the QR code with four end points in the above input image;

[0060] S2, establishing straight line equations A and B according to the image coordinates of one end point in each of the above QR codes and the adjacent two end points of the end point;

[0061] S3, judging the orientation of the checkered area relative to the end point corresponding to the QR code according to the straight line equations A and B, and obtaining the checkered boundary line of the checkered area 2 and the corresponding straight line;

[0062] S4, selecting an arbitrary point from the checkered boundary line, and establishing the detection area 3 according to the selected point and the end point of the QR code;

[0063] S5, any selected detection point in the above-mentioned detection area 3, the straight line fitting is carried out to all the selected detection points, and the two adjacent edges of the end point are obtained;

[0064] S6, the intersection of the adjacent edges of the end point is calculated, and the pixel coordinates of the end point are obtained;

[0065] All the end points of the above-mentioned QR code are traversed, and the pixel coordinates of the four end points corresponding to the above-mentioned QR code are obtained;

[0066] S7, the world coordinates of the center point of the QR code are stored in the above-mentioned QR code, the world coordinates of the center point of the QR code stored in the QR code are read by Zbar, and the world coordinates of the four end points corresponding to the above-mentioned QR code are calculated according to the side length of the above-mentioned QR code and the world coordinates of the center point of the QR code;

[0067] S8, according to the pixel coordinates and world coordinates of the four end points corresponding to the above-mentioned QR code, a mapping relationship is established, and the calibration parameters are obtained.

[0068] The above-mentioned end point is Figure 4 A, the above-mentioned QR code with four end points in the checkerboard is regarded as an independent area, the input image includes a plurality of independent areas, and the mapping of the independent areas is carried out respectively, so that the calibration parameters of each independent area are obtained; the effective error of the mapping is reduced, and the precision of the calibration result is improved.

[0069] Specific embodiments, as shown in Figure 2 The QR code with four end points in the checkerboard in the above-mentioned input image is extracted, including the following steps:

[0070] S11, the image coordinates of all the end points of the above-mentioned input image are read by Zbar;

[0071] S12, the four end points corresponding to the above-mentioned QR code are extracted respectively, four regions 1 are generated respectively with the four above-mentioned end points as the centers, and the side length of the above-mentioned region 1 is 1:2 of the side length of the QR code; the calculation difficulty of the area and the threshold value of the region 1 is simplified, so that the algorithm is more stable.

[0072] S13, the four above-mentioned regions 1 are filtered, binarized and morphologically processed respectively, and a plurality of connected regions are obtained;

[0073] S14, the areas of the above-mentioned connected regions are calculated respectively, whether the area of the above-mentioned connected region is greater than the corresponding threshold value is judged, and all the connected regions corresponding to the above-mentioned region 1 are traversed;

[0074] S15, if the area of the above-mentioned connected region is greater than the threshold value, it is determined that the end point corresponding to the above-mentioned connected region exists in the checkerboard;

[0075] If the area of the connected region is not greater than the threshold value, it is determined that the end point corresponding to the connected region does not exist in the chessboard grid.

[0076] S16, it is determined whether the four end points corresponding to the QR code all exist in the chessboard grid.

[0077] S17, if the four end points corresponding to the QR code all exist in the chessboard grid, the QR code is extracted.

[0078] If one or more of the four end points corresponding to the QR code do not exist in the chessboard grid, the QR code is not extracted.

[0079] The QR code in which the four end points all exist in the chessboard grid is extracted from the input image, and the QR code is an independent region. The calibration parameters are established in the independent region. For an input image without linear distortion, the input image is divided into multiple independent regions, a mapping relationship is established according to the independent regions, and the accuracy of the calibration result is effectively improved.

[0080] In a specific embodiment, the threshold value is 1 / 2 of the area of the chessboard grid region 2 in the region 1. The calculation formula of the threshold value is as follows:

[0081] Wherein, L represents the side length of the QR code.

[0082] The threshold value compared with the area of the connected region is calculated according to the side length of the QR code, and the calculation method is simple, so that the algorithm is more stable.

[0083] In a specific embodiment, any detection point in the detection region 3 is selected, a straight line fitting is performed on the selected detection point to obtain the adjacent side of the end point, and the detailed steps are as follows:

[0084] The detection region 3 is divided into multiple equal proportion detection regions, any detection point is selected from each detection region, a detection point set is obtained, and a straight line fitting is performed on all detection points in the detection point set by using the least square method to obtain two adjacent sides of the end point.

[0085] The line obtained by performing the straight line fitting on all detection points in the detection point set is the adjacent side of the end point, and the adjacent side is a side on the chessboard grid passing through the end point.

[0086] For example, the two adjacent sides are straight line 1: A1x+B1y+C1=0 and A2x+B2y+C2=0, and the intersection point of the straight line 1 and the straight line 2 is the pixel coordinates of the end point (the end point of the QR code is adjacent to the chessboard grid, and the end point is the corner point of the corresponding chessboard grid), and the pixel coordinates of the end point are as follows:

[0087]

[0088] In specific embodiments, the to-be-detected point is a point on a boundary line of a chessboard grid.

[0089] In specific embodiments, the four endpoints corresponding to the extracted QR code are taken as a group, a mapping relationship is established, and the calibration parameter is obtained.

[0090] The four endpoints of the QR code existing in the chessboard grid are taken as an independent region, a mapping relationship is established according to the pixel coordinates and the world coordinates of each endpoint of the QR code, the calibration parameter of the independent region is obtained, the input image is divided into multiple independent regions, and the mapping is performed on the independent regions to obtain the calibration parameter of each independent region; since the range of the calibration parameter in each independent region is only in the independent region, the mapping error is effectively reduced, and the precision of the calibration result is improved.

[0091] In specific embodiments, before determining whether the four endpoints corresponding to the QR code exist in the chessboard grid, the following steps are further included:

[0092] Initializing a counter;

[0093] Receiving a determination result of an endpoint corresponding to the QR code;

[0094] If the determination result is that the chessboard grid exists, the counter is incremented by 1, and the four endpoints corresponding to the QR code are traversed;

[0095] Determining whether the counter is equal to 4;

[0096] If yes, the four endpoints corresponding to the QR code all exist in the chessboard grid;

[0097] If no, at least one of the four endpoints corresponding to the QR code does not exist in the chessboard grid.

[0098] In specific embodiments, the world coordinates of each endpoint of the QR code are calculated as follows:

[0099] If the world coordinates of the center point of the QR code are (x, y), and the side length of the QR code is L, the world coordinates of the upper left corner of the QR code are The world coordinates of the lower left corner of the QR code are The world coordinates of the upper right corner of the QR code are The world coordinates of the lower right corner of the QR code are

[0100] Embodiment 2

[0101] Any point P in the above space, the world coordinates of P point is (X, Y, Z), the camera coordinates is (x, y, z), the conversion relationship between the world coordinates and the camera coordinates is as follows by calling the calibration coefficient:

[0102]

[0103] Wherein, R is a 3*3 rotation matrix, t is a 3*1 translation vector.

[0104] The above space, the world coordinates of P point is (X, Y, Z), the camera coordinates is (x, y, z), the conversion relationship between the world coordinates and the camera coordinates is as follows by calling the calibration coefficient:

[0105] Embodiment 3

[0106] The embodiment 3 provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program makes the computer execute the camera calibration method.

[0107] The above specific embodiments, the purpose, technical scheme and beneficial effects of the present application are further described in detail, it should be understood that the above description is only a specific embodiment of the present application, and is not used to limit the protection scope of the present application, any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A camera calibration method, characterized by, The method comprises the following steps: After receiving the input image, a QR code with a checkboard pattern at each of the four corners of the QR code is extracted from the input image; A straight line equation is established according to the image coordinates of one corner of each of the QR codes and the adjacent two corners of the one corner; According to the straight line equation, the orientation of the checkboard pattern relative to the corner of the QR code corresponding to the checkboard pattern is determined, and a checkboard pattern boundary line in the checkboard pattern region (2) that is collinear with the straight line is obtained; An arbitrary point on the checkboard pattern boundary line is selected, and a detection region (3) is established according to the selected point and the corner of the QR code; An arbitrary detection point in the detection region (3) is selected, and a straight line fitting is performed on all the selected detection points to obtain the adjacent edge of the corner. The intersection of the adjacent edges of the corner is calculated to obtain the pixel coordinates of the corner. All the corners of the QR code are traversed to obtain the pixel coordinates of all the corners of the QR code. The world coordinates of all the corners of the QR code corresponding to the corner are calculated to obtain the world coordinates of all the corners of the QR code. A mapping relationship is established according to the pixel coordinates and the world coordinates of all the corners of the QR code to obtain the calibration parameters.

2. The camera calibration method of claim 1, wherein, The QR code with a checkboard pattern at each of the four corners of the QR code is extracted from the input image, which comprises the following steps: All the corner image coordinates of the QR code in the input image are read; Four regions (1) are generated respectively by taking the four corners of the QR code as the centers; The regions (1) are filtered, binarized and morphologically processed respectively to obtain a plurality of connected regions; The areas of the connected regions (1) are calculated respectively, and it is determined whether the area of the connected region is greater than a corresponding threshold value; If the area of the connected region is greater than the threshold value, it is determined that the corner of the region (1) corresponding to the connected region has a checkboard pattern; If the area of the connected region is not greater than the threshold value, it is determined that the corner of the region (1) corresponding to the connected region does not have a checkboard pattern; It is determined whether the four corners of the QR code have a checkboard pattern; If the four corners of the QR code have a checkboard pattern, the QR code is extracted; If one or more of the four corners of the QR code do not have a checkboard pattern, the QR code is not extracted.

3. The camera calibration method of claim 2, wherein, The side length of the region (1) is 1:2 of the side length of the QR code.

4. The camera calibration method of claim 3, wherein, The threshold is a multiple of the area of the checkerboard region within region (1) .

5. The camera calibration method of claim 1, wherein, The detailed steps of the straight line fitting on all the selected detection points to obtain the adjacent edge of the corner are as follows: The detection region (3) is divided into a plurality of equal proportion detection areas, and an arbitrary detection point is selected from each of the detection areas to obtain a set of detection points. The least square method is used to perform straight line fitting on all the detection points in the set of detection points to obtain the adjacent edge of the corner.

6. The camera calibration method of claim 5, wherein, The detection point is a point on the checkboard pattern boundary line.

7. The camera calibration method of claim 1, wherein, The four corners of the extracted QR code are taken as a group, a mapping relationship is established, and the calibration parameters are obtained.

8. The camera calibration method of claim 2, wherein, Before determining whether the four corners of the QR code have a checkboard pattern, the following steps are further included: The counter is initialized; The determination result of one corner of the QR code is received; If the determination result is that the chessboard exists, the counter is incremented by 1, and four endpoints corresponding to the QR code are traversed; It is determined whether the counter is equal to 4; If yes, the four endpoints corresponding to the QR code all exist the chessboard; If no, at least one of the four endpoints corresponding to the QR code does not exist the chessboard.

9. The camera calibration method of claim 1, wherein, The world coordinates of the center point of the QR code are stored in the QR code; and the world coordinates of the four endpoints corresponding to the QR code are calculated according to the side length of the QR code and the world coordinates of the center point of the QR code.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program causes the computer to execute the camera calibration method according to any one of claims 1-9.

Images