Method for calibrating binocular camera by sub-region

Abstract

This invention provides a method for calibrating a stereo camera in different regions, including image acquisition and calibration processes. The working area is divided into multiple sub-regions for separate calibration. Different calibration parameters are used for coordinate calculation in different sub-regions. During calibration, corner points are first retrieved, followed by single-target calibration, and then dual-target calibration is performed considering camera distortion. After calibration, the data is saved, and the calibration results are used for practical applications. The calibration method provided by this invention overcomes the problem of significant differences in measurement accuracy of a single calibration parameter in different regions.

Description

Technical Field

[0001] This invention relates to the field of visual measurement, specifically a method for calibrating a binocular camera by dividing the area. Background Technology

[0002] In visual measurement systems, camera calibration accuracy is crucial, affecting the overall system precision. For dual-camera calibration, specialized calibration equipment such as dot, checkerboard, or QR code checkerboard calibration boards are typically used, employing algorithms like the Zhang Zhengyou calibration method. This allows for obtaining a relatively ideal positional relationship between the distortion model parameters (intrinsic parameters) and the camera (extrinsic parameters).

[0003] Traditional camera calibration methods calibrate the entire image frame to obtain a set of camera parameters, which are then used for pixel correction. Since these calibration parameters are calculated from the pixels across the entire image, they represent an average correction for the entire image. However, deviations inevitably occur during camera and lens manufacturing processes. Even with finely crafted lenses, it's impossible to guarantee that the distortion of the image projected onto the sensor will be uniform across every point. Therefore, in scenarios requiring high precision, using only a set of average correction parameters to calculate measurement results often results in inconsistent accuracy across different areas, failing to guarantee consistently high-precision measurements globally. Summary of the Invention

[0004] To address the problems of existing technologies, this invention provides a method for calibrating a binocular camera by dividing the working area into multiple sub-regions for separate calibration. Different calibration parameters are used for coordinate calculation in different sub-regions, overcoming the problem that the measurement accuracy of a single calibration parameter varies significantly in different regions.

[0005] A method for calibrating a stereo camera by region, the specific process of which is as follows:

[0006] S1. Determine the size of the working area of ​​the stereo camera. Mark the overlapping area of ​​the images captured by the two cameras at the working distance of the stereo camera as the working area. Divide the working area into N*M sub-regions according to its size, and assign an independent label to each sub-region. Determine the size of the calibration board according to the size of each sub-region, requiring that each sub-region can accommodate the complete calibration board, and that the area of ​​the calibration board is not less than one-third of the area of ​​each sub-region.

[0007] S2. The binocular camera will capture photos of the calibration board at different angles in each sub-region. Each sub-region must capture photos containing the complete calibration tool, and the number of photos captured in each sub-region should be sufficient to complete a calibration independently.

[0008] S3, Searching for Corner Points:

[0009] The checkerboard image is uniformly adjusted to the set resolution and converted to grayscale; the corner positions in the checkerboard image are determined by searching using the Harris algorithm; the obtained corner positions are refined using a sub-pixel precision method to obtain the image coordinates of the corners.

[0010] S4. Calibrate the photos corresponding to each sub-region of the binocular camera separately, and calibrate each sub-region's photo set as an independent calibration image set separately.

[0011] To calculate the spatial position of a photographed object from an image, it is necessary to determine the mapping relationship between the image coordinate system and the world coordinate system. The image is transformed from the world coordinate system to the pixel coordinate system, including the transformation from the world coordinate system M... w Transform to camera coordinate system M c Transformation from camera coordinate system to image coordinate system M xy Transformation from image coordinate system to pixel coordinate system M p Three processes;

[0012] (1) Transformation from world coordinate system to camera coordinate system:

[0013]

[0014] Where R is the orthogonal selection matrix

[0015]

[0016] T is the translation matrix.

[0017]

[0018] (2) Transformation from camera coordinate system to image coordinate system:

[0019] Let the homogeneous coordinates of point P in space in the camera coordinate system be...

[0020]

[0021] Its image point's homogeneous coordinates in the image coordinate system are

[0022]

[0023] in

[0024]

[0025] In practical applications, the principal point may not be at the origin of the image coordinate system. Let the coordinates of the principal point in the image coordinate system be...

[0026]

[0027] Then there is

[0028]

[0029] (3) From image coordinate system to pixel coordinate system:

[0030] Let the length and width of each pixel in the photosensitive unit be d, respectively. x and d y Let the coordinates of point P in the pixel coordinate system be...

[0031]

[0032] If u and v are in pixels, then Therefore, there is

[0033]

[0034] Combining equations 1, 2, and 3, we can obtain

[0035]

[0036] make

[0037]

[0038]

[0039] but

[0040]

[0041] make

[0042]

[0043] K is the intrinsic parameter matrix of the monocular camera.

[0044] Therefore, the unknown variables that need to be calibrated to transform an image from the world coordinate system to the pixel coordinate system include the rotation matrix R, the translation matrix T, and the intrinsic parameter matrix K.

[0045] S5, Camera Distortion

[0046] Camera distortion models include radial distortion and centrifugal distortion.

[0047] Radial distortion model is

[0048]

[0049] Where r 2 =u 2 +v 2 k1, k2, and k3 are distortion coefficients that need to be obtained through calibration.

[0050] Centrifugal distortion model is

[0051]

[0052] p1 and p2 are centrifugal distortion parameters that need to be obtained through calibration.

[0053] Assuming the actual image point coordinates are (u, v), and the ideal pixel point is (u... u ,v u Then the distortion compensation model

[0054]

[0055] Using a distortion matrix D to represent the above distortion parameters, it can be expressed as follows:

[0056] D = [k1 k2 p1 p2 k3].

[0057] S6. Binocular camera calibration:

[0058] Let the extrinsic parameter matrices of the left and right cameras obtained from single-target calibration be R and R, respectively. r T r and R l T l Therefore, the relationship matrix between the left and right cameras can be represented as:

[0059]

[0060] S7, Data Saving

[0061] After completing the monocular and binocular camera calibrations, save the acquired N*M sets of calibration results. Each set of calibration results should include the intrinsic parameter matrices of the left and right cameras, the distortion matrices of the left and right cameras, the rotation and translation matrices between the left and right cameras, and the identifier of each sub-region.

[0062] The beneficial effects of this invention are that it divides the working area into multiple sub-regions for calibration, and uses different calibration parameters for coordinate calculation in different sub-regions, thus overcoming the problem that the measurement accuracy of a single calibration parameter varies greatly in different regions. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart of the regional calibration process;

[0065] Figure 2 This is a schematic diagram illustrating the operation of an embodiment;

[0066] Figure 3 This is a schematic diagram of the calibration photos collected in the sub-region of the embodiment;

[0067] Figure 4 This is a flowchart of the calibration data application process. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] This embodiment provides a method for calibrating a binocular camera in a regional manner. The implementation process includes three parts: image acquisition, parameter calibration, and application of calibration parameters.

[0070] 1. Photo Collection

[0071] 1.1 Divide the work area into sub-regions. Based on the work area size, divide it into N*M sub-regions and assign each sub-region a unique identifier. For example... Figure 2 As shown, in this embodiment, the working area is divided into 9 sub-areas of 3*3, and each area is labeled as 1 to 9.

[0072] 1.2 The binocular camera will capture images of the calibration board at different angles in each sub-region. Each sub-region's captured images must contain the complete calibration tool, and the number of images captured in each sub-region should be sufficient to complete one calibration independently. For example... Figure 3 As shown, the photo sets taken in each sub-region are labeled I1, I2, I3, ...

[0073] 2. Calibration

[0074] Photo set I for each sub-region acquired by the binocular camera n Calibration is performed separately, treating each sub-region's image set as an independent calibration image set and calibrating it completely once to obtain the corresponding calibration parameters. The calibration results need to include the intrinsic parameter matrices K of the left and right cameras. l K r The distortion matrices D of the left and right cameras l D r The rotation matrix R and translation matrix T between the left and right cameras.

[0075] 2.1 Corner Detection

[0076] In this embodiment, the Harris algorithm is used to locate corner points, and a sub-pixel precision method is used to refine the obtained data.

[0077] The corner positions are refined to obtain the image coordinates of the corners. Furthermore, this embodiment uses two two-dimensional arrays to store the image coordinates of the corners of the checkerboard images obtained by the left and right cameras, respectively. Specifically, corner searches are performed on each checkerboard image sequentially according to the storage order of the checkerboard images.

[0078] 2.2 Single Target Setting

[0079] Using the pixel coordinates of the calibration board images captured by the left and right cameras, and the world coordinates of the calibration board, solve for the projection matrix M of the corner points from the world coordinate system to the pixel coordinate system; let the camera's pixel calibration system be a coordinate system M with the top left corner of the image as the origin. p (x,y), the world coordinate system is a coordinate system M with the coordinates of the top left corner of the calibration plate as the origin. w (x,y,z); Therefore, the process of transforming from the world coordinate system to the pixel coordinate system can be represented as:

[0080] M p =MM c

[0081] It should be noted that the image is transformed from the world coordinate system to the pixel coordinate system, including the transformation from the world coordinate system M... w Transform to camera coordinate system M c Transformation from camera coordinate system to image coordinate system M xy Transformation from image coordinate system to pixel coordinate system M p Three processes;

[0082] 2.2.1 Transformation from world coordinate system to camera coordinate system:

[0083] The coordinate transformation between two different coordinate systems can be regarded as a transformation consisting of two parts: rotation and translation. Let the rotation matrix of a single camera be R and the translation matrix be T.

[0084]

[0085] Where R is the orthogonal selection matrix

[0086]

[0087] T is the translation matrix.

[0088]

[0089] 2.2.2 Transformation from camera coordinate system to image coordinate system:

[0090] The xy-plane of the camera coordinate system lies on the same plane as the image coordinate system. According to the pinhole imaging principle, the transformation between the camera coordinate system and the image coordinate system can be regarded as a similar triangle transformation related to the focal length f. Let the homogeneous coordinates of point P in space in the camera coordinate system be...

[0091]

[0092] Its image point's homogeneous coordinates in the image coordinate system are

[0093]

[0094] Then there is

[0095]

[0096] Therefore, the transformation from the camera coordinate system to the image coordinate system can be expressed in matrix form as follows:

[0097]

[0098] 2.2.3. From image coordinate system to pixel coordinate system:

[0099] Let the length and width of each pixel in the photosensitive unit be d, respectively. x and d y Let u and v represent the number of pixels per millimeter along the x and y axes, respectively, and let u0 and v0 represent the coordinates of the origin of the image coordinate system in the pixel coordinate system. Then the coordinates of point P in the pixel coordinate system can be expressed as:

[0100]

[0101] The transformation from the image coordinate system to the pixel coordinate system can be expressed in matrix form as follows:

[0102]

[0103] Combining Equations 1, 2, and 3, it can be seen that the transformation of an image from the world coordinate system to the pixel coordinate system can be expressed by a matrix expression as follows:

[0104]

[0105] make

[0106]

[0107] but

[0108]

[0109] make

[0110]

[0111] Therefore, the unknown variables that need to be obtained through calibration when transforming an image from the world coordinate system to the pixel coordinate system include the rotation matrix R, the translation matrix T, and the intrinsic parameter matrix K. These unknowns can be solved through camera calibration.

[0112] 2.3 Camera Distortion

[0113] Camera distortion models include radial distortion and centrifugal distortion.

[0114] Radial distortion model is

[0115]

[0116] Where r 2 =u 2 +v 2 k1, k2, and k3 are distortion coefficients that need to be obtained through calibration.

[0117] Centrifugal distortion model is

[0118]

[0119] p1 and p2 are centrifugal distortion parameters that need to be obtained through calibration.

[0120] Assuming the actual image point coordinates are (u, v), and the ideal pixel point is (u... u ,v u Then the distortion compensation model

[0121]

[0122] Using a distortion matrix D to represent the above distortion parameters, it can be expressed as follows:

[0123] D = [k1 k2 p1 p2 k3]

[0124] 2.4 Binocular camera calibration:

[0125] Let the extrinsic parameter matrices of the left and right cameras obtained from single-target calibration be R and R, respectively. r T r and R l T l Therefore, the relationship matrix between the left and right cameras can be represented as:

[0126]

[0127] In summary, the parameters that need to be determined through calibration include the calibration results, and the results that need to be obtained include the intrinsic parameter matrices K of the left and right cameras. l K rThe distortion matrices D of the left and right cameras l D r The rotation matrix R between the left and right cameras l R r and the translation matrix T l T r .

[0128] This embodiment uses the Zhang Zhengyou calibration method to solve for the above parameters. Since the Zhang Zhengyou calibration method is a well-known and mature method in the industry, it will not be described in detail in this embodiment.

[0129] 3. Data storage

[0130] After calibration, each sub-region's image generates a set of calibration parameters, including the intrinsic parameter matrices K for the left and right cameras. l K r The distortion matrices D of the left and right cameras l D r The rotation matrix R between the left and right cameras l R r and the translation matrix T l T r These parameters and their corresponding region labels need to be saved to a file. In this embodiment, the nine sets of calibration parameters calculated for the nine sub-regions and their corresponding numbers are saved.

[0131] 4. Use of calibration results

[0132] like Figure 4 As shown, when a binocular camera performs a measurement, it first calculates the position of the object being measured in the photograph, determines the sub-region to which the image of the measured object belongs, and then uses the calibration parameters relative to the sub-region to calculate the world coordinates M of the measured object. w (x,y,z).

[0133] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, for the device embodiments, the above descriptions are merely preferred embodiments of the present invention. Since they are fundamentally similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention, without departing from the principle of the present invention, should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for calibrating a stereo camera by region, characterized in that: The process includes photo acquisition and calibration, and the specific steps are as follows: 1) Photo Acquisition: 1.1) Determine the size of the working area of ​​the binocular camera. Mark the overlapping area of ​​the images captured by the two cameras at the working distance of the binocular camera as the working area. Divide the working area into N*M sub-regions according to the size of the working area and assign an independent label to each sub-region. 1.2) The binocular camera captures images of the calibration board at different angles in each sub-region, and the images captured in each sub-region contain complete calibration tools; 2) Calibration: 2.1) Corner Retrieval: The Harris algorithm is used to search and determine the corner positions in the checkerboard image; the obtained corner positions are refined using a sub-pixel precision method to obtain the image coordinates of the corners; 2.2) Single-target calibration: Each sub-region of the stereo camera's images is calibrated separately, with each sub-region's image set treated as an independent calibration set for individual calibration. During calibration, to calculate the spatial position of the captured object from the image, the mapping relationship between the image coordinate system and the world coordinate system needs to be determined. The image is converted from the world coordinate system to the pixel coordinate system, including the conversion from the world coordinate system M... w Transform to camera coordinate system M c Transformation from camera coordinate system to image coordinate system M xy Transformation from image coordinate system to pixel coordinate system M p Three processes; 2.3) Camera distortion: including radial distortion and centrifugal distortion; 2.4) Binocular Camera Calibration: Let the extrinsic parameter matrices of the left and right cameras obtained from the single-camera calibration be R0 and R1 respectively. r T r and R l T l Therefore, the relationship matrix between the left and right cameras can be represented as:

2. The binocular camera calibration method by region according to claim 1, characterized in that: The size of the calibration plate mentioned in step 1.2) is determined according to the size of the sub-regions. Each sub-region can accommodate the complete calibration plate, and the area of ​​the calibration plate is not less than one-third of the area of ​​each sub-region.

3. The binocular camera calibration method by region according to claim 1, characterized in that: Before the corner point retrieval process described in step 2.1), the chessboard image is uniformly adjusted to the set resolution and converted to grayscale.

4. The binocular camera calibration method by region according to claim 1, characterized in that: The three processes described in step 2.2) for converting the image from the world coordinate system to the pixel coordinate system are as follows: 2.21) From the world coordinate system M w Transform to camera coordinate system M c The process is as follows: Where R is the orthogonal selection matrix: T is the translation matrix: 2.22) Transform from camera coordinate system to image coordinate system M xy The process is as follows: Let the homogeneous coordinates of point P in space in the camera coordinate system be: Its homogeneous coordinates in the image coordinate system are: in, In practical applications, the principal point may not be at the origin of the image coordinate system. Let the coordinates of the principal point in the image coordinate system be... Then there is 2.23) Transform from image coordinate system to pixel coordinate system M p The specific process is as follows: Let the length and width of each pixel in the photosensitive unit be dx and dy, respectively, and let the coordinates of point P in the pixel coordinate system be... If u and v are in pixels, then Therefore, there is 2.24) According to the above formulas (1), (2), and (3), we can obtain: make: but: make: K is the intrinsic parameter matrix of the monocular camera.

5. The region-based binocular camera calibration method according to claim 1, characterized in that: The radial distortion model described in step 2.3) is Where r 2 =u 2 +v 2 k1, k2, and k3 are distortion coefficients that need to be obtained through calibration.

6. The region-based binocular camera calibration method according to claim 1 or 5, characterized in that: The centrifugal distortion model described in step 2.3) is p1 and p2 are centrifugal distortion parameters that need to be obtained through calibration. Assuming the actual image point coordinates are (u, v), and the ideal pixel point is (u... u ,v u Then the distortion compensation model Using a distortion matrix D to represent the above distortion parameters, it can be expressed as follows: D = [k1 k2 p1 p2 k3].

Images