Coordinate transformation calibration method for binocular vision system camera and space rectangular coordinate system

Abstract

The invention discloses a coordinate transformation calibration method for a binocular vision system camera and a space rectangular coordinate system. The method comprises the following steps: 1) designing and manufacturing a checkerboard calibration plate, placing the checkerboard calibration plate in a binocular vision system, collecting left and right images of a plurality of checkerboard calibration plates by moving and rotating the checkerboard calibration plate, and calculating internal and external parameters of a left camera and a right camera by adopting a Zhang Zhengyou calibration method; 2) placing the checkerboard calibration plate on the horizontal ground, collecting left and right images of the checkerboard calibration plate once, extracting pixel coordinates of each angular point in the left and right images of the checkerboard calibration plate, and calculating three-dimensional coordinates of each angular point in a left camera coordinate system according to the internal and external parameters of the left and right cameras obtained in the step 1); and 3) randomly selecting a certain angular point of the checkerboard calibration plate as a base point, constructing any space rectangular coordinate system based on the three-dimensional coordinate of each angular point in the left camera coordinate system obtained in the step 2), and establishing a coordinate transformation relationship between the camera coordinate system and the space rectangular coordinate system.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and more particularly to a coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system. Background Technology

[0002] In practical applications, pose transformation of a target object can effectively demonstrate its orientation. For example, in motion capture, obtaining a positive human coordinate system is crucial for subsequent data processing and positive display of human joints, and is of great significance. Currently, pose calibration mainly adopts camera-based calibration methods, such as establishing the relationship between the end-effector coordinate system of the camera motion mechanism and the end-effector coordinate system of the robot to achieve object localization and recognition.

[0003] However, these methods rely on both the camera end effector and the robotic arm end effector being in fixed positions, which has certain limitations and makes them unsuitable for situations where the target object's position is uncertain. Furthermore, although the pose can be approximated by measuring the distance and angle between the camera end effector and the target object's position, precise results cannot be obtained, often leading to a significant reduction in the accuracy of subsequent data processing results.

[0004] Currently, there is a lack of effective methods for calibrating the coordinate transformation between the camera coordinate system of a binocular vision system and any spatial rectangular coordinate system. Summary of the Invention

[0005] Purpose of the invention: To address the shortcomings and defects of existing technologies, this invention provides a coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system. By reconstructing the three-dimensional coordinates of the corner points of a checkerboard calibration board, a spatial rectangular coordinate system is constructed. The formulas for axis shifting and rotation transformations between the spatial rectangular coordinate system and the camera coordinate system are calculated to realize their coordinate transformation relationship, thereby improving the accuracy of the results.

[0006] Technical solution: The coordinate transformation calibration method of the binocular vision system camera and the spatial rectangular coordinate system of the present invention is characterized by the following steps:

[0007] 1) Design and fabricate a checkerboard calibration board, place the checkerboard calibration board in the binocular vision system, and acquire multiple left and right images of the checkerboard calibration board by moving and rotating the checkerboard calibration board. Use Zhang Zhengyou calibration method to calculate the intrinsic and extrinsic parameters of the left and right cameras.

[0008] 2) Place the checkerboard calibration board on a horizontal surface, acquire left and right images of the checkerboard calibration board, extract the pixel coordinates of each corner point in the left and right images of the checkerboard calibration board, and calculate the three-dimensional coordinates of each corner point in the left camera coordinate system based on the internal and external parameters of the left and right cameras obtained in step 1).

[0009] 3) Arbitrarily select a corner point of the chessboard calibration board as the base point, and based on the three-dimensional coordinates of each corner point in the left camera coordinate system obtained in step 2), construct an arbitrary spatial rectangular coordinate system and establish the coordinate transformation relationship between the camera coordinate system and the spatial rectangular coordinate system.

[0010] In step 1), the checkerboard calibration board is arranged in 7 rows and 8 columns with a spacing of 130mm. First, the checkerboard calibration board is moved to the first position and placed vertically. Left and right images of the calibration board are captured using the left and right cameras. Then, the calibration board is placed in forward tilt, backward tilt, left turn, and right turn positions, and left and right images are captured again for each position. The calibration board is then moved to the second position, and the above acquisition process is repeated until the next step. N One position.

[0011] Each movement interval shall not exceed 300mm, the rotation angle shall not exceed 30°, and the number of movements shall not exceed 6. It is ensured that the checkerboard calibration plate is within the field of view of the left and right cameras during each acquisition. Therefore, the intrinsic parameter matrices of the left and right cameras are respectively... and ,Right now: ; ;

[0012] The spatial transformation matrix from the right camera coordinate system to the left camera coordinate system is: ,Right now: .

[0013] In step 2), the Matlab function detectCheckerboardPoints() is used to extract the pixel coordinates of each corner point in the left and right images of the checkerboard calibration board. and ,in, l Indicates the left camera. r Indicates the right camera. m Indicates the number of columns ( m =1,…,8), n Indicates the number of rows ( n =1,..,7), then the first m List n The corner points of the line are in the left camera coordinate system The three-dimensional coordinates below are: .

[0014] Step 3) of constructing an arbitrary spatial rectangular coordinate system includes the following steps:

[0015] 31) Based on the obtained m List n 3D coordinates of the corner point in the left camera coordinate system The spatial plane containing the corner points of the chessboard calibration board is determined by polynomial fitting. The general equation is Ax + By + Cz + D =0;

[0016] 32) Let the base point of step 3) be the first point of the chessboard calibration board. m List n If the point is a corner point, then the origin of a spatial rectangular coordinate system can be constructed. O ,and Spatial rectangular coordinate system Z The axis passes through the origin. O And with the spatial plane Perpendicular, its direction vector can be represented as =( A , B , C );

[0017] 33) Select the chessboard grid calibration board. m +1 column n The corner point of the row can be obtained E ,and ,Pass OE It can construct a spatial rectangular coordinate system X The axis, whose direction vector can be represented as ;

[0018] 34) Passing the point O And with X axis, Z A perpendicular axis can be used to construct a spatial rectangular coordinate system. Y The axis, whose direction vector can be represented as Thus, a spatial rectangular coordinate system is constructed. O - XYZ .

[0019] Step 3) of establishing the coordinate transformation relationship between the camera coordinate system and the spatial rectangular coordinate system includes the following steps:

[0020] 35) Let spatial points P The coordinates in the left camera coordinate system are: In a Cartesian coordinate system O - XYZ The coordinates below are The origin of the spatial rectangular coordinate system obtained according to claim 4 O The formula for the inverse axis-shifting transformation from the left camera coordinate system to the spatial rectangular coordinate system can be obtained as follows: ;

[0021] 36) Left camera coordinate system axis, axis, The direction vectors of the axes can each be taken as their unit vectors. , , According to claim 4, the direction vectors of the X, Y, and Z axes of the spatial rectangular coordinate system can be used to obtain the angles between any two coordinate axes of the left camera coordinate system and the spatial rectangular coordinate system. Then, the formula for the inverse transformation of the rotation axis from the left camera coordinate system to the spatial rectangular coordinate system is: .

[0022] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0023] 1. This invention is based on a horizontally placed checkerboard calibration board. It acquires left and right images of the checkerboard calibration board, extracts the pixel coordinates of each corner point in the left and right images, and calculates the three-dimensional coordinates of each corner point in the left camera coordinate system. By arbitrarily selecting a corner point of the checkerboard calibration board as a base point, an arbitrary spatial Cartesian coordinate system can be constructed. This invention only needs to calculate the axis-shifting and axis-rotation transformation formulas between the spatial Cartesian coordinate system and the camera coordinate system to achieve the coordinate transformation relationship.

[0024] 2. The chessboard calibration plate of this invention is simple and easy to use, and can be printed, thus reducing costs. This invention can select any corner point of the chessboard as a base point to construct a spatial rectangular coordinate system. This invention has strong applicability. This invention is non-contact, the operation process is simple and convenient, and the calibration results have high accuracy and can meet the requirements of coordinate system transformation. Attached Figure Description

[0025] Figure 1 This is a flowchart of the present invention;

[0026] Figure 2 This is a schematic diagram of the chessboard calibration plate of the present invention;

[0027] Figure 3 This is a schematic diagram of the image acquisition position of the chessboard calibration board of the present invention;

[0028] Figure 4 The figure shows the three-dimensional coordinates and fitting plane diagram of the corner points of the chessboard grid according to the present invention; (a) is the three-dimensional coordinate diagram of the corner points of the chessboard grid, and (b) is the fitting plane diagram of the corner points of the chessboard grid.

[0029] In the image, 1 represents the vertical position; 2 represents the forward tilt position; 3 represents the backward tilt position; 4 represents the left turn position; 5 represents the right turn position; 6 represents the left camera; 7 represents the right camera; 8 represents the first position; 9 represents the second position; and 10 represents the third position. NOne position. Detailed Implementation

[0030] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0031] The coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system of the present invention includes the following steps:

[0032] 1) Design and fabricate a checkerboard calibration board, place the checkerboard calibration board in the binocular vision system, and acquire multiple left and right images of the checkerboard calibration board by moving and rotating it. Use Zhang Zhengyou calibration method to calculate the intrinsic and extrinsic parameters of the left and right cameras.

[0033] The checkerboard calibration board has 7 rows and 8 columns with a spacing of 130mm. First, move the calibration board to position 8 and place it in vertical position 1. Capture left and right images of the calibration board using the left camera 6 and right camera 7. Then, place the calibration board in forward tilt position 2, backward tilt position 3, left turn position 4, and right turn position 5, and capture left and right images of the calibration board again for each position. Next, move the calibration board to position 9 and repeat the above acquisition process until the next position. N 10 positions.

[0034] Each movement interval shall not exceed 300mm, the rotation angle shall not exceed 30°, and the number of movements shall not exceed 6. It is ensured that the checkerboard calibration plate is within the field of view of the left and right cameras during each acquisition. Therefore, the intrinsic parameter matrices of the left and right cameras are respectively... and ,Right now: ; ;

[0035] The spatial transformation matrix from the right camera coordinate system to the left camera coordinate system is: ,Right now: .

[0036] 2) Place the checkerboard calibration board on a horizontal surface, acquire left and right images of the checkerboard calibration board, extract the pixel coordinates of each corner point in the left and right images of the checkerboard calibration board, and calculate the three-dimensional coordinates of each corner point in the left camera coordinate system based on the internal and external parameters of the left and right cameras obtained in step 1).

[0037] The `detectCheckerboardPoints()` function in Matlab was used to extract the pixel coordinates of each corner point in the left and right images of the checkerboard calibration board, respectively. and ,in, l Indicates the left camera. r Indicates the right camera. m Indicates the number of columns ( m=1,…,8), n Indicates the number of rows ( n =1,..,7), then the first m List n The corner points of the line are in the left camera coordinate system The three-dimensional coordinates below are: .

[0038] 3) Arbitrarily select a corner point of the chessboard calibration board as the base point, and based on the three-dimensional coordinates of each corner point in the left camera coordinate system obtained in step 2), construct an arbitrary spatial rectangular coordinate system and establish the coordinate transformation relationship between the camera coordinate system and the spatial rectangular coordinate system.

[0039] Constructing an arbitrary spatial rectangular coordinate system includes the following steps:

[0040] 31) Based on the obtained m List n 3D coordinates of the corner point in the left camera coordinate system The spatial plane containing the corner points of the chessboard calibration board is determined by polynomial fitting. The general equation is Ax + By + Cz + D =0;

[0041] 32) Let the base point of step 3) be the first point of the chessboard calibration board. m List n If the point is a corner point, then the origin of a spatial rectangular coordinate system can be constructed. O ,and Spatial rectangular coordinate system Z The axis passes through the origin. O And with the spatial plane Perpendicular, its direction vector can be represented as =( A , B , C );

[0042] 33) Select the chessboard grid calibration board. m +1 column n The corner point of the row can be obtained E ,and ,Pass OE It can construct a spatial rectangular coordinate system X The axis, whose direction vector can be represented as ;

[0043] 34) Passing the point O And with X axis, ZA perpendicular axis can be used to construct a spatial rectangular coordinate system. Y The axis, whose direction vector can be represented as Thus, a spatial rectangular coordinate system is constructed. O - XYZ .

[0044] Establishing the coordinate transformation relationship between the camera coordinate system and the spatial rectangular coordinate system includes the following steps:

[0045] 35) Let spatial points P The coordinates in the left camera coordinate system are: In a Cartesian coordinate system O - XYZ The coordinates below are The origin of the spatial rectangular coordinate system obtained according to claim 4 O The formula for the inverse axis-shifting transformation from the left camera coordinate system to the spatial rectangular coordinate system can be obtained as follows: ;

[0046] 36) Left camera coordinate system axis, axis, The direction vectors of the axes can each be taken as their unit vectors. , , According to claim 4, the direction vectors of the X, Y, and Z axes of the spatial rectangular coordinate system can be used to obtain the angles between any two coordinate axes of the left camera coordinate system and the spatial rectangular coordinate system. Then, the formula for the inverse transformation of the rotation axis from the left camera coordinate system to the spatial rectangular coordinate system is: .

[0047] This invention is based on a horizontally placed checkerboard calibration board. It acquires left and right images of the calibration board, extracts the pixel coordinates of each corner point in the images, and calculates the 3D coordinates of each corner point in the left camera coordinate system. By arbitrarily selecting a corner point of the checkerboard calibration board as a base point, an arbitrary spatial Cartesian coordinate system can be constructed. This invention only requires calculating the axis-shifting and axis-rotation transformation formulas between the spatial Cartesian coordinate system and the camera coordinate system to achieve the coordinate transformation relationship.

[0048] The chessboard calibration plate of this invention is simple and easy to use, and can be printed, thus reducing costs. This invention can select any corner point of the chessboard as a base point to construct a spatial rectangular coordinate system. This invention has strong applicability. This invention is non-contact, the operation process is simple and convenient, and the calibration results have high accuracy and can meet the requirements of coordinate system transformation.

Claims

1. A coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system, characterized in that: Includes the following steps: 1) Design and fabricate a checkerboard calibration board, place the checkerboard calibration board in the binocular vision system, and acquire multiple left and right images of the checkerboard calibration board by moving and rotating the checkerboard calibration board. Use Zhang Zhengyou calibration method to calculate the intrinsic and extrinsic parameters of the left and right cameras. 2) Place the checkerboard calibration board on a horizontal surface, acquire left and right images of the checkerboard calibration board, extract the pixel coordinates of each corner point in the left and right images of the checkerboard calibration board, and calculate the three-dimensional coordinates of each corner point in the left camera coordinate system based on the internal and external parameters of the left and right cameras obtained in step 1). 3) Arbitrarily select a corner point of the chessboard calibration board as the base point, and based on the three-dimensional coordinates of each corner point in the left camera coordinate system obtained in step 2), construct an arbitrary spatial rectangular coordinate system and establish the coordinate transformation relationship between the camera coordinate system and the spatial rectangular coordinate system.

2. The coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system according to claim 1, characterized in that: In step 1), the checkerboard calibration plate has 7 rows and 8 columns with a spacing of 130mm. First, move the checkerboard calibration plate to the first position (8) and place it in a vertical position (1). Then, capture the left and right images of the checkerboard calibration plate once through the left camera (6) and the right camera (7). Then, place the checkerboard calibration plate in the forward tilt position (2), backward tilt position (3), left turn position (4), and right turn position (5), and capture the left and right images of the checkerboard calibration plate again. Then, move the checkerboard calibration plate to the second position (9) and repeat the above acquisition process until the 1st position. N 10 positions.

3. The coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system according to claim 2, characterized in that: Each movement interval shall not exceed 300mm, the rotation angle shall not exceed 30°, and the number of movements shall not exceed 6. It is ensured that the checkerboard calibration plate is within the field of view of the left and right cameras during each acquisition. Therefore, the intrinsic parameter matrices of the left and right cameras are respectively... and ,Right now: ; The spatial transformation matrix from the right camera coordinate system to the left camera coordinate system is: ,Right now: .

4. The coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system according to claim 1, characterized in that: In step 2), the Matlab function detectCheckerboardPoints() is used to extract the pixel coordinates of each corner point in the left and right images of the checkerboard calibration board, respectively. and ,in, l Indicates the left camera. r Indicates the right camera. m Indicates the number of columns ( m =1,…,8), n Indicates the number of rows ( n =1,..,7), then the first m List n The corner points of the line are in the left camera coordinate system The three-dimensional coordinates below are: .

5. The coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system according to claim 1, characterized in that: Step 3) involves constructing an arbitrary spatial rectangular coordinate system, which includes the following steps: 31) Based on the obtained m List n 3D coordinates of the corner point in the left camera coordinate system The spatial plane containing the corner points of the chessboard calibration board is determined by polynomial fitting. The general equation is Ax + By + Cz + D =0; 32) Let the base point of step 3) be the first point of the chessboard calibration board. m List n If the point is a corner point, then the origin of a spatial rectangular coordinate system can be constructed. O ,and Spatial rectangular coordinate system Z The axis passes through the origin. O And with the spatial plane Perpendicular, its direction vector can be represented as =( A , B , C ); 33) Select the chessboard grid calibration board. m +1 column n The corner point of the row can be obtained E ,and ,Pass OE It can construct a spatial rectangular coordinate system X The axis, whose direction vector can be represented as ; 34) Passing the point O And with X axis, Z A perpendicular axis can be used to construct a spatial rectangular coordinate system. Y The axis, whose direction vector can be represented as Thus, a spatial rectangular coordinate system is constructed. O - XYZ.

6. The coordinate transformation calibration method for a binocular vision system camera and a spatial rectangular coordinate system according to claim 1, characterized in that: Step 3) of establishing the coordinate transformation relationship between the camera coordinate system and the spatial rectangular coordinate system includes the following steps: 35) Let spatial points P The coordinates in the left camera coordinate system are: In a Cartesian coordinate system O - XYZ The coordinates below are The origin of the spatial rectangular coordinate system obtained according to claim 4 O The formula for the inverse axis-shifting transformation from the left camera coordinate system to the spatial rectangular coordinate system can be obtained as follows: ; 36) Left camera coordinate system axis, axis, The direction vectors of the axes can each be taken as their unit vectors. , , According to claim 4, the direction vectors of the X, Y, and Z axes of the spatial rectangular coordinate system can be used to obtain the angles between any two coordinate axes of the left camera coordinate system and the spatial rectangular coordinate system. Then, the formula for the inverse transformation of the rotation axis from the left camera coordinate system to the spatial rectangular coordinate system is: .

Images