A relative pose calibration method for binocular cameras based on light plane
By using a light plane and a line structured light laser in a binocular camera, adjusting the light plane to be parallel to the camera coordinate system to form a straight light stripe, and calculating the rotation and translation matrix, the calibration inaccuracy problem caused by the small depth of field in traditional methods is solved, thus improving the calibration accuracy.
Patent Information
- Application Number
- CN202210380420.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-12
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-04-12
AI Technical Summary
Traditional methods for relative pose calibration of binocular cameras are limited by the small depth of field of binocular cameras, making it impossible to achieve clear imaging of the calibration board at the same time, resulting in inaccurate calibration and affecting measurement accuracy.
The light plane method is used to place a line structured light laser between binocular cameras, adjust the light plane to be parallel to the camera coordinate system, and form a straight light stripe on the object surface. The rotation and translation matrix is calculated using the three-dimensional coordinates of the center point of the light stripe and the equation of the straight line to achieve relative pose calibration.
This reduces the requirement for a clear imaging range from both binocular cameras and improves the accuracy of relative pose calibration.
Smart Images

Figure CN114758010B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of machine vision measurement, in particular to a binocular camera relative posture calibration method. Background Art
[0002] Stereo vision sensors are often used to reconstruct three-dimensional objects. The sensor mainly includes two cameras with fixed relative postures (hereinafter referred to as binocular cameras), which obtain the three-dimensional geometric information of the object by taking two images of the object from different angles. Before use, the stereo vision sensor needs to calibrate the relative posture of the binocular cameras, that is, the rotation and translation matrix between the two camera coordinate systems of the binocular cameras. When calibrating with traditional methods, the binocular camera needs to clearly capture all the feature points on the calibration plate at the same time. Since the two main optical axes of the binocular camera usually have a certain angle, their focal planes do not overlap, such as Figure 1 As shown in the figure, when performing microscopic measurement with a large magnification, the binocular camera has a small depth of field and a small area that can be clearly imaged at the same time, making it impossible to clearly image the calibration plate at the same time, which will cause inaccurate relative pose calibration and affect the measurement accuracy. Summary of the invention
[0003] The purpose of the present invention is to provide a binocular camera relative posture calibration method based on a light plane, which reduces the requirement for the binocular camera's simultaneous clear imaging range when performing relative posture calibration and improves the calibration accuracy of the binocular camera's relative posture. The technical solution is as follows:
[0004] A method for relative position and posture calibration of a binocular camera based on a light plane comprises the following steps:
[0005] Step 1: Take the projection center O of camera 1 C1 Establish the camera coordinate system 1 for the origin, X C1 The Y axis is parallel to the column axis of the CCD target surface of camera 1. C1 The Z axis is parallel to the row axis of the CCD target surface and is determined by the right-hand rule. C1 Axis; the projection center O of camera 2 C2 Establish the camera coordinate system 2 for the origin, X C2 The Y axis is parallel to the column axis of the CCD target surface of camera 2. C2 The Z axis is parallel to the row axis of the CCD target surface and is determined by the right-hand rule. C2 Axis; perform internal calibration on the two cameras respectively to obtain the internal parameters of camera 1: principal distance f x1 、f y1 , principal point coordinates [u1 v1] T , Camera 2 internal parameters: principal distance f x2 、f y2 , principal point coordinates [u2 v2] T ;
[0006] Step 2: Place a line structured light laser between the binocular cameras. After turning on the line structured light laser, the laser projects a light plane. Adjust the position of the light plane so that it passes through the area where the binocular cameras simultaneously image clearly and is aligned with the Y coordinate of the camera coordinate system 1. C1 Axis parallel;
[0007] Step 3: Calibrate the relative poses of camera 1, camera 2 and the light plane respectively, and obtain the plane equations of the light plane in camera coordinate system 1 and camera coordinate system 2: A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0, where [A1 B1 C1] T , [A2 B2C2] T are the normal vectors of the light plane in camera coordinate system 1 and camera coordinate system 2 respectively;
[0008] Step 4: Place an object with a flat surface in the area where the light plane intersects with the binocular camera and is clearly imaged at the same time, forming the first straight light strip on the surface of the object;
[0009] Step 5: Camera 1 captures the light strip image and extracts the pixel coordinates of the center of the light strip in the image. The three-dimensional coordinates of the center point of the first straight light strip in camera coordinate system 1 are calculated using formula (1):
[0010]
[0011] Where [X E1 Y E1 ] T is the pixel coordinate of the center of the light bar in the image captured by camera 1, [X C1 Y C1 Z C1 ] T is the three-dimensional coordinate of the center point of the first linear light strip in camera coordinate system 1;
[0012] Step 6: C1 Y C1 Z C1 ] T Perform straight line fitting to obtain the straight line equation L of the first straight line light strip in camera coordinate system 1 11 :
[0013]
[0014] Where [X 11 Y 11 Z 11 ] T For L 11 The three-dimensional coordinates of the point passed, [a 11 b 11 c11 ] T For L 11 The direction vector of
[0015] Step 7: Repeat steps 5 and 6 for camera 2 to obtain the linear equation L of the first linear light strip in camera coordinate system 2. 21 :
[0016]
[0017] Where [X 21 Y 21 Z 21 ] T For L 21 The three-dimensional coordinates of the point passed, [a 21 b 21 c 21 ] T For L 21 The direction vector of
[0018] Step 8: Adjust the inclination angle of the flat surface object to form a second light strip, so that the second light strip intersects with the first light strip;
[0019] Step 9: Repeat steps 5, 6, and 7 to obtain the linear equation L of the second light strip in camera coordinate system 1 and camera coordinate system 2. 12 , L 22 :
[0020]
[0021] Where [X 12 Y 12 Z 12 ] T 、[X 22 Y 22 Z 22 ] T For L 12 , L 22 The three-dimensional coordinates of the point passed, [a 12 b 12 c 12 ] T 、[a 22 b 22 c 22 ] T For L 12 , L 22 The direction vector of
[0022] Step 10: Repeat steps 8 and 9 to obtain the linear equations L of N linear light strips in camera coordinate system 1 and camera coordinate system 2. 1i , L 2i, i=1~N:
[0023]
[0024] Where [X 1i Y 1i Z 1i ] T 、[X 2i Y 2i Z 2i ] T L 1i , L 2i The three-dimensional coordinates of the point passed, [a 1i b 1i c 1i ] T 、[a 2i b 2i c 2i ] T L 1i , L 2i The direction vector of
[0025] Step 11: Calculate L 1i The intersection coordinates [X Cross1j Y Cross1j Z Cross1j ] T and L 2i The intersection coordinates [X Cross2j Y Cross2j Z Cross2 j ] T , j = N(N-1) / 2;
[0026] Step 12: Put [A1 B1 C1] T , [A2 B2 C2] T 、[a 1i b 1i c 1i ] T 、[a 2i b 2i c 2i ] T Substitute (6) into the least squares solution to calculate the rotation matrix R from camera coordinate system 2 to camera coordinate system 1 3×3 :
[0027]
[0028] where [r1 r2 r3] T 、[r4 r5 r6] T 、[r7 r8 r9] T R3×3 1, 2, and 3 of ;
[0029] Step 13: Put [X Cross1j Y Cross1j Z Cross1j ] T 、[X Cross2j Y Cross2j Z Cross2j ] T Substitute (7) into the least squares solution to calculate the translation vector T from camera coordinate system 2 to camera coordinate system 1 3×1 :
[0030]
[0031] Where T 3×1 =[T x T y T z ] T ;
[0032] So far, we get the rotation and translation matrix [R 3×3 |T 3×1 ], realizing the calibration of the relative pose of the binocular camera.
[0033] The optical plane-based binocular camera relative posture calibration method provided by the present invention has the following beneficial effects: (1) reducing the requirement for the simultaneous clear imaging range of the binocular camera when performing relative posture calibration; and (2) improving the calibration accuracy of the relative posture of the binocular camera with a small depth of field. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a schematic diagram of the simultaneous clear imaging area of the binocular camera.
[0035] Figure 2 This is a schematic diagram of the relative pose calibration structure of a binocular camera based on the light plane.
[0036] Figure 3 It is a schematic diagram of the camera coordinate system.
[0037] Figure 4 It is a schematic diagram of adjusting objects to form new light strips.
[0038] Figure 5 This is a flowchart of the relative pose calibration of a binocular camera based on the light plane. DETAILED DESCRIPTION
[0039] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The best embodiment consists of the following steps:
[0040] Step 1: If Figure 3As shown, the projection center O of camera 1 C1 Establish the camera coordinate system 1 for the origin, X C1 The Y axis is parallel to the column axis of the CCD target surface of camera 1. C1 The Z axis is parallel to the row axis of the CCD target surface and is determined by the right-hand rule. C1 Axis; the projection center O of camera 2 C2 Establish the camera coordinate system 2 for the origin, X C2 The Y axis is parallel to the column axis of the CCD target surface of camera 2. C2 The Z axis is parallel to the row axis of the CCD target surface and is determined by the right-hand rule. C2 Axis; perform internal calibration on the two cameras respectively to obtain the internal parameters of camera 1: principal distance f x1 、f y1 , principal point coordinates [u1 v1] T , Camera 2 internal parameters: principal distance f x2 、f y2 , principal point coordinates [u2 v2] T ,The camera intrinsic calibration method can use the Zhang Zhengyou camera calibration method and other camera intrinsic calibration methods;
[0041] Step 2: If Figure 2 As shown in the figure, a line structured light laser is placed in the middle of the binocular camera. After the laser is turned on, the laser projects a light plane. The position of the light plane is adjusted so that it passes through the area where the binocular cameras simultaneously image clearly and is aligned with the Y coordinate of the camera coordinate system 1. C1 Axis parallel;
[0042] Step 3: Calibrate the relative poses of camera 1, camera 2 and the light plane respectively, and obtain the plane equations of the light plane in camera coordinate system 1 and camera coordinate system 2: A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0, where [A1 B1 C1] T , [A2 B2C2] Tare the normal vectors of the light plane in camera coordinate system 1 and camera coordinate system 2 respectively; the calibration method of the light plane can refer to relevant papers and patents, among which the steps of a simpler light plane calibration method are roughly as follows: first, place the checkerboard calibration plate in the clear imaging area of camera 1, and the light plane and the checkerboard intersect to form an intersection. The three-dimensional coordinates of the intersection in camera coordinate system 1 can be calculated through the invariance of the intersection ratio. The position of the calibration plate is changed to form an intersection at other positions of the light plane and its three-dimensional coordinates in camera coordinate system 1 are calculated. Since the intersections are all on the light plane, the plane equation of the light plane in camera coordinate system 1 can be obtained by plane fitting the three-dimensional coordinates of the intersection. The light plane calibration steps of camera 2 are the same as those of camera 1. Since the two cameras are calibrated for the light plane separately, the clarity of the image captured during the light plane calibration is only affected by the depth of field of a single camera, and is not constrained by the simultaneous clear imaging area of the binocular camera. This overcomes the disadvantage that the traditional method cannot simultaneously image the calibration plate clearly due to the small simultaneous clear imaging area of the binocular camera;
[0043] Step 4: Use an object with a relatively high surface flatness (a gauge block is used in this embodiment) to intersect the light plane in the area where the binocular cameras simultaneously clearly image, and form the first straight light strip on the surface of the object. Since the light strip is very thin, even if the binocular cameras simultaneously clearly image a small area, the light strip can be clearly imaged in both cameras at the same time;
[0044] Step 5: Camera 1 captures the light strip image and extracts the pixel coordinates of the center of the light strip in the image. The three-dimensional coordinates of the center point of the first straight light strip in camera coordinate system 1 are calculated using formula (1):
[0045]
[0046] Where [X E1 Y E1 ] T is the pixel coordinate of the center of the light bar in the image captured by camera 1, [X C1 Y C1 Z C1 ] T is the three-dimensional coordinate of the center point of the first linear light strip in camera coordinate system 1;
[0047] Step 6: C1 Y C1 Z C1 ] T Perform straight line fitting to obtain the straight line equation L of the first straight line light strip in camera coordinate system 1 11 :
[0048]
[0049] Where [X 11 Y 11Z 11 ] T For L 11 The three-dimensional coordinates of the point passed, [a 11 b 11 c 11 ] T For L 11 The direction vector of
[0050] Step 7: Repeat steps 5 and 6 for camera 2 to obtain the linear equation L of the first linear light strip in camera coordinate system 2. 21 :
[0051]
[0052] Where [X 21 Y 21 Z 21 ] T For L 21 The three-dimensional coordinates of the point passed, [a 21 b 21 c 21 ] T For L 21 The direction vector of
[0053] Step 8: Figure 4 As shown, the inclination angle of the flat surface object is adjusted to form a second light strip, so that the second light strip intersects with the first light strip;
[0054] Step 9: Repeat steps 5, 6, and 7 to obtain the linear equation L of the second light strip in camera coordinate system 1 and camera coordinate system 2. 12 , L 22 :
[0055]
[0056] Where [X 12 Y 12 Z 12 ] T 、[X 22 Y 22 Z 22 ] T For L 12 , L 22 The three-dimensional coordinates of the point passed, [a 12 b 12 c 12 ] T 、[a 22 b 22 c 22 ] T For L 12 , L22 The direction vector of
[0057] Step 10: Repeat steps 8 and 9 to obtain the linear equations L of N linear light strips in camera coordinate system 1 and camera coordinate system 2. 1i , L 2i (i=1~N):
[0058]
[0059] Where [X 1i Y 1i Z 1i ] T 、[X 2i Y 2i Z 2i ] T For L 1i , L 2i The three-dimensional coordinates of the point passed, [a 1i b 1i c 1i ] T 、[a 2i b 2i c 2i ] T For L 1i , L 2i The direction vector of
[0060] Step 11: Calculate L 1i The intersection coordinates [X Cross1j Y Cross1j Z Cross1j ] T and L 2i The intersection coordinates [X Cross2j Y Cross2j Z Cross2 j ] T (j=N(N-1) / 2);
[0061] Step 12: Put [A1 B1 C1] T , [A2 B2 C2] T 、[a 1i b 1i c 1i ] T 、[a 2i b 2i c 2i ] T Substitute (6) into the least squares solution to calculate the rotation matrix R from camera coordinate system 2 to camera coordinate system 1 3×3 :
[0062]
[0063] where [r1 r2 r3] T 、[r4 r5 r6] T 、[r7 r8 r9] T R 3×3 1, 2, and 3 of ;
[0064] Step 13: Put [X Cross1j Y Cross1j Z Cross1j ] T 、[X Cross2j Y Cross2j Z Cross2j ] T Substitute (7) into the least squares solution to calculate the translation vector T from camera coordinate system 2 to camera coordinate system 1 3×1 :
[0065]
[0066] Where T 3×1 =[T x T y T z ] T ;
[0067] So far, we get the rotation and translation matrix [R 3×3 |T 3×1 ], realizing the calibration of the relative pose of the binocular camera.
Claims
1. A method for relative position and posture calibration of a binocular camera based on a light plane, comprising the following steps: Step 1: Take the projection center O of camera 1 C1 Establish the camera coordinate system 1 for the origin, X C1 The Y axis is parallel to the column axis of the CCD target surface of camera 1. C1 The Z axis is parallel to the row axis of the CCD target surface and is determined by the right-hand rule. C1 Axis; the projection center O of camera 2 C2 Establish the camera coordinate system 2 for the origin, X C2 The Y axis is parallel to the column axis of the CCD target surface of camera 2. C2 The Z axis is parallel to the row axis of the CCD target surface and is determined by the right-hand rule. C2 Axis; perform internal calibration on the two cameras respectively to obtain the internal parameters of camera 1: principal distance f x1 、f y1 , principal point coordinates [u1 v1] T , Camera 2 internal parameters: principal distance f x2 、f y2 , principal point coordinates [u2 v2] T ; Step 2: Place a line structured light laser between the binocular cameras. After turning on the line structured light laser, the laser projects a light plane. Adjust the position of the light plane so that it passes through the area where the binocular cameras simultaneously image clearly and is aligned with the Y coordinate of the camera coordinate system 1. C1 Axis parallel; Step 3: Calibrate the relative poses of camera 1, camera 2 and the light plane respectively, and obtain the plane equations of the light plane in camera coordinate system 1 and camera coordinate system 2: A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0, where [A1 B1 C1] T , [A2 B2 C2] T are the normal vectors of the light plane in camera coordinate system 1 and camera coordinate system 2 respectively; Step 4: Place an object with a flat surface in the area where the light plane intersects with the binocular camera and is clearly imaged at the same time, forming the first straight light strip on the surface of the object; Step 5: Camera 1 captures the light strip image and extracts the pixel coordinates of the center of the light strip in the image. The three-dimensional coordinates of the center point of the first straight light strip in camera coordinate system 1 are calculated using formula (1): Where [X E1 Y E1 ] T is the pixel coordinate of the center of the light bar in the image captured by camera 1, [X C1 Y C1 Z C1 ] T is the three-dimensional coordinate of the center point of the first linear light strip in camera coordinate system 1; Step 6: C1 Y C1 Z C1 ] T Perform straight line fitting to obtain the straight line equation L of the first straight line light strip in camera coordinate system 1 11 : Where [X 11 Y 11 Z 11 ] T For L 11 The three-dimensional coordinates of the point passed, [a 11 b 11 c 11 ] T For L 11 The direction vector of Step 7: Repeat steps 5 and 6 for camera 2 to obtain the linear equation L of the first linear light strip in camera coordinate system 2. 21 : Where [X 21 Y 21 Z 21 ] T For L 21 The three-dimensional coordinates of the point passed, [a 21 b 21 c 21 ] T For L 21 The direction vector of Step 8: Adjust the inclination angle of the flat surface object to form a second light strip, so that the second light strip intersects with the first light strip; Step 9: Repeat steps 5, 6, and 7 to obtain the linear equation L of the second light strip in camera coordinate system 1 and camera coordinate system 2. 12 , L 22 : Where [X 12 Y 12 Z 12 ] T 、[X 22 Y 22 Z 22 ] T For L 12 , L 22 The three-dimensional coordinates of the point passed, [a 12 b 12 c 12 ] T 、[a 22 b 22 c 22 ] T For L 12 , L 22 The direction vector of Step 10: Repeat steps 8 and 9 to obtain the linear equations L of N linear light strips in camera coordinate system 1 and camera coordinate system 2. 1i , L 2i , i=1~N: Where [X 1i Y 1i Z 1i ] T 、[X 2i Y 2i Z 2i ] T L 1i , L 2i The three-dimensional coordinates of the point passed, [a 1i b 1i c 1i ] T 、[a 2i b 2i c 2i ] T L 1i , L 2i The direction vector of Step 11: Calculate L 1i The intersection coordinates [X Cross1j Y Cross1j Z Cross1j ] T and L 2i The intersection coordinates [X Cross2j Y Cross2j Z Cross2j ] T , j = N(N-1) / 2; Step 12: Put [A1 B1 C1] T , [A2 B2 C2] T 、[a 1i b 1i c 1i ] T 、[a 2i b 2i c 2i ] T Substitute (6) into the least squares solution to calculate the rotation matrix R from camera coordinate system 2 to camera coordinate system 1 3×3 : where [r1 r2 r3] T 、[r4 r5 r6] T 、[r7 r8 r9] T R 3×3 1, 2, and 3 of ; Step 13: Put [X Cross1j Y Cross1j Z Cross1j ] T 、[X Cross2j Y Cross2j Z Cross2j ] T Substitute (7) into the least squares solution to calculate the translation vector T from camera coordinate system 2 to camera coordinate system 1 3×1 : Where T 3×1 =[T x T y T z ] T ; So far, we get the rotation and translation matrix [R 3×3 |T 3×1 ], realizing the calibration of the relative pose of the binocular camera.
