A stripe structured light based single-line array camera three-dimensional calibration method

Abstract

The application discloses a kind of stripe structure light-based single line array camera three-dimensional calibration method, the relationship coefficient between image coordinates-phase-camera coordinates is calculated, and line array camera and projector are calibrated by simple process and lower cost, provide implementation scheme for high-precision three-dimensional measurement using line array camera, use the sawtooth pattern two-dimensional calibration plate of simple design, convenient production, under static pose, using line array camera shooting, both can provide enough high-precision feature points, and can avoid making three-dimensional calibration object and using high-precision displacement device, the method can increase the number of calibration plate pose, obtain more feature points to improve calibration precision, when specific operation, precision and speed between the balance of actual requirement is obtained, it has certain robustness, the relationship between line array camera and calibration plate, only requires that calibration plate is located in the imaging range of camera, and the placement mode of calibration plate has certain tolerance.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional measurement, and in particular, is a three-dimensional calibration method for a single-line array camera based on stripe structured light. Background Art

[0002] Line scan cameras are representative of linear array image sensors, with their imaging pixels arranged in a one-dimensional pattern. Compared to widely used area scan cameras, they easily achieve higher lateral resolution and faster acquisition speeds at the same process level. They can be used for real-time 3D topography measurement under high-speed motion conditions, acquiring large, high-density point clouds. In practical applications, camera calibration is necessary to determine the relationship between the spatial coordinates of object points and corresponding points in the image and achieve 3D measurement. As a key technology in machine vision, camera calibration accuracy is directly related to 3D measurement accuracy. While area scan camera calibration techniques are already diverse and mature, high-precision calibration solutions are still limited for line scan cameras due to their unique imaging method.

[0003] The Horaud dynamic calibration method uses a linear imaging model and uses a precise displacement platform to move a two-dimensional calibration plate multiple times in a specific direction to achieve feature point acquisition and calibration; the Luna static calibration method uses a three-dimensional calibration object, avoiding the need for a precise displacement platform; the Li calibration method introduces an area array camera, obtains feature points, and then uses Zhang Zhengyou's area array camera calibration method to achieve joint calibration.

[0004] The existing 3D calibration methods for linear array cameras have the following technical problems:

[0005] 1. Static method: Relying on a three-dimensional calibration object of a specific shape, with strict production accuracy requirements;

[0006] 2. Dynamic method: requires a high-precision displacement platform, which is costly and difficult to control precision;

[0007] 3. Joint calibration using area array cameras: The process is more complicated and the cost is higher;

[0008] 4. Structured light method: The projector optical axis is required to be parallel to the camera optical axis, and the projector optical axis is required to be parallel to the camera optical axis, and extremely high installation accuracy is required. Summary of the Invention

[0009] The purpose of the present invention is to address the shortcomings of the existing technology and propose a three-dimensional calibration method for a single line array camera based on stripe structured light, which can achieve high accuracy, simple calibration process and low cost.

[0010] The present invention is achieved through the following technical solutions:

[0011] The present invention discloses a three-dimensional calibration method for a single-line array camera based on stripe structured light. The calibration method is implemented by the following devices:

[0012] Line scan camera, used to capture images;

[0013] a projector for projecting sinusoidal fringes;

[0014] a computer for processing images from the line array camera and providing fringe images to the projector;

[0015] The calibration plate is used to provide feature points, and its sawtooth pattern is composed of several triangles arranged in an array;

[0016] The computer is connected to the line array camera and the projector respectively;

[0017] The shooting direction of the line array camera is consistent with the projection direction of the projector;

[0018] The calibration plate is located within the imaging range of the linear array camera;

[0019] The calibration method includes the following steps:

[0020] A two-dimensional calibration plate with a sawtooth pattern is used to capture images at different positions using a linear array camera. The imaging surface of the linear array camera intersects the calibration plate on a straight line to obtain a first calibration image.

[0021] For the first calibration image, k feature points (Q1, Q2, ..., Q k ); Take the leftmost end of the image as the origin and get the coordinates of each feature point in the image coordinate system (m1, m2, ..., m k ), in pixels;

[0022] For each feature point's image coordinates (m1, m2, ..., m k ) and make a difference between them, and get the length of k-1 intervals (Δ i =m i+1 -m i ,i=1,2,…,k-1), in pixels;

[0023] According to the interval length Δ i The ratio of each feature point on the calibration plate is obtained (x i ,y i ,i=1,2,…,k-1), in mm;

[0024] For each feature point, calculate the distance to the leftmost feature point Q1, establish a calibration plate coordinate system with Q1 as the origin and the straight line where the feature point is located as the axis, and obtain the calibration plate coordinate system coordinate of the feature point (x bi =((x i -x1)2 +

[0025] (y i -y1) 2 ) 1 / 2 ,i=1,2,…,k-1), in mm;

[0026] The image coordinate system coordinates m of each feature point i , calibration plate coordinate system coordinate x bi , solve the transformation coefficient matrix G between the image coordinate system and the camera coordinate system;

[0027] The coordinates (x ci ,z ci );

[0028] Keeping the calibration plate in a fixed position, a projector is used to project sinusoidal stripes onto the calibration plate. The stripe direction is perpendicular to the linear array camera pixels. The linear array camera is used to capture the second calibration image.

[0029] For the second calibration image, use the phase shift method or Fourier method to solve each feature point Q i The phase φ at i ;

[0030] Change the calibration plate posture and repeat the above process;

[0031] For each feature point Q i , by obtaining the camera coordinate system coordinates (x ci ,x ci ) and phase φ i , solve the relationship coefficient a between the phase and the coordinate in the camera coordinate system i ;

[0032] Through the relationship coefficient a i , and the coordinates (x ci ,z ci ).

[0033] As a further improvement, the device of the present invention further includes a mounting bracket for fixing the line array camera and the projector.

[0034] As a further improvement, the present invention provides the i The ratio of each feature point on the calibration plate is obtained, specifically:

[0035] Assume that the long side of each triangle is a, the short side is b, and the unit is mm; establish a plane rectangular coordinate system with the right-angled vertex of the leftmost black triangle as the origin, the short side direction as the x-axis, and the long side direction as the y-axis;

[0036] For feature point Q2, its coordinates (x2, y2) in mm satisfy the following relationship:

[0037]

[0038] For the remaining feature points on the hypotenuse of the triangle (Q i ,i=2,4,…,k), all have similar relationships, and the solution is (x i ,y i ,i=2,4,…,k);

[0039] For each feature point on the hypotenuse, the least square method is used to fit the straight line to obtain the feature points on each right-angled edge (Q i ,i=1,3,…,k-1) position (x i ,y i ,i=1,3,…,k-1).

[0040] As a further improvement, the image coordinate system coordinate m of each feature point in the present invention i , calibration plate coordinate system coordinate x bi , the transformation coefficient matrix G between the image coordinate system and the camera coordinate system is solved, including:

[0041] The image coordinate system and the camera coordinate system have the following relationship, where x c ,z c is the coordinate in the camera coordinate system, in mm; m is the coordinate in the image coordinate system, in pixels; f x is the pixel focal length of the linear array camera in the x direction, m0 is the offset of the optical axis of the linear array camera in the image coordinate system, and these two constitute the internal parameter matrix of the camera, which is recorded as A c , in pixels:

[0042]

[0043] When the calibration plate is in a certain position, there is a certain rotation and translation relationship between it and the linear array camera, which can be described by the transformation matrix, where θ is the rotation angle and t1 and t2 are the translation amounts:

[0044]

[0045] From the above relationship, we can get the image coordinate system m and the calibration plate coordinate system x b The relationship between the image coordinate system m and the camera coordinate system (x c ,z c )

[0046]

[0047] g i(i=1,2,3,4) form the transformation coefficient matrix G. We get the equation mg1+g2-mx b g3-x b g4=0. Set the image coordinate system coordinates of each feature point m i The coordinate x of the calibration plate coordinate system bi Substituting, we get an overdetermined homogeneous linear system of equations;

[0048] Notice Among them, cos 2 θ+sin 2 θ=1 is the constraint condition of the overdetermined homogeneous linear equations, and the transformation coefficient matrix G is solved.

[0049] As a further improvement, the present invention describes the i , by obtaining the camera coordinate system coordinates (x ci ,z ci ) and phase φ i , solve the relationship coefficient a between the phase and the camera coordinate system i ,include:

[0050] Coordinates in the camera coordinate system (x c ,z c ) has the following relationship with the phase φ:

[0051]

[0052] For the relationship, transform it into: x c a1+z c a2+a3-φx c a4-φz c a5-φa6=0;

[0053] Substitute at least 6 feature points obtained from the calibration plate in at least two positions into the transformed relationship to obtain an overdetermined homogeneous linear equation system. Use the least squares method as a constraint to solve a i (i=1,2,…,6).

[0054] As a further improvement, the present invention provides a relationship coefficient a i , and the coordinates (x ci ,z ci ),include:

[0055] According to the relationship between the image coordinate system and the camera coordinate system, and the relationship between the phase and the camera coordinate system, we can get the phase φ-image coordinate m-camera coordinate (x ci ,z ci )

[0056]

[0057] The beneficial effects of the present invention are as follows:

[0058] 1. This method calculates the relationship coefficients between image coordinates, phase, and camera coordinates. It calibrates line scan cameras and projectors through a simple process and at a low cost, providing a solution for high-precision 3D measurement using line scan cameras.

[0059] 2. This method uses a simple and easy-to-manufacture 2D calibration plate with a sawtooth pattern, and uses a linear array camera to capture the image in a static position. This not only provides sufficient high-precision feature points, but also avoids the need to produce 3D calibration objects and use high-precision displacement devices.

[0060] 3. This method requires at least two calibration images of the calibration plate in two different poses. Compared with the joint calibration method using an area array camera, it greatly simplifies the calibration process and achieves rapid calibration.

[0061] 4. Compared with existing structured light methods, this method does not require the line connecting the projection center and the camera optical center to be parallel to the projection surface, nor does it require the projector optical axis to be parallel to the camera optical axis, which greatly reduces the installation requirements of the projector and camera;

[0062] 5. This method can improve the calibration accuracy by increasing the number of calibration plate poses and obtaining more feature points. The specific operation allows for a balance between accuracy and speed according to actual requirements.

[0063] 6. This method has a certain degree of robustness. The relationship between the linear array camera and the calibration plate only requires that the calibration plate be located within the camera's imaging range. The placement of the calibration plate has a certain degree of tolerance.

[0064] 7. This method has certain scalability and can theoretically be used to calibrate multi-line array camera-projector systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flow chart of the calibration method of the present invention;

[0066] Figure 2 It is a schematic diagram of the structure of the device on which the calibration method of the present invention is implemented;

[0067] Figure 3 It is a schematic diagram of the calibration plate pattern style;

[0068] Figure 4 is a schematic diagram of the calibration operation;

[0069] Figure 5 It is a schematic diagram of the intersection between the imaging surface of the linear array camera and the calibration plate;

[0070] In the figure, 1. mounting bracket 2. line array camera 3. projector 4. computer 5. storage surface 6. calibration plate. DETAILED DESCRIPTION

[0071] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0072] The present invention discloses a 2D and 3D calibration method for a single line array camera based on stripe structured light. The calibration method is implemented by the following technical solutions:

[0073] Linear array camera 2, used for capturing images;

[0074] Projector 3, used for projecting sinusoidal fringes;

[0075] Computer 4, used for processing the image of line array camera 2 and providing fringe image to projector 3;

[0076] Calibration plate 6, used to provide feature points, its sawtooth pattern is composed of a number of triangles arranged;

[0077] The computer 4 is connected to the line array camera 2 and the projector 3 respectively;

[0078] The shooting direction of the line array camera 2 is consistent with the projection direction of the projector 3;

[0079] The calibration plate 6 is located within the imaging range of the linear array camera 2;

[0080] In a specific embodiment, a mounting frame 1 is used to fix the projector 3 and the line scan camera 2. The projector 3 and the line scan camera 2 are fixed to the beam of the mounting frame 1. The projection direction of the projector 3 is the same as the shooting direction of the line scan camera 2. Figure 2 It is not required that the line connecting the projection center and the optical center of the line scan camera 2 be parallel to the projection plane.

[0081] In a specific embodiment, the production Figure 3 The plane sawtooth pattern calibration plate 6 shown is composed of 8 black right-angled triangles, the longer right-angled side of the triangle is 20 mm long and the shorter right-angled side is 5 mm long.

[0082] The present invention also discloses a 2D calibration method for a single linear array camera based on stripe structured light, the process is as follows: Figure 1 As shown, the following steps are included:

[0083] 1. Place the calibration plate 6 on the object plane 5 and shoot within the imaging range of the line array camera 2, as shown in the figure. Figure 4 The imaging surface of the linear array camera 2 intersects with the eight triangles on a straight line, as shown in Figure 5 ;

[0084] 2. From the intersection of the straight line and the right angle and hypotenuse of each triangle, 16 feature points (Q1, Q2, ..., Q 16 ). Consider the leftmost end of the image as the origin and obtain the image coordinates of 16 points (m1, m2, ..., m 16 ), in pixels;

[0085] 3. Subtract the image coordinates of each point and get the length of 15 black and white intervals (Δ i =m i+1 -m i ,i=1,2,…,15), in pixels;

[0086] 4. Based on the length ratio of each adjacent black and white interval, the position of each feature point on the calibration plate 6 can be obtained. Take the feature point Q2 on the hypotenuse of the leftmost black triangle as an example: With the leftmost black triangle right-angle vertex as the origin, the short side as the x-axis, and the long side as the y-axis, a plane rectangular coordinate system is established. Then the coordinates (x2, y2) of Q2 in mm satisfy the following relationship:

[0087]

[0088] By analogy, the characteristic points on each oblique edge (Q i ,i=2,4,…,16) position (x i ,y i ,i=2,4,…,16). By fitting the straight line using the least square method, we can get the characteristic points (Q i ,i=1,3,…,15) position (x i ,y i ,i=1,3,…,15);

[0089] 5. Calculate the distance between Q1 and other points, establish the calibration plate 6 coordinate system with q1 as the origin and the straight line where each feature point is located as the axis, and obtain the calibration plate 6 coordinate system coordinate (x bi =((x i -x1) 2 +(y i -y1) 2 ) 1 / 2 ,i=1,2,…,16). All the above are in mm;

[0090] 6. The relationship between the line scan camera 2 coordinate system and the image coordinate system of the line scan camera 2 is as follows, where x c ,z c is the coordinate in the coordinate system of the linear array camera 2, in mm; m is the coordinate in the image coordinate system, in pixels; f xis the pixel focal length of the line array camera 2 in the x direction, m0 is the offset of the optical axis of the line array camera 2 in the image coordinate system, and these two constitute the internal parameter matrix of the line array camera 2, denoted as A c , in pixels:

[0091]

[0092] When the calibration plate 6 is in a certain position, there is a certain rotation and translation relationship between it and the linear array camera 2, which can be described by the transformation matrix:

[0093]

[0094] Based on the above relationship, the relationship between the image coordinate system and the calibration plate 6 coordinate system, and the relationship between the image coordinate system and the line array camera 2 coordinate system can be obtained through matrix transformation:

[0095]

[0096] g i (i=1,2,3,4) form the transformation coefficient matrix G. We get the equation mg1+g2-mx b g3-x b g4=0. i ,x bi (i=2,3,…,16) into the equations, and we get an overdetermined homogeneous linear system. Note that Among them, cos 2 θ+sin 2 θ=1 is the constraint condition of the overdetermined homogeneous linear equations, and the matrix G is solved;

[0097] 7. For each feature point, the coordinates (x ci ,z ci );

[0098] 8. Keep the calibration plate 6 in the same position, use projector 3 to project the sinusoidal fringe pattern, and use line array camera 2 to capture it. The fringe direction is perpendicular to the pixel arrangement direction of line array camera 2.

[0099] 9. Use phase shift method or Fourier transform method to solve the phase φ of each feature point on the image i .

[0100] 10. Change the calibration plate 6 pose and repeat steps 3 to 6 to obtain another set of 16 feature points of the linear array camera 2 coordinates (x ci ,z ci ) and phase φ i ;

[0101] 11. The relationship between the phase and the coordinates of line array camera 2 is as follows:

[0102]

[0103] Right now:

[0104] x c a1+z c a2+a3-φx c a4-φz c a5-φa6=0#(7)

[0105] The phases of 32 feature points and the coordinates of the linear array camera 2 (φ i ,x ci ,z ci ,i=1,2,…,32) into equation (7) to obtain an overdetermined homogeneous linear system of equations. Use the least squares method to solve a i (i=1,2,…,6).

[0106] 12. Based on equations (2) and (6), the relationship between phase, image coordinates, and line array camera 2 coordinates is established as follows:

[0107]

[0108] So far, the coordinates of the linear array camera 2 coordinate system (x c ,z c ), calibration is completed and can be used for structured light 3D reconstruction.

[0109] The above is not a limitation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several changes, modifications, additions or substitutions can be made without departing from the essential scope of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A three-dimensional calibration method for a single-line array camera based on stripe structured light, characterized in that: The calibration method is implemented by the following devices: Line scan camera, used to capture images; a projector for projecting sinusoidal fringes; a computer, configured to process the image from the line array camera and provide a fringe image for the projector; The calibration plate is used to provide feature points, and its sawtooth pattern is composed of several triangles arranged in an array; The computer is connected to the line array camera and the projector respectively; The shooting direction of the line array camera is consistent with the projection direction of the projector; The calibration plate is located within the imaging range of the linear array camera; The calibration method comprises the following steps: A two-dimensional calibration plate with a sawtooth pattern is used to capture images at different positions using a linear array camera. The imaging surface of the linear array camera intersects the calibration plate on a straight line to obtain a first calibration image. For the first calibration image, k feature points (Q1, Q2, ..., Q k ); Taking the leftmost end of the image as the origin, obtain the coordinates of each feature point in the image coordinate system (m1, m2, ..., m k ), in pixels; The image coordinates of each feature point (m1, m2, ..., m k ) and make a difference between them, and get the length of k-1 intervals (Δ i =m i+1 -m i ,i=1,2,…,k-1), in pixels; According to the interval length Δ i The ratio of each feature point on the calibration plate is obtained (x i ,y i ,i=1,2,…,k-1), in mm; For each feature point, calculate the distance to the leftmost feature point Q1, establish a calibration plate coordinate system with Q1 as the origin and the straight line where the feature point is located as the axis, and obtain the calibration plate coordinate system coordinate (x bi =((x i -x1) 2 +(y i -y1) 2 ) 1 / 2 ,i=1,2,…,k-1), in mm; The image coordinate system coordinates m of each feature point i , calibration plate coordinate system coordinate x bi , solve the transformation coefficient matrix G between the image coordinate system and the camera coordinate system; The coordinates (x ci ,z ci ); Keeping the calibration plate in a constant position, a projector is used to project sinusoidal stripes onto the calibration plate, with the stripe direction being perpendicular to the line array camera pixels, and the line array camera is used to capture a second calibration image; For the second calibration image, use the phase shift method or Fourier method to solve each feature point Q i The phase φ at i ; Change the calibration plate posture and repeat the above process; For each feature point Q i , by obtaining the camera coordinate system coordinates (x ci ,z ci ) and phase φ i , solve the relationship coefficient a between the phase and the coordinate in the camera coordinate system i ; By the relationship coefficient a i , and the coordinates (x ci ,z ci ).

2. The 3D calibration method for a single line array camera based on stripe structured light according to claim 1, characterized in that: The device also includes a mounting frame for fixing the line array camera and the projector.

3. The 3D calibration method for a single line array camera based on stripe structured light according to claim 2, characterized in that: The length of the interval Δ i The ratio of each feature point on the calibration plate is obtained, specifically: Assume that the long side of each triangle is a, the short side is b, and the unit is mm; establish a plane rectangular coordinate system with the right-angled vertex of the leftmost black triangle as the origin, the short side direction as the x-axis, and the long side direction as the y-axis; For feature point Q2, its coordinates (x2, y2) in mm satisfy the following relationship: For the remaining feature points on the hypotenuse of the triangle (Q i ,i=2,4,…,k), all have similar relationships, and the solution is (x i ,y i ,i=2,4,…,k); For the feature points on the hypotenuse, the least square method is used to fit the straight line to obtain the feature points on each right-angled edge (Q i ,i=1,3,…,k-1) position (x i ,y i ,i=1,3,…,k-1).

4. The 3D calibration method for a single line array camera based on stripe structured light according to claim 3, characterized in that: The image coordinate system coordinates m of each feature point i , calibration plate coordinate system coordinate x bi , the transformation coefficient matrix G between the image coordinate system and the camera coordinate system is obtained, including: The image coordinate system and the camera coordinate system have the following relationship, where x c ,z c is the coordinate in the camera coordinate system, in mm; m is the coordinate in the image coordinate system, in pixels; f x is the pixel focal length of the linear array camera in the x direction, m0 is the offset of the optical axis of the linear array camera in the image coordinate system, and these two constitute the internal parameter matrix of the camera, which is recorded as A c , in pixels: When the calibration plate is in a certain position, there is a certain rotation and translation relationship between it and the linear array camera, which can be described by the transformation matrix, where θ is the rotation angle and t1 and t2 are the translation amounts: From the above relationship, we can get the image coordinate system m and the calibration plate coordinate system x b The relationship between the image coordinate system m and the camera coordinate system (x c ,z c ) g i (i=1,2,3,4) form the transformation coefficient matrix G, and we get the equation mg1+g2-mx b g3-x b g4=0,then the image coordinate system coordinates of each feature point m i The coordinate x of the calibration plate coordinate system bi Substituting, we get an overdetermined homogeneous linear system of equations; Notice Among them, cos 2 θ+sin 2 θ=1 is the constraint condition of the overdetermined homogeneous linear equations, and the transformation coefficient matrix G is solved.

5. The 3D calibration method for a single line array camera based on stripe structured light according to claim 4, characterized in that: For each feature point Q i , by obtaining the camera coordinate system coordinates (x ci ,z ci ) and phase φ i , solve the relationship coefficient a between the phase and the camera coordinate system i ,include: Coordinates in the camera coordinate system (x c ,z c ) has the following relationship with the phase φ: For the relationship, the transformation is: x c a1+z c a2+a3-φx c a4-φz c a5-φa6=0; Substitute at least 6 feature points obtained from the calibration plate in at least two positions into the transformed relationship to obtain an overdetermined homogeneous linear equation system, and use the least squares method as a constraint to solve a i (i=1,2,…,6).

6. The 3D calibration method for a single line array camera based on stripe structured light according to claim 5, characterized in that: By the relationship coefficient a i , and the coordinates (x ci ,z ci ),include: According to the relationship between the image coordinate system and the camera coordinate system, and the relationship between the phase and the camera coordinate system, the phase φ - the image coordinate m - the camera coordinate (x ci ,z ci )

Images