A line structured light plane calibration method based on a two-dimensional circular target

Abstract

The present application relates to the technical field of non-contact three-dimensional measurement, in particular to a line structured light plane calibration method based on a two-dimensional circular target. The present application uses a light strip extraction algorithm to obtain sub-pixel coordinate values of a light strip irradiated at any position of a two-dimensional circular calibration target; then all points on the light strip are fitted into a straight line, and each column of target mark connecting lines intersects at a point; after obtaining the intersection point of each column of target mark connecting lines and the fitted light strip center line, a straight line is formed by connecting the intersection point and the optical center of the camera; by simultaneously solving the straight line equation, the two straight line equations in the pixel coordinate system, and the plane equation of the two-dimensional circular calibration target, the coordinates of each intersection point in the camera coordinate system can be solved; finally, a plane fitting is performed using a least squares fitting algorithm to obtain the calculated light plane. The light plane obtained by using the present application has high precision and high robustness, and the calibration process is very simple, which is suitable for general engineering applications and mechanical processing processes, and has very important application value.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of non-contact three-dimensional measurement, in particular to a line structured light plane calibration method based on a two-dimensional circular target. BACKGROUND

[0002] In the field of non-contact three-dimensional measurement, line structured light sensors have the advantages of simple structure, high precision and fast speed. Therefore, it is also widely used in industrial measurement and three-dimensional reconstruction of point clouds. The light plane calibration method is the key research content of line structured light three-dimensional measurement technology.

[0003] Only accurate calibration of the line structured light plane can make the three-dimensional measurement accuracy meet the requirements. This research has undergone many technical changes since the 1980s, from the traditional wire drawing method to determine the target point of the structured light plane to fit the light plane, to the use of a specific shape of three-dimensional calibration object to determine the light plane, and to the current two-dimensional and one-dimensional targets. Therefore, accurate calibration of the light plane is the basis for the accuracy requirements of the line structured light sensor.

[0004] Under the prior art, when using a traditional two-dimensional target for light plane calibration, there are the disadvantages of low light plane precision, low robustness, and cumbersome calibration process.

[0005] The existing technology often uses a fixed distance between the intersection points to calculate the camera coordinate values of each intersection point using the intersection ratio invariance principle. This method depends on the machining accuracy of the two-dimensional target and the distortion of the camera lens. Or want to use more feature points to fit the light plane, so all the center points on the light bar are used without distinction to calculate the camera coordinate system values. Obviously, this method will use some noise points on the light plane to fit the light plane, resulting in poor results. SUMMARY

[0006] To solve the problems of the prior art, the present application provides a line structured light plane calibration method based on a two-dimensional circular target, which has high light plane precision, high robustness, and a very simple calibration process, and is suitable for general engineering applications and mechanical processing processes, and has very important application value.

[0007] The technical solutions adopted by the present application are as follows:

[0008] A line structured light plane calibration method based on a two-dimensional circular target, comprising the following steps:

[0009] A. Take pictures of the calibration target at different positions in the camera field of view;

[0010] B. Calibrate the monocular camera to obtain the camera internal and external parameters and calculate the equation of the plane where each two-dimensional circular calibration target is located in the camera coordinate system;

[0011] C. Subtracting the two calibration target pictures with and without laser light strips to obtain the light strip center of the light strip region;

[0012] D. Linear fitting the light strip center to obtain a linear equation, and solving the expression of the intersection point of the calibration target circle point and the linear equation;

[0013] E. Solving the coordinate values of the intersection point in the camera coordinate system by simultaneously solving the equation set;

[0014] F. After obtaining the camera coordinate system values of multiple intersection points by moving the calibration target multiple times, fitting the light plane using the least square method.

[0015] Preferably, in step C, the specific method for obtaining the light strip center is to use a light strip extraction algorithm to obtain the light strip sub-pixel coordinate values irradiated on the two-dimensional circular calibration target at any position.

[0016] Preferably, step D specifically includes: then fitting all the light strip center points into a straight line, intersecting each column of target markers with a point to obtain the intersection point of each column of target markers and the fitted light strip center line, and connecting the intersection point and the camera optical center to form a straight line.

[0017] Preferably, step E specifically includes:

[0018] Using the obtained intersection point, connecting the camera optical center, and combining the previously obtained equation of the two-dimensional circular calibration target plane in the camera coordinate system to solve the coordinate values of the intersection point in the camera coordinate system.

[0019] The linear structured light plane calibration method based on a two-dimensional circular target can improve the accuracy and efficiency of linear structured light plane calibration, improve the universality of the linear structured light plane calibration method, and the method also verifies the effectiveness by building an experimental system, and the results show that the light plane precision obtained by using the method is high, the robustness is high, and the calibration process is very simple, which is suitable for general engineering application and mechanical processing process, and has very important application value.

[0020] In addition, the method uses all points on the light strip to fit a straight line, which can better optimize the errors caused by environmental light, laser strip light spot noise, lens distortion and calibration target processing precision, so that the calibration result has higher accuracy and stronger anti-interference ability. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0022] Figure 1 In the line structure light plane calibration method based on a two-dimensional circular target of the present application, a schematic diagram of the relationship between the camera imaging model and the coordinate systems is shown.

[0023] Figure 2 In the line structure light plane calibration method based on a two-dimensional circular target of the present application, a schematic diagram of the light strip center extraction result is shown, wherein, Figure 2 (a) is a state diagram of the light strip of the line structure light irradiated on the calibration target; Figure 2 (b) is a schematic diagram of the light strip center result extracted by using the light strip center extraction algorithm;

[0024] Figure 3 In the line structure light plane calibration method based on a two-dimensional circular target of the present application, a schematic diagram of the intersection of the calibration target circle center connecting line and the light strip is shown, wherein, Figure 3 (a) is a schematic diagram of each circle center being connected into a straight line; Figure 3 (b) is a schematic diagram of the intersection point of the circle center connecting line and the light strip center line;

[0025] Figure 4 In the line structure light plane calibration method based on a two-dimensional circular target of the present application, a perspective projection model diagram of the intersection point of the two lines on the calibration plate is shown.

[0026] Figure 5 In the line structure light plane calibration method based on a two-dimensional circular target of the present application, a schematic diagram of the experimental system structure is shown.

[0027] Figure 6 In the line structure light plane calibration method based on a two-dimensional circular target of the present application, a system calibration method flowchart is shown.

[0028] Figure 7 In the line structure light plane calibration method based on a two-dimensional circular target of the present application, a schematic diagram of using the light strip center extraction algorithm is shown.

[0029] Figure 8 In the line structure light plane calibration method based on a two-dimensional circular target of the present application, a schematic diagram of the displacement of the gauge block in different positions is shown. DETAILED DESCRIPTION

[0030] In order to make the objects, technical solutions and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the drawings.

[0031] The method comprises the following steps:

[0032] 1. Camera calibration and relationship between coordinate systems

[0033] Before the line structured light plane calibration, the camera needs to be calibrated for internal and external parameters. Here, the camera is regarded as a pinhole imaging model, by which the relationship between a point A (X w ,Y w ,Z w ) in the world coordinate system and the camera optical center can be obtained, as shown in Figure 1

[0034] In Figure 1 , the world coordinate system is described as O w -X w Y w Z w , the camera coordinate system is described as O wc -X c Y c Z c , the image coordinate system is described as O-xy, and the pixel coordinate system is described as O-uv. Therefore, the conversion of the point A (X w ,Y w ,Z w ) in the world coordinate system to the camera coordinate system A c (X c ,Y c ,Z c ) can be obtained by rigid transformation, as follows,

[0035]

[0036] wherein R is a rotation matrix describing the rigid transformation, and T is a translation vector; the point A c can be converted from the camera coordinate system to the image coordinate system O-xy, as follows,

[0037]

[0038] wherein f is the focal length. When the image coordinate system is converted to the pixel coordinate system, it should be noted that the pixel coordinate origin is usually represented at the top left corner of the picture, while the image coordinate origin is in the middle of the picture, so the conversion relationship between the image coordinate system and the pixel coordinate system is,

[0039]

[0040] ​d in the formula x , d y represents the pixel size of the pixel element, u0, v0 represents the offset of the principal point position in the pixel coordinate system. Get the conversion relationship between each coordinate system, so that monocular camera calibration can be carried out, that is, the required camera internal and external parameter matrix can be obtained.

[0041] 2. Equation solving of the calibration target in the camera coordinate system

[0042] Suppose there is a point B (X w1 , Y w1 , Z w1 ) on the calibration target plane in the world coordinate system, and the world coordinate system is established on the calibration target plane, then the plane equation in the world coordinate system where the calibration target is located can be described as,

[0043] AX w1 +BY w1 +CZ w1 +D=0 (4)

[0044] The expression of the plane in the camera coordinate system can be obtained from formula (1) as,

[0045]

[0046] Because the world coordinate system is established on the calibration target plane, formula (4) can be transformed into Z w1 =0. From this, it can be deduced that formula (5) in the camera coordinate system is,

[0047]

[0048] From the above formula, R, T are the translation and rotation matrices of each calibration target in different poses, which can uniquely determine the equation of the plane where each calibration target is located in the camera coordinate system in different poses as,

[0049] A1X c1 +B1Y c1 +C1Z c1 +D1=0 (7)

[0050] Wherein, A1, B1, C1, D1 are constants that can be solved.

[0051] 3. Intersection camera coordinate calculation and light plane calibration

[0052] First, the light bar sub-pixel center is calculated using the light bar center sub-pixel extraction algorithm, and the result is as follows, Figure 2As shown, the calibration target used here is a circular calibration target. Then, using RANSAC (Random Sample Consensus) for straight line fitting on all the sub-pixel center points extracted, the straight line equation λ n It is described as,

[0053] a0u+b0v+c0=0 (8)

[0054] Because the calibration target used is a circular calibration target, the pixel coordinates of each circular center can be well obtained, and then the straight line equation L of each center connecting line can be obtained. n As shown in Figure 3 L n It can be expressed as,

[0055] a1u+b1v+c1=0 (9)

[0056] Further, as shown in Figure 3 (b), the intersection point Point cross (u, v) of the light strip and the center connecting line is expressed as,

[0057]

[0058] From the perspective projection model of the camera, the connecting line of all points on the calibration board within the camera field of view and the camera optical center has and only has one intersection point with the calibration board. Therefore, from equation (2), the intersection point Point cross (u, v) of the light strip and the target connecting line can be described in the camera coordinate system as,

[0059]

[0060] where (X c0 , Y c0 , Z c0 ) T is the coordinate value of the intersection point in the camera coordinate system, and the coefficient matrix is the camera intrinsic matrix obtained during camera calibration. After inverting the camera intrinsic matrix in the above equation and left multiplying to the left side of the equation, the following relationship is obtained,

[0061]

[0062] After simplification, equation (12) is further expressed as,

[0063]

[0064] The perspective projection model of the intersection point of two lines on the calibration board is as shown in Figure 4As shown, by analysis, the image coordinates of the point E" in the image coordinate system α and the line connecting the camera optical center and the point E" are collinear with the line connecting the camera optical center and the point E in the world coordinate system.

[0065] Therefore, the rotation matrix R between the camera coordinate system and the image coordinate system α at a certain distance from the camera optical center is a unit matrix, and the translation vector T = (0, 0, n) T , where n is a constant. Further, the camera coordinate value of the intersection of the two lines in the coordinate system α is

[0066]

[0067] For convenience of calculation, n is set to 1 here, and equation (14) is simplified to equation (15):

[0068]

[0069] Therefore, Figure 2 The projection coordinates E" of the point E and the camera optical center O c form a straight line equation,

[0070]

[0071] By combining equations (7), (10) and (16), the coordinates of the point E in the camera coordinate system are obtained as

[0072]

[0073] In the above equation, the right side of the equal sign is a known quantity, where a0, a1, b0, b1, c0, c1 can be obtained from the 7 straight line equations of the light strip straight line equation and the target connecting line on the N different pose calibration board. Fix the light plane unchanged, move the N times calibration board to obtain N x 7 intersection points, and then use the least squares method to fit to obtain the light plane equation.

[0074] The line structure light plane calibration method based on a two-dimensional circular target of the embodiment can be tested by the following system:

[0075] In order to test the accuracy of the light plane calibration and the stability of the algorithm, a line structure light three-dimensional measurement system as Figure 5 shown is designed and built. The system uses a MER2-503-36U3M black and white industrial camera of Daheng Image, and is equipped with a Computar M0828-MPW3 8mm fixed focus lens for imaging. A 405nm linear structure light laser of Beijing Yihua Mingsheng Technology Co., Ltd. is used as the linear structure light source, and a circular target with a pattern number of 7 x 7 and a target center distance of 2cm is used as the target.

[0076] In order to make the calibration more simple and accurate, a picture subtraction method is used to locate the region of the light bar during the system calibration. When taking the picture needed for the calibration of the monocular camera, the laser is also turned on to irradiate the calibration target and take another picture. In this way, the difference between the two pictures taken at the same position of the calibration target is only the light bar, and thus the ROI extraction of the light bar is completed.

[0077] The calibration process of the whole light plane is roughly divided into:

[0078] (1) taking images of the calibration target at different positions;

[0079] (2) calibrating the monocular camera;

[0080] (3) extracting the light bar center by subtracting the calibration target picture with and without the light bar of the structured light;

[0081] (4) performing linear fitting on the light bar center and obtaining the expression of the intersection point of the calibration target circle and the straight line;

[0082] (5) obtaining the expression in the camera coordinate system of the plane where the calibration target is located;

[0083] (6) obtaining the expression of formula (15) in Chapter 2, and solving the coordinate values of the intersection point in the camera coordinate system by solving all equations simultaneously;

[0084] (7) after obtaining the camera coordinate system values of multiple intersection points by moving the calibration target multiple times, fitting the light plane using the least square method. The specific calibration process is shown in Figure 6 .

[0085] The results of the above experiment are as follows:

[0086] After moving the calibration target 10 times and taking 10 groups of calibration target pictures with and without light bars, the intrinsic matrix of the monocular camera obtained by calibration is

[0087]

[0088] The light bar used for all calibration is shown in Figure 7 (a), and the center line corresponding to each light bar obtained by using the light bar center extraction algorithm is shown in Figure 7 (b).

[0089] The result of the light plane calibration is

[0090] Z c = 0.260958X c + 1.833015Y c + 68.194519 (19)

[0091] A standard gauge block with a length of 40 mm is used to test the calibration accuracy of the light plane, and the actual picture of the gauge block is as followsFigure 8 (a) shown. The images of the gauge block at different positions of the light bar are taken by the camera. The center points of the light bar on the gauge block are converted to world coordinates using the planar light calibration results and the distance between the two ends of the gauge block is calculated.

[0092] The measurement results of the gauge block using the method proposed in this embodiment are shown in Table 1. The 16 measurement results of the gauge block at different positions of the same light plane using the method proposed in this embodiment are listed in the table.

[0093] As can be easily seen in Table 1, the method proposed in this embodiment still maintains good robustness in measuring the width of the gauge block at the last ten light bars and the edge positions of the camera field of view. The measurement results using the Zhang method are slightly better than the method proposed in this embodiment when the light bar quality is good and the measured object is in the center region of the camera imaging during measurement. However, the accuracy and robustness under non-ideal conditions cannot reach the performance of the method proposed in this embodiment.

[0094] Table 1: Measurement results of a 40mm gauge block at different positions (unit: cm)

[0095]

[0096] From the above two tables, we can see that the measurement errors of the two methods at the center of the camera field of view and at the center of the laser exit angle are relatively small. Under the unfavorable conditions of imaging at the edge position, the method proposed in this embodiment can still achieve relatively high precision, and the RMSE (Root Mean Square Error) and the average measurement error are also relatively small, proving that the method has strong robustness.

[0097] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A two-dimensional circular target-based line structured light plane calibration method, comprising the following steps: A. Taking pictures of the calibration target at different positions in the camera field of view; B. Calibrating the monocular camera to obtain the camera internal and external parameters and to obtain the equation of the plane of each two-dimensional circular calibration target in the camera coordinate system; In the step B, the specific method for obtaining the camera internal and external parameters comprises the following steps: The camera is regarded as a pinhole imaging model, from which the relationship between a point A(X w ,Y w ,Z w ) in the world coordinate system and the optical center of the camera can be obtained; The world coordinate system is described as O w -X w Y w Z w The camera coordinate system is described as O wc -X c Y c Z c The image coordinate system is described as O-xy, and the pixel coordinate system is described as O-uv, so the process of converting a point A (X w ,Y w ,Z w ) in the world coordinate system to A c (X c ,Y c ,Z c ) in the camera coordinate system by a rigid body transformation is as follows: where R is a rotation matrix describing the rigid body transformation, and T is a translation vector; point A c The conversion from camera coordinates to image coordinates system O-xy can be described as: In the formula, f is the focal length; when the image coordinate system is converted to the pixel coordinate system, the pixel coordinate origin is usually represented at the upper left corner of the picture, while the image coordinate origin is at the middle of the picture, so the conversion relationship between the image coordinate system and the pixel coordinate system is: d in the formula x d in the formula y pixel size of a single pixel, u0, v0 represent offset of the principal point position in the pixel coordinate system, conversion relationship between the coordinate systems is obtained, so that the camera internal and external parameter matrices required for mutual conversion between the above formulas are obtained; The specific method for obtaining the equation of the plane of each two-dimensional circular calibration target in the camera coordinate system comprises the following steps: Suppose there is a point B(X w1 ,Y w1 ,Z w1 ) on the calibration target plane in the world coordinate system, and the world coordinate system is established on the calibration target plane, then the plane equation of the calibration target in the world coordinate system can be described as: AX w1 +BY w1 +CZ w1 +D = 0 (4) From formula (1), the expression of the plane in the camera coordinate system can be obtained as, Because the world coordinate system is established on the calibration target plane, equation (4) can be transformed into Z w1 = 0, so it can be derived that equation (5) is expressed as: As can be seen from the above formula, R and T are the translation and rotation matrices of each calibration target at different poses obtained during camera calibration, so the equation of the plane of each calibration target at different poses in the camera coordinate system can be uniquely determined as: A1X c1 +B1Y c1 +C1Z c1 +D1=0 (7) Wherein, A1, B1, C1 and D1 are constants that can be obtained; C. Subtracting the two calibration target pictures with and without laser stripes to obtain the light stripe center of the light stripe region; In the step C, the specific method for obtaining the light stripe center is to obtain the light stripe sub-pixel coordinate value irradiated at any position of the two-dimensional circular calibration target by using a light stripe extraction algorithm; D. Fitting the light stripe center to obtain a straight line equation, and obtaining the expression of the intersection point of the calibration target circle point and the straight line; The step D specifically comprises: then fitting all the points of the light stripe center into a straight line, intersecting each column of target markers with a point to obtain the intersection point of each column of target marker connecting lines and the fitted light stripe center line, and connecting the intersection point and the camera optical center to form a straight line; E. Solving the coordinate value of the intersection point in the camera coordinate system by using the equation set; The step E specifically comprises: Using the obtained intersection point, connecting the camera optical center, and combining the previously obtained equation of the plane of the two-dimensional circular calibration target in the camera coordinate system to solve the coordinate value of the intersection point in the camera coordinate system; F. After obtaining the camera coordinate system values of multiple intersection points by moving the calibration target multiple times, fitting the light plane using the least square method.

2. The line structured light plane calibration method based on two-dimensional circular target according to claim 1, characterized in that, The steps C-F specifically comprise the following steps: a0u + b0v + c0 = 0 (8) Firstly, the sub-pixel center of light stripe is calculated by using the light stripe sub-pixel extraction algorithm, and the calibration target used here is a circular calibration target; then, the RANSAC algorithm is used to perform line fitting on all the extracted sub-pixel center points, and the line equation of the light line of the line structured light can be obtained n It is described as, a1u + b1v + c1 = 0 (9) Because the calibration target used is a circular calibration target, the pixel coordinates of each circular center can be well obtained, and then the straight line equation L of each center connecting line is obtained n , L n is expressed as: After simplification, formula (12) is further expressed as, Further, the intersection point Point of the light bar and the line connecting the circle center cross (u, v) is represented as: From the perspective projection model of the camera, it is known that the line connecting all points on the calibration board within the field of view of the camera and the optical center of the camera has only one intersection point with the calibration board; therefore, from equation (2), the intersection point Point cross In the camera coordinate system, it can be described as: wherein (X c0 ,Y c0 ,Z c0 ) T denotes the coordinate value of the intersection point in the camera coordinate system, the coefficient matrix thereof is the camera intrinsic matrix obtained during camera calibration, and after the camera intrinsic matrix in the above formula is inverted and multiplied to the left of the equal sign, the following relationship is obtained: In order to facilitate calculation, n is set to 1 here, and formula (14) is simplified as, Since the line connecting the image coordinate of the point E" in the image coordinate system α at a certain distance from the camera optical center and the camera optical center is collinear with the line connecting the point and the camera optical center in the world coordinate system, the rotation matrix R between the camera coordinate system and the image coordinate system α at a certain distance from the camera optical center is a unit matrix, and the translation vector T=(0, 0, n) T , wherein n is a constant; and the camera coordinate value of the intersection point of the two lines in the coordinate system α is obtained as follows: By combining formulas (7), (10) and (16), the coordinates of point E in the camera coordinate system are obtained as, The projection coordinates E" of point E and the optical center O of the camera c The equation of the straight line formed is, In the above formula, the right side of the equal sign is a known quantity, wherein a0, a1, b0, b1, c0 and c1 can be obtained from the light stripe straight line equation and the 7 straight line equations of the target marker connecting lines on the N different pose calibration plates, so that the light plane is fixed and unchanged, and N times of moving the calibration plate obtains N*7 intersection points, and the light plane equation can be obtained by using the least square method. ​

Images