Line laser three-dimensional scanning-based reference coordinate system calibration method

Abstract

The invention relates to the technical field of line laser three-dimensional scanning, in particular to a reference coordinate system calibration method based on line laser three-dimensional scanning, which comprises the following steps of: calibrating camera parameters; calibrating a light plane equation of the laser; obtaining a calibration whiteboard image; establishing a reference coordinate system; and calculating a conversion matrix from the camera coordinate system to the reference coordinate system. The reference coordinate system of the line laser three-dimensional scanner can be determined only by additionally acquiring a white board image during calibration, and whether the current object is located in the measurement range or not can be determined according to the z coordinate value of the point cloud data by converting the acquired three-dimensional point cloud data of the measured object to the reference coordinate system. And the efficiency and the accuracy of subsequent point cloud processing and three-dimensional measurement are greatly improved. Therefore, the problem that an existing calibration method is low in measurement precision is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of line laser three-dimensional scanning, and in particular to a reference coordinate system calibration method based on line laser three-dimensional scanning. Background Art

[0002] With the rapid development of China's industrial manufacturing, the line laser 3D scanning measuring instrument is becoming an important tool on the modern production line due to its non-contact, high measurement accuracy and high speed. The line laser 3D scanner consists of a camera and a single-line laser. By controlling the laser to project a beam of laser line onto the object to be measured, and then the camera shoots the deformed laser line and extracts the center point of the laser line, the 3D point cloud data of the object to be measured at the current scanning position can be obtained. By mounting the scanner on a linear motor and controlling the motor to move, the complete 3D point cloud data of the object to be measured can be obtained. Based on the obtained 3D point cloud data, the 3D measurement of the object to be measured can be achieved.

[0003] For single-line laser 3D scanners, the reference distance of the scanner is generally determined by the hardware structure. However, due to actual installation errors, the reference distance is often not equal to the design value, so calibration is required to obtain a more accurate reference distance. In addition, since the calibrated light plane equation is often in the camera coordinate system, the scanned 3D point cloud data will also be in the camera coordinate system, and the camera coordinate system is tilted relative to the object being measured. Therefore, the 3D points in the camera coordinate system cannot well determine whether the current object being measured is within the optimal working range of the scanner, which will affect the measurement accuracy to a certain extent. Summary of the invention

[0004] The purpose of the present invention is to provide a reference coordinate system calibration method based on line laser three-dimensional scanning, aiming to solve the problem of low measurement accuracy of existing calibration methods.

[0005] To achieve the above object, the present invention provides a reference coordinate system calibration method based on line laser three-dimensional scanning, comprising the following steps:

[0006] Calibrate camera parameters;

[0007] Calibrate the optical plane equation of the laser;

[0008] Get the calibration whiteboard image;

[0009] Establish a reference coordinate system;

[0010] Calculate the transformation matrix from the camera coordinate system to the reference coordinate system.

[0011] Among them, in "calibrating camera parameters", the following steps are included:

[0012] Use the checkerboard calibration plate to calibrate the camera;

[0013] Control the camera to take multiple images of the calibration plate in different postures;

[0014] Extract the coordinates of the pixel corner points on the calibration plate, and establish a world coordinate system based on the calibration plate to obtain the three-dimensional coordinates of each corner point in the world coordinate system;

[0015] The extracted corner pixel coordinates and three-dimensional coordinates are brought into the camera calibration function to obtain the camera's intrinsic and extrinsic parameter matrix and distortion coefficient matrix.

[0016] Among them, in the "optical plane equation for calibrating laser", the following steps are included:

[0017] Control the camera to capture two sets of calibration plate images in different postures and their corresponding calibration plate images projected with laser lines;

[0018] Corner point extraction is performed on the calibration plate image without laser lines, and laser line extraction is performed on the calibration plate image with laser lines to obtain the laser center pixel coordinates;

[0019] Calculate the three-dimensional coordinates corresponding to the center pixel coordinates of the laser through the camera intrinsic parameter matrix and the camera extrinsic parameter matrix corresponding to the current calibration plate posture;

[0020] Fit the light plane equation based on the three-dimensional point of the laser center.

[0021] Among them, in "obtaining a calibration whiteboard image", the following steps are included:

[0022] Controlling the camera to capture an image of the whiteboard on which the laser line is projected;

[0023] Extract the laser center pixel coordinates based on the whiteboard image;

[0024] Obtain the three-dimensional coordinates of the laser center according to the calibrated camera intrinsic parameter matrix and light plane equation;

[0025] The X-axis of the reference coordinate system is obtained by fitting the laser center line equation.

[0026] Among them, in "Establishing a reference coordinate system", the following steps are included:

[0027] The normal vector of the light plane equation is used as the Y axis of the reference coordinate system;

[0028] The Z axis of the reference coordinate system is obtained by cross-producting the X axis and Y axis direction vectors of the reference coordinate system;

[0029] Determine the coordinate axis direction of the base coordinate system.

[0030] Among them, in "calculating the transformation matrix from the camera coordinate system to the reference coordinate system", the following steps are included:

[0031] According to the obtained three-dimensional coordinates of the three coordinate axes of the reference coordinate system in the camera coordinate system and the three-dimensional coordinates of the three coordinate axes of the camera coordinate system itself, the rotation matrix from the camera coordinate system to the reference coordinate system is obtained;

[0032] Based on the three-dimensional coordinates of the origin of the reference coordinate system in the camera coordinate system, a translation matrix from the camera coordinate system to the reference coordinate system is obtained;

[0033] Based on the rotation matrix and translation matrix, the three-dimensional point cloud data in the camera coordinate system is transformed into the reference coordinate system.

[0034] The present invention provides a reference coordinate system calibration method based on line laser three-dimensional scanning, comprising the following steps: calibrating camera parameters; calibrating the optical plane equation of the laser; acquiring a calibration whiteboard image; establishing a reference coordinate system; and calculating a conversion matrix from the camera coordinate system to the reference coordinate system. The present invention only needs to acquire an additional whiteboard image during calibration to determine the reference coordinate system of the line laser three-dimensional scanner. By converting the acquired three-dimensional point cloud data of the measured object into the reference coordinate system, it is possible to determine whether the current object is within the measurement range based on the z coordinate value of the point cloud data, thereby greatly improving the efficiency and accuracy of subsequent point cloud processing and three-dimensional measurement. This solves the problem of low measurement accuracy of existing calibration methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 It is a chessboard calibration board.

[0037] Figure 2 This is a structural diagram of a line laser 3D scanner.

[0038] Figure 3 This is the result diagram of laser center extraction.

[0039] Figure 4 It is a flow chart of a reference coordinate system calibration method based on line laser three-dimensional scanning provided by the present invention.

[0040] Figure 5 This is the flow chart for calibrating camera parameters.

[0041] Figure 6 It is a flow chart of the light plane equation for calibrating the laser.

[0042] Figure 7 This is the flow chart for obtaining the calibration whiteboard image.

[0043] Figure 8 It is a flow chart for establishing a reference coordinate system.

[0044] Fig. 9 It is a flow chart for calculating the transformation matrix from the camera coordinate system to the reference coordinate system. DETAILED DESCRIPTION

[0045] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0046] See also Figures 1 to 9 The present invention provides a reference coordinate system calibration method based on line laser three-dimensional scanning, comprising the following steps:

[0047] S1 calibrates camera parameters;

[0048] S11 uses a checkerboard calibration plate to calibrate the camera;

[0049] Specifically, a checkerboard calibration plate is used to calibrate the camera.

[0050] S12 controls the camera to take multiple images of the calibration plate in different positions;

[0051] Specifically, firstly, the camera is controlled to capture multiple calibration plate images in different positions and postures, and then the captured calibration plate images are used.

[0052] S13 extracts the coordinates of the pixel corner points on the calibration plate, and establishes a world coordinate system according to the calibration plate to obtain the three-dimensional coordinates of each corner point in the world coordinate system;

[0053] Specifically, the pixel coordinates of the corner points on the calibration plate image are obtained, and then the normal vector of the plane where the calibration plate is located is used as the Z axis, and the upper left corner point of the calibration plate is used as the origin to establish a world coordinate system.

[0054] S14 brings the extracted corner pixel coordinates and three-dimensional coordinates into the camera calibration function to obtain the camera's internal and external parameter matrix and distortion coefficient matrix.

[0055] Specifically, by bringing the pixel coordinates and three-dimensional coordinates of the corner points of the calibration plate under multiple different poses into the camera calibration function of Opencv, the camera's intrinsic parameter matrix and distortion coefficient matrix can be obtained.

[0056] S2 calibrates the optical plane equation of the laser;

[0057] S21 controls the camera to capture two sets of calibration plate images in different positions and their corresponding calibration plate images projected with laser lines;

[0058] Specifically, the calibration plate is placed within the field of view of the camera, the camera is controlled to capture the current calibration plate image, and then the laser line is projected onto the calibration plate. The camera exposure is reduced and the current calibration plate image is captured. The calibration plate is moved and the above operation is repeated to obtain two sets of calibration plate images with and without laser lines.

[0059] S22 performs corner point extraction on the calibration plate image without laser lines, and performs laser line extraction on the calibration plate image with laser lines, to obtain the laser center pixel coordinates;

[0060] Specifically, for the calibration plate image without laser lines, the two images are added together for calibration when calibrating the camera, and the camera extrinsic matrix corresponding to the two calibration plate poses can be obtained. For the calibration plate image with laser lines, the image is firstly distorted, and then the center coordinates of the laser line are extracted using the grayscale centroid method, such as Figure 3 As shown, (a) is the original image at position 1; (b) is the laser center extraction result at position 1; (c) is the original image at position 2; and (d) is the laser center extraction result at position 2.

[0061] S23 calculates the three-dimensional coordinates corresponding to the laser center pixel coordinates through the camera intrinsic parameter matrix and the camera extrinsic parameter matrix corresponding to the current calibration plate posture;

[0062] Specifically, according to the camera intrinsic matrix and the camera extrinsic matrix corresponding to the current calibration plate posture, the three-dimensional coordinates (X c ,Y c ,Z c ), the equation is as follows:

[0063]

[0064] In the above formula, (X w ,Y w ,Z w ) represents the coordinates of the laser center in the world coordinate system, K represents the intrinsic parameter matrix of the camera, R represents the rotation matrix between the camera coordinate system and the world coordinate system under the current calibration plate posture, and t represents the translation matrix between the camera coordinate system and the world coordinate system under the current calibration plate posture. Since the laser line is projected on the calibration plate, Z w =0, the above formula can be simplified to the following equation:

[0065]

[0066] By solving the above equation, we can get Zc The value of the laser center three-dimensional coordinates (X c ,Y c ,Z c ) can be solved by the following equation:

[0067]

[0068] S24 fits the light plane equation according to the three-dimensional point of the laser center.

[0069] Specifically, by performing plane fitting on the three-dimensional points of the laser center at two different positions, the calibration of the light plane can be completed. The fitted light plane equation is as follows:

[0070] aX+bY+cZ+d=0

[0071] In the above formula, (X, Y, Z) represents any point on the plane, (a, b, c) represents the normal vector of the plane, and d represents the coefficient of the plane.

[0072] S3 obtains the calibration whiteboard image;

[0073] S31 controls the camera to capture the image of the whiteboard with the laser line projected;

[0074] Specifically, a standard whiteboard is first placed on a horizontal plane, a laser line is projected onto the whiteboard, and then the camera captures the current whiteboard image.

[0075] S32 extracts the laser center pixel coordinates based on the whiteboard image;

[0076] Specifically, the grayscale centroid method is used to extract the laser center line of the image to obtain the pixel coordinates (u, v) of the laser center.

[0077] S33 obtains the three-dimensional coordinates of the laser center according to the calibrated camera internal parameter matrix and the light plane equation;

[0078] Specifically, according to the calibrated camera intrinsic parameter matrix K and the light plane equation of the laser, the three-dimensional coordinates (X c ,Y c ,Z c ), the equations are as follows:

[0079]

[0080] S34 obtains the X-axis of the reference coordinate system by fitting the laser center straight line equation.

[0081] Specifically, by performing straight line fitting on the acquired three-dimensional point of the laser center, the straight line equation of the three-dimensional point of the laser center can be obtained, and the equation is as follows:

[0082]

[0083] In the above formula, (X0, Y0, Z0) represents a point on the straight line, (l x ,l y ,l z ) represents the direction vector of the line.

[0084] S4 establishes a reference coordinate system;

[0085] S41 takes the normal vector of the light plane equation as the Y axis of the reference coordinate system;

[0086] Specifically, the calibrated light plane normal vector is used as the Y-axis direction vector of the reference coordinate system, and Y = (y1, y2, y3). The direction vector of the whiteboard laser straight line equation (l x ,l y ,l z ) as the X-axis direction vector of the reference coordinate system, we can get X=(x1,x2,x3).

[0087] S42 obtains the Z axis of the reference coordinate system by cross-producting the X axis and Y axis direction vectors of the reference coordinate system;

[0088] Specifically, the Z-axis direction vector Z = (z1, z2, z3) of the reference coordinate system can be obtained by cross-producting the vector X and the vector Y. The equation is as follows:

[0089]

[0090] S43 determines the coordinate axis direction of the reference coordinate system.

[0091] Specifically, after the three coordinate axes of the reference coordinate system are determined, since the directions of the coordinate axes are random, the directions of the coordinate axes need to be determined. The upward direction of the Z axis is uniformly used as the positive direction of the Z axis. By judging the angle between the positive direction of the Z axis of the camera coordinate system and the Z axis of the reference coordinate system, if the angle is less than 90°, it indicates that the Z axis of the reference coordinate system is upward at this time, and the direction of the Z axis does not need to be adjusted. If the angle is greater than 90°, it indicates that the Z axis of the reference coordinate system is downward at this time, and the direction of the Z axis needs to be reversed. At this time, Z=(-z1,-z2,-z3) TThe positive direction of the X-axis can be determined by taking a whiteboard laser image. Select the first laser center pixel point and the last laser center pixel point extracted from the whiteboard laser image, and calculate their corresponding three-dimensional points P1 and P2 respectively. Solve the vector from P2 to P1 to get v=P2-P1. The normalized direction of vector v is taken as the positive direction of the X-axis. Calculate the angle between vector v and the X-axis direction vector of the reference coordinate system. If the angle is less than 90°, it indicates that the direction of the X-axis is in the same direction as vector v. At this time, there is no need to adjust the direction of the X-axis. If the angle is greater than 90°, it indicates that the direction of the X-axis is opposite to vector v. The direction of the X-axis needs to be reversed. At this time, X=(-x1,-x2,-x3) T After the positive directions of the X-axis and the Z-axis are determined, the positive direction of the Y-axis can be obtained by cross-producting the vector X and the vector Z, thus completing the establishment of the reference coordinate system.

[0092] S5 calculates the transformation matrix from the camera coordinate system to the reference coordinate system.

[0093] S51 obtains a rotation matrix from the camera coordinate system to the reference coordinate system according to the obtained three-dimensional coordinates of the three coordinate axes of the reference coordinate system in the camera coordinate system and the three-dimensional coordinates of the three coordinate axes of the camera coordinate system itself;

[0094] Specifically, through calibration, the direction vectors of the three coordinate axes of the reference coordinate system in the camera coordinate system are X = (x1, x2, x3), Y = (y1, y2, y3) and Z = (z1, z2, z3), and the direction vectors of the three coordinate axes of the camera coordinate system can be defined as X c =(1,0,0),Y c =(0,1,0) and Z c =(0,0,1), according to the one-to-one correspondence between the two, the rotation matrix R from the camera coordinate system to the reference coordinate system b This can be solved by the following equation:

[0095]

[0096] Solving the above equations gives R b =(X,Y,Z), and then by solving the difference between the origin of the camera coordinate system and the reference coordinate system, we can get the translation matrix from the camera coordinate system to the reference coordinate system. c = (0,0,1) as the direction vector of the straight line, passing through the origin of the camera coordinate system O c = (0,0,0) and the intersection of the light plane is the coordinate origin O of the required reference coordinate system. The equation is as follows:

[0097]

[0098] In the above formula, O = (x, y, z) represents the intersection of the line and the light plane, r represents the origin O of the reference coordinate system and the origin O of the camera coordinate system c The distance between them, n = (a, b, c) T represents the normal vector of the light plane equation, and d represents the coefficient of the light plane equation.

[0099] S52 obtains a translation matrix from the camera coordinate system to the reference coordinate system based on the three-dimensional coordinates of the coordinate origin of the reference coordinate system in the camera coordinate system;

[0100] Specifically, r can be solved through the above equations, and r can be substituted into the first expression of the equation to obtain the origin of the reference coordinate system O = (0,0,r) T Therefore, the translation matrix t from the camera coordinate system to the reference coordinate system is b =O c -O=(0,0,-r) T .

[0101] S53 transforms the three-dimensional point cloud data in the camera coordinate system into the reference coordinate system based on the rotation matrix and the translation matrix.

[0102] Specifically, after the rotation matrix and translation matrix of the camera coordinate system to the reference coordinate system are calibrated, the point cloud P in the camera coordinate system obtained by the line laser 3D scanner can be converted into c Convert to the reference coordinate system and get the point cloud P in the reference coordinate system. The equation is as follows:

[0103] P=R b (P c +t b )

[0104] Beneficial effects:

[0105] 1. Improve measurement accuracy: The accuracy of 3D scanning data is ensured by accurately calibrating camera parameters and laser light plane equations.

[0106] 2. Improve efficiency: The automated calibration process reduces manual operations and improves overall work efficiency.

[0107] 3. Improve reliability: Through precise calibration and conversion, the reliability of measurement results is improved and errors are reduced.

[0108] What is disclosed above is only a preferred embodiment of the reference coordinate system calibration method based on line laser three-dimensional scanning of the present invention. Of course, this cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A reference coordinate system calibration method based on line laser three-dimensional scanning, characterized in that: The following steps are involved: Calibrate camera parameters; Calibrate the optical plane equation of the laser; Get the calibration whiteboard image; Establish a reference coordinate system; Calculate the transformation matrix from the camera coordinate system to the reference coordinate system.

2. The reference coordinate system calibration method based on line laser three-dimensional scanning according to claim 1, characterized in that: In "Calibrate Camera Parameters", the following steps are included: Use the checkerboard calibration plate to calibrate the camera; Control the camera to take multiple images of the calibration plate in different postures; Extract the coordinates of the pixel corner points on the calibration plate, and establish a world coordinate system based on the calibration plate to obtain the three-dimensional coordinates of each corner point in the world coordinate system; The extracted corner pixel coordinates and three-dimensional coordinates are brought into the camera calibration function to obtain the camera's intrinsic and extrinsic parameter matrix and distortion coefficient matrix.

3. The reference coordinate system calibration method based on line laser three-dimensional scanning according to claim 1, characterized in that: In the "Light plane equation for calibrating the laser", the following steps are included: Control the camera to capture two sets of calibration plate images in different postures and their corresponding calibration plate images projected with laser lines; Corner point extraction is performed on the calibration plate image without laser lines, and laser line extraction is performed on the calibration plate image with laser lines to obtain the laser center pixel coordinates; Calculate the three-dimensional coordinates corresponding to the center pixel coordinates of the laser through the camera intrinsic parameter matrix and the camera extrinsic parameter matrix corresponding to the current calibration plate posture; Fit the light plane equation based on the three-dimensional point of the laser center.

4. The reference coordinate system calibration method based on line laser three-dimensional scanning according to claim 1, characterized in that: In "Acquiring a calibration whiteboard image", the following steps are included: Controlling the camera to capture an image of the whiteboard on which the laser line is projected; Extract the laser center pixel coordinates based on the whiteboard image; Obtain the three-dimensional coordinates of the laser center according to the calibrated camera intrinsic parameter matrix and light plane equation; The X-axis of the reference coordinate system is obtained by fitting the laser center line equation.

5. The reference coordinate system calibration method based on line laser three-dimensional scanning according to claim 1, characterized in that: In "Establishing a Base Coordinate System", the following steps are included: The normal vector of the light plane equation is used as the Y axis of the reference coordinate system; The Z axis of the reference coordinate system is obtained by cross-producting the X axis and Y axis direction vectors of the reference coordinate system; Determine the coordinate axis direction of the base coordinate system.

6. The reference coordinate system calibration method based on line laser three-dimensional scanning according to claim 1, characterized in that: In "Calculating the transformation matrix from the camera coordinate system to the reference coordinate system", the following steps are included: According to the obtained three-dimensional coordinates of the three coordinate axes of the reference coordinate system in the camera coordinate system and the three-dimensional coordinates of the three coordinate axes of the camera coordinate system itself, the rotation matrix from the camera coordinate system to the reference coordinate system is obtained; Based on the three-dimensional coordinates of the origin of the reference coordinate system in the camera coordinate system, a translation matrix from the camera coordinate system to the reference coordinate system is obtained; Based on the rotation matrix and translation matrix, the three-dimensional point cloud data in the camera coordinate system is transformed into the reference coordinate system.

Images