Light plane calibration method based on multi-line laser three-dimensional scanning

Abstract

The invention relates to the technical field of multi-line laser three-dimensional scanning, in particular to a light plane calibration method based on multi-line laser three-dimensional scanning, which comprises the following steps of: calibrating parameters of two cameras; calculating a laser center three-dimensional coordinate based on the calibration plate image; fitting a plurality of laser linear equations; classifying laser center three-dimensional points; a plurality of light plane equations are fitted. According to the method, only the two calibration plate images projected with the laser rays and not projected with the laser rays need to be shot, the light planes to which the multiple laser rays in the images belong do not need to be manually recognized, the laser rays can be automatically classified, then the light plane equations of all the laser rays are calibrated, the method is high in robustness and speed, and the method is suitable for large-scale popularization and application. Therefore, the problem that an existing light plane calibration method is low in accuracy is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-line laser three-dimensional scanning, and in particular to a light plane calibration method based on multi-line laser three-dimensional scanning. Background Art

[0002] With the improvement of intelligent manufacturing and automation, the market demand for efficient and accurate measurement solutions has increased dramatically. Handheld multi-line laser 3D scanners have been widely used in various industries due to their advantages such as non-contact, high precision and high speed.

[0003] At present, when calibrating the light plane of multi-line lasers, since there are multiple projected laser lines and the extracted three-dimensional points of the laser center are disordered, it is impossible to determine the light plane to which each three-dimensional point of the laser center belongs, and it is also impossible to match the laser lines at two different calibration positions one by one, which leads to errors in the calibration of the light plane. The traditional method can achieve the light plane calibration of multi-line lasers based on the classified laser lines by segmenting each laser line and then manually classifying them, but this method requires manual classification, which is a cumbersome process and has low stability and accuracy. Summary of the invention

[0004] The object of the present invention is to provide a light plane calibration method based on multi-line laser three-dimensional scanning, aiming to solve the problem of low accuracy of existing light plane calibration methods.

[0005] To achieve the above object, the present invention provides a method for calibrating a light plane based on multi-line laser three-dimensional scanning, comprising the following steps:

[0006] Calibrate the parameters of the two cameras;

[0007] Calculate the three-dimensional coordinates of the laser center based on the calibration plate image;

[0008] Fitting multiple laser line equations;

[0009] Laser center 3D point classification;

[0010] Fitting multiple light plane equations.

[0011] Among them, in "calibrating the parameters of the two cameras", the following steps are included:

[0012] Get the camera calibration image;

[0013] Calculate the pixel coordinates of the center of the ellipse based on the calibration image;

[0014] Sort the pixel coordinates of the ellipse center and establish a world coordinate system to obtain the three-dimensional coordinates of the ellipse center in the world coordinate system;

[0015] Input the pixel coordinates of the ellipse center and its corresponding three-dimensional coordinates into the camera calibration function of Opencv to obtain the intrinsic parameter matrix and distortion coefficient matrix of the two cameras;

[0016] Calibrate the transformation matrix between two cameras.

[0017] Among them, in "calculating the three-dimensional coordinates of the laser center based on the calibration plate image", the following steps are included:

[0018] Place the calibration plate in the field of view of the two cameras, and control the two cameras to capture two sets of images of the calibration plate with and without laser lines;

[0019] Binarize the image, keep only the image data of the laser line, set the pixels of other data to 0, and then use the grayscale centroid method to extract the laser center;

[0020] According to the camera intrinsic parameter matrix and the camera extrinsic parameter matrix at the positions of the two calibration plates, the three-dimensional coordinates of the laser center in the camera coordinate system are obtained.

[0021] Among them, in "Fitting multiple laser line equations", the following steps are included:

[0022] Fit a straight line and extract the inner point data of the straight line;

[0023] The other three-dimensional points except the inner points of the current fitting line are fitted again, and a threshold of the number of inner points is set. If the number of inner points of the current line is less than the threshold, the current fitting line is considered invalid; if it is greater than the threshold, the line is retained and the above steps are repeated until seven laser line equations are extracted.

[0024] Among them, in the "laser center 3D point classification", the following steps are included:

[0025] Transfer two points on the seven laser lines to the two-dimensional pixel plane, and calculate the two-dimensional linear direction vector of the first line;

[0026] The direction vector of the first straight line obtains its corresponding normal direction vector, and the normal equation is calculated;

[0027] The distances between the seven intersection points of the normal line and the laser line and the intersection point of the normal line and the left boundary line are calculated, and the distances are sorted to achieve the classification of the laser center line.

[0028] Among them, in "Fitting multiple light plane equations", the following steps are included:

[0029] Perform light plane fitting on the three-dimensional points on the two groups of laser lines to obtain the light plane equation of the current laser line;

[0030] The three-dimensional coordinates of the laser center point are obtained according to the corresponding light plane equation and camera internal parameters.

[0031] The present invention discloses a method for calibrating a light plane based on multi-line laser three-dimensional scanning, comprising the following steps: calibrating the parameters of two cameras; calculating the three-dimensional coordinates of the laser center based on the calibration plate image; fitting multiple laser straight line equations; classifying the three-dimensional points of the laser center; and fitting multiple light plane equations. The present invention only needs to take two calibration plate images with and without projecting laser lines, and does not need to manually identify which light planes the multiple laser lines in the image belong to. The method can automatically classify the laser lines and then calibrate the light plane equations of all the laser lines. The method has high robustness and high speed, thereby solving the problem of low accuracy of the existing light plane calibration method. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 It is a schematic diagram of the circular array calibration plate.

[0034] Figure 2 This is a flow chart of a light plane calibration method based on multi-line laser three-dimensional scanning provided by the present invention.

[0035] Figure 3 It is a structural diagram of a multi-line laser 3D scanning system.

[0036] Figure 4 This is the original image of position 1.

[0037] Figure 5 This is the result of laser center extraction at position 1.

[0038] Figure 6 It is the original image of position 2

[0039] Figure 7 This is the result of laser center extraction at position 2.

[0040] Figure 8 It is a flow chart for calibrating the parameters of two cameras.

[0041] Fig. 9 It is a flow chart for calculating the three-dimensional coordinates of the laser center based on the calibration plate image.

[0042] Fig.10 It is a flow chart for fitting multiple laser line equations.

[0043] Fig.11 It is a flow chart of the classification of three-dimensional points at the laser center.

[0044] Fig.12 is a flow chart for fitting multiple light plane equations. DETAILED DESCRIPTION

[0045] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0046] See also Figures 1 to 12 The present invention provides a method for calibrating a light plane based on multi-line laser three-dimensional scanning, comprising the following steps:

[0047] S1 calibrates the parameters of the two cameras;

[0048] S11 obtains a camera calibration image;

[0049] Specifically, obtain the camera calibration image. Place the calibration plate in the field of view of the two cameras, control the two cameras to capture the current calibration plate image, then change the position of the calibration plate, continue to capture the current calibration plate image, and repeat the above operation until the calibration plate image is acquired.

[0050] S12 calculates the pixel coordinates of the center of the ellipse based on the calibration image;

[0051] Specifically, the pixel coordinates of the center of the ellipse are calculated. First, the captured calibration plate image is binarized, and then contour extraction is performed to filter the contour belonging to the marker circle. Then, the sub-pixel edge extraction of the marker circle contour is performed to obtain the sub-pixel contour of the marker circle. Finally, an ellipse is fitted based on the sub-pixel marker circle contour to obtain the pixel coordinates of the center of the ellipse.

[0052] S13 sorts the pixel coordinates of the ellipse center and establishes a world coordinate system to obtain the three-dimensional coordinates of the ellipse center in the world coordinate system;

[0053] Specifically, the pixel coordinates of the ellipse center are sorted. The ellipse center at the upper left of the calibration plate is used as the starting point to sort the ellipse centers, and then a world coordinate system is established based on the plane where the calibration plate is located to obtain the three-dimensional coordinates of the ellipse center in the world coordinate system.

[0054] S14 inputs the pixel coordinates of the ellipse center and the corresponding three-dimensional coordinates into the camera calibration function of Opencv to obtain the intrinsic parameter matrix and the distortion coefficient matrix of the two cameras;

[0055] Specifically, the pixel coordinates of the ellipse center and their corresponding three-dimensional coordinates are input into the camera calibration function of Opencv to obtain the intrinsic parameter matrices and distortion coefficient matrices of the two cameras.

[0056] S15 calibrates the transformation matrix between two cameras.

[0057] Specifically, calibrate the transformation matrix between the two cameras. Input the ellipse center pixel coordinates, intrinsic parameter matrix and distortion coefficient matrix of the left and right cameras into the stereo calibration function of Opencv to obtain the transformation matrix between the two cameras.

[0058] S2 calculates the three-dimensional coordinates of the laser center based on the calibration plate image;

[0059] S21 places the calibration plate in the field of view of the two cameras, and controls the two cameras to capture two sets of images of the calibration plate with and without laser lines;

[0060] Specifically, the calibration plate is placed in the field of view of the two cameras, the two cameras are controlled to capture the current calibration plate image, and then the laser is controlled to project seven laser lines onto the calibration plate, the camera exposure is reduced, the current calibration plate image is captured, the position of the calibration plate is changed, and the above operations are repeated to collect two sets of calibration plate images with and without laser lines.

[0061] S22 binarizes the image, retains only the image data of the laser line part, sets the pixels of the other data parts to 0, and then uses the grayscale centroid method to extract the laser center;

[0062] Specifically, for the calibration plate image without laser lines, add it to the camera calibration, and then obtain the camera extrinsic matrix at the positions of the two calibration plates. For the calibration plate image with laser lines, first binarize the image, only retain the image data of the laser line part, set the pixels of the other data to 0, and then use the grayscale centroid method to extract the laser center; the result is as follows Figures 4 to 7 shown.

[0063] S23 obtains the three-dimensional coordinates of the laser center in the camera coordinate system according to the camera intrinsic parameter matrix and the camera extrinsic parameter matrix at the positions of the two calibration plates.

[0064] Specifically, according to the camera intrinsic parameter matrix and the camera extrinsic parameter matrix at the positions of the two calibration plates, the laser center (u c ,v c ) is the three-dimensional coordinate (x c ,y c ,z c ), the equation is as follows:

[0065]

[0066] In the above formula, (x w ,y w ,z w ) represents the three-dimensional coordinates of the laser center in the world coordinate system, K represents the intrinsic parameter matrix of the camera, R represents the rotation matrix of the camera coordinate system and the world coordinate system corresponding to the current calibration plate posture, and t represents the translation matrix of the camera coordinate system and the world coordinate system corresponding to the current calibration plate posture.

[0067] Since the XY plane of the world coordinate system is the plane where the calibration plate is located, and the laser line is projected onto the calibration plate, z w =0, the above formula can be simplified to the following equation:

[0068]

[0069] By solving the above equation, we can get z c The value of the laser center three-dimensional coordinates (x c ,y c ,z c ) can be solved by the following equation:

[0070]

[0071] S3 fits multiple laser line equations;

[0072] S31 fits a straight line and extracts the inner point data of the straight line;

[0073] Specifically, since the laser projects seven laser lines, there will be seven laser lines at the calculated three-dimensional point of the laser center. These seven laser lines need to be distinguished in order to perform straight line fitting on the laser lines. Figure 4-Figure 7 As can be seen from the figure, the seven laser lines are theoretically parallel to each other, and there is a certain distance between the laser lines, so the RANSAC (Random Sampling Consensus) method can be used for straight line fitting. First, fit a straight line and extract the inner point data of the line, where the inner point refers to the three-dimensional point that is finally used to fit the line.

[0074] S32 performs straight line fitting again for the other three-dimensional points except the inner points of the current fitting straight line, and sets an inner point number threshold. If the inner point number of the current straight line is less than the threshold, the current fitting straight line is considered invalid; if it is greater than the threshold, the straight line is retained and the above steps are repeated until seven laser straight line equations are extracted.

[0075] Specifically, the other three-dimensional points except the inner points of the current fitting line are fitted again. By setting an inner point number threshold, if the inner point number of the current line is less than this threshold, the current fitting line is considered invalid. If it is greater than the threshold, the line is retained and the above steps are repeated until seven laser line equations are extracted. The extracted laser line equations are as follows:

[0076]

[0077] In the above formula, i = 1, 2, ..., 7, p i =(x i ,y i ,z i ) represents a point on the i-th straight line, Represents the direction vector of the i-th line.

[0078] S4 laser center 3D point classification;

[0079] S41 transfers two points on the seven laser lines to a two-dimensional pixel plane, and calculates a two-dimensional linear direction vector of the first line;

[0080] Specifically, the seven laser line equations obtained by fitting are out of order, so they must be sorted to match the three-dimensional points of the laser center obtained under two different postures and realize the light plane fitting of multiple laser lines. According to the camera's intrinsic parameter matrix, the two points on these seven laser lines can be and Switching to the two-dimensional pixel plane, the formula is as follows:

[0081]

[0082] By solving the above equation, we can get the corresponding two points on the two-dimensional line and From this, the corresponding two-dimensional straight line direction vector is The formula is as follows:

[0083]

[0084] S42 The direction vector of the first straight line obtains the corresponding normal direction vector, and calculates the normal equation;

[0085] Specifically, according to the direction vector of the first straight line The corresponding normal direction vector can be obtained as Then we can calculate p 1 =(u 1 ,v 1 ) is the normal line equation, which is as follows:

[0086]

[0087] S43 calculates the distances between the seven intersection points of the normal line and the laser straight line and the intersection point of the normal line and the left boundary straight line, sorts the distances, and implements classification of the laser center line.

[0088] Specifically, since the seven laser lines are substantially parallel, the normal line must intersect with all seven laser lines. The equation for calculating the intersection of the normal line and the laser line is as follows:

[0089]

[0090] In the above formula, i = 1, 2, ..., 7, (m i ,n i ) represents the intersection point of the i-th laser line and the normal line.

[0091] The left boundary line of the image can be calculated through the points (0,0) and (0,h). The equation of the line is u=h, where h is the height of the image. Substituting the left boundary line into the normal equation can get the intersection of the two lines.

[0092]

[0093] Finally, by calculating the seven intersection points (m i ,n i ) intersects the normal and the left boundary line

[0094]

[0095] By sorting the distances, the laser center lines can be classified.

[0096] S5 fits multiple light plane equations.

[0097] S51 performs light plane fitting on the three-dimensional points on the two groups of laser lines to obtain the light plane equation of the current laser line;

[0098] Specifically, by classifying the three-dimensional points of the laser center at two different positions, the two groups of laser lines can be matched one by one. By fitting the light plane of the three-dimensional points on the two groups of laser lines, the light plane equation of the current laser line can be obtained. The equation is as follows:

[0099] a i x+b i y+c i z+d=0

[0100] In the above formula, i = 1, 2, ..., 7, (a i ,b i ,c i ) represents the normal vector of the i-th light plane, di represents the coefficient of the i-th light plane.

[0101] S52 obtains the three-dimensional coordinates of the laser center point according to the corresponding light plane equation and the camera internal parameters.

[0102] Specifically, during actual reconstruction, by determining which light plane the current laser center point belongs to, the three-dimensional coordinates of the laser center point can be calculated based on the corresponding light plane equation and the camera intrinsic parameters, thereby achieving multi-line laser three-dimensional reconstruction.

[0103] Beneficial effects:

[0104] The present invention only needs to take two calibration plate images with and without projecting laser lines, and there is no need to manually identify which light planes multiple laser lines in the image belong to. The method can automatically classify the laser lines and then calibrate the light plane equations of all laser lines. The method has high robustness and high speed.

[0105] What is disclosed above is only a preferred embodiment of the optical plane calibration method based on multi-line laser three-dimensional scanning of the present invention. Of course, this cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for optical plane calibration based on multi-line laser three-dimensional scanning, characterized in that: The following steps are involved: Calibrate the parameters of the two cameras; Calculate the three-dimensional coordinates of the laser center based on the calibration plate image; Fitting multiple laser line equations; Laser center 3D point classification; Fitting multiple light plane equations.

2. The optical plane calibration method based on multi-line laser three-dimensional scanning according to claim 1, characterized in that: In "Calibrate the parameters of two cameras", the following steps are included: Get the camera calibration image; Calculate the pixel coordinates of the center of the ellipse based on the calibration image; Sort the pixel coordinates of the ellipse center and establish a world coordinate system to obtain the three-dimensional coordinates of the ellipse center in the world coordinate system; Input the pixel coordinates of the ellipse center and its corresponding three-dimensional coordinates into the camera calibration function of Opencv to obtain the intrinsic parameter matrix and distortion coefficient matrix of the two cameras; Calibrate the transformation matrix between two cameras.

3. The optical plane calibration method based on multi-line laser three-dimensional scanning according to claim 2, characterized in that: In "Calculating the 3D coordinates of the laser center based on the calibration plate image", the following steps are included: Place the calibration plate in the field of view of the two cameras, and control the two cameras to capture two sets of images of the calibration plate with and without laser lines; Binarize the image, keep only the image data of the laser line, set the pixels of other data to 0, and then use the grayscale centroid method to extract the laser center; According to the camera intrinsic parameter matrix and the camera extrinsic parameter matrix at the positions of the two calibration plates, the three-dimensional coordinates of the laser center in the camera coordinate system are obtained.

4. The optical plane calibration method based on multi-line laser three-dimensional scanning according to claim 3, characterized in that: In "Fitting Multiple Laser Line Equations", the following steps are included: Fit a straight line and extract the inner point data of the straight line; The other three-dimensional points except the inner points of the current fitting line are fitted again, and a threshold of the number of inner points is set. If the number of inner points of the current line is less than the threshold, the current fitting line is considered invalid; if it is greater than the threshold, the line is retained and the above steps are repeated until seven laser line equations are extracted.

5. The optical plane calibration method based on multi-line laser three-dimensional scanning according to claim 4, characterized in that: In "Laser Center 3D Point Classification", the following steps are included: Transfer two points on the seven laser lines to the two-dimensional pixel plane, and calculate the two-dimensional linear direction vector of the first line; The direction vector of the first straight line obtains its corresponding normal direction vector, and the normal equation is calculated; The distances between the seven intersection points of the normal line and the laser line and the intersection point of the normal line and the left boundary line are calculated, and the distances are sorted to achieve the classification of the laser center line.

6. The optical plane calibration method based on multi-line laser three-dimensional scanning according to claim 5, characterized in that: In "Fitting Multiple Light Plane Equations", include the following steps: Perform light plane fitting on the three-dimensional points on the two groups of laser lines to obtain the light plane equation of the current laser line; The three-dimensional coordinates of the laser center point are obtained according to the corresponding light plane equation and camera internal parameters.

Images