A high-precision line laser three-dimensional reconstruction calibration method

Abstract

The present invention provides a high-precision line laser three-dimensional reconstruction calibration method, including: performing multi-height calibration through a stepped standard gauge block and a two-dimensional target, selecting corner points adjacent to the laser, and using Zhang's calibration method for solution to complete camera calibration; emitting laser to the surface of an object and collecting corresponding original images with the calibrated camera; obtaining an initial centerline point set based on a centerline extraction algorithm combining improved gray centroid and Jacobian matrix, and denoising the point set with a denoising algorithm to obtain a centerline point set after filtering out noise points; obtaining an optimal optical plane equation based on the pinhole imaging principle and the constraints of the special orthogonal group, combined with the least squares and random parameter search methods; solving a linear equation system according to the centerline point set, internal parameters, external parameters, distortion coefficients, and the optimal optical plane equation to obtain three-dimensional point cloud data of the object contour, and completing three-dimensional reconstruction. The present invention improves the accuracy of line laser three-dimensional reconstruction, simplifies the calibration process, and reduces the calibration cost.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and particularly to a high-precision line laser three-dimensional reconstruction calibration method. Background Art

[0002] The three-dimensional reconstruction technology based on line laser mainly projects laser onto the surface of an object. The laser is modulated due to the change in the depth of the object surface and possible gaps, which is manifested as the change and discontinuity of the light stripe in the image. The degree of change is proportional to the depth, and the discontinuity shows the physical gap on the object surface. By reconstructing this modulation process through a mathematical model, the three-dimensional coordinates can be obtained from the two-dimensional laser stripe image of the intersection line between the laser plane and the outer surface of the object. Due to its advantages of fast reconstruction speed, simple structure, high precision, and strong anti-interference ability, it is widely used in product size detection, weld seam tracking, workpiece contour measurement, robot trajectory guidance and many other scenarios in various fields such as automobile production and electronic manufacturing. The three-dimensional reconstruction technology based on line laser mainly includes camera calibration, laser stripe centerline extraction, laser plane calibration, etc. Among them, there are many mature solutions for laser stripe centerline extraction, such as the gray center of gravity method and the Steger algorithm based on the Hessian matrix. Camera calibration mainly uses Zhang's calibration method for two-dimensional targets and its improved methods to complete.

[0003] The laser plane calibration method is the most important part of the entire three-dimensional reconstruction technology based on line laser, directly affecting the accuracy of the three-dimensional reconstruction algorithm. Scholars at home and abroad have conducted in-depth research and proposed methods such as the wire drawing method and the tooth type method that use the movement of precision mechanical structures to solve the problems of control point extraction and light plane calibration, the self-calibration method that does not require a special calibration target and uses the high-precision movement of the robot itself to construct additional constraints to complete light plane calibration, the parallel line target method that uses the principle of obtaining the plane vanishing line from known equally spaced parallel lines to complete light plane calibration, and the three-dimensional target method based on the cross-ratio invariance theorem. However, the calibration methods in the prior art have problems such as low reconstruction accuracy, complex calibration process, and high calibration cost. Summary of the Invention

[0004] Based on this, it is necessary to provide a high-precision line laser three-dimensional reconstruction calibration method for the above technical problems.

[0005] A high-precision line laser three-dimensional reconstruction calibration method includes the following steps: Through step standard blocks and two-dimensional targets of different sizes, multi-height calibration is carried out within the full range of the line laser. Select the corner points adjacent to the laser as the solution domain, and use the Zhang's calibration method to solve the external parameters, internal parameters, and distortion coefficients of the camera to complete camera calibration; The laser emitter emits laser to the object surface, and the calibrated camera is used to collect the original laser stripe images of the laser irradiated on the object surface; Based on the centerline extraction algorithm of improved gray center of gravity and Jacobian matrix, the centerline of the original image is extracted to obtain the initial centerline point set, and a denoising algorithm based on the random sample consensus algorithm is used to filter the noise points in the initial centerline point set to obtain the centerline point set after filtering the noise points; Based on the principle of pinhole imaging and the properties of the special orthogonal group, the coordinates of all points in the centerline point set in the camera coordinate system are obtained, and combined with the least squares and random parameter search methods, the optimal optical plane equation is obtained; According to the centerline point set, internal parameters, external parameters, distortion coefficients, and the optimal optical plane equation, the linear equations are solved to obtain the coordinates of the object surface contour points in the camera coordinate system and transformed into the world coordinate system to obtain the three-dimensional point cloud data of the object surface contour points, completing the three-dimensional reconstruction.

[0006] In one embodiment, the multi-height calibration is carried out within the full range of the line laser through step standard blocks and two-dimensional targets of different sizes, specifically including: Adjust the height of the workbench so that the working distance of the laser emitter covers the full range; Adjust the pose of the two-dimensional target and save the photo of the two-dimensional target after adjusting the pose; Place the first gauge block, and place the two-dimensional target on the first gauge block. After adjusting the pose, save the photo to complete the calibration of the height of the first gauge block; Repeat placing different standard gauge blocks until the calibration of all heights within the full range is completed.

[0007] In one embodiment, the corner points adjacent to the laser are selected as the solution domain, and the Zhang's calibration method is used to solve the external parameters, internal parameters, and distortion coefficients of the camera to complete camera calibration, specifically including: Select the corner points adjacent to the laser as the solution domain of the Zhang's calibration method; Add the constraint of the special orthogonal group to the unconstrained least squares problem of nonlinear optimization to transform it into a constrained least squares problem; Use the Lagrange multiplier method to solve the constrained least squares problem, and use the LM algorithm to obtain the internal parameter matrix, external parameter matrix, and distortion coefficients; Reproject the rotation matrix in the external parameter matrix from the matrix space to the SE3 manifold space to obtain the optimal rotation matrix, and obtain the external parameters, internal parameters, and distortion coefficients of the camera according to the optimal rotation matrix to complete camera calibration.

[0008] In one embodiment, the centerline extraction algorithm based on improved gray - scale centroid and Jacobi matrix extracts the centerline from the original image to obtain an initial centerline point set, which specifically includes: performing 16 - fold down - sampling on the original image using max - pooling; performing line scanning on the down - sampled image to obtain the ROI region; performing image processing on the ROI region and calculating the Jacobi matrix in the y - direction; using preset high and low thresholds to judge the Jacobi matrix, selecting candidate center points; performing weighted averaging on all candidate center points in the y - direction with the weight being the gray - scale value to obtain the final center points, and forming an initial centerline point set according to all the final center points.

[0009] In one embodiment, the denoising algorithm based on the random sample consensus (RANSAC) algorithm filters the noise points in the initial centerline point set to obtain a centerline point set after filtering the noise points, which specifically includes: randomly selecting two points in the initial centerline point set, calculating the straight - line equation passing through the two points, denoted as the standard straight - line; based on the standard straight - line, setting a threshold T, where the threshold T represents the distance from a point to the standard straight - line; calculating the distances from all points in the initial centerline point set to the standard straight - line, counting the number of points with distances less than the threshold T, denoted as n1; repeatedly and with replacement selecting different two points and calculating the number of points in the initial centerline point set with distances less than the threshold T to the standard straight - line, obtaining n1, n2…n k ; selecting the maximum value n k from n1, n2…n max = max i∈[1，k] {n j}, and identifying the points in the initial centerline point set with distances less than the threshold T to the standard straight - line as center points and the points with distances greater than or equal to the threshold T as noise points, obtaining a centerline point set after filtering the noise points.

[0010] In one embodiment, based on the principle of pinhole imaging and the properties of the special orthogonal group, the coordinates of all points in the centerline point set in the camera coordinate system are obtained, which specifically includes: according to the pinhole imaging model, the correspondence between points in the pixel coordinate system and the camera coordinate system is:

[0011]

[0012] where f x , f y are the focal lengths of the camera in the x - axis and y - axis directions respectively, (u, v) are the coordinates of the centerline point set in the pixel coordinate system, and (x c , y c , z c ) are the coordinates of the corresponding points in the camera coordinate system; based on the principle of pinhole imaging, there is:

[0013]

[0014] The points in the center line point set are located on the target plane in the camera coordinate system. According to the properties of the rotation matrix, the third column vector of the optimal rotation matrix is as follows:

[0015]

[0016] Among them, the third column vector of the optimal rotation matrix is the component of the Z-axis of the world coordinate system in the camera coordinate system. Then r3 is the normal vector of the target plane in the camera coordinate system, and the coordinates of the origin of the original world coordinate system in the camera coordinate system are (t1, t2, t3). Then the cross product of the two is 0, that is:

[0017] r 13 (x c -t1)+r 23 (y c -t2)+r 33 (z c -t3)=0 (4)

[0018] After arrangement, the target plane equation is obtained as:

[0019] r 13 x c +r 23 y c +r 33 z c -(r 13 t1+r 23 t2+r 33 t3)=0 (5)

[0020] By combining the transformation from the pixel coordinate system to the camera coordinate system and the target plane equation, we get:

[0021]

[0022] The coordinates of the center line in the camera coordinate system are solved as:

[0023]

[0024] According to Equation (7), the coordinates of the points projected by the laser emitter on the target plane in the camera coordinate system are calculated.

[0025] In one embodiment, the optimal optical plane equation is obtained by combining the least squares method and the random parameter search method, which specifically includes: setting the number of random parameter searches as m and the number of feature points as f, and setting the optical plane equation as:

[0026] z=ax+by+c (8)

[0027] Randomly extract f feature points from the centerline point set based on the truncated normal distribution, and the corresponding probability distribution is:

[0028]

[0029] Solve based on the f feature points extracted by the least squares method, and the formula is:

[0030]

[0031] Adopt the random parameter search strategy to calculate the average distance ∈i from all points to the light plane equation, and complete the first random search:

[0032]

[0033] Repeat the least squares fitting plane equation and random search until m parameter searches are completed; calculate the minimum average distance during the search to obtain the optimal solution in the parameter space, denoted as the optimal parameter, which is:

[0034]

[0035] Substitute the optimal parameter into the light plane equation to obtain the optimal light plane equation.

[0036] In one of the embodiments, according to the centerline point set, internal parameters, external parameters, distortion coefficients, and the optimal light plane equation, solve the linear equations to obtain the coordinates of the object surface contour points in the camera coordinate system and convert them to the world coordinate system to obtain the three-dimensional point cloud data of the object surface, and complete the two-dimensional to three-dimensional reconstruction. Specifically, it includes: when the optimal parameters are A, B, and C respectively, the optimal light plane equation of the laser plane in the camera coordinate system is:

[0037] z c = Ax c + By c + C (13)

[0038] Combine equations (1) and (13) to obtain the calculation formula for the points in the pixel coordinate system to the camera coordinate system, which is:

[0039]

[0040] Calculate the coordinates of all points in the centerline point set in the camera coordinate system according to the above formula; use the calibrated external parameters to convert the coordinates of all points in the centerline point set in the camera coordinate system to the coordinates in the world coordinate system, and the formula is:

[0041]

[0042] According to the coordinates of all points in the center line point set in the world coordinate system, the three-dimensional point cloud data of the object contour is obtained, and based on the three-dimensional point cloud data of the object contour, the three-dimensional reconstruction of the object is completed.

[0043] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: Through stepped standard blocks and two-dimensional targets of different sizes, multi-height calibration is carried out within the full range of the on-line laser. The corner points adjacent to the laser are selected as the solution domain, and the Zhang's calibration method is adopted to ensure the reconstruction accuracy at different heights, and the external parameters, internal parameters and distortion parameters of the camera are solved to complete camera calibration; The laser emitter emits laser to the object surface, and the calibrated camera is used to collect the original image of the laser irradiated on the object surface; Based on the improved gray center of gravity algorithm and the center line extraction algorithm based on the Jacobian matrix, the center line of the original image is extracted to obtain the initial center line point set, which improves the accuracy and efficiency of the laser stripe center line extraction. Further, a denoising algorithm based on the random sample consensus algorithm is adopted to filter the noise points of the initial center line point set to obtain the center line point set after filtering the noise points, improving the accuracy of the center line extraction algorithm; Based on the constraints of pinhole imaging and special orthogonal group, the coordinates of all points in the center line point set in the camera coordinate system are obtained, and the optimal optical plane equation is obtained by combining the least squares method and the random parameter search method; According to the center line point set, internal parameters, external parameters, distortion coefficients and the optimal optical plane equation, the linear equations are solved to obtain the coordinates of the object surface contour points in the camera coordinate system and convert them to the world coordinate system to obtain the three-dimensional point cloud data of the object surface contour, completing the three-dimensional reconstruction. This method improves the accuracy of line laser three-dimensional reconstruction, simplifies the calibration process and reduces the calibration cost. Description of the Drawings

[0044] Figure 1 It is a schematic flow chart of a high-precision line laser three-dimensional reconstruction calibration method in an embodiment;

[0045] Figure 2 It is a schematic structural diagram of a line laser three-dimensional reconstruction system in an embodiment;

[0046] Figure 3 It is the relationship between points and normal vectors on a checkerboard in an embodiment;

[0047] Figure 4 It is a schematic diagram of the effect of the traditional center line extraction algorithm;

[0048] Figure 5 It is a schematic diagram of the effect of the method based on the improved gray center of gravity and Jacobian matrix in an embodiment. Detailed Embodiment

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] In one embodiment, as Figures 1 to 5 shown, a high-precision line laser three-dimensional reconstruction calibration method is provided, including the following steps:

[0051] Step S101, through step standard blocks and two-dimensional targets of different sizes, perform multi-height calibration within the full range of the line laser, select the corner points adjacent to the laser as the solution domain, and use the Zhang calibration method to solve the external parameters, internal parameters and distortion coefficients of the camera, and complete the camera calibration.

[0052] Specifically, the essence of camera calibration lies in solving the mapping from three-dimensional points in space to two-dimensional points on the image plane. Therefore, the detection accuracy of corner points on the image plane determines the upper limit of the accuracy of the entire three-dimensional reconstruction system.

[0053] At the same time, since there may be a large height difference on the surface of the object when the line laser scans, different from the traditional calibration method, step standard blocks and two-dimensional targets of different sizes are innovatively used to perform height-by-height calibration within the full range of the line laser to ensure the reconstruction accuracy at different heights within the range.

[0054] In addition, due to the characteristics that the camera has clear imaging near the laser plane, the more distant from the laser, the more noise and the more serious image distortion. Therefore, when calibrating the camera, the corner points adjacent to the laser are selected as the solution domain of the Zhang calibration method, and the solved rotation matrix is re-projected from the matrix space to the SE3 manifold space, so as to ensure that the rotation matrix satisfies the constraints of the special orthogonal group and is optimal in this solution domain.

[0055] Through the above calibration method, corner point selection strategy and processing idea of the rotation matrix, the camera calibration can be completed simply and efficiently, and the calibration accuracy is relatively high.

[0056] Among them, the steps of performing camera height calibration specifically include: adjusting the height of the workbench so that the working distance of the laser emitter covers the full range; adjusting the pose of the two-dimensional target and saving the photo of the two-dimensional target with the adjusted pose; placing the first gauge block, placing the two-dimensional target on the first gauge block, and after adjusting the pose, saving the photo to complete the calibration of the height of the first gauge block; repeating the placement of different standard gauge blocks until the calibration of all heights within the full range is completed.

[0057] Specifically, when the line laser scans, there may be a large height difference on the surface of the object. Therefore, stepped standard blocks of different sizes, such as 1 mm, 5 mm, and 10 mm standard blocks, can be used. Combined with a two-dimensional target, height-by-height calibration is performed within the full range of the line laser to ensure the reconstruction accuracy at different heights. The specific calibration process is as follows: adjust the height of the workbench so that the working distance of the laser emitter can cover the full range; adjust the pose of the two-dimensional target and save the corresponding photos; place the 1 mm standard block, place the two-dimensional target on the block, adjust the pose, and save the corresponding photos to complete the calibration at a height of 1 mm; and so on, complete the calibration at all heights within the full range. According to this calibration method, combined with the corner point selection strategy and the processing idea of the rotation matrix, camera calibration can be completed simply and efficiently, and the calibration accuracy is relatively high.

[0058] Among them, the steps for solving camera calibration specifically include: selecting the corner points adjacent to the laser as the solution domain of the Zhang's calibration method; adding the constraint of the special orthogonal group to the unconstrained least squares problem of nonlinear optimization to transform it into a constrained least squares problem; using the Lagrange multiplier method to solve the constrained least squares problem and using the LM algorithm to obtain the internal parameter matrix, external parameter matrix, and distortion coefficients; re-projecting the rotation matrix from the matrix space to the SE3 manifold space to obtain the optimal rotation matrix, and further obtaining the translation matrix, internal parameters, and distortion coefficients of the camera to complete camera calibration.

[0059] Specifically, the traditional solution is to add the constraint of the special orthogonal group to the unconstrained least squares problem of nonlinear optimization to transform it into a constrained least squares problem, and then use the Lagrange multiplier method to solve this constrained least squares problem or directly perform QR decomposition on the solved rotation matrix, and use the obtained orthonormal matrix Q as the rotation matrix. However, due to the high distortion of the camera in the line laser three-dimensional reconstruction system, when performing camera calibration based on the Zhang's calibration method, the detection error of the checkerboard corner points is relatively high, resulting in a large error in the subsequent solved internal parameter matrix, and the rotation matrix in the external parameters no longer satisfies the constraint of the special orthogonal group SO(3). At this time, the initial solution is no longer near the optimal solution space, and the accuracy of the solved internal and external parameter matrices is poor. Therefore, it is necessary to re-project the solved rotation matrix from the matrix space to the SE(3) manifold. The formula is as follows:

[0060]

[0061] Use the re-projected rotation matrix as the optimal rotation matrix, and calculate the external parameters, internal parameters, and distortion coefficients of the camera according to the optimal rotation matrix, so as to complete camera calibration.

[0062] Step S102: Emit laser from the laser emitter to the surface of the object, and use the calibrated camera to collect the original laser stripe image of the laser irradiated on the surface of the object.

[0063] Specifically, through a three-dimensional reconstruction system as shown in Figure 2 , a laser emitter emits laser light onto the surface of the object to be reconstructed, and a calibrated camera collects the original laser stripe images of the laser light irradiated on the object surface. At this time, image processing such as distortion correction can be performed on the original images to improve the reconstruction accuracy. Through the collected original images, centerline extraction and three-dimensional reconstruction of the object are performed, and the complete three-dimensional shape of the object is obtained by continuously collecting, extracting, and reconstructing, thereby realizing the three-dimensional reconstruction of the object.

[0064] Step S103: Based on the centerline extraction algorithm of improved gray center of gravity and Jacobian matrix, extract the centerline from the original image to obtain an initial centerline point set, and use a denoising algorithm based on the random sample consensus algorithm to filter out the noise points in the initial centerline point set, obtaining a centerline point set after filtering the noise points.

[0065] Specifically, in the three-dimensional reconstruction method in the prior art, there is a problem that the laser stripe centerline extraction algorithm has low robustness in a complex environment, and the extracted centerline contains irrelevant noise points, which affects the reconstruction accuracy. Based on the problem that the centerline extracted by the centerline extraction algorithm in a complex environment contains noise points, this application combines the traditional gray center of gravity method and the Steger algorithm, and proposes a centerline extraction algorithm based on improved gray center of gravity and Jacobian matrix, which greatly improves the extraction accuracy and extraction efficiency of the laser stripe centerline. Among them, the effects of the Steger algorithm and the improved algorithm in this application for extracting the centerline are respectively as shown in Figure 4 and Figure 5 shown.

[0066] To further improve the accuracy of the centerline extraction algorithm, a denoising algorithm based on the random sample consensus algorithm is used to filter out the noise points therein, thereby obtaining a centerline point set after filtering the noise points.

[0067] Among them, the steps of obtaining the initial centerline point set specifically include: performing 16-fold downsampling on the original image using max pooling; performing line scanning on the downsampled image in the y direction to obtain the ROI region; performing image processing on the ROI region and calculating the Jacobian matrix in the y direction; using preset high and low thresholds to judge the Jacobian matrix, selecting candidate center points; performing weighted averaging on all candidate center points in the y direction, with the weight being the gray value, to obtain the final center points, and forming an initial centerline point set according to all the final center points.

[0068] Specifically, the center extraction algorithm based on improved gray - scale centroid and Jacobian matrix is as follows: First, perform max - pooling on the original image, take the point with the largest value in the pooling area, and perform 16 - fold down - sampling; then perform line scanning on the down - sampled image to obtain the ROI (region of interest); perform image processing on the ROI area, such as mean filtering, histogram equalization, etc., which is convenient for subsequent processing and enhances the robustness of the algorithm at the same time, and calculate the Jacobian matrix in the y - direction of the ROI image; use preset high and low thresholds to judge the Jacobian matrix, select candidate center points; perform weighted average on all candidate center points in the y - direction, with the weight being the gray - scale value, so as to obtain the final center point. Repeat the above steps to obtain all the final center points and form the initial center - line point set.

[0069] Among them, the steps of filtering noise points from the initial center - line point set specifically include: Randomly select two points in the initial center - line point set, calculate the straight - line equation passing through the two points, denoted as the standard straight - line; based on the standard straight - line, set a threshold T, where the threshold T represents the distance from a point to the standard straight - line; calculate the distances from all points in the initial center - line point set to the standard straight - line, and count the number of points with distances less than the threshold T, denoted as n1; repeatedly select different two points with replacement and calculate the number of points in the initial center - line point set with distances less than the threshold T to the standard straight - line, obtaining n1, n2…n k ; Select the maximum value among n1, n2…n k which is n max = max i∈[1，k] {n i}, identify the points in the initial center - line point set whose distances to the standard straight - line are less than the threshold T as center points, and the points greater than or equal to the threshold T as noise points, obtaining the center - line point set after filtering noise points.

[0070] Specifically, randomly select two points in the initial center - line point set, obtain the corresponding straight - line equation according to the selected two points, denoted as the standard straight - line; based on the standard straight - line, set a threshold T, where the threshold T represents the distance from a point in the initial center - line point set to the standard straight - line, and count the number of points with distances less than the threshold T; repeatedly select different two points with replacement to fit the straight - line equation and count the number of points with distances less than the threshold T, select the threshold T with the largest number of points with distances less than the threshold T among them, denote the points in the corresponding initial center - line point set with distances less than the threshold T to the standard straight - line as correct center points, denote the size of the point set at this time as m, and the remaining points as noise points. Then the center - line point set after filtering noise points is γ, and there is |γ| = m. Through the above method, the initial center - line point set is filtered for noise points, thereby further improving the accuracy of the center - line extraction algorithm.

[0071] Step S104: Based on the principle of pinhole imaging and the properties of the special orthogonal group, obtain the coordinates of all points in the centerline point set in the camera coordinate system, and combine the least squares method and the random parameter search method to obtain the optimal optical plane equation.

[0072] Specifically, based on the principle of pinhole imaging and the constraints of the special orthogonal group between coordinate systems, obtain the coordinates of the center point of the laser stripe in the camera coordinate system. Since the accuracy of the optical plane has a significant impact on the final reconstruction effect, further combine the least squares method and the random parameter search method to solve and obtain the optimal optical plane equation, thereby improving the 3D reconstruction accuracy.

[0073] Among them, the steps to obtain the coordinates of all points in the centerline point set in the camera coordinate system specifically include: According to the pinhole imaging model, the correspondence between the pixel coordinate system and the camera coordinate system is:

[0074]

[0075] In the formula, f x and f y are the focal lengths of the camera in the x-axis and y-axis directions respectively, (u, v) are the coordinates of the points in the centerline point set, and (x c , y c , z c ) are the coordinates of the corresponding points in the camera coordinate system;

[0076] Based on the principle of pinhole imaging, we have:

[0077]

[0078] The points in the centerline point set are located on the target plane in the camera coordinate system. Obtain the third column vector of the optimal rotation matrix from the properties of the rotation matrix, which is:

[0079]

[0080] Among them, the third column vector of the optimal rotation matrix is the component of the Z-axis of the world coordinate system in the camera coordinate system. Then r3 is the normal vector of the target plane in the camera coordinate system, and the coordinates of the origin of the original world coordinate system in the camera coordinate system are (t1, t2, t3). Then the vector product of the two is 0, that is:

[0081] r 13 (x c -t1) + r 23 (y c -t2) + r 33 (z c -t3) = 0 (4)

[0082] After arrangement, the equation of the target plane is obtained as:

[0083] r13 x c +r 23 y c +r 33 z c -(r 13 t1+r 23 t2+r 33 t3) = 0 (5)

[0084] By simultaneously solving the transformation from the pixel coordinate system to the camera coordinate system and the target plane equation, we get:

[0085]

[0086] The coordinates of the points on the center line in the camera coordinate system are solved as:

[0087]

[0088] According to Equation (7), the coordinates of the points projected by the laser emitter on the target plane in the camera coordinate system are calculated.

[0089] Specifically, based on the filtered center line point set, the relationship between the pixel coordinate system and the camera coordinate system is obtained according to the pinhole imaging model, so that the points in the center line point set can be converted into coordinates in the camera coordinate system. Since the points in the center line point set are located on the target plane in the camera coordinate system, according to the properties of the rotation matrix, the third column vector of the optimal rotation matrix is the component of the z-axis of the world coordinate system in the camera coordinate system. Therefore, r3 is the direction vector of the target plane in the camera coordinate system, as Figure 3 shown, and according to the point-normal form of the plane equation, the vector product between the coordinates of the origin of the original world coordinate system in the camera coordinate system and r3 is 0, thus obtaining the coordinates of the center point of the laser stripe in the camera coordinate system.

[0090] Among them, the steps to obtain the optimal light plane equation specifically include: Let the number of random parameter searches be m, the number of feature points be f, and the light plane equation be:

[0091] z = ax + by + c (8)

[0092] Based on the truncated normal distribution, f feature points are randomly extracted from the center line point set, and the corresponding probability distribution is:

[0093]

[0094] By the least squares method, based on the extracted f feature points, the solution is obtained, and the formula is:

[0095]

[0096] Using a random parameter search strategy, the average distance ∈i from all points to the light plane equation is calculated to complete the first random search:

[0097]

[0098] Repeat the least squares and random search until m parameter searches are completed; calculate the minimum average distance in the search process to obtain the optimal solution in the parameter space, which is recorded as the optimal parameter:

[0099]

[0100] Substituting the optimal parameters into the light plane equation, the optimal light plane equation is obtained.

[0101] Specifically, according to the calculated coordinates of the points projected by the laser emitter on the target plane in the camera coordinate system, and these points are all located on the light plane, theoretically, the light plane equation can be directly fitted based on these points. However, since there is still a certain error in the centerline extraction, it is necessary to further extract feature points from the centerline point set, fit according to the feature points, and combine the random parameter search strategy to obtain the optimal solution of the parameter space, thereby obtaining the optimal light plane equation. The steps of the algorithm specifically include: setting the number of searches and the number of feature points of the random parameters to obtain the corresponding light plane equation, based on the truncated normal distribution, randomly selecting feature points in the center point set; and using the least squares method to process the selected feature points to obtain the light plane equation, calculating the average distance from all feature points to the light plane equation, completing the first random search, repeating the least squares and average distance calculations until all searches are completed; calculating the minimum average distance in all search processes to obtain the optimal parameters, and obtaining the corresponding optimal light plane equation based on the optimal parameters. The optimal light plane equation is obtained by the above method, which improves the reconstruction accuracy.

[0102] Step S105, based on the centerline point set, internal parameters, external parameters, distortion coefficients and optimal light plane equation, solve the linear equation group to obtain the coordinates of the object surface contour points in the camera coordinate system and convert them to the world coordinate system to obtain the three-dimensional point cloud data of the object contour and complete the three-dimensional reconstruction.

[0103] Specifically, according to the centerline point set, internal parameters, external parameters, distortion coefficients and optimal light plane equation, a group of simultaneous equations is established to solve the linear equations to obtain the coordinates of the object surface contour points in the camera coordinate system, and then convert them into the world coordinate system to obtain the three-dimensional point cloud data of the object surface contour. At the same time, based on the three-dimensional point cloud data of the object surface contour, the three-dimensional reconstruction of the object is completed, thereby simplifying the calibration process, reducing the calibration cost, and improving the three-dimensional reconstruction accuracy.

[0104] Among them, step S105 specifically includes: when the optimal parameters are A, B, and C respectively, the optimal optical plane equation of the laser plane in the camera coordinate system is:

[0105] z c = Ax c + By c + C (13)

[0106] By combining equations (1) and (13), the calculation formula for the points in the pixel coordinate system to the camera coordinate system is obtained as:

[0107]

[0108] According to the above formula, the coordinates of all points in the center line point set in the camera coordinate system are calculated; using the calibrated external parameters, the coordinates of all points in the center line point set in the camera coordinate system are converted into the coordinates in the world coordinate system, and the formula is:

[0109]

[0110] Based on the coordinates of all points in the center line point set in the world coordinate system, the three-dimensional point cloud data of the object contour is obtained, and based on the three-dimensional point cloud data of the object contour, the three-dimensional reconstruction of the object is completed.

[0111] Specifically, according to the optimal parameters, the corresponding optimal optical plane equation is obtained. According to the pinhole imaging model, the conversion from pixel coordinates to camera coordinates is realized, and combined with the optimal optical plane equation, external parameters, internal parameters, distortion coefficients, and the center line point set, the linear equations are solved to obtain the coordinates of the object surface contour points in the camera coordinate system and convert them to the world coordinate system, obtaining the three-dimensional point cloud data of the object surface contour. The three-dimensional reconstruction of the object is completed based on the three-dimensional point cloud coordinates, and the accuracy of the three-dimensional reconstruction is high.

[0112] In this embodiment, multi-height calibration is performed within the full range of the on-line laser through stepped standard gauge blocks and two-dimensional targets of different sizes. The corner points adjacent to the laser are selected as the solution domain, and the Zhang's calibration method is used to ensure the reconstruction accuracy at different heights, and the external parameters, internal parameters, and distortion parameters of the camera are solved to complete camera calibration. The laser emitter emits laser to the surface of the object, and the calibrated camera is used to collect the original image of the laser irradiated on the surface of the object. Based on the improved gray center-of-gravity algorithm and the centerline extraction algorithm based on the Jacobian matrix, the centerline of the original image is extracted to obtain the initial centerline point set, which improves the accuracy and efficiency of laser stripe centerline extraction. Further, a denoising algorithm based on the random sample consensus algorithm is used to filter the noise points of the initial centerline point set to obtain the centerline point set after noise filtering, improving the accuracy of the centerline extraction algorithm. Based on the constraints of pinhole imaging and special orthogonal group, the coordinates of all points in the centerline point set in the camera coordinate system are obtained, and the optimal optical plane equation is obtained by combining the least squares method and the random parameter search method. According to the centerline point set, internal parameters, external parameters, distortion coefficients, and the optimal optical plane equation, the linear equations are solved to obtain the coordinates of the object surface contour points in the camera coordinate system and convert them to the world coordinate system to obtain the three-dimensional point cloud data of the object surface contour, completing the three-dimensional reconstruction. This method improves the accuracy of line laser three-dimensional reconstruction, simplifies the calibration process, and reduces the calibration cost.

[0113] In one embodiment, the height of the standard gauge block can also be reconstructed by using this method and the traditional three-dimensional reconstruction method respectively, and their accuracies are compared. For example, standard gauge blocks with sizes of 1 mm, 5 mm, 10 mm, and 20 mm are selected, and three-dimensional reconstruction is performed by using this method and the traditional method respectively, and the height of the image after three-dimensional reconstruction is measured, as shown in Table 1:

[0114] Table 1 Height dimension table of the standard gauge block reconstructed by this method and the traditional method

[0115]

[0116] According to Table 1, the height of the standard gauge block reconstructed by this method is closer to the actual height of the standard gauge block than that of the traditional method. Therefore, this method has a smaller error and higher accuracy than the traditional line laser three-dimensional reconstruction calibration method, and the three-dimensional reconstruction effect is better.

[0117] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0118] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device. Thus, they can be stored in a computer storage medium (ROM / RAM, magnetic disk, optical disk) and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to be implemented. Therefore, the present invention is not limited to any specific combination of hardware and software.

[0119] The above content is a further detailed description of the present invention in combination with specific implementation manners. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention belongs, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A high-precision line laser three-dimensional reconstruction calibration method, characterized in that, Including the following steps: Perform multi-height calibration within the full range of the on-line laser through stepped standard blocks and two-dimensional targets of different sizes, including: adjusting the height of the workbench so that the working distance of the laser emitter covers the full range; adjusting the pose of the two-dimensional target and saving the photo of the two-dimensional target after the pose is adjusted; placing the first block and placing the two-dimensional target on the first block, and after adjusting the pose, saving the photo to complete the calibration of the height of the first block; repeating the placement of different standard blocks until the calibration of all heights within the full range is completed; Select the corner points adjacent to the laser as the solution domain, and use the Zhang's calibration method to solve the external parameters, internal parameters and distortion coefficients of the camera to complete the camera calibration; Emit laser from the laser emitter to the object surface, and use the calibrated camera to collect the original laser stripe image of the laser irradiated on the object surface; Based on the centerline extraction algorithm of improved gray center of gravity and Jacobian matrix, extract the centerline of the original image to obtain the initial centerline point set, including: performing 16-fold downsampling on the original image using max pooling; performing line scanning on the downsampled image to obtain the ROI region; performing image processing on the ROI region to calculate the Jacobian matrix in the y direction; using preset high and low thresholds to judge the Jacobian matrix, and selecting candidate center points; performing weighted average on all candidate center points in the y direction with the weight being the gray value to obtain the final center point, and forming the initial centerline point set according to all the final center points; Use the denoising algorithm based on the random sample consensus algorithm to filter the noise points in the initial centerline point set to obtain the centerline point set after filtering the noise points; Based on the principle of pinhole imaging and the properties of the special orthogonal group, obtain the coordinates of all points in the centerline point set in the camera coordinate system, and combine the least squares theory and the random parameter search method to obtain the optimal optical plane equation; According to the centerline point set, internal parameters, external parameters, distortion coefficients and the optimal optical plane equation, solve the linear equations to obtain the coordinates of the object surface contour points in the camera coordinate system and convert them to the world coordinate system to obtain the three-dimensional point cloud data of the object surface contour and complete the three-dimensional reconstruction.

2. The high-precision line laser three-dimensional reconstruction calibration method according to claim 1, characterized in that, The step of selecting the corner points adjacent to the laser as the solution domain and using the Zhang's calibration method to solve the external parameters, internal parameters and distortion coefficients of the camera to complete the camera calibration specifically includes: Select the corner points adjacent to the laser as the solution domain of the Zhang's calibration method; Add the constraint of the special orthogonal group to the unconstrained least squares problem of nonlinear optimization to transform it into a constrained least squares problem; Use the Lagrange multiplier method to solve the constrained least squares problem and use the LM algorithm to obtain the internal parameter matrix, external parameter matrix and distortion coefficients; Reproject the rotation matrix in the external parameter matrix from the matrix space to the SE3 manifold space to obtain the optimal rotation matrix, and obtain the external parameters, internal parameters and distortion coefficients of the camera according to the optimal rotation matrix to complete the camera calibration.

3. The high-precision line laser three-dimensional reconstruction calibration method according to claim 1, characterized in that, The step of using the denoising algorithm based on the random sample consensus algorithm to filter the noise points in the initial centerline point set to obtain the centerline point set after filtering the noise points specifically includes: Randomly select two points from the initial centerline point set, calculate the straight-line equation passing through the two points, and denote it as the standard straight line; Based on the standard straight line, set a threshold T, where the threshold T represents the distance from a point to the standard straight line; Calculate the distances from all points in the initial centerline point set to the standard straight line, count the number of points with distances less than the threshold T, and denote it as n1; Select two different points repeatedly with replacement, and calculate the number of points in the initial centerline point set whose distance to the standard line is less than the threshold T, obtaining n1, n2... n k ; Select n1, n2…n k The maximum value n in max = max i∈[1,k] {n i}, and identify the points in the initial center line point set corresponding to the maximum value whose distance to the standard line is less than the threshold T as the center points, and the points greater than or equal to the threshold T as noise points, so as to obtain the center line point set after filtering the noise points.

4. The high-precision line laser three-dimensional reconstruction calibration method according to claim 2, characterized in that, Based on the principle of pinhole imaging and the properties of the special orthogonal group, obtain the coordinates of all points in the centerline point set in the camera coordinate system, specifically including: According to the pinhole imaging model, the correspondence between points in the pixel coordinate system and the camera coordinate system is: where f x , f y are the focal lengths of the camera in the x-axis and y-axis directions respectively, (u, v) are the coordinates of the center line point set in the pixel coordinate system, and (x c , y c , z c ) are the corresponding coordinates in the camera coordinate system; Based on the principle of pinhole imaging, we have: The points in the centerline point set are located on the target plane in the camera coordinate system. The third column vector of the optimal rotation matrix is obtained from the properties of the rotation matrix, which is: Among them, the third column vector of the optimal rotation matrix is the component of the Z-axis of the world coordinate system in the camera coordinate system. Then r3 is the normal vector of the target plane in the camera coordinate system, and the coordinates of the origin of the original world coordinate system in the camera coordinate system are (t1, t2, t3). Then the vector product of the two is 0, that is: r 13 (x c - t1) + r 23 (y c - t2) + r 33 (z c - t3) = 0 (4) After arrangement, the target plane equation is obtained as: r 13 x c +r 23 y c +r 33 z c -(r 13 t1+r 23 t2+r 33 t3) = 0 (5) Combine the transformation from the pixel coordinate system to the camera coordinate system and the target plane equation to get: Solve to obtain the coordinates of the points on the centerline in the camera coordinate system as: Calculate the coordinates of the points projected by the laser emitter on the target plane in the camera coordinate system according to Equation (7).

5. A high-precision line laser three-dimensional reconstruction calibration method according to claim 4, characterized in that, Combine the least squares theory and the random parameter search method to obtain the optimal light plane equation, specifically including: Let the number of random parameter searches be m, and the number of feature points be f. Let the light plane equation be: z = ax + by + c (8) Randomly extract f feature points from the centerline point set based on the truncated normal distribution, and the corresponding probability distribution is: Solve based on the extracted f feature points by the least squares method, and the formula is: Using a random parameter search strategy, calculate the average distance of all points to the light plane equation ∈ i , and complete the first random search: Repeat the least squares and random search until m parameter searches are completed; Calculate the minimum average distance during the search process to obtain the optimal solution in the parameter space, denoted as the optimal parameter, which is: Substitute the optimal parameter into the light plane equation to obtain the optimal light plane equation.

6. A high-precision line laser three-dimensional reconstruction calibration method according to claim 5, characterized in that, According to the centerline point set, internal parameters, external parameters, distortion coefficients, and the optimal light plane equation, solve the linear equations to obtain the coordinates of the object surface contour points in the camera coordinate system and convert them to the world coordinate system to obtain the three-dimensional point cloud data of the object surface, and complete the three-dimensional reconstruction, specifically including: When the optimal parameters solved are A, B, and C respectively, the optimal light plane equation of the laser plane in the camera coordinate system is: z c = Ax c + By c + C (13) Combine Equation (1) and (13) to obtain the calculation formula for the points in the pixel coordinate system to the points in the camera coordinate system, which is: Calculate the coordinates of all points in the centerline point set in the camera coordinate system according to the above formula; Use the calibrated external parameters to convert the coordinates of all points in the centerline point set in the camera coordinate system to the coordinates in the world coordinate system, and the formula is: According to the coordinates of all points in the centerline point set in the world coordinate system, obtain the three-dimensional point cloud data of the object surface and complete the three-dimensional reconstruction of the object.

Images