A 3D shape measurement method for complex structural parts based on a robot

Abstract

The present invention discloses a three-dimensional shape measurement method for complex structural parts based on a robot. First, a mobile robot drives a binocular camera to collect left and right camera images of a measurement object at multiple viewing poses, reconstruct single-viewpoint cloud data, and record the robot pose information. Then, hand-eye calibration is performed on the conversion relationship from the end flange coordinate system of the robot to the binocular camera coordinate system, and further, the conversion matrix from the binocular camera coordinate system to the robot base coordinate system is obtained, and the single-viewpoint measurement point cloud obtained by the binocular camera is converted to a unified coordinate system to complete the rough registration of multi-viewpoint cloud. Finally, the ICP algorithm is used to perform fine registration on the measurement point cloud to achieve three-dimensional shape measurement. Throughout the process, there is no need to stick fiducial points on the surface of the object to be measured, which simplifies the point cloud registration process; at the same time, the accuracy of the hand-eye relationship is corrected, which can ensure the rough registration accuracy of multi-viewpoint cloud data and greatly improve the efficiency of multi-view three-dimensional shape measurement.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional shape measurement, and more specifically, relates to a three-dimensional shape measurement method for complex structural parts based on a robot. Background Art

[0002] Multi-view three-dimensional shape measurement technology refers to the process of matching and aligning single-view point cloud data obtained at different times and different shooting orientations, which is a very basic and important problem in three-dimensional measurement. The most common registration method in practical applications is the feature-based registration method, such as artificial marking points. However, due to interferences such as moving objects, repetitive structures, noise, and occlusion, the robustness of feature description and feature matching is poor, and the registration accuracy is low. In order to simultaneously satisfy multi-view point cloud data acquisition and markerless registration, an industrial robot is used as a motion carrier, so that while the measurement system retains the characteristics of non-contact and fast of vision measurement technology, the flexibility of the entire system is enhanced due to the fast and flexible characteristics of the robot.

[0003] In order to complete multi-view three-dimensional shape measurement, it is necessary to transform the measured point clouds under different views to a unified coordinate system through rigid transformations such as rotation and translation. This requires the calculation of hand-eye calibration from the measurement device coordinate system to the end flange coordinate system of the industrial robot. The current method of hand-eye calibration for industrial robots mainly realizes it by taking pictures of the calibration object by the camera to identify the pose, obtaining the camera parameters and the robot pose information, and then solving the hand-eye calibration equation. The hand-eye calibration accuracy obtained by such hand-eye calibration methods is easily affected by the accuracy of the camera and the robot, the pose recognition accuracy, and the accuracy of the hand-eye calibration equation solving method, and cumulative errors are formed.

[0004] The existing methods for improving hand-eye calibration accuracy can only be optimized from the pose recognition of the calibration object and the hand-eye calibration equation algorithm, but there is still a gap between the calculated hand-eye relationship matrix and the true value, which cannot meet the requirements of the point cloud registration task. Therefore, it is necessary to adopt a more practical and effective hand-eye relationship correction method to improve the hand-eye calibration accuracy in order to achieve higher-quality point cloud registration. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a three-dimensional shape measurement method for complex structural parts based on a robot, so as to realize that no marker points need to be pasted on the surface of the measurement object throughout the process, simplify the point cloud registration process, and at the same time correct the hand-eye relationship accuracy to ensure the rough registration accuracy of multi-view point cloud data, and greatly improve the efficiency of multi-view three-dimensional shape measurement.

[0006] To achieve the above invention purpose, the three-dimensional shape measurement method for complex structural parts based on a robot of the present invention is characterized in that it includes:

[0007] (1) The binocular camera is driven by the robot to complete the acquisition of left and right camera images of the measurement object at multiple viewing poses, reconstruct the single-viewpoint cloud, and record the robot pose information.

[0008] 1.1) Mount the binocular camera on the fixing device of the robot end flange, so that the left and right cameras are kept at the same horizontal position and a certain distance is reserved.

[0009] 1.2) Debug the binocular camera so that it can clearly capture the measurement object, and debug the robot so that it can carry the binocular camera to perform the 3D measurement task, ensure that the entire picture of the measurement object is captured, and use the Zhang's calibration method for camera calibration. For the measurement object at the k-th, k = 1, 2,..., n robot poses, a grayscale image is taken by the left and right cameras respectively.

[0010] 1.3) Take a grayscale image for the left and right cameras respectively, perform point cloud reconstruction according to the binocular camera principle, and obtain the single-viewpoint cloud at the k-th robot pose. And obtain the robot pose matrix at this viewing angle.

[0011] (2) Perform hand-eye calibration on the conversion relationship from the robot end flange coordinate system to the binocular camera coordinate system, and then obtain the conversion matrix from the binocular camera coordinate system to the robot base coordinate system, and convert the single-viewpoint cloud obtained by the binocular camera to the unified coordinate system to complete the rough registration of the multi-viewpoint cloud.

[0012] 2.1) Mount the binocular camera on the fixing device of the robot end flange, control the robot to move to the shooting viewing angle, and use the binocular camera to take a calibration board picture at this shooting viewing angle, while recording the robot pose information.

[0013] 2.2) Repeat step 2.1) to obtain m groups of calibration board pictures and the corresponding robot pose information.

[0014] 2.3) Obtain the rotation vector R of the calibration board relative to the camera in each group of calibration board pictures according to the Zhang's calibration method. ci And the translation vector T ci , and convert them into a rotation-translation matrix, that is, obtain the external camera parameter matrix H ci , i = 1, 2,..., m; calculate the rotation vector R gi And the translation vector T gi relative to the base according to the robot pose information, and obtain the robot pose matrix H gi , i = 1, 2,..., m, where:

[0015]

[0016] 2.4), Since the relative pose relationship between the robot base and the calibration board, i.e., the base calibration board relationship matrix H, remains fixed between any two transformation poses i and j, and the relative pose relationship between the camera and the robot end flange, i.e., the hand-eye relationship matrix H, remains fixed, there is the following conversion relationship: BW H cg = H

[0017] H BW H gi H cg H ci

[0018] H BW = H gj H cg H cj

[0019] By means of the camera extrinsic parameter matrix H ci , i = 1, 2,..., n and the robot pose matrix H gi , i = 1, 2,..., m, combined with the fixed relationship between the robot base coordinate system and the calibration board coordinate system, a hand-eye calibration equation is established:

[0020] AX = XB;

[0021] where A = H gj -1 H gi , B = H cj H ci -1 , X = H cg .

[0022] And the hand-eye calibration function is used to solve this equation to obtain the rotation and translation matrix of the camera coordinate system relative to the robot end flange coordinate system, i.e., the hand-eye relationship matrix H cg ;

[0023] 2.5), The calculated hand-eye relationship matrix H cg is corrected to obtain the hand-eye relationship matrix for multi-viewpoint cloud rough registration

[0024] 2.5.1), Fix the probe device on the robot end flange, establish the tool coordinate through the five-point method and the teach pendant, obtain the position coordinate ΔP0 of the probe tip relative to the robot end flange coordinate system. The robot drives the probe to move to directly above the four corner points of the first grid in the upper left corner of the calibration board, and record the coordinate of the robot end flange relative to the base coordinate system at this time from the teach pendant. Continuously drive the probe to move to directly above the four corner points of the grids on the calibration board to obtain the coordinates of the robot end flange relative to the base coordinate system directly above the corner points And calculate the true coordinates of all the corner points of the calibration board:

[0025]

[0026] Among them, is the three-dimensional coordinate of the robot end flange relative to the base coordinate system directly above the l-th corner point, Δx0, Δy0, Δz0 are the position coordinates of the probe end relative to the robot end flange coordinate system, L is the number of corner points on the calibration board, T represents the transpose, and B represents the robot base coordinate system;

[0027] 2.5.2) Take and reconstruct the three-dimensional point cloud of the calibration board under a certain robot pose. After reconstructing the three-dimensional point cloud of the calibration board, click on and obtain the point cloud coordinates of the four corner points of the first grid in the upper left corner of the calibration board relative to the binocular camera coordinate system in sequence. According to the calculation, all the point cloud coordinates P1 of the calibration board corner points corresponding to the true coordinates of the corner points can be obtained l :

[0028]

[0029] Among them, is the three-dimensional coordinate representation of the point cloud coordinate P1 l and C represents the binocular camera coordinate system;

[0030] 2.5.3) According to the robot pose information, obtain the robot pose matrix when shooting the calibration board point cloud as H g , and from the known true coordinates of the corner points P0 l , obtain the corner point coordinates in the flange coordinate system converted to this robot pose

[0031]

[0032] Among them, is the three-dimensional coordinate representation of the corner point coordinates and G represents the robot end flange coordinate system; thus, a set of corresponding point sets is obtained

[0033] 2.5.4) Change the robot pose J - 1 times, repeat steps 2.5.2) and 2.5.3), obtain J sets of corresponding point sets between the true coordinates of the corner points and the point cloud coordinates, and re - record them as

[0034] 2.5.5) Let the corrected hand - eye relationship matrix be:

[0035]

[0036] Among them, T′cg = [x′, y′, z′] T is the translation vector to be corrected, and R cg is the calculated hand-eye relationship matrix H cg rotation matrix;

[0037] From the rigid body transformation relationship between a set of corresponding point sets establish a correction function for optimization:

[0038]

[0039] Let the correction function f(P j ) = 0, and substitute each set of point sets Calculate the J translation vectors T′ corresponding to the J sets of point sets cg , and obtain the average value to get the corrected translation vector That is, the final corrected hand-eye calibration matrix is obtained

[0040]

[0041] 2.6), According to the point cloud data reconstructed in step (1) and each perspective pose matrix and the finally corrected hand-eye calibration matrix Calculate the rigid body transformation matrix for 3D point cloud registration

[0042]

[0043]

[0044] Among them, the rigid body transformation matrix reflects the transformation relationship from the robot base coordinate system to the binocular camera;

[0045] Thus, the single-view measurement point cloud data obtained by the binocular camera is converted to the unified coordinate system of the robot base, realizing the rough registration of the point cloud:

[0046]

[0047] Among them, is the single-view point cloud at the k-th pose after rough registration;

[0048] (3), Perform fine registration on the measured point cloud through the ICP algorithm (Iterative Closest Point algorithm) to realize 3D shape measurement

[0049] 3.1), Take the two roughly registered single-view point clouds P homo_point_cloud at adjacent poses as the ICP source point cloud and the ICP target point cloud;

[0050] 3.2) Perform ICP point cloud registration on two single-view point clouds, search for the nearest corresponding points, and iteratively calculate to obtain the rigid body transformation matrix that minimizes the average distance between the corresponding point pairs. When the point cloud distance is less than the set threshold or the set number of iterations is reached, the rigid body transformation matrix between the source point cloud and the target point cloud is output, and the target point cloud is converted to the source point cloud by this matrix to obtain the registered point cloud;

[0051] 3.3) Use the above-mentioned registered point cloud as the ICP source point cloud and the single-view point cloud in the adjacent pose as the target ICP point cloud. Repeat the above steps until all point clouds are accurately registered to obtain the reconstructed 3D shape point cloud data of the measured object.

[0052] The object of the invention of the present invention is achieved like this:

[0053] The present invention is a robot-based three-dimensional shape measurement method for complex structural parts. First, a mobile robot drives a binocular camera to complete left and right camera image acquisition of the measured object at multiple viewing angles and postures, reconstruct single-view point cloud data, and record the robot's posture information. Then, a hand-eye calibration is performed on the conversion relationship between the robot's end flange coordinate system and the binocular camera coordinate system, thereby obtaining a conversion matrix from the binocular camera coordinate system to the robot base coordinate system. The single-view measurement point cloud acquired by the binocular camera is converted to a unified coordinate system to complete the coarse registration of the multi-view point cloud. Finally, the measurement point cloud is finely registered using the ICP algorithm to achieve three-dimensional shape measurement. The entire process does not require the placement of marker points on the surface of the measured object, simplifying the point cloud registration process. At the same time, the hand-eye relationship accuracy is corrected to ensure the coarse registration accuracy of the multi-view point cloud data, greatly improving the efficiency of multi-view three-dimensional shape measurement.

[0054] The related advantages and novelties of the present invention are:

[0055] (1) The present invention uses a high-resolution binocular camera to achieve high-precision point cloud reconstruction;

[0056] (2) The present invention uses a robot to perform three-dimensional measurement tasks, which can reconstruct the three-dimensional appearance of the measured object to a greater extent;

[0057] (3) The present invention adopts a contactless registration method in the coarse registration of point clouds, and at the same time corrects the hand-eye calibration method, which retains more real information of the measured object while achieving rapid splicing. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of a specific embodiment of the robot-based three-dimensional shape measurement method for complex structural parts of the present invention;

[0059] Figure 2It is a schematic diagram of the measurement method and the hand-eye relationship in the embodiments of the present invention;

[0060] Figure 3 It is a flowchart of calculating and correcting the hand-eye calibration matrix of the present invention. Specific embodiments

[0061] The following describes the specific embodiments of the present invention in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be particularly noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0062] Figure 1 It is a flowchart of a specific embodiment of the three-dimensional shape measurement method for complex structural parts based on a robot in the present invention.

[0063] In this embodiment, as Figure 1 shown, the three-dimensional shape measurement method for complex structural parts based on a robot in the present invention includes the following steps:

[0064] 1. Reconstruct single-viewpoint measurement point cloud data:

[0065] The robot drives the binocular camera to complete the acquisition of left and right camera images of the measurement object at multiple viewpoint poses, reconstructs the single-viewpoint point cloud, and records the robot pose information. Specifically, it includes the following steps:

[0066] 1.1. Binocular camera calibration

[0067] Install the binocular camera on the fixed instrument at the end flange of the robot so that the left and right cameras are kept at the same horizontal position and a certain distance is reserved.

[0068] 1.2. Left and right camera image acquisition

[0069] Debug the binocular camera so that it can clearly capture the measurement object, and debug the robot so that it can carry the binocular camera to perform the three-dimensional measurement task, ensure that the entire view of the measurement object is captured, and use the Zhang's calibration method for camera calibration. For the measurement object at the kth, k = 1, 2,..., n robot poses, a grayscale image is taken by the left and right cameras respectively.

[0070] 1.3. Image processing

[0071] For the grayscale images taken by the left and right cameras respectively, according to the principle of the binocular camera, point cloud reconstruction is performed to obtain the single-viewpoint point cloud at the kth robot pose and obtain the robot pose matrix at this viewpoint

[0072] 2. Coarse matching of the same point cloud coordinate system

[0073] Perform hand-eye calibration on the transformation relationship from the end flange coordinate system of the robot to the binocular camera coordinate system, and then obtain the transformation matrix from the binocular camera coordinate system to the robot base coordinate system. Convert the single-viewpoint cloud obtained by the binocular camera to the unified coordinate system to complete the rough registration of multi-viewpoint clouds. Specifically:

[0074] 2.1. As Figure 2 shown, install the binocular camera on the fixed instrument of the end flange of the robot, control the robot to move to the shooting perspective, and use the binocular camera to take pictures of the calibration board at this shooting perspective, while recording the robot pose information.

[0075] 2.2. Repeat step 2.1) to obtain m groups of calibration board pictures and the corresponding robot pose information.

[0076] 2.3. Obtain the rotation vector R ci of the calibration board relative to the camera and the translation vector T ci for each group of calibration board pictures according to Zhang's calibration method, and convert them into a rotation-translation matrix, that is, obtain the external camera parameter matrix H ci , i = 1, 2,..., m; calculate the rotation vector R gi of the robot relative to the base and the translation vector T gi according to the robot pose information, and obtain the robot pose matrix H gi , i = 1, 2,..., m, where:

[0077]

[0078] 2.4. Since the relative pose relationship between the robot base and the calibration board, that is, the base calibration board relationship matrix H BW is fixed and unchanged between any two transformation shooting poses i, j, and the relative pose relationship between the camera and the end flange of the robot, that is, the hand-eye relationship matrix H cg is fixed and unchanged, there is the following transformation relationship:

[0079] H BW = H gi H cg H ci .

[0080] H BW = H gj H cg H cj

[0081] Through the external camera parameter matrix H ci , i = 1, 2,..., n and the robot pose matrix H gi, for \(i = 1, 2, \ldots, m\), combining the fixed relationship between the robot base coordinate system and the calibration board coordinate system, establish the hand-eye calibration equation:

[0082] AX = XB;

[0083] where \(A = H\) gj -1 H gi , \(B = H\) cj H ci -1 , \(X = H\) cg .

[0084] And use the hand-eye calibration function to solve this equation to obtain the rotation and translation matrix of the camera coordinate system relative to the robot end flange coordinate system, that is, the hand-eye relationship matrix \(H\) cg .

[0085] 2.5. Correct the calculated hand-eye relationship matrix \(H\) cg to obtain the hand-eye relationship matrix for multi-viewpoint cloud rough registration As Figure 3 shown, it includes the following steps:

[0086] 2.5.1. Fix the probe device on the robot end flange, establish the tool coordinate through the five-point method and the teach pendant, and obtain the position coordinate \(\Delta P_0\) of the probe end relative to the robot end flange coordinate system. The robot drives the probe to move to directly above the four corner points of the first grid in the upper left corner of the calibration board, and record the coordinate of the robot end flange relative to the base coordinate system at this time from the teach pendant. Continuously drive the probe to move to directly above the four corner points of the grids on the calibration board to obtain the coordinates of the robot end flange relative to the base coordinate system directly above the corner points And calculate the true coordinates of all the corner points on the calibration board:

[0087]

[0088] where, is the three-dimensional coordinate of the robot end flange relative to the base coordinate system directly above the \(l\)-th corner point, \(\Delta x_0, \Delta y_0, \Delta z_0\) are the position coordinates of the probe end relative to the robot end flange coordinate system, \(L\) is the number of corner points on the calibration board, \(T\) represents transpose, and \(B\) represents the robot base coordinate system.

[0089] 2.5.2. Take and reconstruct the three-dimensional point cloud of the calibration board under a certain robot pose. After reconstructing the three-dimensional point cloud of the calibration board, click and obtain the point cloud coordinates of the four corner points of the first grid in the upper left corner of the calibration board relative to the binocular camera coordinate system in sequence. According to the calculation, the point cloud coordinates \(P_1\) of all the corner points on the calibration board corresponding to the true coordinates of the corner points can be obtained l :

[0090]

[0091] Among them, is the three-dimensional coordinate representation of the point cloud coordinate P1 l , and C represents the binocular camera coordinate system.

[0092] 2.5.3. According to the robot pose information, the robot pose matrix when shooting the calibration board point cloud is H g , and from the known true coordinates P0 of the corner points l , the corner point coordinates in the flange coordinate system converted to this robot pose are obtained

[0093]

[0094] Among them, is the three-dimensional coordinate representation of the corner point coordinates , and G represents the flange coordinate system at the end of the robot; thus, a set of corresponding point sets is obtained

[0095] 2.5.4. Change the robot pose J - 1 times, repeat steps 2.5.2) and 2.5.3), obtain J sets of corresponding point sets between the true corner point coordinates and the point cloud coordinates, and re - record them as

[0096] 2.5.5. Let the corrected hand - eye relationship matrix be:

[0097]

[0098] Among them, T′ cg = [x′, y′, z′] T is the translation vector to be corrected, and R cg is the rotation matrix of the calculated hand - eye relationship matrix H cg .

[0099] Based on the rigid - body transformation relationship between a set of corresponding point sets , establish a correction function for optimization:

[0100]

[0101] Let the correction function f(P j ) = 0, substitute each set of point sets respectively to calculate J translation vectors T′ cg corresponding to J sets of point sets, and calculate the average value to obtain the corrected translation vector That is, the finally corrected hand - eye calibration matrix

[0102]

[0103] 2.6) The point cloud data reconstructed according to step (1) and the pose matrices of each perspective as well as the finally corrected hand-eye calibration matrix Calculate the rigid body transformation matrix for 3D point cloud registration

[0104]

[0105]

[0106] Among them, the rigid body transformation matrix reflects the transformation relationship from the robot base coordinate system to the binocular camera.

[0107] Thus, the single-view measurement point cloud data obtained by the binocular camera is converted to the unified coordinate system of the robot base, realizing the rough registration of the point cloud:

[0108]

[0109] Among them, is the single-view point cloud at the k-th pose after rough registration.

[0110] 3. Perform fine registration on the measured point cloud through the ICP algorithm to realize 3D shape measurement

[0111] 3.1. Take two single-view point clouds P homo_point_cloud after rough registration at adjacent poses as the ICP source point cloud and the ICP target point cloud;

[0112] 3.2. Perform ICP point cloud registration on the two single-view point clouds, search for the nearest corresponding points, and through iterative calculation, obtain the rigid body transformation matrix that minimizes the average distance of the above corresponding point pairs. When the point cloud distance is less than the set threshold or reaches the set number of iterations, output the rigid body transformation matrix between the source point cloud and the target point cloud, and use this matrix to realize the transformation of the target point cloud to the source point cloud to obtain the registered point cloud;

[0113] 3.3. Take the above registered point cloud as the ICP source point cloud, and the single-view point cloud at its adjacent pose as the target ICP point cloud, and repeat the above steps until all point clouds are finely registered to obtain the 3D shape point cloud data of the reconstructed measurement object.

[0114] Fine registration is prior art and will not be elaborated here.

[0115] Although the above description of the illustrative embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions made using the concept of the present invention are within the scope of protection.

Claims

1. A three-dimensional shape measurement method for complex structural parts based on a robot, characterized by comprising: (1) Driving a binocular camera by the robot to complete the acquisition of left and right camera images of the measurement object at multiple viewing poses, reconstructing a single-viewpoint cloud, and recording the robot pose information 1.1), Mount the binocular camera on the fixing device of the robot end flange, so that the left and right cameras are kept at the same horizontal position and a certain distance is reserved; 1.2), Debug the binocular camera so that it can clearly capture the measurement object, and debug the robot so that it can carry the binocular camera to perform three-dimensional measurement tasks, ensure that the whole picture of the measurement object is captured, and use the Zhang's calibration method for camera calibration. For the measurement object at the k-th, k = 1, 2,..., n robot poses, a grayscale image is captured by the left and right cameras respectively; 1.3) Take a grayscale image for each of the left and right cameras respectively, and perform point cloud reconstruction according to the binocular camera principle to obtain the single-viewpoint cloud at the pose of the k-th robot. And obtain the robot pose matrix of this view. (2), Perform hand-eye calibration on the conversion relationship from the robot end flange coordinate system to the binocular camera coordinate system, and then obtain the conversion matrix from the binocular camera coordinate system to the robot base coordinate system, convert the single-viewpoint cloud obtained by the binocular camera to the unified coordinate system, and complete the rough registration of the multi-viewpoint cloud 2.1), Mount the binocular camera on the fixing device of the robot end flange, control the robot to move to the shooting angle, and use the binocular camera to capture the calibration board picture at this shooting angle, and record the robot pose information at the same time; 2.2), Repeat step 2.1) to obtain m groups of calibration board pictures and the corresponding robot pose information; 2.3) Obtain the rotation vector R of the calibration board relative to the camera in each group of calibration board pictures according to Zhang's calibration method ci and the translation vector T ci , and convert them into a rotation-translation matrix, that is, obtain the external camera parameter matrix H ci , i = 1, 2,..., m; calculate the rotation vector R gi relative to the base according to the robot pose information gi and the translation vector T gi , i = 1, 2,..., m, where: 2.4), since the relative pose relationship between the robot base and the calibration board, i.e., the base calibration board relationship matrix H, remains fixed between any two transformation poses i and j BW and the relative pose relationship between the camera and the robot end flange, i.e., the hand-eye relationship matrix H cg remains fixed, there is the following conversion relationship: H BW = H gi H cg H ci H BW = H gj H cg H cj Through the external camera parameter matrix H ci , i = 1, 2, ..., n and the robot pose matrix H gi , i = 1, 2, ..., m, combined with the fixed relationship between the robot base coordinate system and the calibration board coordinate system, establish the hand-eye calibration equation: AX = XB; Among them, A = H gj -1 H gi , B = H cj H ci -1 , X = H cg And use the hand-eye calibration function to solve this equation to obtain the rotation and translation matrix of the camera coordinate system relative to the robot end flange coordinate system, that is, the hand-eye relationship matrix H cg ; 2.5), correct the calculated hand-eye relationship matrix H cg to obtain the hand-eye relationship matrix for multi-view point cloud rough registration 2.5.1) Fix it on the end flange of the robot with a probe device, establish the tool coordinate by the five-point method and the teach pendant, and obtain the position coordinate ΔP0 of the probe end relative to the coordinate system of the robot end flange. The robot drives the probe to move directly above the four corner points of the first grid in the upper left corner of the calibration board, and record the coordinate of the robot end flange relative to the base coordinate system at this time from the teach pendant. Continuously drive the probe to move directly above the four corner points of the grids on the calibration board to obtain the coordinates of the robot end flange relative to the base coordinate system directly above the corner points. And calculate the true coordinates of all the corner points of the calibration board: Among them, It is the three-dimensional coordinates of the robot end flange directly above the l-th corner point relative to the base coordinate system. Δx0, Δy0, and Δz0 are the position coordinates of the probe end relative to the robot end flange coordinate system. L represents the number of corner points on the calibration plate, T represents the transpose, and B represents the robot base coordinate system; 2.5.2) At a certain robot pose, capture and reconstruct the 3D point cloud of the calibration board. After the reconstruction of the 3D point cloud of the calibration board, click and obtain the point cloud coordinates of the four corner points of the first grid in the upper left corner of the calibration board relative to the binocular camera coordinate system in sequence. According to the calculation, the point cloud coordinates P1 of all calibration board corner points corresponding to the true coordinates of the corner points can be obtained. l : Among them, is the three-dimensional coordinate representation of the point cloud coordinate P1 l and C represents the binocular camera coordinate system; 2.5.3) According to the robot pose information, the robot pose matrix when capturing the calibration board point cloud is H g , from the known true coordinates P0 of the corner points l , the corner point coordinates in the flange coordinate system converted to this robot pose are obtained Among them, is the corner coordinate in three-dimensional coordinate representation, and G represents the coordinate system of the robot end flange; A set of corresponding point sets is thus obtained 2.5.4), Change the pose of the robot by J times, repeat steps 2.5.2) and 2.5.3), obtain the corresponding point set between the true coordinates of J sets of corner points and the point cloud coordinates, and re - record it as 2.5.5), Let the corrected hand - eye relationship matrix be: Among them, T′ cg = [x′, y′, z′] T is the translation vector to be corrected, and R cg is the rotation matrix of the calculated hand-eye relationship matrix H cg ; From a set of corresponding point sets Establish a correction function for optimization based on the rigid body transformation relationship between them: Let the correction function f(P j ) = 0, and substitute each group of point sets respectively to calculate J translation vectors T′ corresponding to J groups of point sets cg , and calculate the average value to obtain the corrected translation vector That is, the final corrected hand-eye calibration matrix is obtained 2.6) The point cloud data reconstructed according to step (1) and the pose matrices of each perspective as well as the final corrected hand-eye calibration matrix Calculate the rigid body transformation matrix for 3D point cloud registration Among them, Rigid body transformation matrix Reflect the transformation relationship from the base coordinate system of the robot to the binocular camera; Thus, the single-viewpoint cloud measurement data obtained by the binocular camera is converted to the unified coordinate system of the robot base, and the rough registration of the point cloud is realized: Among them, is the single-viewpoint point cloud at the k-th pose after coarse registration; (3), Perform fine registration on the measurement point cloud through the ICP algorithm (Iterative Closest Point algorithm) to realize three-dimensional shape measurement 3.1), Take two coarsely registered single-view point clouds P under adjacent poses homo_point_cloud as the ICP source point cloud and the ICP target point cloud; 3.2), Perform ICP point cloud registration on the two single-viewpoint clouds, search for the nearest corresponding points, and through iterative calculation, obtain the rigid body transformation matrix that minimizes the average distance of the above corresponding point pairs. When the point cloud distance is less than the set threshold or reaches the set number of iterations, output the rigid body transformation matrix between the source point cloud and the target point cloud, and realize the conversion of the target point cloud to the source point cloud by this matrix to obtain the registered point cloud; 3.3), Take the above registered point cloud as the ICP source point cloud, and the single-viewpoint cloud at its adjacent pose as the target ICP point cloud, and repeat the above steps until all point clouds are finely registered, and obtain the three-dimensional shape point cloud data of the reconstructed measurement object.

Images