Step-by-step collaborative calibration method for mechanical arm hand-eye external parameters of large-scale three-dimensional reconstruction

Abstract

This invention discloses a stepwise collaborative calibration method for hand-eye extrinsic parameters of a robotic arm in large-scale 3D reconstruction. It acquires 3D point cloud data from multiple observation poses and simultaneously generates initial hand-eye extrinsic parameters. Based on geometric consistency constraints of point-to-surface residuals, the hand-eye extrinsic parameters are iteratively optimized. In each iteration, the point cloud is transformed to the robot's base coordinate system based on the current hand-eye extrinsic parameters and matched, resulting in a set of matched point pairs in the robot's base coordinate system. A point-to-surface residual based on consistency geometric constraints is constructed using this set of matched point pairs as the target. The translation vector and rotation matrix are updated alternately. Finally, the optimal hand-eye extrinsic parameters are selected from those obtained in each optimization round. This invention constructs geometric consistency constraints based on point-to-surface residuals and performs block-based alternating updates of translation and rotation components to achieve posterior compensation for residual errors in the hand-eye extrinsic parameters, thereby improving the consistency and stability of initial registration of multi-view point clouds.

Description

Technical Field

[0001] This invention belongs to the field of robot vision measurement and 3D reconstruction technology. More specifically, it relates to a step-by-step collaborative calibration method for the hand-eye extrinsic parameters of a robotic arm for large-scale 3D reconstruction. Background Technology

[0002] With the development of aerospace, high-end equipment manufacturing, precision testing, and digital modeling of complex irregular components, the demand for high-precision acquisition of the 3D geometry of complex targets is constantly increasing. For targets that are large in size, complex in structure, severely occluded in some areas, or have uneven distribution of surface features, visual measurement using a single viewpoint often suffers from problems such as limited field of view, obvious occlusion, and missing local information, making it difficult to obtain complete, continuous, and accurate 3D point cloud data. Therefore, using a robotic arm to drive a vision sensor to actively scan the target from multiple different observation poses, and then uniformly converting the point cloud data acquired from multiple viewpoints to the same coordinate system for stitching, has become a common technique in the 3D reconstruction of complex targets.

[0003] In eye-to-hand measurement systems based on robotic arms and vision sensors, hand-eye calibration is a fundamental step in achieving a unified coordinate representation of point clouds from multiple perspectives. Hand-eye calibration establishes a rigid transformation relationship between the vision sensor coordinate system and the robotic arm end effector coordinate system, enabling point cloud data acquired by the vision sensor under different observation poses to be transformed into a unified robot base coordinate system based on the robotic arm end effector pose. Current technologies typically employ checkerboard calibration boards, dot array calibration boards, or other manual calibration targets, combining multiple sets of calibration images with corresponding robotic arm pose data to obtain initial hand-eye extrinsic parameters.

[0004] However, in practical industrial applications, due to factors such as calibration plate feature extraction errors, camera imaging errors, pose calculation errors, robotic arm repetitive positioning errors, sensor installation errors, and minor deformations of the mechanical structure, the initial hand-eye extrinsic parameters obtained usually still have certain residual errors. When the point cloud data collected under different observation poses is converted to a unified coordinate system using the initial hand-eye extrinsic parameters, these residual errors accumulate and propagate during the initial registration process of multi-view point clouds, resulting in phenomena such as layering, misalignment, boundary ghosting, or poor local fitting between point clouds from different viewpoints. This affects the quality of subsequent fine registration, global optimization, and 3D reconstruction results.

[0005] On the other hand, most existing hand-eye calibration techniques focus on solving the initial hand-eye extrinsic parameters using standard parts. After obtaining the initial hand-eye extrinsic parameters, they are usually directly applied to the multi-view point cloud coordinate transformation process of the actual object under test, with little further integration of geometric consistency information from subsequent observation data to perform a posteriori correction on the initial hand-eye extrinsic parameters. Even in some improved schemes, attempts are made to continue using standard parts independent of the object under test as auxiliary reference objects to recalibrate the initial hand-eye extrinsic parameters. If the rotation and translation components in the hand-eye extrinsic parameters are still treated as overall variables for joint nonlinear optimization, problems such as strong parameter coupling, high optimization dimensionality, sensitivity to initial values, and instability in the solution process often arise. On the one hand, rotation and translation errors often influence each other in geometrically constrained models, easily leading to mutual constraints on parameter update directions. On the other hand, when rotation and translation deviations exist simultaneously in the initial extrinsic parameters, the overall joint solution is prone to local convergence, large result fluctuations, or insufficient stability in engineering implementation, which is not conducive to stable application in complex industrial scenarios.

[0006] Therefore, how to use the stable geometric constraint information provided by the standard parts independent of the object under test, based on the existing initial hand-eye calibration results, to perform block-by-block alternating correction of the rotation and translation components in the hand-eye extrinsic parameters, reduce the coupling degree between parameters, reduce the solution difficulty brought about by the overall joint optimization, improve the stability and reliability of the a posteriori correction process, and further improve the initial registration accuracy and spatial consistency of multi-view point clouds in a unified coordinate system has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a step-by-step collaborative calibration method for the hand-eye extrinsic parameters of a large-scale 3D reconstruction robotic arm. Based on the existing initial hand-eye extrinsic parameters, the invention utilizes the effective matching relationship of the overlapping areas of multi-view point clouds in real scanning data to construct a geometric consistency constraint based on the point-to-surface residual. The translation and rotation components are updated alternately in blocks to achieve step-by-step collaborative calibration, thereby achieving posterior compensation for the residual error of the hand-eye extrinsic parameters and improving the consistency and stability of the initial registration of multi-view point clouds.

[0008] To achieve the above-mentioned objectives, the present invention provides a stepwise collaborative calibration method for extrinsic parameters of a robotic arm's hand and eye in large-scale 3D reconstruction, comprising the following steps:

[0009] S1: Control the industrial robotic arm to drive the camera mounted on its end effector to scan the target from multiple different observation poses. Arrange all observation poses sequentially according to the scanning path, and record the first... The pose matrix of the robotic arm end effector for each observed pose is as follows: :

[0010] ,

[0011] in, , Indicates the number of observed poses. Indicates the first The rotation matrix of the robotic arm's end effector relative to the robot's base coordinate system at each observed pose. This represents the corresponding translation vector;

[0012] Record No. The 3D point cloud data acquired under each observation pose is :

[0013] ,

[0014] in, Indicates the first The first observation pose The three-dimensional coordinates of a point in the camera coordinate system , Indicates the first The number of points in the point cloud under each observation pose;

[0015] S2: Generate the initial hand-eye extrinsic parameters of the camera coordinate system relative to the robot arm's end effector coordinate system. :

[0016] ,

[0017] in, Represents the initial rotation matrix. Indicates the initial translation vector;

[0018] S3: Let the number of iterations be... ;

[0019] S4: For each point in the camera coordinate system at each observation pose The coordinates of the point cloud in the robot's base coordinate system are obtained by performing point cloud transformation using the following formula. :

[0020] ,

[0021] in, Indicates the first The first observation pose The three-dimensional coordinates of a point in the robot's base coordinate system;

[0022] S5: Based on the robot's base coordinate system, determine the poses of two adjacent observation positions. Match the points to obtain the set of matching point pairs in the robot's base coordinate system for this round. :

[0023] ,

[0024] in, Represents the first in the robot's base coordinate system The first observation pose One point, Represents the first in the robot's base coordinate system The first observation pose One point, Point The corresponding local unit normal vector, , They represent the first The indexes of the two points in the corresponding observed pose point cloud of each matching point pair. , This indicates that this round is in the The observed pose and the first The number of matching point pairs extracted between observation poses;

[0025] S6: Fixed rotation matrix Using the point-to-surface residual as the objective function, the translation vector is updated to obtain the updated translation vector. The formula for calculating the point-to-surface residual is as follows:

[0026] ,

[0027] S7: Fixed translation vector Similarly, using the point-to-surface residual as the objective function, the rotation matrix is ​​updated to obtain the updated rotation matrix. ;

[0028] S8: Determine whether the iteration termination condition has been met. If not, proceed to step S9; otherwise, end the iteration optimization and proceed to step S10.

[0029] S9: Order Return to step S4;

[0030] S10: Select the optimal hand-eye extrinsic parameters from the hand-eye extrinsic parameters obtained in each round of optimization according to the preset indicators.

[0031] This invention presents a stepwise collaborative calibration method for hand-eye extrinsic parameters of a robotic arm in large-scale 3D reconstruction. It acquires 3D point cloud data under multiple observation poses and simultaneously generates initial hand-eye extrinsic parameters. Based on the geometric consistency constraint of point-to-surface residuals, the hand-eye extrinsic parameters are iteratively optimized. In each iteration, the point cloud is transformed to the robot's base coordinate system and matched according to the current hand-eye extrinsic parameters, thus obtaining a set of matched point pairs in the robot's base coordinate system. Based on this set of matched point pairs, a point-to-surface residual based on consistent geometric constraints is constructed as the target. The translation vector and rotation matrix are alternately updated. Finally, the optimal hand-eye extrinsic parameters are selected from those obtained in each optimization round.

[0032] The present invention has the following beneficial effects:

[0033] 1) Based on the existing initial hand-eye extrinsic parameters, this invention introduces geometric consistency information of the overlapping area of ​​multi-view point clouds in real scanning data to further correct the hand-eye extrinsic parameters, which can reduce the propagation effect of the residual error of the initial extrinsic parameters in the initial registration process of multi-view point clouds.

[0034] 2) This invention uses point-to-surface residuals as a form of cross-view geometric consistency constraint. It not only utilizes the initial hand-eye calibration results, but also further utilizes the local geometric information of the overlapping areas in the real scan data, so that the optimization of hand-eye extrinsic parameters can be directly aimed at the application goal of initial registration of multi-view point clouds.

[0035] 3) This invention adopts a step-by-step collaborative calibration strategy of updating the translation vector with a fixed rotation matrix and updating the rotation matrix with a fixed translation vector. This strategy can decompose the joint solution of hand-eye extrinsic parameters into two relatively independent sub-problems, thereby reducing the solution difficulty caused by parameter coupling.

[0036] 4) During the iteration process, this invention continuously updates the cross-view correspondence and effective matching set based on the current hand-eye extrinsic parameter estimation, so that the geometric constraints can be adjusted synchronously with the extrinsic parameter results, thereby improving the stability and constraint effectiveness of the optimization process. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the specific implementation of the stepwise collaborative calibration method for the extrinsic parameters of the robotic arm's hand and eye in large-scale three-dimensional reconstruction according to the present invention.

[0038] Figure 2 This is a flowchart of the matching method based on normal consistency in this embodiment;

[0039] Figure 3 This is a flowchart of the translation vector update in this embodiment;

[0040] Figure 4 This is a flowchart of the rotation matrix update in this embodiment;

[0041] Figure 5 This is a diagram showing the evolution of the initial registration state of the multi-view point cloud under different iteration rounds in this embodiment;

[0042] Figure 6 This is a comparison image of the registration results before and after optimization of hand-eye extrinsic parameters in this embodiment. Detailed Implementation

[0043] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0044] Example

[0045] Figure 1 This is a flowchart illustrating a specific implementation of the stepwise collaborative calibration method for the hand-eye extrinsic parameters of a robotic arm for large-scale 3D reconstruction according to the present invention. Figure 1 As shown, the stepwise collaborative calibration method for the hand-eye extrinsic parameters of a robotic arm in large-scale 3D reconstruction according to the present invention includes the following steps:

[0046] S101: Acquiring multi-view point cloud datasets:

[0047] The industrial robotic arm is controlled to drive a camera mounted on its end effector to scan the target from multiple different observation poses. All observation poses are arranged sequentially according to the scanning path, and the first is recorded as... The pose matrix of the robotic arm end effector for each observed pose is as follows: :

[0048] ,

[0049] in, , Indicates the number of observed poses. Indicates the first The rotation matrix of the robotic arm's end effector relative to the robot's base coordinate system at each observed pose. This represents the corresponding translation vector.

[0050] Record No. The 3D point cloud data acquired under each observation pose is :

[0051] ,

[0052] in, Indicates the first The first observation pose The three-dimensional coordinates of a point in the camera coordinate system , Indicates the first The number of points in the point cloud at each observation pose.

[0053] In this embodiment, a six-axis industrial robotic arm is used, and its end-effector pose matrix is... It is generated using the following method:

[0054] Record the original six-dimensional pose parameters of the robotic arm's end effector under the corresponding observed pose, and record the first... The original six-dimensional pose parameters of each observed pose are denoted as follows: ,in, The position parameters of the robotic arm's end effector. The pose angle parameters of the robotic arm's end effector are given, with the superscript T indicating transpose. Then, based on the original six-dimensional pose parameters... Perform attitude angle analysis and construct homogeneous transformation matrix to obtain the first... The robotic arm end-effector pose matrix under each observation pose .

[0055] S102: Generate initial hand-eye extrinsic parameters:

[0056] Generate initial hand-eye extrinsic parameters of the camera coordinate system relative to the robot arm end effector coordinate system. :

[0057] ,

[0058] in, Represents the initial rotation matrix. This represents the initial translation vector.

[0059] In this embodiment, the initial hand-eye extrinsic parameters are obtained using a calibration method, that is, the initial hand-eye extrinsic parameters are generated based on the pre-completed hand-eye calibration. This method is a common approach, and its specific process will not be described in detail here.

[0060] S103: Set the iteration count .

[0061] S104: Point cloud transformation:

[0062] For each point in the camera coordinate system at each observation pose The coordinates of the point cloud in the robot's base coordinate system are obtained by performing point cloud transformation using the following formula. :

[0063] ,

[0064] in, Indicates the first The first observation pose The three-dimensional coordinates of a point in the robot's base coordinate system.

[0065] By transforming point clouds, point clouds from various perspectives can be uniformly represented in the robot's base coordinate system, forming a unified expression of multi-view point clouds under the current hand-eye extrinsic parameter estimation conditions. This provides a unified spatial reference for subsequent cross-view correspondence construction and objective function calculation.

[0066] S105: Obtain matching point pairs of overlapping observation poses:

[0067] Based on the robot's base coordinate system, the poses of two adjacent observations are... Match the points to obtain the set of matching point pairs in the robot's base coordinate system for this round. :

[0068] ,

[0069] in, Represents the first in the robot's base coordinate system The first observation pose One point, Represents the first in the robot's base coordinate system The first observation pose One point, Point The corresponding local unit normal vector, , They represent the first The indexes of the two points in the corresponding observed pose point cloud of each matching point pair. , This indicates that this round is in the The observed pose and the first The number of matching point pairs extracted between observation poses.

[0070] In this embodiment, a matching method based on normal consistency is used to obtain the set of matching point pairs. . Figure 2 This is a flowchart of the matching method based on normal consistency in this embodiment. Figure 2 As shown, the specific steps of the matching method based on normal consistency in this embodiment include:

[0071] S201: Search for nearest neighbor:

[0072] In the robot's base coordinate system, adjacent observation poses are... One point cloud of the observed pose is used as the source point cloud, and the other as the target point cloud. Each point in the source point cloud... For the query point, search for its nearest neighbor in the target point cloud. , Indicates the point number in the source point cloud. This represents the index of a point in the target point cloud, thus obtaining candidate matching point pairs. This allows for the establishment of cross-perspective nearest neighbor candidate correspondences.

[0073] S202: Filtering matching point pairs based on distance:

[0074] For each candidate matching point pair Calculate the distance between two points in the robot's base coordinate system. If the distance is greater than a preset threshold, the candidate matching point pair is discarded. Otherwise, retain the candidate matching point pair. .

[0075] S203: Filtering matching point pairs based on normal consistency:

[0076] To improve matching reliability, the candidate matching point pairs retained in step S202... Calculate the source point normal vector With the target point normal vector The cosine similarity between the candidates is used to determine the match. If the cosine similarity is greater than a preset threshold, the candidate matching pair is discarded. Otherwise, the candidate matching point pair As valid matching point pairs, thus obtaining the set of matching point pairs. .

[0077] Furthermore, to improve the effectiveness of geometric constraints, this embodiment also performs a set of matching point pairs. Further filtering: Determine the set of matching point pairs If the number of matching point pairs is less than a preset threshold, then set the number of matching point pairs to [a certain threshold]. Otherwise, no action will be taken.

[0078] S106: Translation vector update:

[0079] Fixed rotation matrix Using the point-to-surface residual as the objective function, the translation vector is updated to obtain the updated translation vector. .

[0080] To characterize the local geometric consistency of multi-view point clouds within overlapping regions, a set of matching point pairs is used. Construct the following objective function for point-to-surface residuals:

[0081] .

[0082] The objective function described above is used to measure the degree of local fit of point clouds from different viewpoints in the robot's base coordinate system under the current hand-eye extrinsic parameter estimation conditions.

[0083] In this embodiment, the translation vector is updated by constructing and solving an overdetermined system of linear equations about the translation vector. Figure 3 This is a flowchart of the translation vector update in this embodiment. For example... Figure 4 As shown, the specific steps for updating the translation vector in this embodiment are as follows:

[0084] S301: Constructing point-to-surface residuals:

[0085] With the current rotation matrix fixed Under the given conditions, for the current round of matching point pair set Any matching point pair in Construct the point-to-surface residual corresponding to the matching point pair. :

[0086] .

[0087] The point-to-surface residual is used to characterize the normal distance of the source point from the local tangent plane of the target point under the current rotation estimation conditions.

[0088] S302: Constructing the linear expression for the translation vector:

[0089] With the current rotation matrix fixed Under the condition that, the point-to-surface residuals corresponding to each matching point pair in step S301 are... Rearranged into a linear expression for the translation vector:

[0090] ,

[0091] in, Indicates the first Each pair of matching points corresponds to a translation coefficient term. This represents a constant term, which is determined by the current fixed rotation matrix, the coordinates of the matching point pair, the pose of the robotic arm's end effector, and the normal vector of the target point.

[0092] S303: Construct an overdetermined system of linear equations with respect to the translation vector:

[0093] By superimposing the linear constraints corresponding to all valid matching points, a translation vector is constructed. Overdetermined linear equations:

[0094] ,

[0095] in, This represents the translation coefficient term corresponding to each matching point. The coefficient matrix formed by superposition, This indicates that each matching point corresponds to a constant term. The right-hand term vector formed by superposition.

[0096] S304: Solving for the updated translation vector:

[0097] The overdetermined linear equations constructed in step S303 are solved using the least squares method to obtain the equations with the current rotation matrix fixed. Updated translation vector under the given conditions .

[0098] S107: Rotation matrix update:

[0099] After the translation vector is updated, the rotation matrix is ​​updated, i.e., the translation vector is fixed. Similarly, using the point-to-surface residual as the objective function, the rotation matrix is ​​updated to obtain the updated rotation matrix. .

[0100] In this embodiment, the rotation matrix is ​​also updated by constructing and solving an overdetermined system of linear equations about the rotation matrix. Figure 4 This is a flowchart of the rotation matrix update in this embodiment. For example... Figure 4 As shown, the specific steps for updating the rotation matrix in this embodiment are as follows:

[0101] S401: Parameterize the rotation increment:

[0102] With the current translation vector fixed Under the condition of using Lie algebraic small perturbation to increment the rotation of the current rotation matrix Perform parameterization:

[0103] ,

[0104] in, , , These represent the rotation increments in the three dimensions.

[0105] S402: Constructing a linear expression for the rotation increment:

[0106] Under the small perturbation approximation condition, for the set of matching point pairs in the current round Any matching point pair in The corresponding point-to-surface residuals are linearized to obtain the point-to-surface residuals. Regarding rotation increment The linear expression:

[0107] ,

[0108] in, Indicates the first The linearized coefficients corresponding to each matching point pair This represents the corresponding constant term.

[0109] S403: Constructing an overdetermined linear system of equations with rotational increments:

[0110] By superimposing the linearized residuals corresponding to all valid matching point pairs, an overdetermined system of linear equations with respect to the rotation increment is constructed:

[0111] ,

[0112] in, This represents the linear coefficient terms corresponding to each matching point. The coefficient matrix formed by superposition, This indicates that each matching point corresponds to a constant term. The right-hand term vector formed by superposition.

[0113] S404: Solving for the rotation increment:

[0114] The overdetermined linear equations constructed in step S403 are solved using the least squares method to obtain the solution with the current translation vector fixed. Rotation increment under conditions .

[0115] S405: Update rotation matrix:

[0116] Based on rotation increment The updated rotation matrix is ​​calculated. :

[0117] ,

[0118] in, Indicates the rotation increment Construct an antisymmetric matrix.

[0119] S108: Determine whether the iteration termination condition has been met. If not, proceed to step S109; otherwise, the iteration optimization ends and proceed to step S110.

[0120] In practical applications, the iteration termination condition can generally be the reaching of the maximum number of iterations or the convergence of the hand-eye extrinsic parameters. In this embodiment, the hand-eye extrinsic parameters converge when both the translation increment and the rotation increment are less than the preset threshold, or when the change in the objective function is less than the preset threshold.

[0121] S109: Order Return to step S104.

[0122] S110: Selecting the optimal hand-eye extrinsic parameters:

[0123] The optimal hand-eye extrinsic parameters are selected from the hand-eye extrinsic parameters obtained in each round of optimization based on preset indicators.

[0124] Considering that the local matching relationship will change dynamically during different iteration stages, the final result is not simply the value of the last iteration, but rather the optimal hand-eye extrinsic parameter is retained from all completed iteration results. In this embodiment, the root mean square error of the point-to-surface residual and the point cloud matching rate of each optimized hand-eye extrinsic parameter are calculated, and then the optimal hand-eye extrinsic parameter is selected according to the criterion of residual priority and matching rate as the second best.

[0125] To better illustrate the technical effects of this invention, this embodiment first compares it theoretically with two prior art technologies. Prior patent CN115588053B uses a robotic arm-assisted binocular vision system to photograph a standard sphere, uses point cloud fitting on the sphere to obtain the coordinates of the sphere's center from each viewpoint, and combines this with the prior pose information of the robotic arm to construct optimization equations to solve for hand-eye extrinsic parameters. Its optimization is essentially based on the correspondence established around the geometric center of the standard sphere; that is, each viewpoint is mainly solved by the single-point feature of the sphere's center. Furthermore, prior patent CN115546289B, after completing the initial hand-eye calibration, fixes a probe device to the robot's end effector, uses the tool coordinate system and the robot's taught pose to determine the true spatial coordinates of discrete corner points on the calibration board point by point, and corrects the hand-eye relationship based on these finite reference points. Both of these prior art technologies primarily rely on standard reference objects, finite discrete reference features, or corresponding pose relationships established during the calibration stage. Their constraints are relatively sparse, and their optimization objectives are mainly focused on the accuracy of parameter solving during the calibration stage.

[0126] Unlike the two schemes mentioned above, this invention does not continue to optimize around the standard reference error in the calibration stage. Instead, it uses the initial hand-eye extrinsic parameters obtained by existing hand-eye calibration methods as prior values. Furthermore, it addresses the spatial inconsistency problem caused by the propagation of residual errors in the extrinsic parameters during the initial registration of real multi-view point clouds, establishing a posterior optimization mechanism for hand-eye extrinsic parameters based on geometric consistency constraints in overlapping regions of multi-view points. The optimization information in this invention directly originates from a large number of actual 3D points and their geometric relationships in overlapping regions of point clouds from different perspectives. The constraint information is richer, and the optimization objective is no longer limited to the geometric errors, reprojection errors, or pose relationship errors in the calibration stage, but directly addresses the spatial consistency and initial stitching quality of multi-view point clouds in a unified coordinate system. Furthermore, this invention performs block processing on the rotation and translation components in the hand-eye extrinsic parameters and updates them separately using an alternating correction method, thereby reducing the parameter coupling degree and solution complexity in the overall joint nonlinear optimization and improving the stability of the posterior optimization process. Based on the aforementioned technical means, this invention can improve problems such as layering, misalignment, and poor local fitting that occur after unifying multi-view point clouds to the robot base coordinate system, providing a more reliable initial foundation for subsequent fine registration and 3D reconstruction. Therefore, this invention differs from the aforementioned prior patents in terms of the technical problems it addresses, the source of constraint information, the optimization implementation stage, the optimization objectives, and the parameter solution methods.

[0127] To better illustrate the technical effects of the present invention, specific examples are used to experimentally verify the present invention. Figure 5 This is an image showing the evolution of the initial registration state of the multi-view point cloud under different iteration rounds in this embodiment. Figure 5 (a) shows the initial registration results of the multi-view point cloud under the initial hand-eye extrinsic parameters. Figure 5 (b) and Figure 5 (c) represents intermediate results during the iteration process. Figure 5 (d) shows the initial registration result of the multi-view point cloud after optimization. For example... Figure 5 As shown, with the iterative optimization of hand-eye extrinsic parameters, the degree of overlap and fit between point clouds from different perspectives, uniformly transformed to the robot base coordinate system, gradually increases, and the original layering, misalignment, and local ghosting phenomena gradually weaken. This indicates that the method described in this embodiment can improve the spatial consistency of the initial registration of multi-view point clouds.

[0128] To further illustrate the applicability of the optimization method described in this embodiment to different data batches, the optimized hand-eye extrinsic parameters were applied to the second set of independently acquired point cloud data and compared with the initial registration results before optimization. Figure 6 This is a comparison image of the registration results before and after optimization of hand-eye extrinsic parameters in this embodiment. For example... Figure 6As shown, after using optimized hand-eye extrinsic parameters for coordinate transformation, the overlap and fit of the second set of point cloud data under the unified robot base coordinate system is improved, and the original layering, misalignment and local ghosting phenomena are reduced. This indicates that the method described in this embodiment is applicable to different batches of data under the condition that the hand-eye installation relationship remains unchanged.

[0129] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A stepwise collaborative calibration method for hand-eye extrinsic parameters of a robotic arm for large-scale 3D reconstruction, characterized in that, Includes the following steps: S1: Control the industrial robotic arm to drive the camera mounted on its end effector to scan the target from multiple different observation poses. Arrange all observation poses sequentially according to the scanning path, and record the first... The pose matrix of the robotic arm end effector for each observed pose is as follows: : ， in, , Indicates the number of observed poses. Indicates the first The rotation matrix of the robotic arm's end effector relative to the robot's base coordinate system at each observed pose. This represents the corresponding translation vector; Record No. The 3D point cloud data acquired under each observation pose is : ， in, Indicates the first The first observation pose The three-dimensional coordinates of a point in the camera coordinate system , Indicates the first The number of points in the point cloud under each observation pose; S2: Generate the initial hand-eye extrinsic parameters of the camera coordinate system relative to the robot arm's end effector coordinate system. : ， in, Represents the initial rotation matrix. Indicates the initial translation vector; S3: Let the number of iterations be... ; S4: For each point in the camera coordinate system at each observation pose The coordinates of the point cloud in the robot's base coordinate system are obtained by performing point cloud transformation using the following formula. : ， in, Indicates the first The first observation pose The three-dimensional coordinates of a point in the robot's base coordinate system; S5: Based on the robot's base coordinate system, determine the poses of two adjacent observation positions. Match the points to obtain the set of matching point pairs in the robot's base coordinate system for this round. : ， in, Represents the first in the robot's base coordinate system The first observation pose One point, Represents the first in the robot's base coordinate system The first observation pose One point, Point The corresponding local unit normal vector, , They represent the first The indexes of the two points in the corresponding observed pose point cloud of each matching point pair. , This indicates that this round is in the The observed pose and the first The number of matching point pairs extracted between observation poses; S6: Fixed rotation matrix Using the point-to-surface residual as the objective function, the translation vector is updated to obtain the updated translation vector. The formula for calculating the point-to-surface residual is as follows: ， S7: Fixed translation vector Similarly, using the point-to-surface residual as the objective function, the rotation matrix is ​​updated to obtain the updated rotation matrix. ; S8: Determine whether the iteration termination condition has been met. If not, proceed to step S9; otherwise, end the iteration optimization and proceed to step S10. S9: Order Return to step S4; S10: Select the optimal hand-eye extrinsic parameters from the hand-eye extrinsic parameters obtained in each round of optimization according to the preset indicators.

2. The step-by-step collaborative calibration method for extrinsic parameters of a robotic arm's hand and eye according to claim 1, characterized in that, In step S1, the pose matrix of the robotic arm end effector It is generated using the following method: Record the original six-dimensional pose parameters of the robotic arm's end effector under the corresponding observed pose, and record the first... The original six-dimensional pose parameters of each observed pose are denoted as follows: ,in, The position parameters of the robotic arm's end effector. The attitude angle parameters of the robotic arm's end effector are given, with the superscript T indicating transpose; then, based on the original six-dimensional pose parameters... Perform attitude angle analysis and construct homogeneous transformation matrix to obtain the first... The robotic arm end-effector pose matrix under each observation pose .

3. The step-by-step collaborative calibration method for extrinsic parameters of a robotic arm's hand and eye according to claim 1, characterized in that, In step S5, the set of matching point pairs The matching method based on normal consistency is used to obtain the data. The specific method is as follows: S5.1: In the robot's base coordinate system, convert adjacent observed poses One point cloud of the observed pose is used as the source point cloud, and the other is used as the target point cloud; Each point in the source point cloud For the query point, search for its nearest neighbor in the target point cloud. , Indicates the point number in the source point cloud. This represents the index of a point in the target point cloud, thus obtaining candidate matching point pairs. ; S5.2: For each candidate matching point pair Calculate the distance between two points in the robot's base coordinate system. If the distance is greater than a preset threshold, the candidate matching point pair is discarded. Otherwise, retain the candidate matching point pair. ; S5.3: For the candidate matching point pairs retained in step S5.2 Calculate the source point normal vector With the target point normal vector The cosine similarity between the candidates is used to determine the match. If the cosine similarity is greater than a preset threshold, the candidate matching pair is discarded. Otherwise, the candidate matching point pair As valid matching point pairs, thus obtaining the set of matching point pairs. .

4. The step-by-step collaborative calibration method for extrinsic parameters of a robotic arm's hand and eye according to claim 1, characterized in that, Step S5 also involves each set of matching point pairs. Filtering: Determine the set of matching point pairs If the number of matching point pairs is less than a preset threshold, then set the number of matching point pairs to [a certain threshold]. Otherwise, no action will be taken.

5. The step-by-step collaborative calibration method for extrinsic parameters of a robotic arm's hand and eye according to claim 1, characterized in that, The specific method for updating the translation vector in step S6 is as follows: S6.1: With the current rotation matrix fixed Under the given conditions, for the current round of matching point pair set Any matching point pair in Construct the point-to-surface residual corresponding to the matching point pair. : ； S6.2: Convert the point-to-surface residuals for each matched point pair Rearranged into a linear expression for the translation vector: ， in, Indicates the first Each pair of matching points corresponds to a translation coefficient term. Represents a constant term; S6.3: Superimpose the linear constraints corresponding to all valid matching point pairs to construct a translation vector. Overdetermined linear equations: ， in, This represents the translation coefficient term corresponding to each matching point. The coefficient matrix formed by superposition, This indicates that each matching point corresponds to a constant term. The right-hand term vector formed by superposition; S6.4: Solve the overdetermined linear equation system constructed in step S6.33 using the least squares method to obtain the solution with the current rotation matrix fixed. Updated translation vector under the given conditions .

6. The step-by-step collaborative calibration method for extrinsic parameters of a robotic arm's hand and eye according to claim 1, characterized in that, The specific method for updating the rotation matrix in step S7 is as follows: S7.1: With the current translation vector fixed Under the condition of using Lie algebraic small perturbation to increment the rotation of the current rotation matrix Perform parameterization: ， in, , , These represent the rotation increments in the three dimensions, respectively. S7.2: Under the small perturbation approximation condition, for the set of matching point pairs in the current round Any matching point pair in The corresponding point-to-surface residuals are linearized to obtain the point-to-surface residuals. Regarding rotation increment The linear expression: ， in, Indicates the first The linearized coefficients corresponding to each matching point pair Indicates the corresponding constant term; S7.3: Superimpose the linearized residuals corresponding to all valid matching point pairs to construct an overdetermined system of linear equations about the rotation increment: ， in, This represents the linear coefficient terms corresponding to each matching point. The coefficient matrix formed by superposition, This indicates that each matching point corresponds to a constant term. The right-hand term vector formed by superposition; S7.4: Solve the overdetermined linear equation system constructed in step S7.3 using the least squares method to obtain the solution with the current translation vector fixed. Rotation increment under conditions ; S7.5: Based on rotation increment The updated rotation matrix is ​​calculated. : ， in, Indicates the rotation increment Construct an antisymmetric matrix.

7. The step-by-step collaborative calibration method for extrinsic parameters of a robotic arm's hand and eye according to claim 1, characterized in that, The specific method for selecting the optimal hand-eye extrinsic parameters in step S10 is as follows: calculate the root mean square error of the point-to-surface residual and the point cloud matching rate for each optimized hand-eye extrinsic parameter, and then select the optimal hand-eye extrinsic parameters according to the principle of residual priority and matching rate as the second best.

Images