A Lie group hand-eye calibration method based on optimal transmission theory

Abstract

This invention proposes a Lie group-based hand-eye calibration method based on optimal transmission theory, comprising the following steps: S1, after the robotic arm performs multiple non-coplanar pose transformations, synchronously acquire the pose data of the actuator end effector and the camera after each pose transformation; S2, map the rotation matrix in the acquired pose data to the Lie algebra space to construct discrete probability distributions of the two sets of pose data; S3, calculate the weights of the pose data based on optimal transmission theory according to the discrete probability distributions of the two sets of pose data; S4, solve the rotation and translation matrices in the hand-eye calibration matrix using weighted averages to obtain the hand-eye calibration matrix. This invention utilizes optimal transmission theory to achieve global consistency optimization of calibration parameters, overcoming the limitations of local optimization in traditional least squares methods, ensuring that the solution results of the rotation matrix and translation vector are not affected by abnormal initial values, maintaining extremely high calibration accuracy even under complex motion trajectories, and completely avoiding systematic failures caused by error accumulation.

Description

Technical Field

[0001] This invention relates to the field of hand-eye calibration technology, and in particular to a Lie group hand-eye calibration method based on optimal transmission theory. Background Technology

[0002] Robot vision systems are among the most common control systems in the field of robot control. They provide robots with rich environmental information, guiding them to make correct decisions and complete predetermined actions. To analyze the image data captured by the vision system and the 3D motion data in the robot's workspace within a unified coordinate framework, hand-eye calibration of the robot vision system is necessary to obtain the transformation relationship between the vision sensor coordinate system and the robot's end effector coordinate system.

[0003] The core bottleneck of traditional hand-eye calibration methods (such as the Park-Martin method and the Navy method) lies in their over-reliance on the least squares optimization framework. This framework treats noise and outliers in motion data (such as outliers caused by mechanical vibration or camera occlusion) equally with valid data. This leads to the destruction of orthogonality constraints due to noise pollution when solving the rotation matrix, resulting in distortion of rotation parameters. At the same time, the error of the translation component continues to accumulate and amplify during the iteration process, eventually causing the calibration result to deviate significantly from the true value. This theoretical vulnerability is directly manifested in the surge in calibration failure rate in industrial scenarios. The rotation and translation errors far exceed the tolerance threshold of high-precision tasks, forcing on-site operations into an inefficient cycle of repeated retries, making it difficult to meet the high-precision operation requirements of industrial robots. Summary of the Invention

[0004] This invention proposes a Lie group-based hand-eye calibration method based on optimal transmission theory, which solves the problems of traditional hand-eye calibration methods in the prior art, such as over-reliance on the least squares optimization framework, treating all data points equally, being sensitive to motion noise, easily leading to the accumulation of calibration deviations, and failing to meet the high-precision operation requirements of industrial robots.

[0005] The technical solution of this invention is implemented as follows:

[0006] This invention provides a Lie group hand-eye calibration method based on optimal transmission theory, comprising the following steps:

[0007] S1, after the robotic arm performs multiple non-coplanar pose transformations, synchronously collect the pose data of the actuator end effector and the camera after each pose transformation.

[0008] S2, map the rotation matrix in the acquired pose data to the Lie algebra space to construct the discrete probability distribution of the two sets of pose data;

[0009] S3, calculate the weights of the pose data based on the optimal transmission theory according to the discrete probability distribution of the two sets of pose data;

[0010] S4. Based on the weights calculated in S3, the rotation and translation matrices in the hand-eye calibration matrix are solved by weighted calculation to obtain the hand-eye calibration matrix.

[0011] Specifically, in step S1, the pose data of the actuator end is obtained through the rotary encoders of each joint of the robotic arm, and the pose data of the camera is obtained through visual detection using a calibration plate.

[0012] Specifically, in step S2, the method for mapping the rotation matrix to the Lie algebra space includes:

[0013] Let the actuator end of the first i The pose data after the next pose transformation is A i The corresponding rotation and translation matrices are respectively and ; Make the camera the first i The pose data after the next pose transformation is B i The corresponding rotation and translation matrices are respectively and ;

[0014] Perform a Lie algebra mapping on the rotation matrix at the actuator end:

[0015] ;

[0016] in, Indicates the end of the actuator i Rotation angle of the next pose transformation tr (.) denotes the trace of a matrix; Indicates the end of the actuator i Rotation vector for next pose transformation. T This is the transpose of the matrix;

[0017] Perform a Lie algebra mapping on the camera's rotation matrix:

[0018] ;

[0019] in, Indicates the camera's number i Rotation angle of the next pose transformation Indicates the camera's number i Rotation vector for the next pose transformation.

[0020] Furthermore, in step S2, constructing the discrete probability distribution of the pose data includes:

[0021] Create six-dimensional vectors for the actuator end effector and camera pose data respectively:

[0022] ;

[0023] Construct discrete probability distributions for the pose data of the actuator end effector and the camera, respectively:

[0024] ;

[0025] in , These are the six-dimensional vectors of the actuator end and the camera, respectively. P A , P B These are the discrete probability distributions of the pose data of the actuator end effector and the camera, respectively. N This represents the total number of pose transformations at the actuator end effector.

[0026] Specifically, step S3 includes:

[0027] The local cost of each pose data set is defined as the weighted deviation between the rotation and translation matrices, and the large cost matrix is ​​calculated. C Each element in c ij :

[0028] ;

[0029] in, c ij Indicate each set of motion pairs ( A i , B j The cost of ) The F-norm of a vector. The second norm of a vector; λ These are the translation weighting coefficients;

[0030] Large cost matrix C Divide the data into 3×3 sub-blocks, and apply the entropy-regularized Sinkhorn algorithm to each sub-block:

[0031] Generate the exponential kernel matrix: ,in, σ This is a temperature parameter used to control the sparsity of the transmission schedule;

[0032] Initialization vector u , v It is a vector consisting entirely of 1s;

[0033] Perform Sinkhorn iterative updates until convergence:

[0034] ;

[0035] Calculate the transfer matrix P ij : ;

[0036] in, This indicates that the vector is converted into a diagonal matrix;

[0037] Calculate the weights: ;

[0038] in, w i Indicates the first i Weights of pose data for the next pose transformation.

[0039] Furthermore, in step S4, the method for weightedly solving the rotation matrix in the hand-eye calibration matrix includes the following steps:

[0040] Construct a weighted cumulative matrix M : ;

[0041] For matrix M Perform SVD decomposition: ;

[0042] in, U , V Σ is an orthogonal matrix, and Σ is a diagonal matrix;

[0043] Calculate the rotation matrix: ;

[0044] in, R X Hand-eye calibration matrix X The rotation matrix in the matrix.

[0045] Furthermore, in step S4, the method for weightedly solving the translation matrix in the hand-eye calibration matrix includes the following steps:

[0046] Based on the hand-eye calibration equation, construct a system of linear equations:

[0047] For the i Next pose transformation, we have:

[0048] ;

[0049] Where I is a 3×3 identity matrix. t X Hand-eye calibration matrix X Translation matrix in;

[0050] The first i Multiply both sides of the equation by The weighted equation is obtained as follows:

[0051] ;

[0052] All N The equations are stacked into a matrix form: ;

[0053] in, , ;

[0054] Then, hand-eye calibration matrix X Translation matrix in t X for:

[0055] .

[0056] Furthermore, in step S4, the final hand-eye calibration matrix is ​​obtained. X for:

[0057] ;

[0058] Among them, hand-eye calibration matrix X It is a 4×4 homogeneous transformation matrix. R X Let the rotation matrix be the camera relative to the actuator end. t X Let be the translation matrix of the camera relative to the actuator end.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0060] (1) This invention utilizes the optimal transmission theory to achieve global consistency optimization of calibration parameters, breaking through the local optimization limitations of the traditional least squares method, ensuring that the solution results of the rotation matrix and translation vector are not affected by abnormal initial values, maintaining extremely high calibration accuracy under complex motion trajectories, and completely avoiding systemic failures caused by error accumulation.

[0061] (2) This invention creatively solves the memory explosion problem of optimal transmission theory by dividing the cost matrix into sub-blocks and introducing entropy regularization constraints. This design enables the algorithm to maintain real-time computing efficiency in the scenario of large-scale motion data of hundreds of levels. At the same time, the smoothing characteristics of entropy regularization suppress the excessive sparsity of the transmission plan, ensure the physical rationality of weight allocation, and provide feasible support for the high-frequency calibration needs of industrial sites.

[0062] (3) The present invention dynamically adjusts the optimized weights of rotation and translation components based on the weights of motion data generated by optimal transmission, giving the calibration process a self-perceived reliability capability. High weights focus on effective data in the core region of the probability distribution, while low weights automatically suppress outliers, enabling the SVD decomposition of the rotation matrix and the least squares solution of the translation vector to have adaptive noise resistance characteristics, significantly reducing the manual retry rate. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating a Lie group hand-eye calibration method based on optimal transmission theory according to the present invention.

[0065] Figure 2 This is a schematic diagram of the hand-eye calibration system according to an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram showing the comparison of rotation error between the hand-eye calibration algorithm and the traditional hand-eye calibration algorithm in this embodiment of the invention;

[0067] Figure 4 This is a schematic diagram showing the comparison of translation error between the hand-eye calibration algorithm and the traditional hand-eye calibration algorithm in this embodiment of the invention. Detailed Implementation

[0068] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0069] To address the robustness issue of traditional hand-eye calibration methods in noisy environments, this invention proposes a hand-eye calibration method that integrates Optimal Transport (OT) theory and Lie algebra mapping. By mapping the matrix logarithms of robot motion and camera motion to the Lie algebra space and constructing their probability distribution representation, globally consistent calibration parameters are solved using Wasserstein distance minimization, significantly improving calibration accuracy and robustness.

[0070] Reference Figure 1 , 2This invention provides a Lie group hand-eye calibration method based on optimal transmission theory, comprising the following steps:

[0071] S1, after the robotic arm performs multiple non-coplanar pose transformations, synchronously collect the pose data of the actuator end effector and the camera after each pose transformation.

[0072] S2, map the rotation matrix in the acquired pose data to the Lie algebra space to construct the discrete probability distribution of the two sets of pose data;

[0073] S3, calculate the weights of the pose data based on the optimal transmission theory according to the discrete probability distribution of the two sets of pose data;

[0074] S4. Based on the weights calculated in S3, the rotation and translation matrices in the hand-eye calibration matrix are solved by weighted calculation to obtain the hand-eye calibration matrix.

[0075] Specifically, in step S1, the pose data of the actuator end is obtained through the rotary encoders of each joint of the robotic arm, and the pose data of the camera is obtained through visual detection using a calibration plate.

[0076] Specifically, in step S2, the method for mapping the rotation matrix to the Lie algebra space includes:

[0077] Let the actuator end of the first i The pose data after the next pose transformation is A i The corresponding rotation and translation matrices are respectively and ; Make the camera the first i The pose data after the next pose transformation is B i The corresponding rotation and translation matrices are respectively and ;

[0078] Perform a Lie algebra mapping on the rotation matrix at the actuator end:

[0079] ;

[0080] in, Indicates the end of the actuator i Rotation angle of the next pose transformation tr (.) denotes the trace of a matrix; Indicates the end of the actuator i The rotation vector for the next pose transformation (the direction of the vector is the rotation direction, and the length of the vector is the rotation angle). T This is the transpose of the matrix;

[0081] Perform a Lie algebra mapping on the camera's rotation matrix:

[0082] ;

[0083] in, Indicates the camera's number i Rotation angle of the next pose transformation Indicates the camera's number i The rotation vector for the next pose transformation (the direction of the vector is the rotation direction, and the length of the vector is the rotation angle).

[0084] Furthermore, in step S2, constructing the discrete probability distribution of the pose data includes:

[0085] Create six-dimensional vectors for the actuator end effector and camera pose data respectively:

[0086] ;

[0087] Construct discrete probability distributions for the pose data of the actuator end effector and the camera, respectively:

[0088] ;

[0089] in , These are the six-dimensional vectors of the actuator end and the camera, respectively. P A , P B These are the discrete probability distributions of the pose data of the actuator end effector and the camera, respectively. N This represents the total number of pose transformations at the actuator end effector.

[0090] This invention provides an operational basis for optimal transmission theory by upscaling discrete motion data to a probability space; it clusters normal data in the main probability region and distributes abnormal data to the edge region, transforming discrete data into an overall distribution, which facilitates the identification of outliers.

[0091] Specifically, step S3 includes:

[0092] The local cost of each pose data set is defined as the weighted deviation between the rotation and translation matrices, and the large cost matrix is ​​calculated. C Each element in c ij :

[0093] ;

[0094] in, c ij Indicate each set of motion pairs ( A i , B j The cost of ) The F-norm of a vector. The second norm of a vector; λ This is the translation weighting factor (default is 0.5);

[0095] Large cost matrix C Divide the data into 3×3 sub-blocks, and apply the entropy-regularized Sinkhorn algorithm to each sub-block:

[0096] Generate the exponential kernel matrix: ,in, σ This is a temperature parameter (default is 0.1), used to control the sparsity of the transmission schedule;

[0097] Initialization vector u , v It is a vector consisting entirely of 1s;

[0098] Perform Sinkhorn iterative updates until convergence:

[0099] ;

[0100] Calculate the transfer matrix P ij : ;

[0101] in, This indicates that the vector is converted into a diagonal matrix;

[0102] Calculate the weights: ;

[0103] in, w i Indicates the first i Weights of pose data in the next pose transformation. w i ≈1, indicating that the first... i The pose data from the next pose transformation is reliable. w i ≈0, indicating that the first... i The pose data from the next pose transformation is abnormal.

[0104] Furthermore, in step S4, the method for weightedly solving the rotation matrix in the hand-eye calibration matrix includes the following steps:

[0105] Construct a weighted cumulative matrix M : ;

[0106] For matrix M Perform SVD decomposition: ;

[0107] in,U , V Σ is an orthogonal matrix, and Σ is a diagonal matrix;

[0108] Calculate the rotation matrix: ;

[0109] in, R X Hand-eye calibration matrix X The rotation matrix in the matrix.

[0110] Furthermore, in step S4, the method for weightedly solving the translation matrix in the hand-eye calibration matrix includes the following steps:

[0111] Based on the hand-eye calibration equation, construct a system of linear equations:

[0112] For the i Next pose transformation, we have:

[0113] ;

[0114] Where I is a 3×3 identity matrix. t X Hand-eye calibration matrix X Translation matrix in;

[0115] The first i Multiply both sides of the equation by The weighted equation is obtained as follows:

[0116] ;

[0117] All N The equations are stacked into a matrix form: ;

[0118] in, , ;

[0119] Then, hand-eye calibration matrix X Translation matrix in t X for:

[0120] .

[0121] Furthermore, in step S4, the final hand-eye calibration matrix is ​​obtained. X for:

[0122] ;

[0123] Among them, hand-eye calibration matrix X It is a 4×4 homogeneous transformation matrix. R XLet the rotation matrix be the camera relative to the actuator end. t X Let be the translation matrix of the camera relative to the actuator end.

[0124] This embodiment compares the hand-eye calibration method (OT-Navy) of the present invention with the traditional calibration method (Navy) through experiments. The comparison results of the rotation error between the OT-Navy algorithm of the present invention and the traditional Navy algorithm are as follows: Figure 3 As shown, Figure 3 In the figure, the horizontal axis represents the number of pose transformations performed by the robotic arm, and the vertical axis represents the error between the rotation angle obtained by the calibration method and the actual rotation angle; the translation error comparison results between the OT-Navy algorithm of this invention and the traditional Navy algorithm are as follows: Figure 4 As shown, Figure 4 In the diagram, the horizontal axis represents the number of pose changes performed by the robotic arm, and the vertical axis represents the error between the translation amount obtained from the calibration method and the actual translation amount; from... Figure 3 , Figure 4 It can be seen that the OT-Navy algorithm of this invention is generally superior to the traditional Navy algorithm in terms of rotation error and translation error, and can achieve higher hand-eye calibration accuracy.

[0125] Theoretical analysis shows that the hand-eye calibration method of this invention can maintain high calibration accuracy even in the presence of Gaussian noise or motion data deviation, significantly improving the accuracy of rotation matrix calibration. R X The estimation stability is improved; while the block-based Sinkhorn algorithm solves the memory bottleneck problem caused by quadratic memory growth in traditional optimal transfer theory by using entropy regularization and independent computation of sub-blocks.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A Lie group hand-eye calibration method based on optimal transport theory, characterized in that, The method comprises the following steps: S1, after multiple non-coplanar pose transformations of a mechanical arm, synchronously collecting pose data of an end effector and pose data of a camera after each pose transformation; S2, mapping the rotation matrix in the collected pose data to Lie algebra space, and constructing discrete probability distributions of the two sets of pose data; S3, calculating the weight of the pose data based on optimal transport theory according to the discrete probability distributions of the two sets of pose data, specifically comprising: The local cost of each set of pose data is defined as a weighted deviation of the rotation matrix and the translation matrix, and each element in the large cost matrix is calculated C c ij :​ ； wherein, c ij denotes the cost of each group of motion pairs A i , B j ; and are the pose data of the end-effector after the first i pose transformation A i are the corresponding rotation matrix and translation matrix, respectively; and are the pose data of the camera after the first i pose transformation B i are the corresponding rotation matrix and translation matrix, respectively; denotes the F-norm of a vector, denotes the two-norm of a vector; Lambda is a translation weight coefficient; Divide the large cost matrix C into 3x3 sub-blocks, and apply the Sinkhorn algorithm with entropy regularization to each sub-block: Generating an exponential kernel matrix: wherein, Sigma is a temperature parameter for controlling the sparsity of the transmission plan; Initialization vector u , v is a vector of all ones; updating the Sinkhorn iteration until convergence: ； Computing a transport matrix P ij : ; wherein denotes the conversion of a vector into a diagonal matrix; Computing the weights: ; wherein, w i denotes the pose data of the pose transformation of the i weight of the pose data of the pose transformation of the S4, weighting the rotation matrix and the translation matrix in the hand-eye calibration matrix according to the weight calculated in S3, to obtain the hand-eye calibration matrix.

2. The Lie group hand-eye calibration method based on optimal transmission theory according to claim 1, wherein, In step S1, the pose data of the end effector is obtained through the rotary encoder of each joint of the mechanical arm, and the pose data of the camera is obtained through visual detection with the aid of a calibration board.

3. The Lie group hand-eye calibration method based on optimal transmission theory according to claim 1, wherein, In step S2, the method of mapping the rotation matrix to Lie algebra space comprises: Let the pose data of the end of the actuator after the first i pose transformation be A i The corresponding rotation matrix and translation matrix are and respectively; let the pose data of the camera after the first i pose transformation be B i The corresponding rotation matrix and translation matrix are and respectively; Lie algebra mapping of the rotation matrix of the end effector: ； wherein denotes the rotation angle of the pose transformation of the end effector, i tr (.) denotes the trace of a matrix; denotes the rotation vector of the pose transformation of the end effector, i T is the transpose of a matrix;​​ Lie algebra mapping of the rotation matrix of the camera: ； wherein, represents a rotation angle of the first i pose transformation of the camera, represents a rotation vector of the first i pose transformation of the camera.

4. The Lie group hand-eye calibration method based on optimal transmission theory according to claim 3, characterized in that, In step S2, the construction of the discrete probability distribution of the pose data comprises: respectively creating six-dimensional vectors of the pose data of the end effector and the camera: ； respectively constructing discrete probability distributions of the pose data of the end effector and the camera: ； wherein , are six-dimensional vectors of the end-effector and camera, respectively, P A , P B are discrete probability distributions of the pose data of the end-effector and camera, respectively, N is the total number of end-effector pose transformations.

5. The Lie group hand-eye calibration method based on optimal transmission theory according to claim 1, characterized in that, In step S4, the method of weighting the rotation matrix in the hand-eye calibration matrix comprises the following steps: constructing a weighted cumulative matrix M : ; On the matrix M SVD decomposition is performed: ; wherein U , V is an orthogonal matrix and∑ is a diagonal matrix. Compute rotation matrix: ; wherein, R X is a hand-eye calibration matrix X is a rotation matrix in 6. The Lie group hand-eye calibration method based on optimal transmission theory according to claim 5, characterized in that, In step S4, the method of weighting the translation matrix in the hand-eye calibration matrix comprises the following steps: Based on the hand-eye calibration equation, a linear equation set is constructed: For the first i pose transformation, there is: ； where I is a 3x3 identity matrix, t X is a hand-eye calibration matrix X is a translation matrix in Simultaneously multiplying both sides of the first equation by i and the second equation by yields the weighted equations: ； Stack all the equations in matrix form: N ;​ wherein , ; Then, the hand-eye calibration matrix X is t X is: 。 7. The Lie group hand-eye calibration method based on optimal transmission theory according to claim 6, characterized in that, In step S4, the hand-eye calibration matrix H is finally obtained X is: ； wherein the hand-eye calibration matrix X is a 4x4 homogeneous transformation matrix, R X is a rotation matrix of the camera with respect to the end of the effector, t X is a translation matrix of the camera with respect to the end of the effector.

Images