Method for detecting hand-eye calibration analysis solution singularity of robot

Abstract

The invention relates to the technical field of robot vision calibration, and discloses a method for detecting robot hand-eye calibration analytical solution singularity, which comprises the following steps: S1, acquiring a homogeneous transformation matrix of a first group of mechanical arm tool system relative to a basic system and a calibration plate pose matrix under a camera coordinate system; s2, collecting a second group of homogeneous transformation matrixes and calibration plate pose matrixes under different configurations; s3, the camera rotates around the preset axis by adjusting the joint of the mechanical arm, and a third group of data is obtained; s4, calculating a mechanical arm transition matrix and a camera relative pose matrix between any two groups; s5, setting a sampling count value, and repeating the previous steps until the count meets a preset value; and S6, acquiring a preset number of measurement data pair sets by repeating the acquisition operation. According to the invention, a solution is provided by revealing and analyzing the singular phenomenon and the reason thereof when the rotation angle is close to 180 degrees in the prior art, so that the accuracy and the safety of the calibration method are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot vision calibration, and in particular to a method for detecting singularities of an analytical solution for robot hand-eye calibration. Background Art

[0002] In modern industry and manufacturing, robot hand-eye calibration technology is widely used to improve the accuracy and reliability of robots, particularly in high-precision tasks. For example, robots rely on visual sensors to accurately grasp objects or perform precision assembly, a process that relies on coordination between the robot and the vision system. Hand-eye calibration technology calculates the spatial transformation relationship between the end-of-arm and the camera, enabling the robot to accurately perform tasks.

[0003] Currently, hand-eye calibration technology has been widely used in many fields, including visual grasping operations for space manipulators, hand-eye coordination control for minimally invasive surgical robots, industrial manufacturing and assembly, and autonomous navigation for underwater robots. There are two main approaches to this problem, numerical and analytical. The accuracy and computational complexity of numerical solutions for hand-eye calibration are heavily dependent on the given initial conditions.

[0004] According to the differences in the parameterization of the rotation matrix, the analytical solutions for hand-eye calibration mainly include solutions based on the parameters of the Direction Cosine Matrix, closed solutions based on the parameters of the Euler Axis-Angle, closed solutions based on the parameters of the Modified Rodrigues, closed solutions based on the parameters of the Quaternion Algebra, closed solutions based on the parameters of the Euclidean Group, closed solutions based on the parameters of the Dual Quaternion / Screw Theory, and closed solutions based on the parameters of the Orthogonal Dual Tensor. Given that the hand-eye calibration method based on the parameters of the Direction Cosine Matrix cannot guarantee that the posture part of the unknown hand-eye matrix obtained is always an orthogonal matrix with a determinant equal to +1, this paper only considers analytical solutions for hand-eye calibration based on the remaining six parameters.

[0005] If, under certain specific conditions, one or more of the above-mentioned hand-eye calibration analytical solutions can only give a completely wrong estimate of the unknown hand-eye matrix, the corresponding hand-eye calibration method will fail (herein referred to as "hand-eye calibration singularity"), which will directly lead to the failure of the robot visual operation task. From a safety perspective, whether the hand-eye calibration analytical solution can avoid singularity under certain specific conditions is still a key to evaluating the success of robot hand-eye calibration. Therefore, it is particularly important to analyze the singularity of existing hand-eye calibration analytical solutions, which can not only provide the triggering conditions of singularity, but also improve the security of hand-eye calibration algorithms to a certain extent. However, there is no relevant research on the singularity of existing robot hand-eye calibration analytical solutions in the existing literature. In order to remedy the above-mentioned defects, the present invention will for the first time point out the singularity common to existing hand-eye calibration analytical solutions, propose a method for analyzing the singularity of existing hand-eye calibration analytical solutions, and provide the triggering conditions when the singularity occurs, which can improve the security of existing robot hand-eye calibration analytical solutions to a certain extent. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the present invention provides a method for detecting the singularity of the analytical solution of robot hand-eye calibration, which solves the problem that the analytical solution of hand-eye calibration in the existing technology fails under specific conditions.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for detecting singularities in an analytical solution of a robot hand-eye calibration, comprising the following steps:

[0008] S1. Obtain the homogeneous transformation matrix of the first set of manipulator tool systems relative to the base system and the calibration plate pose matrix in the camera coordinate system;

[0009] S2, collecting the second set of homogeneous transformation matrices and calibration plate pose matrices under different configurations;

[0010] S3, adjusting the joints of the robotic arm to achieve the rotation of the camera around a preset axis to obtain a third set of data;

[0011] S4, calculating the manipulator transition matrix and camera relative pose matrix between any two groups;

[0012] S5. Set the sampling count value and repeat S1 to S4 until the count meets the preset value, including:

[0013] Set a minimum sampling number threshold N to ensure that the rotation axis covers different directions;

[0014] After each sampling is completed, the angle distribution between the rotation axis pairs in the currently collected data is calculated;

[0015] Determine whether the rotation axis angle coverage width threshold is met;

[0016] If not satisfied, continue to execute S1 to S4 until convergence or the maximum sampling number limit is reached;

[0017] S6. Repeat the acquisition operation for not less than 60 times to obtain not less than 160 sets of candidate measurement data sets, including:

[0018] Evaluate the quality of the collected data and eliminate data with too small translation vector length or insufficient rotation angle;

[0019] Use K-means clustering or the furthest distance sampling method to select representative sample data;

[0020] Construct a set of K sample pairs with non-parallel rotation axes for subsequent algorithm error evaluation;

[0021] The rotation axis angle should be no less than a set threshold and the distribution should cover three-dimensional space;

[0022] S7. Randomly select three sets of data with non-parallel rotation axes from no less than 160 sets of data, input them into multiple hand-eye calibration analytical solution algorithms, and calculate the hand-eye matrix estimation value and its rotational error;

[0023] S8. Repeat S7 for no less than 90 times to obtain the logarithmic values ​​of the errors corresponding to different algorithms and determine whether there is singularity.

[0024] The present invention provides a method for detecting singularities in the analytical solution of robot hand-eye calibration.

[0025] Beneficial effects:

[0026] 1. This invention reveals for the first time that the existing analytical solution for robot hand-eye calibration is not applicable to all conditions. In particular, when the rotation angle of the unknown hand-eye matrix is ​​close to radians, the estimated value of the unknown hand-eye matrix given by the existing analytical solution for calibration will seriously deviate from the true value, resulting in singular phenomena.

[0027] 2. The present invention further explains the cause of this strange phenomenon, that is, when the rotation angle of the measurement data is close to radians, the rotation axis and rotation angle values ​​of the measurement data no longer meet the constraints of the hand-eye calibration equation, thereby causing distortion of the calibration results. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of the method of the present invention;

[0029] Figure 2 This is a diagram of a robot hand-eye calibration device based on a checkerboard calibration board;

[0030] Figure 3 is the constant logarithm of the rotation matrix error. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the specification of the present invention. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0032] Please see the attached Figure 1-3 , an embodiment of the present invention provides a method for detecting singularities of an analytical solution for robot hand-eye calibration, comprising the following steps:

[0033] S1, initialize j = 1, such as Figure 2 As shown, within the visible range of the camera, adjust the robot arm to a certain configuration and record the homogeneous transformation matrix of the robot arm tool system relative to the base system (denoted as At the same time, the image of the checkerboard calibration plate is captured, and the position and posture (abbreviated as pose) matrix of the calibration plate in the camera coordinate system is extracted using the Camera Calibration toolkit of Matlab R6016 a software (denoted as ).

[0034] S2. If 1≤j≤3, adjust and swing the robot arm to another configuration, and use the method in step S1 to record the homogeneous transformation matrix of the robot arm tool system relative to the base system (denoted as ) and the pose matrix of the calibration plate in the camera coordinate system (denoted as );

[0035] S3. If j = 3, adjust the displacement of each joint of the robot arm to ensure that the camera held by the robot arm can rotate 178° around its axis. Use the method of step S1 to record the homogeneous transformation matrix of the robot arm tool system relative to its base system (denoted as ) and the pose matrix of the calibration plate in the camera coordinate system (denoted as ).

[0036] S4. Solve the transition matrix of the robot tool coordinate system between two adjacent movements And the relative pose matrix of the camera coordinate system held by the robotic arm between two adjacent movements

[0037] S5. Increase the value of j (j=j+1), and repeat steps S10 to S40 until j=3;

[0038] S6, repeat steps S1 to S5 for no less than 60 times to obtain no less than 160 sets of candidate measurement data sets ( a A j , a B j );

[0039] S7. Randomly select 3 groups of data with non-parallel rotation axes from the no less than 160 groups of data obtained in S6, and substitute them into the existing robot hand-eye calibration method in turn to obtain the corresponding hand-eye matrix estimation value and the rotation part error value.

[0040] Preferably, considering that the pose between the robot tool system and the camera coordinate system it holds is unknown, in order to further evaluate the accuracy of the hand-eye matrix solved by different hand-eye calibration methods, it is necessary to introduce a rotation error measurement criterion Among them, the matrix R Ai With R Bi The homogeneous coordinate matrix is a A j and a B j The rotating part; is the unknown hand-eye matrix; N is the number of measurements performed for the entire calibration, thereby obtaining the error of the estimated value of the rotation part of the hand-eye matrix.

[0041] Using the three selected data sets, the corresponding rotation axis and angle values ​​and translation vectors are extracted in turn, converted into Modified Rodrigues parameters and substituted into the existing hand-eye calibration analytical solution based on such parameters, thereby determining the estimated value of the unknown hand-eye matrix.

[0042] Preferably, according to the above rotation error metric E r , and then obtain the rotation matrix error value given by the analytical solution of the Modified Rodrigues parameter hand-eye calibration.

[0043] Using the three sets of data selected by S7, the corresponding rotation axis, rotation angle value and translation vector are extracted in turn, converted into Quaternion Algebra parameters and introduced into the existing hand-eye calibration analytical solution based on such parameters to determine the estimated value of the unknown hand-eye matrix; according to the above rotation error measurement criterion E r , and then obtain the rotation matrix error value given by the analytical solution of the hand-eye calibration based on QuaternionAlgebra parameters.

[0044] Using the three sets of data selected in S7, the corresponding rotation axis, rotation angle value and translation vector are extracted in turn, converted into Euclidean Group parameters and substituted into the existing hand-eye calibration analytical solution based on this type of parameters to uniquely determine the estimated value of the unknown hand-eye matrix; according to the above rotation error measurement criterion E r , and then obtain the rotation matrix error value given by the analytical solution of hand-eye calibration based on Euclidean Group parameters.

[0045] Using the three sets of data selected in S7, the corresponding rotation axis, rotation angle value and translation vector are extracted in turn, converted into Dual Quaternion parameters and substituted into the existing hand-eye calibration analytical solution based on this type of parameters, and then the estimated value of the hand-eye matrix is ​​obtained; according to the above rotation error measurement criterion E r , and then obtain the rotation matrix error value given by the analytical solution of hand-eye calibration based on Dual Quaternion parameters.

[0046] Using the three sets of data selected in S7, the corresponding rotation axis and rotation angle values ​​and translation vectors are extracted in turn, converted into orthogonal dual tensor parameters and introduced into the existing hand-eye calibration analytical solution based on such parameters to uniquely determine the estimated value of the unknown hand-eye matrix; according to the above rotation error measurement criterion E r , and then obtain the rotation matrix error value given by the analytical solution of hand-eye calibration based on Orthogonal Dual Tensor parameters.

[0047] Using the three sets of data selected in S7, the existing hand-eye calibration analytical solution method based on Euler Axis-Angle parameters is used to uniquely determine the estimated value of the unknown hand-eye matrix; according to the above rotation error measurement criterion E r , and then obtain the rotation matrix error value given by the analytical solution of hand-eye calibration based on Euler Axis-Angle parameters.

[0048] S8: Repeat S7 for not less than 90 times, such as Figure 3 As shown in the figure, 90 sets of logarithmic values ​​of rotation errors can be obtained. Among them, "MRP", "QA", "EG", "ODT", "DQ" and "EAA" represent the errors of the rotation matrix estimates given by the hand-eye calibration solution based on Modified Rodrigues parameters, Quaternion Algebra parameters, Euclidean Group parameters, Orthogonal Dual Tensor parameters, DualQuaternion parameters and Euler Axis-Angle parameters, respectively.

[0049] Depend on Figure 3 It can be seen that for the remaining 5309 experiments except the 691st calibration experiment, the rotation error values ​​corresponding to the existing hand-eye calibration analytical solution are all within the normal error range.

[0050] On the contrary, in the 691st calibration experiment, the hand-eye matrix rotation error values ​​"MRP", "QA", "EG", "ODT", "DQ" and "EAA" given by the existing hand-eye calibration analytical solution are all between 0.31 radians and 1.25 radians. At this moment, the rotation matrix estimation values ​​given by the existing hand-eye calibration analytical solution are seriously deviated from their true values, that is, the hand-eye calibration analytical solution has a singular phenomenon, thereby verifying the existence of the singular phenomenon shared by the existing hand-eye calibration analytical solution. Repeat the run for not less than 90 times S7, as shown in FIG. Figure 3 As shown in the figure, 90 sets of rotation error constant logarithm values ​​can be obtained. Among them, "MRP", "QA", "EG", "ODT", "DQ" and "EAA" represent the errors of the rotation matrix estimation value given by the hand-eye calibration solution algorithm based on Modified Rodrigues parameters, Quaternion Algebra parameters, Euclidean Group parameters, Orthogonal Dual Tensor parameters, Dual Quaternion parameters and Euler Axis-Angle parameters, respectively.

[0051] Among them, the experimental data corresponding to the 691st calibration are shown in Table 1 and Table 2 respectively;

[0052] Table 1 shows the homogeneous coordinate matrix obtained by three acquisitions. a A j :

[0053] parameter <![CDATA[ a A1]]> <h2 style=";text-align:left;direction:ltr"><![CDATA[ <h2 style=";text-align:left;direction:ltr"> a <h2 style=";text-align:left;direction:ltr"> A2]]><h2 style=";text-align:left;direction:ltr"> <h2 style=";text-align:left;direction:ltr"><![CDATA[ <h2 style=";text-align:left;direction:ltr"> a <h2 style=";text-align:left;direction:ltr"> A3]]><h2 style=";text-align:left;direction:ltr"> <![CDATA[k x ]]> -0.290338 -0.595077 0.358435 <![CDATA[k y ]]> -0.369307 0.001449 -0.000066 <![CDATA[k z ]]> -0.882788 -0.803666 0.933554 θ(rad) 0.271343 3.13651 3.141329 <![CDATA[t x (m)]]> 0.166751 -0.441234 -0.572892 <![CDATA[t y (m)]]> 0.134360 -0.207974 -0.029405 <![CDATA[t z (m)]]> 0.367902 -0.015153 0.261216

[0054] Table 2: Homogeneous coordinate matrix obtained for 3 acquisitions a B j :

[0055]

[0056]

[0057] In the table, k x With k y and k z are the rotation axes of the corresponding matrices, θ is the rotation angle of the corresponding matrix, and t x With t y and t z The projections of the corresponding matrix bit shift vectors on the X, Y, and Z axes are shown in Tables 1 and 2.

[0058] For the remaining experiments except the 691st calibration test, the rotation error values ​​corresponding to the existing hand-eye calibration analytical solutions are all within the normal error range.

[0059] On the contrary, in the 691st calibration experiment, the hand-eye matrix rotation error values ​​"MRP", "QA", "EG", "ODT", "DQ" and "EAA" given by the existing hand-eye calibration analytical solutions are all between 0.31 radians and 1.25 radians. At this moment, the rotation matrix estimates given by the existing hand-eye calibration analytical solutions seriously deviate from their true values, that is, the hand-eye calibration analytical solutions all have singular phenomena, thereby verifying the existence of singular phenomena shared by the existing hand-eye calibration analytical solutions.

[0060] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting singularities in an analytical solution for robot hand-eye calibration, characterized in that: The following steps are involved: S1. Obtain the homogeneous transformation matrix of the first set of manipulator tool systems relative to the base system and the calibration plate pose matrix in the camera coordinate system; S2, collecting the second set of homogeneous transformation matrices and calibration plate pose matrices under different configurations; S3, adjusting the joints of the robotic arm to achieve the rotation of the camera around a preset axis to obtain a third set of data; S4, calculating the manipulator transition matrix and camera relative pose matrix between any two groups; S5. Set the sampling count value and repeat S1 to S4 until the count meets the preset value, including: Set a minimum sampling number threshold N to ensure that the rotation axis covers different directions; After each sampling is completed, the angle distribution between the rotation axis pairs in the currently collected data is calculated; Determine whether the rotation axis angle coverage width threshold is met; If not satisfied, continue to execute S1 to S4 until convergence or the maximum sampling number limit is reached; S6. Repeat the acquisition operation for not less than 60 times to obtain not less than 160 sets of candidate measurement data sets, including: Evaluate the quality of the collected data and eliminate data with too small translation vector length or insufficient rotation angle; Use K-means clustering or the furthest distance sampling method to select representative sample data; Construct a set of K sample pairs with non-parallel rotation axes for subsequent algorithm error evaluation; The rotation axis angle should be no less than a set threshold and the distribution should cover three-dimensional space; S7. Randomly select three sets of data with non-parallel rotation axes from no less than 160 sets of data, input them into multiple hand-eye calibration analytical solution algorithms, and calculate the hand-eye matrix estimation value and its rotational error; S8. Repeat S7 for no less than 90 times to obtain the logarithmic values ​​of the errors corresponding to different algorithms and determine whether there is singularity.

2. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, characterized in that: The hand-eye calibration analytical solution algorithm includes: Analytical solution algorithm based on modified Rodriguez parameter; Analytical solution algorithm based on quaternion algebra; Analytical solution algorithm based on Euclidean group parameters; Analytical solution algorithm based on dual quaternion expression; Analytical solution algorithm based on orthogonal double tensor; Analytical solution algorithm based on Euler axis-angle model.

3. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, wherein: The obtaining of the first set of data comprises: Control the robotic arm to move to the initial configuration; Use an image sensor to collect image information of the calibration plate; Use calibration software to extract the position and posture of the calibration plate in the camera coordinate system; Get the homogeneous transformation matrix of the tool coordinate system relative to the base coordinate system in the current configuration of the robot arm.

4. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, wherein: The collecting of the second set of data comprises: Control the robotic arm to move to different configurations; Repeat the image acquisition and calibration plate pose extraction operations; Synchronously record the transformation matrix of the robot arm tool coordinate system; Determine whether the rotation axis in the current configuration is parallel to that in the previous configuration.

5. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, characterized in that: The camera rotation is realized by: Control the robotic arm to change multiple joint angles while maintaining the gripping posture; Make the clamped camera rotate around the set axis; Record the robot arm transformation matrix and camera pose matrix before and after rotation respectively; Ensure that the recorded data has a rotation angle sufficient to cover the three-dimensional space.

6. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, characterized in that: The calculation of the transition matrix and relative pose includes: Solve the relative transformation of the robot arm transformation matrix obtained from two adjacent acquisitions; Correspondingly extract the transformation relationship between two adjacent camera pose matrices; The above transformations are uniformly converted into the form of rotation vectors and translation vectors for subsequent analysis; Each transformation matrix is ​​normalized to reduce the impact of numerical errors.

7. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, characterized in that: The selecting of the non-parallel data of the rotation axis and substituting it into the algorithm includes: A number of groups of data with rotation axis angles greater than a set threshold are selected from all collected data by random sampling; Extract the rotation axis direction, rotation angle and corresponding translation vector respectively; Convert the rotation information into a parameter expression supported by the target algorithm; The converted parameters are input into the corresponding analytical solution algorithm to calculate the hand-eye matrix estimation result and its rotation error.

8. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, characterized in that: The repeated error calculation process includes: Based on the aforementioned selected data set, each hand-eye calibration analytical solution method is called respectively; Repeat the calculation process, changing the random combination data each time; Record the rotation error value obtained for each execution; The error of each algorithm is logarithmized and normalized.

9. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, characterized in that: The singularity identification step comprises: Perform statistical analysis on the error values ​​of all algorithms under each set of data; Establish error normal interval threshold model; Detect situations where there are abnormal and sudden increases in errors of multiple algorithms at the same time; Determine if the analytical solution input for this set of data has singular behavior.

10. The method for detecting singularities in an analytical solution of a robot hand-eye calibration according to claim 1, characterized in that: The error judgment basis is the rotation matrix error, which is calculated based on a geometric consistency criterion in the form of a selected rotation parameter, which includes one or more of the following: Corrected Rodriguez parameter rotation error metric; Rotational consistency criteria for quaternion algebra; Distance error evaluation model for Euclidean groups; Geometric error projection criteria for quaternions; Minimum norm error judgment method for intersection double tensors; The rotation angle difference calculation method of the Euler axis-angle model.

Images