A six-axis industrial robot hand-eye calibration method based on a three-dimensional ball calibrator

Abstract

The application discloses a six-axis industrial robot hand-eye calibration method based on a three-dimensional ball calibrator and belongs to the technical field of six-axis industrial robot vision. The three-dimensional ball calibrator comprises a bottom plate, 11 standard balls and 11 standard rods, a plane coordinate system is arranged on the bottom plate, the 11 standard balls are respectively arranged on the bottom plate 12 in a vertical manner through the 11 standard rods according to the projection position coordinates and the height, and a space distribution is formed. The calibration method is used for determining the pose relationship between the end effector of the six-axis industrial robot and the camera, converting the position coordinates recognized by the vision system into the coordinates in the base coordinate system of the six-axis industrial robot, and thus ensuring that the six-axis industrial robot can accurately grasp the target object according to the information recognized by the vision system. The application improves the accuracy of the hand-eye calibration from two aspects: one is that the camera pose can be accurately obtained through the three-dimensional ball calibrator, which lays a foundation for accurate operation; and the other is that the calibration precision is further improved through the hand-eye equation AX=XB.

Description

Technical Field

[0001] This invention belongs to the field of industrial robot vision technology, specifically relating to a hand-eye calibration method for a six-axis industrial robot, which is used to accurately determine the pose relationship between the end effector and the vision system of a six-axis industrial robot, thereby improving the positioning accuracy of the six-axis industrial robot operation. Background Technology

[0002] With the continuous development of intelligent manufacturing and industrial robot technology, the collaborative operation of industrial robots and vision systems is becoming increasingly common. To ensure that industrial robots can accurately grasp target objects based on information recognized by the vision system, the first step is to convert the position coordinates recognized by the vision system into coordinates in the industrial robot's base coordinate system. Therefore, it is necessary to determine the pose relationship between the industrial robot's end effector and the camera, i.e., hand-eye calibration. Hand-eye systems can be divided into eye-on-hand systems and eye-on-hand systems. In the former, the camera is mounted in a fixed position outside the industrial robot and does not move with the robot during operation. In the latter, the camera is fixed to the end effector and moves with the industrial robot during its movement. The hand-eye calibration system studied in this invention is an eye-on-hand system.

[0003] Based on their geometric shape, 3D calibrators can be classified into spheres, polyhedrons (such as two checkerboard squares placed at a certain angle), cylinders, and composite geometries. Among them, spherical calibrators, due to their geometric symmetry, can maintain the uniqueness and stability of the sphere's center from any viewpoint, providing reliable spatial constraints without adjusting the orientation, significantly simplifying the calibration process. Secondly, they have excellent anti-occlusion capabilities; even if part of the surface is occluded, the sphere's center position can still be accurately fitted using the remaining contour. Furthermore, their uniform reflective properties (such as matte materials) effectively reduce interference from complex lighting, allowing the contour of the spherical calibrator to be reliably and accurately extracted from the image. Therefore, spherical calibrators are widely used in 3D calibrators. For spherical calibrators, the number of spheres also affects the calibrator's performance. Currently, most spherical calibrators use a single sphere. However, single-sphere 3D calibrators have significant drawbacks in hand-eye calibration: First, the rotational symmetry of the sphere makes it impossible to distinguish the rotation angle around the optical axis in the projected image (e.g., when the camera rotates around the center of the sphere, the projections are completely consistent), causing ambiguity in pose estimation. Second, single-sphere systems have poor robustness; when the occlusion rate is too high, the detection accuracy will be greatly reduced, while multi-sphere calibrators can still complete the calibration using the remaining spheres even if some spheres are occluded. In addition, in terms of equation solving, if the multiple poses of the industrial robot cause the projections of the sphere centers in the camera coordinate system to be collinear or coplanar, it may lead to insufficient rank of the equation system and non-unique solutions.

[0004] In terms of algorithms for solving hand-eye calibration equations, traditional analytical methods employ a two-step closed-form solution, which achieves a closed-form solution by solving the rotation matrix and translation vector step by step. However, this method suffers from the problem of rotation error propagation to the translation component. The simultaneous optimization method constructs a joint objective function, which can suppress error propagation, but its non-convexity makes convergence difficult, typically requiring more than 200 iterations. Therefore, other methods are needed to obtain suitable initial values. Moreover, even with reasonable external initial values, such methods exhibit poor convergence. In numerical optimization methods, the introduction of dual quaternions linearizes the nonlinear equations, reducing rotation error, but the accuracy is slightly lower than that of nonlinear optimization methods. Summary of the Invention

[0005] To improve the accuracy of hand-eye calibration for six-axis industrial robots, this invention provides a three-dimensional sphere calibrator for hand-eye calibration of six-axis industrial robots. Simultaneously, this invention provides a six-axis industrial robot hand-eye calibration method based on the three-dimensional sphere calibrator.

[0006] A three-dimensional ball calibrator 15 for hand-eye calibration of a six-axis industrial robot includes a base plate 12, 11 standard balls and 11 standard rods 13;

[0007] The base plate 12 is a rectangular plate. The adjacent long side and wide side of the base plate 12 form a plane coordinate system. The origin is the intersection of the adjacent long side and wide side. The long side is set as the x-axis and the wide side is set as the y-axis.

[0008] The 11 standard balls are the first standard ball 1, the second standard ball 2, the third standard ball 3, the fourth standard ball 4, the fifth standard ball 5, the sixth standard ball 6, the seventh standard ball 7, the eighth standard ball 8, the ninth standard ball 9, the tenth standard ball 10, and the eleventh standard ball 11.

[0009] The projected coordinates of the centers of the first standard sphere 1 to the eleventh standard sphere 11 in the plane coordinate system are (355, 51), (198, 205), (291, 241), (567, 242), (277, 384), (152, 403), (476, 422), (570, 456), (80, 549), (321, 550), and (570, 550), respectively. The first number in the brackets of the projected coordinates is the distance between the coordinate point and the y-axis, and the second number is the distance between the coordinate point and the x-axis, both in millimeters.

[0010] The distances from the center of each of the first standard balls 1 to the eleventh standard balls 11 to the base plate 12 are 50mm, 140mm, 192mm, 194mm, 136mm, 73mm, 355mm, 344mm, 81mm, 190mm, and 359mm, respectively.

[0011] The standard sphere has a diameter of 50mm. The 11 standard spheres are respectively erected on the base plate 12 by 11 standard rods 13 according to the projected position coordinates and height, forming a spatial distribution.

[0012] The technical solution for the further defined three-dimensional sphere calibrator is as follows:

[0013] The base plate 12 is 650mm long and 630mm wide; the standard rod 13 has a diameter of 15mm.

[0014] The projection distance between the centers of any two standard spheres on the plane of base plate 12 is greater than 200mm and less than 550mm; the matrix condition number newCond of the three-dimensional coordinate set of the centers of the 11 standard spheres is less than 10000.

[0015] A six-axis industrial robot hand-eye calibration method based on a 3D sphere calibrator is described. The hand-eye calibration platform includes a six-axis industrial robot 14, a 3D sphere calibrator 15, a binocular depth camera 16, and a host computer. The binocular depth camera 16 is fixedly mounted at the end effector of the six-axis industrial robot 14. The 3D sphere calibrator 15 is located 0.5 meters directly in front of the six-axis industrial robot 14. The calibration operation steps are as follows:

[0016] (1) Obtaining data for constructing the hand-eye calibration equation

[0017] The hand-eye calibration equation is as follows: (1)

[0018] In equation (1), A is the transformation matrix of the end effector of the six-axis industrial robot 14 from position i to position i+1, B is the transformation matrix of the binocular depth camera 16 from position i to i+1, and X is the transformation matrix of the binocular depth camera 16 to the end effector of the six-axis industrial robot 14 to be solved.

[0019] The A-transformation matrix of the end effector of the six-axis industrial robot 14 and the B-transformation matrix of the binocular depth camera 16 are obtained through the 3D sphere calibrator 15, thus obtaining 30 pairs of A-transformation matrices and B-transformation matrices for constructing the hand-eye calibration equation. ;

[0020] (2) Obtain the initial solution of the hand-eye calibration equation

[0021] Obtain the initial solution for the rotation components of the X transformation matrix. That is, by using vectorization operations and the Kronecker product, the rotation component equation in the nonlinear hand-eye calibration equation is transformed into a linear equation, and the initial solution of the rotation component of the X transformation matrix is ​​obtained by solving the linear equation. ;

[0022] Obtain the initial solution for the translation components of the X transformation matrix. That is, the initial solution based on the rotation components of the X transformation matrix. The translation component equations in the hand-eye calibration equations are solved using the least squares method to obtain the initial solution for the translation components of the X transformation matrix. ;

[0023] The initial solution of the rotation component

[0024] (2)

[0025] In equation (2), For matrix The right singular vector corresponding to the minimum singular value.

[0026] Soon Restore to a 3×3 matrix. middle Let i be the rotation component of the i-th transformation matrix A. Let I be the rotation component of the i-th B transformation matrix, and let I be the identity matrix. For Kronecker product;

[0027] The initial solution of the translation component as follows;

[0028] (3)

[0029] In equation (3), Let i be the rotation component of the i-th transformation matrix A. Let i be the translation component of the i-th transformation matrix A. Let i be the translation component of the i-th B transformation matrix. Let I be the initial solution for the rotation components of the X transformation matrix, and let I be the identity matrix. The data of the i-th A transformation matrix is ​​obtained from step (1).

[0030] (3) Solve the hand-eye calibration equation

[0031] (3.1) Obtain the optimization objective function, which is to minimize the rotation error under the orthogonality constraint of the X transformation matrix. Translation error The function;

[0032] The rotational error for ,

[0033] The translation error for ,

[0034] The orthogonality constraint of the X transformation matrix is: ,

[0035] The optimization objective function is as follows:

[0036] (4)

[0037] In equation (4), The translation components of the transformation matrix A. The translation components of the B transformation matrix. The rotation component of the X transformation matrix. Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix, and I be the identity matrix;

[0038] (3.2) Obtain the synchronous optimization function, that is, use quaternions to improve the convergence of the optimization objective function and obtain the quaternion rotation error. Quaternion translation error And auxiliary matrices M, N, C, D, E; quaternion rotation errors are balanced by normalization parameters. Quaternion translation error Order of magnitude;

[0039] The quaternion rotation error for ,

[0040] The quaternion translation error for ,

[0041] The synchronization optimization function is as follows:

[0042] (5)

[0043] In equation (5), and For normalization parameters, Let X be the quaternion corresponding to the rotation component in the X transformation matrix. Let X be the translation component of the transformation matrix. Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the objective function using quaternions. vec() is a vectorized operation.

[0044] Represented as ,

[0045] Represented as ;

[0046] (3.3) Use the Lagrange multiplier method to obtain the incremental solution of the synchronous optimization function.

[0047] The Lagrange multiplier method is used to process equation (5) to construct the Lagrange function; the differential method is used to solve the Lagrange function, and finally the incremental solution of the synchronous optimization function is obtained;

[0048] The Lagrange function is as follows:

[0049] (6)

[0050] In equation (6), For Lagrange multipliers, the meanings of the other variables are the same as those corresponding to those in equation (5);

[0051] The incremental solution is shown in equation (7):

[0052] (7)

[0053] In equation (7),

[0054] (8)

[0055] (9)

[0056] In equations (8) and (9), the variable Q is expressed as:

[0057] (10)

[0058] The meanings of the other variables are the same as those of the variables in equations (5) and (6); the meanings of the variables in equation (10) are the same as those of the variables in equation (5); the incremental solution of the synchronous optimization function can be obtained by calculating according to equations (7), (8), (9) and (10);

[0059] (3.4) Using the incremental solution, the initial solution of the hand-eye calibration equation is optimized by an iterative algorithm to obtain the closed solution of the X transformation matrix from the binocular depth camera 16 to the end effector of the six-axis industrial robot 14;

[0060] When the closed solution of the transformation matrix X satisfies the following three conditions:

[0061] (a) Rotation error is less than 0.15°;

[0062] (b) Translation error is less than 2.5 mm;

[0063] (c) The reconstruction error is less than 5.6 mm;

[0064] (d) The root mean square error of reprojection is less than 1 pixel.

[0065] If the closed solution of the X transformation matrix meets the calibration requirements, then the closed solution of the X transformation matrix does not meet any of the above conditions.

[0066] The further defined technical solution is as follows:

[0067] In step (1), the transformation matrix A, the transformation matrix B, and the transformation matrix X are defined as follows:

[0068] (11)

[0069] (12)

[0070] (13)

[0071] The meanings of each variable in equation (11) are as follows:

[0072] This is the inverse matrix of the transformation matrix from the base coordinate system of the six-axis industrial robot 14 to the coordinate system of the end effector when the end effector is at position i+1. The left superscript {E} represents the coordinate system of the end effector, the left subscript {B} represents the base coordinate system of the six-axis industrial robot 14, the right superscript {-1} represents the matrix inversion operation, and the right subscript i+1 represents the (i+1)th position.

[0073] Let be the transformation matrix from the base coordinate system of the six-axis industrial robot 14 to the coordinate system of the end effector when the end effector is at position i. The left superscript {E} represents the coordinate system of the end effector, the left subscript {B} represents the base coordinate system of the six-axis industrial robot 14, and the right subscript i represents the i-th position.

[0074] The meanings of each variable in equation (12) are as follows:

[0075] The transformation matrix from the camera coordinate system to the world coordinate system based on the 3D sphere calibrator 15 for the end effector of the six-axis industrial robot 14 at position i+1 is given by the superscript {W}, which represents the world coordinate system based on the 3D sphere calibrator 15, and the subscript {C} represents the camera coordinate system. The subscript i+1 represents the (i+1)th position.

[0076] Let {i} be the inverse of the transformation matrix from the base coordinate system of the six-axis industrial robot 14 to the coordinate system of the end effector when the end effector is at position i. The left superscript {E} represents the coordinate system of the end effector, the left subscript {B} represents the base coordinate system of the six-axis industrial robot 14, the right superscript {-1} represents the matrix inversion operation, and the right subscript i represents the i-th position.

[0077] The meanings of each variable in equation (13) are as follows:

[0078] This is the transformation matrix from the end effector coordinate system to the camera coordinate system of the six-axis industrial robot 14, where the superscript {C} represents the camera coordinate system and the subscript {E} represents the end effector coordinate system.

[0079] The detailed steps for obtaining the data of transformation matrix A and transformation matrix B in the hand-eye calibration equation using the 3D sphere calibrator 15 are as follows:

[0080] (1.1) Controlling the movement of the six-axis industrial robot 14 and adjusting it to the initial posture: The optical axis of the binocular depth camera 16 on the end effector is aligned with the center of the upper surface of the base plate 12 of the 3D sphere calibrator 15. The distance between the binocular depth camera 16 and the center of the upper surface of the base plate 12 of the 3D sphere calibrator 15 is 0.7 meters, and the acute angle formed by the optical axis of the binocular depth camera 16 and the ground is 40°. The A transformation matrix of the six-axis industrial robot 14 in the initial posture is directly recorded manually. Simultaneously, the binocular depth camera 16 captures images of the 3D sphere calibrator 15 using its left and right eyes, obtaining left and right images, which are then sent to the host computer. The host computer uses a binocular reconstruction method to process the left and right images, obtaining the B-transform matrix. The transformation matrix A in and the B transformation matrix Construct an initial transformation matrix pair;

[0081] (1.2) Construct an upper hemisphere with the center of the upper surface of the base plate 12 of the three-dimensional ball calibrator 15 as the center and a radius of 0.7 meters. The first standard ball (1) to the eleventh standard ball (11) are all located below the upper hemisphere. Set a sampling area on the upper hemisphere for uniform sampling. The acute angle formed by the line connecting any point in the sampling area and the center of the upper surface of the base plate 12 and the base plate 12 is greater than 45°. Generate 30 uniformly distributed points in the sampling area. Convert the coordinate positions of the 30 uniformly distributed points in the world coordinate system based on the three-dimensional ball calibrator 15 to the coordinate positions in the base coordinate system of the six-axis industrial robot 14 to obtain 30 target coordinates.

[0082] (1.3) Control the six-axis industrial robot 14 to move the end effector to a target coordinate obtained in step (1.2), and use the repositioning method to adjust the posture of the six-axis industrial robot 14 so that the binocular depth camera 16 can capture at least 7 standard balls on the 3D ball calibrator 15 in this posture. Manually record the A transformation matrix of the six-axis industrial robot 14 in this posture. Simultaneously, the binocular depth camera 16 captures images of the 3D sphere calibrator 15 using its left and right eyes, obtaining left and right images, which are then sent to the host computer. The host computer uses a binocular reconstruction method to process the left and right images, obtaining the B-transform matrix. The transformation matrix A in and the B transformation matrix Form a transformation matrix pair;

[0083] (1.4) Repeat step (1.3) until the 30 target coordinates in step (1.2) are traversed to obtain 30 transformation matrix pairs. Based on the 30 transformation matrix pairs and the initial transformation matrix in step (1.1), calculate according to equations (11) and (12) to obtain 30 pairs of A transformation matrix and B transformation matrix data. .

[0084] In step (1.1), the specific steps for processing the left and right images using the binocular reconstruction method are as follows:

[0085] (1.1.1) Extract the coordinates of the sphere center in the camera coordinate system

[0086] For the left and right images under the same pose, the edge detection algorithm is used to extract the sphere contours of all standard spheres in the left and right images. The least squares method is used to fit the ellipse equation to obtain the coordinates of the center of all standard spheres in the left and right images. The coordinates of the center of all standard spheres in the camera coordinate system are calculated using the coordinates of the center of all standard spheres and the camera's internal parameters. Thus, the coordinates of the center of the spheres in the camera coordinate system are obtained.

[0087] (1.1.2) Solving for the transformation matrix B

[0088] The centers of the 11 standard spheres are paired with their world coordinates in the world coordinate system of the 3D sphere calibrator 15 and their center coordinates in the camera coordinate system. The perspective n-point algorithm is then used to solve for the B-transform matrix. , that is, the transformation matrix from the camera coordinate system to the world coordinate system based on the three-dimensional sphere calibrator 15 of the binocular depth camera on the end effector of the six-axis industrial robot 14 at position i;

[0089] The processing of the left and right images using the binocular reconstruction method in step (1.3) is the same as the processing of the left and right images using the binocular reconstruction method in step (1.1).

[0090] The specific steps for step (2) are as follows:

[0091] (2.1) Obtain the initial solution of the rotation component of the X transformation matrix

[0092] The rotation component equation in the hand-eye calibration equation is as follows:

[0093] (14)

[0094] The translation component equations in the hand-eye calibration equations are as follows:

[0095] (15)

[0096] In equations (14) and (15), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, The rotation component of the X transformation matrix. The translation components of the transformation matrix A. The translation components of the B transformation matrix. For the translation component of the X transformation matrix;

[0097] Using vectorized operations to process the rotation component equation, we obtain equation (16):

[0098] (16)

[0099] In equation (16), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix. vec() represents the vectorization operation, which is to perform column vectorization on the matrix, that is, to expand the matrix into a column vector in column order.

[0100] Using the Kronecker product to process equation (16), we obtain equation (17):

[0101] (17)

[0102] In equation (17), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix, and I be the identity matrix. The Kronecker product is an operation between two matrices of arbitrary size.

[0103] From equation (17), we obtain the homogeneous equation (18):

[0104] (18)

[0105] Based on the data of 30 pairs of A transformation matrices and B transformation matrices obtained in step (1) Expanding equation (18) into equation (19):

[0106] (19)

[0107] In equation (19), Let i be the rotation component of the i-th transformation matrix A. Let i be the rotation component of the i-th B transformation matrix. Let X be the rotation component of the transformation matrix, and I be the identity matrix;

[0108] The initial solution of the rotation component of the X transformation matrix is ​​obtained by solving equation (19) using singular value decomposition. The solution to equation (19) is ,in For matrix The right singular vector corresponding to the minimum singular value, the initial solution of the rotation component. for

[0109] (2)

[0110] Soon Restore it to a 3×3 matrix;

[0111] (2.2) Obtain the initial solution of the translation components of the X transformation matrix

[0112] Initial solution based on rotation components of the X-transform matrix The data of 30 pairs of A transformation matrices and B transformation matrices obtained in step (1) Solving equation (15) using the least squares method yields the following results. The least squares solution is used as the initial solution. initial solution The resulting formula (3) is as follows:

[0113] (3).

[0114] In step (3.2), the specific operations are as follows:

[0115] (3.2.1) Using quaternions to improve the convergence of the optimization objective function

[0116] For the optimization objective function obtained in step (3.1), quaternions are introduced. , To improve the convergence of the optimization method: Based on the quaternions corresponding to the transformation matrices A, B, and X, the Kronecker product is used to adjust the rotation error in equation (4). Translation error The quaternion rotation error is obtained through processing. Quaternion translation error as follows:

[0117] (20)

[0118] (twenty one)

[0119] In equations (20) and (21), Let A be the rotation component of the transformation matrix. The translation components of the transformation matrix A. Represents the translation components of the B transformation matrix. Let X be the translation component of the transformation matrix, and I be the identity matrix. Let A be the quaternion corresponding to the rotation component in the transformation matrix A. Let be the quaternion corresponding to the rotation component in the B transformation matrix. Let X be the quaternion corresponding to the rotation component in the X transformation matrix. Represented as , Represented as ,

[0120] Represented as ,

[0121] Represented as ,

[0122] Represented as ,

[0123] Represented as ,

[0124] Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the optimization objective function using quaternions. The auxiliary matrices are defined as follows:

[0125] (twenty two)

[0126] In equation (22), each variable has the same meaning as the corresponding variable in equations (20) and (21);

[0127] That is, the quaternion rotation error is obtained. Quaternion translation error And auxiliary matrices M, N, C, D, E;

[0128] (3.2.2) Balancing quaternion rotation errors Quaternion translation error Order of magnitude to obtain synchronous optimization function

[0129] Use normalization parameters and Quaternion rotation error Quaternion translation error Normalization is performed based on the order of magnitude of the corresponding initial error value to obtain the synchronization optimization function, which includes the quaternion rotation error. Quaternion translation error The weighted combination and unit quaternion constraints; the synchronous optimization function is as follows:

[0130] (5)

[0131] In equation (5), and For normalization parameters, Let X be the quaternion corresponding to the rotation component in the X transformation matrix. Let X be the translation component of the transformation matrix. Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the objective function using quaternions.

[0132] Represented as ,

[0133] Represented as ;

[0134] Equation (5) is the final synchronization optimization function.

[0135] In step (3.3), the specific operations are as follows:

[0136] A Lagrange multiplier μ is introduced to construct the Lagrange function. The incremental solution of the Lagrange function, i.e., the incremental solution of the X-transformation matrix from the stereo depth camera 16 to the end effector, is then solved using the differential method. The detailed operation steps are as follows:

[0137] (3.3.1) Construct the Lagrange function

[0138] The Lagrangian function of equation (5) is constructed as follows:

[0139] (6)

[0140] In equation (6), μ is a Lagrange multiplier, and the meanings of the other variables are the same as those in equation (5).

[0141] (3.3.2) Solving for the Lagrange function using the differential method

[0142] Differentiating equation (6) yields equation (23):

[0143] (twenty three)

[0144] In equation (23), the symbol This indicates that the differential operation is performed on the variable on the right. The meanings of the other variables are the same as those of the corresponding variables in equations (20), (21) and (22).

[0145] The incremental solution of the synchronous optimization function is as follows:

[0146] (7)

[0147] In equation (7), the definitions of H and g are given in equations (8), (9), and (10). The solution direction is determined using backtracking search, and this step ultimately yields the incremental solution of the synchronous optimization function.

[0148] In step (3.4), the specific operations are as follows:

[0149] The initial solution is optimized using an iterative algorithm to obtain a closed-form solution for the X transformation matrix. The iterative algorithm operates as follows:

[0150] (3.4.1) Input 30 sets of matrices ;

[0151] (3.4.2) Calculate the auxiliary matrices M, N, C, D, and E according to equation (22);

[0152] (3.4.3) The initial solution of the rotational component is obtained according to equation (2). ,calculate The corresponding quaternion The initial solution of the translation component is obtained according to equation (3). ,make , , , as well as ;

[0153] (3.4.4) Based on the auxiliary matrices M, N, C, D, E and the current... The incremental solution of the current synchronization optimization function is calculated using equations (7), (8), (9), and (10). The step size β is determined and the parameters are updated by searching the backtracking line. Meanwhile, l increments by 1, where l equals the number of times step (4) has been completed;

[0154] (3.4.5) If Greater than or equal to a predefined threshold And the number of iterations is less than or equal to the maximum limit. If so, return to step (4) and continue executing steps (4) and (5) in sequence; if Less than a predefined threshold or the number of iterations exceeds the maximum limit Then stop the program and obtain the final solution, which is the closed solution of the X transformation matrix;

[0155] This means achieving precise calibration of the X-transform matrix from the binocular depth camera 16 to the end effector of the six-axis industrial robot 14, with the rotation error of the X-transform matrix being less than 0.15°, the reconstruction error less than 5.6mm, the translation error less than 2.5mm, and the root mean square error of reprojection less than 1 pixel.

[0156] The beneficial technical effects of this invention are reflected in the following aspects:

[0157] 1. This invention designs a three-dimensional sphere calibrator with a specific number and three-dimensional position. The three-dimensional sphere calibrator of this invention contains 11 three-dimensional spheres distributed in a specific space. Compared with one-dimensional and two-dimensional calibrators, the three-dimensional sphere calibrator of this invention has higher robustness. When shooting with a binocular depth camera, only seven or more standard spheres need to be captured. Even if not all standard spheres are captured, the final calibration accuracy can still be guaranteed. The three-dimensional sphere calibrator of this invention improves the accuracy of acquiring the A transformation matrix data and can construct a higher precision hand-eye calibration equation. In step (1) of the design process of the three-dimensional calibrator, the number of standard spheres is determined. When only one standard sphere is used, the data error of the A transformation matrix exceeds 120 μm. When the number of standard spheres exceeds 9, the data error of the A transformation matrix decreases slowly. Eleven three-dimensional spheres not only meet the requirements for obtaining a high-precision A transformation matrix, but also facilitate the optimization of the spatial layout of the three-dimensional spheres in step (2). The three-dimensional sphere calibrator of the present invention improves the accuracy of solving the hand-eye calibration equation. In step (2) of the design process of the three-dimensional calibrator, the matrix condition number of the sphere center coordinate set of the three-dimensional sphere calibrator is reduced by optimizing the spatial distribution of the three-dimensional spheres, thereby reducing the error in the actual numerical calculation and improving the accuracy of solving the X transformation matrix. The design process of the three-dimensional calibrator is described in Example 1. The lower the matrix condition number, the smaller the change in the solution of the matrix equation when the coefficient matrix values ​​of the matrix equation change within a small range.

[0158] 2. In step (2) of the six-axis industrial robot hand-eye calibration method of the present invention, an initial value solution for the synchronous solution method of AX=XB is proposed, which has higher accuracy and convergence compared with the existing technology. The method of the present invention first uses the Kronecker product to transform the nonlinear equation AX=XB into a linear equation, providing an initial solution for the rotation and translation components of the hand-eye matrix. Then, the synchronous optimization method proposed by Hollaude is improved by using the Kronecker product technique and quaternions to enhance the convergence of the synchronous optimization equation to obtain a closed solution. The rotation error of Hollaude's synchronous optimization method is greater than 5°, the translation error is greater than 46mm, the reconstruction error is greater than 15mm, and the root mean square error of reprojection is greater than 10 pixels; while the rotation error of the present invention is less than 0.15°, the translation error is less than 2.5mm, the reconstruction error is less than 5.6mm, and the root mean square error of reprojection is less than 1 pixel, which greatly improves the calibration accuracy. Existing mainstream hand-eye calibration methods, such as the Tsai two-step closed-loop solution, have rotation errors greater than 0.25°, translation errors greater than 4.2mm, reconstruction errors greater than 7.5mm, and root mean square errors of reprojection greater than 2.9 pixels, which are still significantly inferior to the method proposed in this invention. This demonstrates that the method proposed in this invention has higher calibration accuracy, especially in the calibration accuracy of rotation errors, root mean square errors of reprojection, and reconstruction errors, making it suitable for high-precision six-axis industrial robot vision systems. Attached Figure Description

[0159] Figure 1 This is a theoretical diagram of the three-dimensional sphere calibrator 15 in Embodiment 1 of the present invention;

[0160] Figure 2 This is a physical image of the three-dimensional sphere calibrator 15 in Embodiment 1 of the present invention;

[0161] Figure 3 This is a flowchart of the process for generating the spatial position of the standard sphere in the 3D sphere calibrator 15.

[0162] Figure 4 This is a graph showing the effect of different numbers of standard spheres on the error of solving the transformation matrix;

[0163] Figure 5 This is a physical image of the hand-eye calibration platform in Example 2;

[0164] Figure 6 This image was taken by the left eye of the Zed 2i binocular depth camera.

[0165] Figure 7 This is a diagram showing the contour fitting result of a single sphere in the 3D sphere calibrator 15;

[0166] Figure 8 The increase in data volume affects the present invention and existing methods. The influence diagram. a) Average rotation error b) Average transfer error c) Root mean square error of reprojection d) Reconstruction error . Detailed Implementation

[0167] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0168] Example 1

[0169] See Figure 1 and Figure 2 A three-dimensional ball calibrator for hand-eye calibration of a six-axis industrial robot includes a base plate 12, 11 standard balls, and 11 standard rods 13.

[0170] The base plate 12 has a length of 650mm and a width of 630mm; it forms a planar coordinate system with adjacent long and wide sides, the origin being the intersection of the adjacent long and wide sides, the long side being set as the x-axis, and the wide side being set as the y-axis.

[0171] The 11 standard balls are respectively the first standard ball 1, the second standard ball 2, the third standard ball 3, the fourth standard ball 4, the fifth standard ball 5, the sixth standard ball 6, the seventh standard ball 7, the eighth standard ball 8, the ninth standard ball 9, the tenth standard ball 10, and the eleventh standard ball 11.

[0172] The projected position coordinates of the centers of the first standard sphere 1 to the eleventh standard sphere 11 in the plane coordinate system are (355, 51), (198, 205), (291, 241), (567, 242), (277, 384), (152, 403), (476, 422), (570, 456), (80, 549), (321, 550), and (570, 550), respectively. The first number in the brackets of the projected position coordinates is the distance between the position coordinate point and the y-axis, and the second number is the distance between the position coordinate point and the x-axis, both in millimeters.

[0173] The distances from the center of each of the first standard balls 1 to the eleventh standard balls 11 to the base plate 12 are 50mm, 140mm, 192mm, 194mm, 136mm, 73mm, 355mm, 344mm, 81mm, 190mm, and 359mm, respectively.

[0174] Eleven standard balls are erected on the base plate 12 according to their projected position coordinates and heights via a standard rod 13, forming a specific spatial distribution. The diameter of each standard ball is 50 mm, and the diameter of the standard rod 13 is 15 mm.

[0175] The projection distance between the centers of any two standard spheres on the plane of base plate 12 is greater than 200mm and less than 550mm; the matrix condition number newCond of the three-dimensional coordinate set of the centers of the 11 standard spheres is less than 10000.

[0176] The design of the three-dimensional sphere calibrator 15 consists of two steps: first, determining the number of standard spheres, and second, determining the spatial position of the standard spheres.

[0177] (1) Determine the number of standard balls

[0178] First, determine the spatial position of a standard sphere. Calculate the error in the transformation matrix after adding random disturbance to one standard sphere. Then, add another standard sphere to the layout of the first standard sphere and calculate the error in the transformation matrix after adding random disturbance to two standard spheres. Continue this process until the error in the transformation matrix after adding random disturbance to 30 standard spheres is calculated. See [link to error calculation]. Figure 4 The final number of standard balls was determined to be 11.

[0179] (2) Determine the spatial position of the standard ball

[0180] See Figure 3 (2.1) Based on the actual processing and measurement conditions, initialize the generation range of the center coordinates of 11 standard spheres, the upper limit of the number of iterations times_limit=20000, the current number of iterations times=0, and the upper limit of the number of matrix conditions of the three-dimensional coordinate set minCond=10000.

[0181] (2.2) If the current iteration count is greater than the upper limit of iteration count t_l ...

[0182] (2.3) Calculate the matrix condition number 𝑛𝑒𝑤𝐶𝑜𝑛d of the three-dimensional coordinate set of the centers of the 11 standard spheres obtained in step (2.2). If newcond is less than minCond, then assign the current value of newCond to 𝑚𝑖𝑛𝐶𝑜𝑛d and record the three-dimensional coordinates of the centers of the 11 standard spheres corresponding to the current minCond. If 𝑛𝑒𝑤𝐶𝑜𝑛d obtained in step (2.3) is less than 𝑚𝑖𝑛𝐶𝑜𝑛d and is greater than or equal to minCond, then do not update the value of 𝑚𝑖𝑛𝐶𝑜𝑛d.

[0183] (2.4) Increment the value of times by 1 and return to step (2.2).

[0184] Example 2

[0185] The following are the steps for six-axis industrial robot hand-eye calibration based on a 3D sphere calibrator:

[0186] The hand-eye calibration platform described in Example 2 is referred to [reference needed]. Figure 5 The system includes a six-axis industrial robot 14, a three-dimensional sphere calibrator 15, a binocular depth camera 16, and a host computer. The host computer is configured with the computer program and related environment for this hand-eye calibration method. The binocular depth camera 16 has been calibrated and is fixed to the end effector of the six-axis industrial robot 14, ensuring that the two are rigidly connected and their relative pose remains constant during the calibration process, thus forming a visual detection system for the eye on the hand.

[0187] The platform uses an ABB IRB2600-20 / 1.65 six-axis industrial robot, which has 6 degrees of freedom and a repeatability of 0.04mm. The camera is a Zed 2i binocular depth camera, with both left and right cameras having a resolution of 2208×1242. Its depth accuracy is less than 1% within 3m. The camera's internal parameters are as follows: Left camera focal length [𝑓 𝑢 , 𝑣 The focal length of the right eye camera is [1074.4203, 1073.4310], the principal point coordinates-pixel coordinates [𝑢0, 𝜈0] are [1108.9122, 630.3229], the radial distortion coefficients [𝑘1, 𝑘2, 𝑘3] are [−0.0034, 0.01455, −0.0100], and the tangential distortion coefficients [𝑝1, 𝑝2] are [0.0007, −0.0002]; the focal length of the right eye camera is [𝑓]. 𝑢 , 𝑣 The coordinates of the principal point are [1073.8537, 1072.9044], the principal point coordinates-pixel coordinates [𝑢0, 𝜈0] are [1110.2186, 630.3606], the radial distortion coefficients [𝑘1, 𝑘2, 𝑘3] are [−0.0020, 0.0082, −0.0047], and the tangential distortion coefficients [𝑝1, 𝑝2] are [0.0003, 0.0003]; the rotation matrix in the solid parameters is... ,

[0188] The translation matrix is .

[0189] (1) Obtaining data for constructing the hand-eye calibration equation

[0190] The hand-eye calibration equation is as follows:

[0191] (1)

[0192] In equation (1), A is the transformation matrix of the end effector of the six-axis industrial robot 14 from position i to position i+1, B is the transformation matrix of the binocular depth camera 16 from position i to i+1, and X is the transformation matrix of the binocular depth camera 16 to the end effector of the six-axis industrial robot 14 to be solved.

[0193] The A-transformation matrix of the end effector of the six-axis industrial robot 14 and the B-transformation matrix of the binocular depth camera 16 are obtained through the 3D sphere calibrator 15, thus obtaining 30 pairs of A-transformation matrices and B-transformation matrices for constructing the hand-eye calibration equation. ;

[0194] The transformation matrix A, transformation matrix B, and transformation matrix X are defined as follows:

[0195] (11)

[0196] (12)

[0197] (13)

[0198] The meanings of each variable in equation (11) are as follows:

[0199] This is the inverse matrix of the transformation matrix from the base coordinate system of the six-axis industrial robot 14 to the coordinate system of the end effector when the end effector is at position i+1. The left superscript {E} represents the coordinate system of the end effector, the left subscript {B} represents the base coordinate system of the six-axis industrial robot 14, the right superscript {-1} represents the matrix inversion operation, and the right subscript i+1 represents the (i+1)th position.

[0200] Let be the transformation matrix from the base coordinate system of the six-axis industrial robot 14 to the coordinate system of the end effector when the end effector is at position i. The left superscript {E} represents the coordinate system of the end effector, the left subscript {B} represents the base coordinate system of the six-axis industrial robot 14, and the right subscript i represents the i-th position.

[0201] The meanings of each variable in equation (12) are as follows:

[0202] The transformation matrix from the camera coordinate system to the world coordinate system based on the 3D sphere calibrator 15 for the end effector of the six-axis industrial robot 14 at position i+1 is given by the superscript {W}, which represents the world coordinate system based on the 3D sphere calibrator 15, and the subscript {C} represents the camera coordinate system. The subscript i+1 represents the (i+1)th position.

[0203] Let {i} be the inverse of the transformation matrix from the base coordinate system of the six-axis industrial robot 14 to the coordinate system of the end effector when the end effector is at position i. The left superscript {E} represents the coordinate system of the end effector, the left subscript {B} represents the base coordinate system of the six-axis industrial robot 14, the right superscript {-1} represents the matrix inversion operation, and the right subscript i represents the i-th position.

[0204] The meanings of each variable in equation (13) are as follows:

[0205] This is the transformation matrix from the end effector coordinate system to the camera coordinate system of the six-axis industrial robot 14, where the superscript {C} represents the camera coordinate system and the subscript {E} represents the end effector coordinate system.

[0206] The detailed steps for obtaining the data of transformation matrix A and transformation matrix B in the hand-eye calibration equation using the 3D sphere calibrator 15 are as follows:

[0207] (1.1) Controlling the movement of the six-axis industrial robot 14 and adjusting it to the initial posture: The optical axis of the binocular depth camera 16 on the end effector is aligned with the center of the upper surface of the base plate 12 of the 3D sphere calibrator 15. The distance between the binocular depth camera 16 and the center of the upper surface of the base plate 12 of the 3D sphere calibrator 15 is 0.7 meters, and the acute angle formed by the optical axis of the binocular depth camera 16 and the ground is 40°. The A transformation matrix of the six-axis industrial robot 14 in the initial posture is directly recorded manually. Simultaneously, the binocular depth camera 16 captures images of the 3D sphere calibrator 15 using its left and right eyes, respectively, resulting in left and right images. For example, see the left image. Figure 6 The left and right images are sent to the host computer, which processes them using a binocular reconstruction method to obtain the B-transform matrix. The transformation matrix A in and the B transformation matrix This forms an initial transformation matrix pair.

[0208] The specific steps for the host computer to process the left and right images using the binocular reconstruction method are as follows:

[0209] (1.1.1) Extract the coordinates of the sphere center in the camera coordinate system

[0210] Edge detection algorithms were used to extract the spherical contours of all standard spheres in the left and right images under the same pose. See [link to individual sphere contour extraction results]. Figure 7 The least squares method is used to fit the ellipse equation to obtain the coordinates of the center of all standard spheres in the image. The coordinates of the center of all standard spheres in the image in the camera coordinate system are calculated using the coordinates of the center of all standard spheres in the image and the camera's internal parameters. Thus, the coordinates of the center of the spheres in the camera coordinate system are obtained.

[0211] (1.1.2) Solving for the transformation matrix B

[0212] The centers of the 11 standard spheres are paired with the 11 world coordinates in the world coordinate system of the 3D sphere calibrator 15 and the sphere center coordinates in the camera coordinate system obtained in step (1.3.1). The perspective n-point algorithm is used to solve the problem, and the B transformation matrix is ​​obtained. , that is, the transformation matrix from the camera coordinate system to the world coordinate system based on the three-dimensional sphere calibrator 15 of the binocular depth camera on the end effector of the six-axis industrial robot 14 at position i.

[0213] (1.2) Construct an upper hemisphere with the center of the upper surface of the base plate 12 of the three-dimensional ball calibrator 15 as the center and a radius of 0.7 meters. The first standard ball 1 to the eleventh standard ball 11 are all located below the upper hemisphere. Set a sampling area on the upper hemisphere for uniform sampling. The acute angle formed by the line connecting any point in the sampling area and the center of the upper surface of the base plate 12 and the base plate 12 is greater than 40°. Generate 30 uniformly distributed points in the sampling area. Convert the coordinate positions of the 30 uniformly distributed points in the world coordinate system based on the three-dimensional ball calibrator 15 to the coordinate positions in the base coordinate system of the six-axis industrial robot 14, that is, obtain 30 target coordinates.

[0214] (1.3) Control the six-axis industrial robot 14 to move the end effector to a target coordinate obtained in step (1.2), and use the repositioning method to adjust the posture of the six-axis industrial robot 14 so that the binocular depth camera 16 can capture at least 7 standard balls on the 3D ball calibrator 15 in this posture. Manually record the A transformation matrix of the six-axis industrial robot 14 in this posture. Simultaneously, the binocular depth camera 16 captures images of the 3D sphere calibrator 15 using its left and right eyes, obtaining left and right images, which are then sent to the host computer. The host computer uses a binocular reconstruction method to process the left and right images, obtaining the B-transform matrix. The transformation matrix A in and the B transformation matrix This forms a transformation matrix pair.

[0215] The specific operations of the host computer in step (1.3) to process the left and right images using the binocular reconstruction method are the same as those in step (1.1).

[0216] (1.4) Repeat step (1.3) until the 30 target coordinates in step (1.2) are traversed to obtain 30 transformation matrix pairs. Based on the 30 transformation matrix pairs and the initial transformation matrix in step (1.1), calculate according to equations (11) and (12) to obtain 30 pairs of A transformation matrix and B transformation matrix data. .

[0217] (2) Obtain the initial solution of the hand-eye calibration equation

[0218] Obtain the initial solution for the rotation components of the X transformation matrix. That is, by using vectorization operations and the Kronecker product, the rotation component equation in the nonlinear hand-eye calibration equation is transformed into a linear equation, and the initial solution of the rotation component of the X transformation matrix is ​​obtained by solving the linear equation. ;

[0219] Obtain the initial solution for the translation components of the X transformation matrix. That is, the initial solution based on the rotation components of the X transformation matrix. The translation component equations in the hand-eye calibration equations are solved using the least squares method to obtain the initial solution for the translation components of the X transformation matrix. ;

[0220] The initial solution of the rotation component

[0221] (2)

[0222] In equation (2), For matrix The right singular vector corresponding to the minimum singular value.

[0223] Soon Restore to a 3×3 matrix. middle Let i be the rotation component of the i-th transformation matrix A. Let I be the rotation component of the i-th B transformation matrix, and let I be the identity matrix. For Kronecker product;

[0224] The initial solution of the translation component as follows;

[0225] (3)

[0226] In equation (3), Let i be the rotation component of the i-th transformation matrix A. Let i be the translation component of the i-th transformation matrix A. Let i be the translation component of the i-th B transformation matrix. Let I be the initial solution for the rotation component of the X transformation matrix, and let I be the identity matrix. The data of the i-th A transformation matrix is ​​obtained from step (1).

[0227] The specific steps for step (2) are as follows:

[0228] (2.1) Obtain the initial solution of the rotation component of the X transformation matrix

[0229] The rotation component equation in the hand-eye calibration equation is as follows:

[0230] (14)

[0231] The translation component equations in the hand-eye calibration equations are as follows:

[0232] (15)

[0233] In equations (14) and (15), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, The rotation component of the X transformation matrix. The translation components of the transformation matrix A. The translation components of the B transformation matrix. For the translation component of the X transformation matrix;

[0234] Using vectorized operations to process the rotation component equation, we obtain equation (16):

[0235] (16)

[0236] In equation (16), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix. vec() represents the vectorization operation, which is to perform column vectorization on the matrix, that is, to expand the matrix into a column vector in column order.

[0237] Using the Kronecker product to process equation (16), we obtain equation (17):

[0238] (17)

[0239] In equation (17), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix, and I be the identity matrix. The Kronecker product is an operation between two matrices of arbitrary size.

[0240] From equation (17), we obtain the homogeneous equation (18):

[0241] (18)

[0242] Based on the data of 30 pairs of A transformation matrices and B transformation matrices obtained in step (1) Expanding equation (18) into equation (19):

[0243] (19)

[0244] In equation (19), Let i be the rotation component of the i-th transformation matrix A. Let i be the rotation component of the i-th B transformation matrix. Let X be the rotation component of the transformation matrix, and I be the identity matrix;

[0245] The initial solution of the rotation component of the X transformation matrix is ​​obtained by solving equation (19) using singular value decomposition. The solution to equation (19) is ,in For matrix The right singular vector corresponding to the minimum singular value, the initial solution of the rotation component. for

[0246] (2)

[0247] Soon Restored to a 3×3 matrix; in this embodiment 2 .

[0248] (2.2) Obtain the initial solution of the translation components of the X transformation matrix

[0249] Initial solution based on rotation components of the X-transform matrix The data of 30 pairs of A transformation matrices and B transformation matrices obtained in step (1) Solving equation (15) using the least squares method yields the following results. The least squares solution is used as the initial solution. initial solution The resulting formula (3) is as follows:

[0250] (3).

[0251] In this embodiment 2, .

[0252] (3) Solve the hand-eye calibration equation

[0253] (3.1) Obtain the optimization objective function, which is to minimize the rotation error under the orthogonality constraint of the X transformation matrix. Translation error The function;

[0254] The rotational error for ,

[0255] The translation error for ,

[0256] The orthogonality constraint of the X transformation matrix is: ,

[0257] The optimization objective function is as follows:

[0258] (4)

[0259] In equation (4), The translation components of the transformation matrix A. The translation components of the B transformation matrix. The rotation component of the X transformation matrix. Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix, and I be the identity matrix;

[0260] (3.2) Obtain the synchronous optimization function, that is, use quaternions to improve the convergence of the optimization objective function and obtain the quaternion rotation error. Quaternion translation error And auxiliary matrices M, N, C, D, E; quaternion rotation errors are balanced by normalization parameters. Quaternion translation error Order of magnitude;

[0261] The quaternion rotation error for ,

[0262] The quaternion translation error for ,

[0263] The synchronization optimization function is as follows:

[0264] (5)

[0265] In equation (5), and For normalization parameters, Let X be the quaternion corresponding to the rotation component in the X transformation matrix. Let X be the translation component of the transformation matrix. Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the objective function using quaternions. vec() is a vectorized operation.

[0266] Represented as ,

[0267] Represented as .

[0268] The specific steps in step (3.2) are as follows:

[0269] (3.2.1) Using quaternions to improve the convergence of the optimization objective function

[0270] For the optimization objective function obtained in step (3.1), quaternions are introduced. , To improve the convergence of the optimization method: Based on the quaternions corresponding to the transformation matrices A, B, and X, the Kronecker product is used to adjust the rotation error in equation (4). Translation error The quaternion rotation error is obtained through processing. Quaternion translation error as follows:

[0271] (20)

[0272] (twenty one)

[0273] In equations (20) and (21), Let A be the rotation component of the transformation matrix. The translation components of the transformation matrix A. Represents the translation components of the B transformation matrix. Let X be the translation component of the transformation matrix, and I be the identity matrix. Let A be the quaternion corresponding to the rotation component in the transformation matrix A. Let be the quaternion corresponding to the rotation component in the B transformation matrix. Let X be the quaternion corresponding to the rotation component in the X transformation matrix.

[0274] Represented as ,

[0275] Represented as ,

[0276] Represented as ,

[0277] Represented as ,

[0278] Represented as ,

[0279] Represented as ,

[0280] Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the optimization objective function using quaternions. The auxiliary matrices are defined as follows:

[0281] (twenty two)

[0282] In equation (22), each variable has the same meaning as the corresponding variable in equations (20) and (21);

[0283] That is, the quaternion rotation error is obtained. Quaternion translation error And auxiliary matrices M, N, C, D, E;

[0284] (3.2.2) Balancing quaternion rotation errors Quaternion translation error Order of magnitude to obtain synchronous optimization function

[0285] Use normalization parameters and Quaternion rotation error Quaternion translation error Normalization is performed based on the order of magnitude of the corresponding initial error value to obtain the synchronization optimization function, which includes the quaternion rotation error. Quaternion translation error The weighted combination and unit quaternion constraints; the synchronous optimization function is as follows:

[0286] (5)

[0287] In equation (5), and For normalization parameters, Let X be the quaternion corresponding to the rotation component in the X transformation matrix. Let X be the translation component of the transformation matrix. Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the objective function using quaternions.

[0288] Represented as ,

[0289] Represented as ;

[0290] Equation (5) is the final synchronization optimization function.

[0291] (3.3) Use the Lagrange multiplier method to obtain the incremental solution of the synchronous optimization function.

[0292] The Lagrange multiplier method is used to process equation (5) to construct the Lagrange function; the differential method is used to solve the Lagrange function, and finally the incremental solution of the synchronous optimization function is obtained;

[0293] The Lagrange function is as follows:

[0294] (6)

[0295] In equation (6), For Lagrange multipliers, the meanings of the other variables are the same as those corresponding to those in equation (5);

[0296] The incremental solution is shown in equation (7):

[0297] (7)

[0298] In equation (7),

[0299] (8)

[0300] (9)

[0301] In equations (8) and (9), the variable Q is expressed as:

[0302] (10)

[0303] The meanings of the remaining variables are the same as those of the variables in equations (5) and (6);

[0304] The variables in equation (10) have the same meaning as the variables in equation (5);

[0305] The incremental solution of the synchronous optimization function can be obtained by calculating according to equations (7), (8), (9) and (10).

[0306] The specific steps for step (3.3) are as follows:

[0307] A Lagrange multiplier μ is introduced to construct the Lagrange function. The incremental solution of the Lagrange function, i.e., the incremental solution of the X-transformation matrix from the stereo depth camera 16 to the end effector, is then solved using the differential method. The detailed operation steps are as follows:

[0308] (3.3.1) Construct the Lagrange function

[0309] The Lagrangian function of equation (5) is constructed as follows:

[0310] (6)

[0311] In equation (6), μ is a Lagrange multiplier, and the meanings of the other variables are the same as those in equation (5).

[0312] (3.3.2) Solving for the Lagrange function using the differential method

[0313] Differentiating equation (6) yields equation (23):

[0314] (twenty three)

[0315] In equation (23), the symbol This indicates that the differential operation is performed on the variable on the right. The meanings of the other variables are the same as those of the corresponding variables in equations (20), (21) and (22).

[0316] The incremental solution of the synchronous optimization function is as follows:

[0317] (7)

[0318] In equation (7), the definitions of H and g are given in equations (8), (9), and (10). The solution direction is determined using backtracking search, and this step ultimately yields the incremental solution of the synchronous optimization function.

[0319] (3.4) Using the incremental solution, the initial solution of the hand-eye calibration equation is optimized by an iterative algorithm to obtain the closed solution of the X transformation matrix from the binocular depth camera 16 to the end effector of the six-axis industrial robot 14.

[0320] In this embodiment 2, the closed-form solution of the X transformation matrix is: .

[0321] The closed solution of the transformation matrix X satisfies the following four conditions:

[0322] (a) Rotation error less than 0.15°; (b) Translation error less than 2.5 mm; (c) Reconstruction error less than 5.6 mm; (d) Reprojection root mean square error less than 1 pixel;

[0323] If the closed-form solution of the X-transform matrix satisfies the calibration requirements, then the closed-form solution of the X-transform matrix does not meet the calibration requirements. The specific steps for using an iterative algorithm to optimize the initial solution to obtain the closed-form solution of the X-transform matrix are as follows:

[0324] (3.4.1) Input 30 sets of matrices ;

[0325] (3.4.2) Calculate the auxiliary matrices M, N, C, D, and E according to equation (22);

[0326] (3.4.3) The initial solution of the rotational component is obtained according to equation (2). ,calculate The corresponding quaternion The initial solution of the translation component is obtained according to equation (3). ,make , , , as well as ;

[0327] (3.4.4) Based on the auxiliary matrices M, N, C, D, E and the current... The incremental solution of the current synchronization optimization function is calculated using equations (7), (8), (9), and (10). The step size β is determined and the parameters are updated by searching the backtracking line. Meanwhile, l increments by 1, where l equals the number of times step (4) has been completed;

[0328] (3.4.5) If Greater than or equal to a predefined threshold And the number of iterations is less than or equal to the maximum limit. If so, return to step (3.4.4) and continue executing steps (3.4.4) and (3.4.5) in sequence; if Less than a predefined threshold or the number of iterations exceeds the maximum limit Then stop the iteration and obtain the final solution, which is the closed solution of the X transformation matrix;

[0329] This means achieving precise calibration of the X-transform matrix from the binocular depth camera 16 to the end effector of the six-axis industrial robot 14, with the rotation error of the X-transform matrix being less than 0.15°, the reconstruction error less than 5.6mm, the translation error less than 2.5mm, and the root mean square error of reprojection less than 1 pixel.

[0330] Six comparative experiments were conducted using existing methods and the method of this invention, with experimental data collected from 30 sets of matrices. Groups of 5, 10, 15, 20, 25, and 30 were selected evenly, respectively. The experimental results are shown in [reference needed]. Figure 8 .like Figure 8 As shown in (a) above, under all matrix group conditions, the rotation error of the calibration results obtained using this invention is the smallest; as Figure 8As shown in (b) above, under all matrix group conditions, the translation error of the calibration results obtained using this invention is close to minimum; Figure 8 As shown in (c), under all matrix group conditions, the root mean square error of the reprojection of the calibration results obtained using this invention is close to the minimum; as Figure 8 As shown in (d) in the figure, the reconstruction error of the calibration results obtained using the present invention is the smallest when the number of matrix groups is 10, 15, 20, 25 and 30.

Claims

1. A hand-eye calibration method for a six-axis industrial robot based on a three-dimensional ball calibrator, wherein the hand-eye calibration platform includes a six-axis industrial robot (14), a three-dimensional ball calibrator (15), a binocular depth camera (16), and a host computer; the binocular depth camera (16) is fixedly installed at the end effector of the six-axis industrial robot (14); the three-dimensional ball calibrator (15) is located 0.5 meters in front of the six-axis industrial robot (14); and includes a base plate (12), 11 standard balls, and 11 standard rods (13). The base plate (12) is a rectangular plate, which forms a plane coordinate system with adjacent long sides and wide sides. The origin is the intersection of adjacent long sides and wide sides. The long side is set as the x-axis and the wide side is set as the y-axis. The 11 standard balls are the first standard ball (1), the second standard ball (2), the third standard ball (3), the fourth standard ball (4), the fifth standard ball (5), the sixth standard ball (6), the seventh standard ball (7), the eighth standard ball (8), the ninth standard ball (9), the tenth standard ball (10), and the eleventh standard ball (11). The projection coordinates of the centers of the first standard sphere (1) to the eleventh standard sphere (11) in the plane coordinate system are (355, 51), (198, 205), (291, 241), (567, 242), (277, 384), (152, 403), (476, 422), (570, 456), (80, 549), (321, 550), and (570, 550), respectively; the first number in the brackets of the projection coordinates is the distance between the coordinate point and the y-axis, and the second number is the distance between the coordinate point and the x-axis, both in millimeters; The distances from the center of each of the first standard ball (1) to the eleventh standard ball (11) to the base plate (12) are 50mm, 140mm, 192mm, 194mm, 136mm, 73mm, 355mm, 344mm, 81mm, 190mm, and 359mm, respectively. The standard sphere has a diameter of 50mm. The 11 standard spheres are respectively erected on the base plate (12) by 11 standard rods (13) according to the projected position coordinates and height, forming a spatial distribution. The base plate (12) has a length of 650mm and a width of 630mm; the standard rod (13) has a diameter of 15mm. The projection distance between the centers of any two standard spheres on the plane of the base plate (12) is greater than 200mm and less than 550mm; the matrix condition number newCond of the three-dimensional coordinate set of the centers of the 11 standard spheres is less than 10000; Its features are, The calibration procedure is as follows: (1) Obtaining data for constructing the hand-eye calibration equation The hand-eye calibration equation is as follows: (1) In equation (1), A is the transformation matrix of the end effector of the six-axis industrial robot (14) from position i to position i+1, B is the transformation matrix of the binocular depth camera (16) from position i to i+1, and X is the transformation matrix of the binocular depth camera (16) to the end effector of the six-axis industrial robot (14) to be solved. The A-transformation matrix of the end effector of the six-axis industrial robot (14) and the B-transformation matrix of the binocular depth camera (16) are obtained by using a three-dimensional sphere calibrator (15), thus obtaining 30 pairs of A-transformation matrices and B-transformation matrices for constructing the hand-eye calibration equation. ; (2) Obtain the initial solution of the hand-eye calibration equation Obtain the initial solution for the rotation components of the X transformation matrix. That is, by using vectorization operations and the Kronecker product, the rotation component equation in the nonlinear hand-eye calibration equation is transformed into a linear equation, and the initial solution of the rotation component of the X transformation matrix is ​​obtained by solving the linear equation. ; Obtain the initial solution for the translation components of the X transformation matrix. That is, the initial solution based on the rotation components of the X transformation matrix. The translation component equations in the hand-eye calibration equations are solved using the least squares method to obtain the initial solution for the translation components of the X transformation matrix. ; The initial solution of the rotation component for: (2) In equation (2), For matrix The right singular vector corresponding to the minimum singular value. Soon Restore to a 3×3 matrix. middle Let i be the rotation component of the i-th transformation matrix A. Let I be the rotation component of the i-th B transformation matrix, and let I be the identity matrix. For Kronecker product; The initial solution of the translation component as follows; (3) In equation (3), Let i be the rotation component of the i-th transformation matrix A. Let i be the translation component of the i-th transformation matrix A. Let i be the translation component of the i-th B transformation matrix. Let I be the initial solution for the rotation components of the X transformation matrix, and let I be the identity matrix. The data of the i-th A transformation matrix is ​​obtained from step (1). (3) Solve the hand-eye calibration equation (3.1) Obtain the optimization objective function, which is to minimize the rotation error under the orthogonality constraint of the X transformation matrix. Translation error The function; The rotational error for , The translation error for , The orthogonality constraint of the X transformation matrix is: , The optimization objective function is as follows: (4) In equation (4), The translation components of the transformation matrix A. The translation components of the B transformation matrix. The rotation component of the X transformation matrix. Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix, and I be the identity matrix; (3.2) Obtain the synchronous optimization function, that is, use quaternions to improve the convergence of the optimization objective function and obtain the quaternion rotation error. Quaternion translation error And auxiliary matrices M, N, C, D, E; quaternion rotation errors are balanced by normalization parameters. Quaternion translation error Order of magnitude; The quaternion rotation error for , The quaternion translation error for , The synchronization optimization function is as follows: (5) In equation (5), and For normalization parameters, Let X be the quaternion corresponding to the rotation component in the X transformation matrix. Let X be the translation component of the transformation matrix. Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the objective function using quaternions. vec() is a vectorized operation. Represented as , Represented as ; (3.3) Use the Lagrange multiplier method to obtain the incremental solution of the synchronous optimization function. The Lagrange multiplier method is used to process equation (5) to construct the Lagrange function; the differential method is used to solve the Lagrange function, and finally the incremental solution of the synchronous optimization function is obtained; The Lagrange function is as follows: (6) In equation (6), For Lagrange multipliers, the meanings of the other variables are the same as those corresponding to those in equation (5); The incremental solution is shown in equation (7): (7) In equation (7), (8) (9) In equations (8) and (9), the variable Q is expressed as: (10) The meanings of the other variables are the same as those of the variables in equations (5) and (6); the meanings of the variables in equation (10) are the same as those of the variables in equation (5); the incremental solution of the synchronous optimization function can be obtained by calculating according to equations (7), (8), (9) and (10); (3.4) Using the incremental solution, the initial solution of the hand-eye calibration equation is optimized by an iterative algorithm to obtain the closed solution of the X transformation matrix from the binocular depth camera (16) to the end effector of the six-axis industrial robot (14); The closed solution of the transformation matrix X satisfies the following four conditions: (a) Rotation error is less than 0.15°; (b) Translation error is less than 2.5 mm; (c) The reconstruction error is less than 5.6 mm; (d) The root mean square error of reprojection is less than 1 pixel; If the closed solution of the X transformation matrix meets the calibration requirements, then the closed solution of the X transformation matrix does not meet any of the above conditions.

2. The six-axis industrial robot hand-eye calibration method based on a three-dimensional sphere calibrator according to claim 1, characterized in that: In step (1), the transformation matrix A, the transformation matrix B, and the transformation matrix X are defined as follows: (11) (12) (13) The meanings of each variable in equation (11) are as follows: Let {i+1} be the inverse of the transformation matrix from the base coordinate system of the six-axis industrial robot (14) to the coordinate system of the end effector when the end effector is at position i+1. The left superscript {E} is the coordinate system of the end effector, the left subscript {B} is the base coordinate system of the six-axis industrial robot (14), the right superscript {-1} is the matrix inversion operation, and the right subscript i+1 is the position of the (i+1)th time. Let be the transformation matrix from the base coordinate system of the six-axis industrial robot (14) to the coordinate system of the end effector when the end effector is at position i. The left superscript {E} is the coordinate system of the end effector, the left subscript {B} is the base coordinate system of the six-axis industrial robot (14), and the right subscript i is the i-th position. The meanings of each variable in equation (12) are as follows: The transformation matrix from the camera coordinate system to the world coordinate system based on the 3D sphere calibrator (15) is given by the end effector of the six-axis industrial robot (14) at position i+1. The left superscript {W} is the world coordinate system based on the 3D sphere calibrator (15), the left subscript {C} is the camera coordinate system, and the right subscript i+1 is the i+1th position. Let {i} be the inverse of the transformation matrix from the base coordinate system of the six-axis industrial robot (14) to the coordinate system of the end effector when the end effector is at position i. The left superscript {E} is the coordinate system of the end effector, the left subscript {B} is the base coordinate system of the six-axis industrial robot (14), the right superscript {-1} is the matrix inversion operation, and the right subscript i is the i-th position. The meanings of each variable in equation (13) are as follows: The transformation matrix from the end effector coordinate system to the camera coordinate system of the six-axis industrial robot (14) is given by the superscript {C}, which represents the camera coordinate system and the subscript {E}, which represents the end effector coordinate system. The detailed steps for obtaining the data of transformation matrix A and transformation matrix B in the hand-eye calibration equation using the three-dimensional sphere calibrator (15) are as follows: (1.1) Control the movement of the six-axis industrial robot (14) and adjust it to the initial posture: The optical axis of the binocular depth camera (16) on the end effector is aligned with the center of the upper surface of the base plate (12) of the three-dimensional sphere calibrator (15). The distance between the binocular depth camera (16) and the center of the upper surface of the base plate (12) of the three-dimensional sphere calibrator (15) is 0.7 meters, and the acute angle formed by the optical axis of the binocular depth camera (16) and the ground is 40°. The A transformation matrix of the six-axis industrial robot (14) in the initial posture is directly recorded manually. Meanwhile, the binocular depth camera (16) captures images of the three-dimensional sphere calibrator (15) with its left and right eyes respectively, obtaining left and right images, and sends them to the host computer. The host computer uses the binocular reconstruction method to process the left and right images to obtain the B transformation matrix. The transformation matrix A in and the B transformation matrix Construct an initial transformation matrix pair; (1.2) Construct an upper hemisphere with the center of the upper surface of the base plate (12) of the three-dimensional ball calibrator (15) as the center and a radius of 0.7 meters. The first standard ball (1) to the eleventh standard ball (11) are all located below the upper hemisphere. Set a sampling area on the upper hemisphere for uniform sampling. The acute angle formed by the line connecting any point in the sampling area and the center of the upper surface of the base plate (12) and the base plate (12) is greater than 45°. Generate 30 uniformly distributed points in the sampling area. Convert the coordinate positions of the 30 uniformly distributed points in the world coordinate system based on the three-dimensional ball calibrator (15) to the coordinate positions in the base coordinate system of the six-axis industrial robot (14) to obtain 30 target coordinates. (1.3) Control the six-axis industrial robot (14) to move the end effector to a target coordinate obtained in step (1.2), and use repositioning to adjust the posture of the six-axis industrial robot (14) so ​​that the binocular depth camera (16) can capture at least 7 standard balls on the 3D ball calibrator (15) in this posture. Manually record the A transformation matrix of the six-axis industrial robot (14) in this posture. Meanwhile, the binocular depth camera (16) captures images of the three-dimensional sphere calibrator (15) with its left and right eyes respectively, obtaining left and right images, and sends them to the host computer. The host computer uses the binocular reconstruction method to process the left and right images to obtain the B transformation matrix. The transformation matrix A in and the B transformation matrix Form a transformation matrix pair; (1.4) Repeat step (1.3) until the 30 target coordinates in step (1.2) are traversed to obtain 30 transformation matrix pairs. Based on the 30 transformation matrix pairs and the initial transformation matrix in step (1.1), calculate according to equations (11) and (12) to obtain 30 pairs of A transformation matrix and B transformation matrix data. .

3. The six-axis industrial robot hand-eye calibration method based on a three-dimensional sphere calibrator according to claim 2, characterized in that: In step (1.1), the specific steps for processing the left and right images using the binocular reconstruction method are as follows: (1.1.1) Extract the coordinates of the sphere center in the camera coordinate system For the left and right images under the same pose, the edge detection algorithm is used to extract the sphere contours of all standard spheres in the left and right images. The least squares method is used to fit the ellipse equation to obtain the coordinates of the center of all standard spheres in the left and right images. The coordinates of the center of all standard spheres in the camera coordinate system are calculated using the coordinates of the center of all standard spheres and the camera's internal parameters. Thus, the coordinates of the center of the spheres in the camera coordinate system are obtained. (1.1.2) Solving for the transformation matrix B ; The center coordinates of the 11 standard spheres in the world coordinate system based on the 3D sphere calibrator (15) are paired with the center coordinates of the spheres in the camera coordinate system. The perspective n-point algorithm is used to solve the problem, and the B transformation matrix is ​​obtained. , that is, the transformation matrix from the camera coordinate system to the world coordinate system based on the three-dimensional sphere calibrator (15) of the binocular depth camera (16) on the end effector of the six-axis industrial robot (14) at position i; The processing of the left and right images using the binocular reconstruction method in step (1.3) is the same as the processing of the left and right images using the binocular reconstruction method in step (1.1).

4. The six-axis industrial robot hand-eye calibration method based on a three-dimensional sphere calibrator according to claim 1, characterized in that: The specific steps for step (2) are as follows: (2.1) Obtain the initial solution of the rotation component of the X transformation matrix The rotation component equation in the hand-eye calibration equation is as follows: (14) The translation component equations in the hand-eye calibration equations are as follows: (15) In equations (14) and (15), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, The rotation component of the X transformation matrix. The translation components of the transformation matrix A. The translation components of the B transformation matrix. For the translation component of the X transformation matrix; Using vectorized operations to process the rotation component equation, we obtain equation (16): (16) In equation (16), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix. vec() represents the vectorization operation, which is to perform column vectorization on the matrix, that is, to expand the matrix into a column vector in column order. Using the Kronecker product to process equation (16), we obtain equation (17): (17) In equation (17), Let A be the rotation component of the transformation matrix. The rotation components of the B transformation matrix, Let X be the rotation component of the transformation matrix, and I be the identity matrix. The Kronecker product is an operation between two matrices of arbitrary size. From equation (17), we obtain the homogeneous equation (18): (18) Based on the data of 30 pairs of A transformation matrices and B transformation matrices obtained in step (1) Expanding equation (18) into equation (19): (19) In equation (19), Let i be the rotation component of the i-th transformation matrix A. Let i be the rotation component of the i-th B transformation matrix. Let X be the rotation component of the transformation matrix, and I be the identity matrix; The initial solution of the rotation component of the X transformation matrix is ​​obtained by solving equation (19) using singular value decomposition. The solution to equation (19) is ,in For matrix The right singular vector corresponding to the minimum singular value, the initial solution of the rotation component. for (2) Soon Restore it to a 3×3 matrix; (2.2) Obtain the initial solution of the translation components of the X transformation matrix Initial solution based on rotation components of the X-transform matrix The data of 30 pairs of A transformation matrices and B transformation matrices obtained in step (1) Solving equation (15) using the least squares method yields the following results. The least squares solution is used as the initial solution. initial solution The resulting formula (3) is as follows: (3)。 5. The six-axis industrial robot hand-eye calibration method based on a three-dimensional sphere calibrator according to claim 1, characterized in that: In step (3.2), the specific operations are as follows: (3.2.1) Using quaternions to improve the convergence of the optimization objective function For the optimization objective function obtained in step (3.1), quaternions are introduced. , To improve the convergence of the optimization method: Based on the quaternions corresponding to the transformation matrices A, B, and X, the Kronecker product is used to adjust the rotation error in equation (4). Translation error The quaternion rotation error is obtained through processing. Quaternion translation error as follows: (20) (21) In equations (20) and (21), Let A be the rotation component of the transformation matrix. The translation components of the transformation matrix A. Represents the translation components of the B transformation matrix. Let X be the translation component of the transformation matrix, and I be the identity matrix. Let A be the quaternion corresponding to the rotation component in the transformation matrix A. Let be the quaternion corresponding to the rotation component in the B transformation matrix. Let X be the quaternion corresponding to the rotation component in the X transformation matrix. Represented as , Represented as , Represented as , Represented as , Represented as , Represented as , Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the optimization objective function using quaternions. The auxiliary matrices are defined as follows: (22) In equation (22), each variable has the same meaning as the corresponding variable in equations (20) and (21); That is, the quaternion rotation error is obtained. Quaternion translation error And auxiliary matrices M, N, C, D, E; (3.2.2) Balancing quaternion rotation errors Quaternion translation error Obtain the synchronization optimization function on the order of magnitude; Use normalization parameters and Quaternion rotation error Quaternion translation error Normalization is performed based on the order of magnitude of the corresponding initial error value to obtain the synchronization optimization function, which includes the quaternion rotation error. Quaternion translation error The weighted combination and unit quaternion constraints; the synchronous optimization function is as follows: (5) In equation (5), and For normalization parameters, Let X be the quaternion corresponding to the rotation component of the X transformation matrix. Let X be the translation component of the transformation matrix. Matrices M, N, C, D, and E are auxiliary matrices used to improve the convergence of the objective function using quaternions. Represented as , Represented as ; Equation (5) is the final synchronization optimization function.

6. The six-axis industrial robot hand-eye calibration method based on a three-dimensional sphere calibrator according to claim 5, characterized in that: In step (3.3), the specific operations are as follows: A Lagrange multiplier μ is introduced to construct a Lagrange function. The incremental solution of the Lagrange function is solved using the differential method, that is, the incremental solution of the X transformation matrix from the binocular depth camera (16) to the end effector. The detailed operation steps are as follows: (3.3.1) Construct the Lagrange function The Lagrangian function of equation (5) is constructed as follows: (6) In equation (6), μ is a Lagrange multiplier, and the meanings of the other variables are the same as those in equation (5). (3.3.2) Solving for the Lagrange function using the differential method Differentiating equation (6) yields equation (23): (23) In equation (23), the symbol This indicates that the differential operation is performed on the variable on the right. The meanings of the other variables are the same as those of the corresponding variables in equations (20), (21) and (22). The incremental solution of the synchronous optimization function is as follows: (7) In equation (7), the definitions of H and g are given in equations (8), (9), and (10). The solution direction is determined using backtracking search, and this step ultimately yields the incremental solution of the synchronous optimization function.

7. The six-axis industrial robot hand-eye calibration method based on a three-dimensional sphere calibrator according to claim 5, characterized in that: In step (3.4), the specific operations are as follows: The initial solution is optimized using an iterative algorithm to obtain a closed-form solution for the X transformation matrix. The iterative algorithm operates as follows: (3.4.1) Input 30 sets of matrices ; (3.4.2) Calculate the auxiliary matrices M, N, C, D, and E according to equation (22); (3.4.3) The initial solution of the rotational component is obtained according to equation (2). ,calculate The corresponding quaternion The initial solution of the translation component is obtained according to equation (3). ,make , , , as well as ; (3.4.4) Based on the auxiliary matrices M, N, C, D, E and the current... The incremental solution of the current synchronization optimization function is calculated using equations (7), (8), (9), and (10). The step size β is determined and the parameters are updated by searching the backtracking line. Meanwhile, l is incremented by 1, where l equals the number of times step (3.4.4) has been completed; (3.4.5) If Greater than or equal to a predefined threshold And the number of iterations is less than or equal to the maximum limit. If so, return to step (3.4.4) and continue executing steps (3.4.4) and (3.4.5) in sequence; if Less than a predefined threshold or the number of iterations exceeds the maximum limit Then stop the iteration and obtain the final solution, which is the closed solution of the X transformation matrix; That is, to achieve accurate calibration of the X-transformation matrix from the binocular depth camera (16) to the end effector of the six-axis industrial robot (14), that is, the rotation error of the X-transformation matrix is ​​less than 0.15°, the reconstruction error is less than 5.6mm, the translation error is less than 2.5mm, and the root mean square error of reprojection is less than 1 pixel.

Images