Line structured light 3D vision hand-eye calibration method for collaborative robot

Abstract

The invention relates to a collaborative robot line structured light 3D vision hand-eye calibration method which comprises the following steps: acquiring a robot motion space, selecting a fixed welding seam feature point in the robot motion space, and determining coordinate values of the fixed welding seam feature point in a robot base coordinate system and a sensor coordinate system; based on the coordinate values of the fixed welding seam feature points under the robot base coordinate system and the sensor coordinate system, the conversion relation from the camera coordinate system to the robot base coordinate system is obtained; obtaining a robot expression posture mode, and determining a conversion matrix between a robot tail end coordinate system and a robot base coordinate system and a conversion matrix between a sensor coordinate system and the robot tail end coordinate system based on the robot expression posture mode, namely a hand-eye matrix; and performing hand-eye calibration through a conversion relation from a camera coordinate system to a robot base coordinate system, a conversion matrix from a robot tail end coordinate system to a robot base coordinate system and a hand-eye matrix.

Description

Technical Field

[0001] The present invention relates to the technical field of robot 3D vision calibration, and in particular to a collaborative robot line structured light 3D vision hand-eye calibration method. Background Art

[0002] At present, there are two common methods for hand-eye calibration: the three-dimensional target method and the two-dimensional target method. The two-dimensional targets mainly include the checkerboard calibration plate, the center of circle calibration plate, etc. During the calibration process, it is necessary to obtain different positions of the target and estimate the hand-eye transformation matrix by identifying the corner points. There are many types of three-dimensional targets. In order to obtain the three-dimensional coordinates of feature points more accurately, the three-dimensional calibration ball method is often used. The standard ball method is based on a local arc formed by the laser line on the spherical surface, and the coordinates of the center of the sphere are calculated through a series of point cloud processing. The following are two existing more mature calibration algorithms:

[0003] 1. Two-step method:

[0004] Tsai two-step method is a classic hand-eye calibration method used to determine the transformation relationship between the robot end and the camera. The main implementation of this method is as follows:

[0005] Step 1: Solve the rotation relationship:

[0006] The calibration plate is placed in the detection plane and remains stationary. The robot's posture is changed to perform a series of movements, and the calibration plate image data is collected at each posture.

[0007] Use the calibration plate to obtain positioning information, and calculate the rotation matrix and translation vector from the calibration plate to the camera coordinate system.

[0008] The camera's internal and external parameter data are obtained through camera calibration, and the relative rotation of the calibration plate between different robot postures is calculated based on this.

[0009] Use the Rodrigues formula or quaternion method to solve the rotation matrix in the hand-eye matrix from the rotation differences between multiple sets of different poses.

[0010] Step 2: Solve the translation relationship:

[0011] Use the rotation matrix obtained in the first step to remove the influence of the rotation component.

[0012] Use the rotation relationship combined with the relative translation obtained previously to establish a system of linear equations.

[0013] The translation part of the hand-eye matrix is ​​solved using the least squares method to solve the system of linear equations.

[0014] 2. Auxiliary calibration ball method:

[0015] The calibration ball method hand-eye calibration mainly uses a calibration ball with known geometric features to assist in positioning and determining the spatial transformation relationship between the robot end and the sensor. The sensor includes a camera and a line laser. The calibration ball method hand-eye calibration has the following steps:

[0016] Prepare a calibration ball and place it in the robot's workspace so that it is within the camera's field of view. The laser emits a line laser that hits the surface of the calibration ball.

[0017] Control the robot end to drive the sensor to move to different positions and directions of the calibration ball, record the end pose matrix of each position of the robot; and use the camera to capture the image of the calibration ball with a linear laser on the surface.

[0018] The calibration sphere image acquired by the camera is analyzed, and the coordinates of the center of the section circle are obtained through point cloud fitting, and then the coordinates of the center of the calibration sphere are solved.

[0019] Using multiple different robot end pose matrices and the corresponding sphere center coordinates, a linear or nonlinear set of equations is constructed to solve the transformation matrix of the robot end relative to the camera coordinate system.

[0020] According to the spatial relationship of the robot's end sensor and the relationship between the position of the sphere center in the camera's field of view, the spatial transformation relationship between the robot's tool coordinate system and the camera coordinate system is solved, which is the result of hand-eye calibration.

[0021] Problems of the above prior art solutions:

[0022] The Tsai two-step hand-eye calibration requires the assistance of a checkerboard calibration board, and the calibration steps are complicated. It requires multiple sets of calibration boards and corresponding robot poses, and has high requirements on the accuracy of the robot pose and calibration board pose data. In addition, the error of the rotation matrix in the calibration process of the Tsai two-step method will be accumulated in the solution of the translation vector.

[0023] The auxiliary calibration sphere method requires the use of an external calibration sphere, which is costly. The accuracy of calculating the center of the sphere through segmentation and fitting of the section arc point cloud is easily affected by noise, and has high requirements on the camera's imaging quality and image processing algorithm. Due to the roughness of the standard sphere surface, not all points on the arc are on the same plane, which will cause the normal vector of the plane to shift, affecting the accuracy of the sphere center coordinates. Summary of the invention

[0024] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to propose a collaborative robot line structured light 3D vision hand-eye calibration method. From the application perspective, the optimal application form of the algorithm is found in practical applications, the steps of hand-eye calibration are simplified, the operation difficulty is reduced, and line structured light is used to assist in obtaining three-dimensional coordinates to improve the accuracy of feature point recognition.

[0025] To achieve the above object, the present invention provides the following solutions:

[0026] A collaborative robot line structured light 3D vision hand-eye calibration method, comprising:

[0027] Acquire the robot motion space, select a fixed weld feature point in the robot motion space, and determine the coordinate value of the fixed weld feature point in the robot base coordinate system and the sensor coordinate system;

[0028] Based on the coordinate values ​​of the fixed weld feature points in the robot base coordinate system and the sensor coordinate system, a conversion relationship from the camera coordinate system to the robot base coordinate system is obtained;

[0029] Acquire the robot's gesture expression mode, and based on the robot's gesture expression mode, determine the conversion matrix between the robot's end coordinate system and the robot's base coordinate system, and the conversion matrix between the sensor coordinate system and the robot's end coordinate system, i.e., the hand-eye matrix;

[0030] The hand-eye calibration is performed through the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix from the robot terminal coordinate system to the robot base coordinate system, and the hand-eye matrix.

[0031] Optionally, the method for obtaining the transformation relationship from the camera coordinate system to the robot base coordinate system is:

[0032] P B =H TB H LT P L

[0033] Among them, P B is the coordinate of the midpoint P in the robot base coordinate system, P L is the coordinate of the midpoint P in the sensor coordinate system, H TB is the transformation matrix between the robot end coordinate system and the robot base coordinate system, H LT It is the transformation matrix between the sensor coordinate system and the robot end coordinate system, that is, the hand-eye matrix.

[0034] Optionally, obtaining the robot's expression posture includes:

[0035] Based on Euler angles and quaternions, different representation postures are converted into posture matrices to obtain the robot's posture expression mode.

[0036] Optionally, determining the hand-eye matrix includes:

[0037]

[0038] Among them, H LTis the transformation matrix between the sensor coordinate system and the robot end coordinate system, that is, the hand-eye matrix, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 All are parameters of the rotation matrix in the hand-eye matrix, and t1, t2, and t3 are parameters of the translation vector in the hand-eye matrix.

[0039] Optionally, performing hand-eye calibration through a conversion relationship from the camera coordinate system to the robot base coordinate system, a conversion matrix from the robot end coordinate system to the robot base coordinate system, and a hand-eye matrix includes:

[0040] According to the hand-eye matrix combined with the conversion relationship from the camera coordinate system to the robot base coordinate system, a first conversion result is obtained:

[0041]

[0042] Based on the full rank of the transformation matrix between the robot end coordinate system and the robot base coordinate system, multiply the first transformation result by H TB -1 , get the second conversion result:

[0043]

[0044] Process the second conversion result to obtain a third conversion result:

[0045]

[0046] The second conversion result and the third conversion result are combined to obtain a combined result:

[0047]

[0048] Among them, x B ,y B 、z B are the coordinate values ​​of the feature points in the robot base coordinate system, H TB is the transformation matrix from the robot end coordinate system to the robot base coordinate system, H TB -1 is the inverse of the transformation matrix from the robot end coordinate system to the robot base coordinate system, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r31 、r 32 、r 33 are all parameters of the rotation matrix in the hand-eye matrix, t1, t2, t3 are all translation vector parameters in the hand-eye matrix, y L ,z L is the coordinate value of the feature point in the sensor coordinate system, X, Y, and Z are all replaced by H TB -1 Left multiplication (x B ,y B ,z B ,1) matrix.

[0049] Optionally, performing hand-eye calibration through the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix from the robot terminal coordinate system to the robot base coordinate system, and the hand-eye matrix further includes:

[0050] The solution model is established using the joint results:

[0051]

[0052] Among them, X1, X2, X3, X4, Y1, Y2, Y3, Y4, Z1, Z2, Z3, and Z4 are all substituted H TB -1 Left multiplication (x B ,y B ,z B ,1) The value of the matrix, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 are all parameters of the rotation matrix in the hand-eye matrix, t1, t2, t3 are all translation vector parameters in the hand-eye matrix, y L1 ,y L2 ,y L3 ,y L4 、z L1 、z L2 、z L3 、z L4 They are all the coordinate values ​​of a set of feature points in the sensor coordinate system;

[0053] Simplify the solution model and take the transposition respectively to obtain the transposition result:

[0054] D T X T =C T

[0055] Among them, D is four groups (X i ,Y i ,Z i ,1), X is the hand-eye matrix, C is the four groups (0,Y Li ,Z Li ,1) The matrix composed of;

[0056] Set the solution condition: A=D T ,B=C T ,X=X T ;

[0057] The transposed result is solved by using singular value decomposition according to the solution condition to obtain a hand-eye matrix.

[0058] Optionally, performing hand-eye calibration through the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix from the robot terminal coordinate system to the robot base coordinate system, and the hand-eye matrix further includes:

[0059] Define a decomposition condition, and use the decomposition condition to perform singular value decomposition on the rotation matrix to be orthogonalized in the hand-eye matrix to obtain the rotation matrix:

[0060] R=UΛV T

[0061] Among them, R is the orthogonalized rotation matrix, U is the square matrix composed of the eigenvectors of P, Λ is the characteristic matrix, V is the square matrix composed of the eigenvectors of W, and T is the transposition symbol;

[0062] The characteristic matrix is ​​replaced with the identity matrix E, and the standard orthogonal hand-eye matrix is ​​further obtained.

[0063] Optionally, define the decomposition condition as:

[0064]

[0065] in, is the rotation matrix in the hand-eye matrix, R LT is the transpose of the rotation matrix in the hand-eye matrix.

[0066] The beneficial effects of the present invention are:

[0067] The present invention does not require any additional calibration objects, and the calibration process is simple. A weld feature point is randomly selected, the robot is moved, the laser line is hit on the feature point, and the sensor recognizes the feature point.

[0068] The present invention uses line structured light to assist in identifying three-dimensional feature points, and the data acquisition accuracy is relatively high.

[0069] The invention has low requirements on the robot posture, is easy to obtain data, and has a small amount of calculation. By changing the robot posture, only four sets of sensor data and the robot posture are needed to solve the hand-eye matrix.

[0070] The present invention has fewer calibration steps, is easy to operate, and introduces as few process errors as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0072] Figure 1 This is a flow chart of a collaborative robot line structured light 3D vision hand-eye calibration method according to an embodiment of the present invention;

[0073] Figure 2 Schematic diagram of a collaborative robot device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] like Figure 1 As shown, this embodiment discloses a collaborative robot line structured light 3D vision hand-eye calibration method, including: obtaining the robot motion space, selecting fixed weld feature points in the robot motion space, and determining the coordinate values ​​of the fixed weld feature points in the robot base coordinate system and the sensor coordinate system; based on the coordinate values ​​of the fixed weld feature points in the robot base coordinate system and the sensor coordinate system, obtaining the conversion relationship from the camera coordinate system to the robot base coordinate system; obtaining the robot's posture expression method, and based on the robot's posture expression method, determining the conversion matrix between the robot's terminal coordinate system and the robot's base coordinate system, and the conversion matrix (hand-eye matrix) between the sensor coordinate system and the robot's terminal coordinate system; performing hand-eye calibration through the conversion relationship from the camera coordinate system to the robot's base coordinate system, the conversion matrix from the robot's terminal coordinate system to the robot's base coordinate system, and the hand-eye matrix.

[0077] Specifically:

[0078] S1, select any weld type, take a fixed weld point as the feature point, adjust the robot posture, align the robot welding torch with the feature point, and obtain the position of the feature point in the robot coordinate system;

[0079] S2, control the robot to drive the line structured light sensor to move to the laser line to identify the feature point, obtain the coordinates of the feature point in the sensor coordinate system; and record the end position of the robot in the current posture;

[0080] S3, change the robot posture to ensure that the laser line recognizes the feature point, repeat step S2 four times, and obtain four sets of feature point coordinates under different postures and their corresponding robot end postures;

[0081] S4, according to steps S1-S3, multiple sets of parameters are obtained, and the hand-eye calibration matrix is ​​calculated using calibration equation (1);

[0082] S5, orthogonalize the hand-eye matrix in step S4 to obtain the final result.

[0083] like Figure 2 As shown, the collaborative robot device mainly includes a collaborative robot, a linear structured light vision sensor and a robot end welding gun; the coordinate system description is shown in Table 1.

[0084] Table 1 Coordinate system description

[0085]

[0086] Furthermore, the method for obtaining the transformation relationship from the camera coordinate system to the robot base coordinate system is:

[0087] P B =H TB H LT P L

[0088] Among them, P B is the coordinate of the midpoint P in the robot base coordinate system, P L is the coordinate of the midpoint P in the sensor coordinate system, H TB is the transformation matrix between the robot end coordinate system and the robot base coordinate system, H LT It is the transformation matrix between the sensor coordinate system and the robot end coordinate system, that is, the hand-eye matrix.

[0089] Furthermore, obtaining the robot's expression posture includes: based on Euler angles and quaternions, determining different expression postures and converting them into posture matrices to obtain the robot's expression posture.

[0090] Specifically, a fixed weld feature point P is selected in the robot motion space. The point is in the robot base coordinate system O B The coordinate value of the B =(X B ,Y B ,Z B ), the coordinate value in the sensor coordinate system is P L =(X L ,Y L ,Z L ), then the transformation relationship between point P from the camera coordinate system to the robot base coordinate system is:

[0091] P B =H TB H LT P L (1)

[0092] In equation (1), P B , P L It can be obtained by measurement. For the robot end pose matrix H TB , the matrix parameters can be obtained directly through the robot and can be expressed as follows:

[0093]

[0094] Among them, R and T represent the rotation matrix and translation vector respectively:

[0095]

[0096] Robots all express their positions in Cartesian coordinates, but different robots have different ways of expressing their postures. There are two types: Euler angles and quaternions. Both need to be converted into posture matrices for use. The following is an explanation of how to convert posture matrices in different ways:

[0097] Quaternion form expression:

[0098] Q=(wxyz) (4)

[0099] Among them, w, x, y, and z are general expressions of quaternions.

[0100] According to formula (4), the conversion expression of rotation matrix and quaternion can be obtained:

[0101]

[0102] Euler angle form:

[0103] Because Euler angles have different combinations and rotations, the way Euler angles represent postures is not unique, and the final expression of the rotation matrix is ​​also different. First, several transformation combinations and rotation methods of Euler angles are explained:

[0104] There are six combinations of different rotation orders for the x, y, and z axes: (xyz, yzx, zxy, xzy, zyx, yxz). When rotating the same angle, different rotation orders will result in different postures.

[0105] Assuming that the angles of rotation around the three axes xyz are αβγ respectively, the calculation method of the rotation matrix of the three rotations is as follows:

[0106]

[0107] The rotation methods used to express the posture of robots from different manufacturers may vary. You need to choose the corresponding calculation method based on the posture rotation method corresponding to the robot used.

[0108] Furthermore, determining the hand-eye matrix includes:

[0109]

[0110] Among them, H LT is the transformation matrix between the sensor coordinate system and the robot end coordinate system, that is, the hand-eye matrix, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 All are parameters of the rotation matrix in the hand-eye matrix, and t1, t2, and t3 are parameters of the translation vector in the hand-eye matrix.

[0111] Furthermore, hand-eye calibration is performed through the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix from the robot end coordinate system to the robot base coordinate system, and the hand-eye matrix, including:

[0112] According to the hand-eye matrix combined with the conversion relationship from the camera coordinate system to the robot base coordinate system, the first conversion result is obtained:

[0113]

[0114] Based on the full rank of the transformation matrix between the robot end coordinate system and the robot base coordinate system, multiply the first transformation result by H TB -1 , get the second conversion result:

[0115]

[0116] The second conversion result is processed to obtain a third conversion result:

[0117]

[0118] The second conversion result and the third conversion result are combined to obtain a combined result:

[0119]

[0120] Among them, x B ,y B 、z B are the coordinate values ​​of the feature points in the robot base coordinate system, H TB is the transformation matrix from the robot end coordinate system to the robot base coordinate system, H TB -1 is the inverse of the transformation matrix from the robot end coordinate system to the robot base coordinate system, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 are all parameters of the rotation matrix in the hand-eye matrix, t1, t2, t3 are all translation vector parameters in the hand-eye matrix, y L ,z L is the coordinate value of the feature point in the sensor coordinate system, X, Y, and Z are all replaced by H TB -1 Left multiplication (x B ,y B ,z B ,1) matrix.

[0121] Furthermore, the hand-eye calibration is performed through the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix between the robot end coordinate system and the robot base coordinate system, and the hand-eye matrix, and further includes:

[0122] Use the combined results to build a solution model:

[0123]

[0124] Among them, X1, X2, X3, X4, Y1, Y2, Y3, Y4, Z1, Z2, Z3, and Z4 are all substituted H TB -1 Left multiplication (x B ,y B,z B ,1) The value of the matrix, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 are all parameters of the rotation matrix in the hand-eye matrix, t1, t2, t3 are all translation vector parameters in the hand-eye matrix, y L1 ,y L2 ,y L3 ,y L4 、z L1 、z L2 、z L3 、z L4 They are all the coordinate values ​​of a set of feature points in the sensor coordinate system;

[0125] Simplify the solution model and take the transposition respectively to obtain the transposition result:

[0126] D T X T =C T

[0127] Among them, D is four groups (X i ,Y i ,Z i ,1), X is the hand-eye matrix, C is the four groups (0,Y Li ,Z Li ,1) The matrix composed of;

[0128] Set the solution condition: A=D T ,B=C T ,X=X T ;

[0129] The transposed result is solved using singular value decomposition according to the solution conditions to obtain the hand-eye matrix.

[0130] Furthermore, the hand-eye calibration is performed through the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix between the robot end coordinate system and the robot base coordinate system, and the hand-eye matrix, and further includes:

[0131] Define the decomposition conditions, use the decomposition conditions to perform singular value decomposition on the rotation matrix to be orthogonalized in the hand-eye matrix, and obtain the rotation matrix:

[0132] R=UΛV T

[0133] Among them, R is the orthogonalized rotation matrix, U is the square matrix composed of the eigenvectors of P, Λ is the characteristic matrix, V is the square matrix composed of the eigenvectors of W, and T is the transposition symbol;

[0134] Replace the feature matrix with the identity matrix E, and further obtain the standard orthogonal hand-eye matrix.

[0135] Furthermore, the decomposition condition is defined as:

[0136]

[0137] in, is the rotation matrix in the hand-eye matrix, R LT is the transpose of the rotation matrix in the hand-eye matrix.

[0138] Specifically,

[0139] The above method can be used to determine H TB All parameters of the hand-eye matrix H LT , which can also be expressed as follows:

[0140]

[0141] Through equations (1) and (7), we can get:

[0142]

[0143] Matrix H TB Full rank, multiply both sides of formula (8) by H TB -1 ,get:

[0144]

[0145] Process the left side of equation (9) and let:

[0146]

[0147] Combining equations (9) and (10), we get:

[0148]

[0149] A group of P B ,P L The corresponding values ​​can be solved to get three equations. There are 12 parameters in X, and at least 4 sets of corresponding values ​​are needed to solve the 12 parameters:

[0150]

[0151] make:

[0152]

[0153] Formula (12) can be simplified as:

[0154] C=XD (13)

[0155] Taking the transposition of both sides of equation (13) we can get:

[0156] C T =(XD) T (14)

[0157] Simplified:

[0158] D T X T =C T

[0159] Let A = D T ,B=C T ,X=X T , then formula (14) is:

[0160] AX=B (15)

[0161] Using singular value decomposition (SVD) to solve equation (15), X T That is the obtained hand-eye matrix H LT .

[0162] Equation (15) is used to obtain the numerical solution of the rotation matrix. Since the sensor coordinates lack X-axis data, an approximate rotation matrix R with errors will be obtained. The orthogonal constraint is not considered, that is, the rotation matrix R is not a standard orthogonal matrix. The rotation matrix R needs to be orthogonalized. LT The rotation matrix R LT for:

[0163]

[0164] definition For the matrix R LT Perform singular value decomposition to obtain the rotation matrix:

[0165] R=UΛV T

[0166] The square matrix U formed by the eigenvectors of P, the square matrix V formed by the eigenvectors of W, and R are the orthogonalized rotation matrices. Replacing the characteristic matrix Λ with the unit matrix E, we can get the orthogonal matrix:

[0167] R=UEV T =UV T

[0168] At this time, the hand-eye matrix H LTis a standard orthogonal matrix to complete the hand-eye calibration process.

[0169] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A collaborative robot line structured light 3D vision hand-eye calibration method, characterized in that: include: Acquire the robot motion space, select a fixed weld feature point in the robot motion space, and determine the coordinate value of the fixed weld feature point in the robot base coordinate system and the sensor coordinate system; Based on the coordinate values ​​of the fixed weld feature points in the robot base coordinate system and the sensor coordinate system, a conversion relationship from the camera coordinate system to the robot base coordinate system is obtained; Acquire the robot's gesture expression mode, and based on the robot's gesture expression mode, determine the conversion matrix between the robot's end coordinate system and the robot's base coordinate system, and the conversion matrix between the sensor coordinate system and the robot's end coordinate system, i.e., the hand-eye matrix; The hand-eye calibration is performed through the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix from the robot terminal coordinate system to the robot base coordinate system, and the hand-eye matrix.

2. The collaborative robot line structured light 3D vision hand-eye calibration method according to claim 1, characterized in that: The method for obtaining the transformation relationship from the camera coordinate system to the robot base coordinate system is: P B =H TB H LT P L Among them, P B is the coordinate of the midpoint P in the robot base coordinate system, P L is the coordinate of the midpoint P in the sensor coordinate system, H TB is the transformation matrix between the robot end coordinate system and the robot base coordinate system, H LT It is the transformation matrix between the sensor coordinate system and the robot end coordinate system, that is, the hand-eye matrix.

3. The collaborative robot line structured light 3D vision hand-eye calibration method according to claim 1, characterized in that: The method of obtaining the robot's expression posture includes: Based on Euler angles and quaternions, different representation postures are converted into posture matrices to obtain the robot's posture expression mode.

4. The collaborative robot line structured light 3D vision hand-eye calibration method according to claim 1, characterized in that: Determining the hand-eye matrix includes: Among them, H LT It is the transformation matrix between the sensor coordinate system and the robot end coordinate system, that is, the hand-eye matrix. r11, r12, r13, r21, r22, r23, r31, r32, and r33 are all parameters of the rotation matrix in the hand-eye matrix. t1, t2, and t3 are all translation vector parameters in the hand-eye matrix.

5. The collaborative robot line structured light 3D vision hand-eye calibration method according to claim 1, characterized in that: The hand-eye calibration is performed by using the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix from the robot end coordinate system to the robot base coordinate system, and the hand-eye matrix, including: According to the hand-eye matrix combined with the conversion relationship from the camera coordinate system to the robot base coordinate system, a first conversion result is obtained: Based on the full rank of the transformation matrix between the robot end coordinate system and the robot base coordinate system, multiply the first transformation result by H TB -1 , get the second conversion result: Process the second conversion result to obtain a third conversion result: The second conversion result and the third conversion result are combined to obtain a combined result: Among them, x B ,y B 、z B are the coordinate values ​​of the feature points in the robot base coordinate system, H TB is the transformation matrix from the robot end coordinate system to the robot base coordinate system, H TB -1 is the inverse of the transformation matrix from the robot end coordinate system to the robot base coordinate system, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 are all parameters of the rotation matrix in the hand-eye matrix, t1, t2, t3 are all translation vector parameters in the hand-eye matrix, y L ,z L is the coordinate value of the feature point in the sensor coordinate system, X, Y, and Z are all replaced by H TB -1 Left multiplication (x B ,y B ,z B ,1) matrix.

6. The collaborative robot line structured light 3D vision hand-eye calibration method according to claim 5, characterized in that: The hand-eye calibration is performed by using the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix from the robot end coordinate system to the robot base coordinate system, and the hand-eye matrix, and further includes: The solution model is established using the joint results: Among them, X1, X2, X3, X4, Y1, Y2, Y3, Y4, Z1, Z2, Z3, and Z4 are all substituted H TB -1 Left multiplication (x B ,y B ,z B ,1) The value of the matrix, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 are all parameters of the rotation matrix in the hand-eye matrix, t1, t2, t3 are all translation vector parameters in the hand-eye matrix, y L1 ,y L2 ,y L3 ,y L4 、z L1 、z L2 、z L3 、z L4 They are all the coordinate values ​​of a set of feature points in the sensor coordinate system; Simplify the solution model and take the transposition respectively to obtain the transposition result: D T X T =C T Among them, D is four groups (X i ,Y i ,Z i ,1), X is the hand-eye matrix, C is the four groups (0,Y Li ,Z Li ,1) The matrix composed of; Set the solution condition: A=D T ,B=C T ,X=X T ; The transposed result is solved by using singular value decomposition according to the solution condition to obtain a hand-eye matrix.

7. The collaborative robot line structured light 3D vision hand-eye calibration method according to claim 6, characterized in that: The hand-eye calibration is performed by using the conversion relationship from the camera coordinate system to the robot base coordinate system, the conversion matrix from the robot end coordinate system to the robot base coordinate system, and the hand-eye matrix, and further includes: Define a decomposition condition, and use the decomposition condition to perform singular value decomposition on the rotation matrix to be orthogonalized in the hand-eye matrix to obtain the rotation matrix: R=UΛV T Among them, R is the orthogonalized rotation matrix, U is the square matrix composed of the eigenvectors of P, Λ is the characteristic matrix, V is the square matrix composed of the eigenvectors of W, and T is the transposition symbol; The characteristic matrix is ​​replaced with the identity matrix E, and the standard orthogonal hand-eye matrix is ​​further obtained.

8. The collaborative robot line structured light 3D vision hand-eye calibration method according to claim 7, characterized in that: The decomposition conditions are defined as: in, is the rotation matrix in the hand-eye matrix, R LT is the transpose of the rotation matrix in the hand-eye matrix.

Images