Line laser camera hand-eye calibration method

Abstract

The invention relates to a line laser camera hand-eye calibration method comprising the following steps: S1, determining a triangular pyramid calibration block, and selecting a fixed calibration point; s2, based on the triangular pyramid calibration block and the fixed calibration point, TCP position calibration and TCP attitude calibration are carried out on the center point of the tail end tool, and a calibrated TCP point is obtained; s3, selecting calibration parameters of the line laser camera, and verifying an internal reference matrix of the line laser camera in combination with the triangular pyramid calibration block; s4, on the basis of the calibrated calibration parameters and the calibrated TCP points, fixed calibration points are measured multiple times at different poses, feature values of the fixed calibration points are extracted, and then a hand-eye calibration transformation matrix is calculated; and S5, performing calibration error verification on the hand-eye calibration transformation matrix, judging whether a precision requirement is met or not, if so, returning to the step S2 to perform calibration again until the precision requirement is met, and if not, not performing any operation. Compared with the prior art, the method has the advantages of high calibration precision and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a hand-eye calibration method for a line laser camera. Background Art

[0002] The application of line laser camera hand-eye calibration technology in welding robots is crucial. By accurately acquiring the three-dimensional information of the workpiece, the precise control and stable operation of the robot welding path can be effectively achieved. This technology can accurately convert the camera coordinate system with the robot coordinate system, improve the level of welding automation, significantly improve welding accuracy, reduce position errors, and ensure the consistency of welding quality. It is particularly suitable for automated welding tasks under complex working conditions. At present, the commonly used line laser camera hand-eye calibration methods include spherical calibration blocks and plane targets. For example, when using a spherical calibration block, although the center of the circle can be solved more accurately by scanning the local arc on the sphere and performing algorithm fitting, this method is limited by the material and roughness of the calibration block surface, and the fitting accuracy is more ideal when the arc radius is in the range of 0.5-0.9 of the standard sphere radius, so the measurement range is limited. In addition, the error accumulation in the fitting process will also affect the calibration results. In contrast, the plane target calibration method can only provide two-dimensional feature point information. Although three-dimensional information can be obtained by shooting at multiple angles, it is more complex and has a greater risk of error accumulation. At the same time, planar targets have high requirements on the camera’s angle and field of view. Any angle deviation will affect the accuracy of feature point recognition, thereby affecting the calibration effect.

[0003] The current hand-eye calibration measures for traditional line laser cameras are:

[0004] 1. One calibration method is to measure the spherical calibration block and indirectly solve the hand-eye calibration matrix by fitting the center position of the circle, such as the literature (Cao Dongwang, Hao Weidong, Yang Daoguo. Research on the hand-eye calibration method of line laser vision measurement robot [J]. Machine Tool and Hydraulics, 2020, 48(21): 31-35.). This article proposes a hand-eye calibration method for line laser vision measurement robot based on calibration sphere. This method uses the fixed position of the center of the calibration sphere in the robot base coordinate system, combines the robot kinematics to establish a matrix transformation relationship, and simplifies the solution of the rotation matrix by introducing quaternions, and then optimizes the measurement data with the least squares method, and finally realizes the robot hand-eye calibration. The experimental results show that this method is highly effective in dealing with the complex transformation relationship between the flange coordinate system and the camera coordinate system. However, this method also has limitations in some cases: although the center position can be solved more accurately by scanning the local arc on the surface of the calibration sphere and performing algorithm fitting, its accuracy is affected by the material and surface roughness of the calibration block, and the fitting effect is best when the radius of the intersecting arc is 0.5 to 0.9 times the radius of the calibration sphere, which limits the measurement range. In addition, when the center of the calibration sphere is close to the light plane of the laser probe or the radius of the intersecting arc is small, error accumulation is prone to occur. Therefore, in order to improve the calibration accuracy, abnormal data can be eliminated to optimize data consistency.

[0005] 2. Another calibration method is the hand-eye calibration method of the line structured light vision sensor robot based on a plane target proposed by Wu Qinghua et al. This article proposes a hand-eye calibration method for the line laser vision measurement robot based on a plane target. This method designs a plane target containing a single circular mark, and makes the robot adjust its posture to ensure that the laser line passes through the center of the target circle. The pixel coordinates of the center of the circle are extracted by image processing, so as to obtain the three-dimensional coordinates of the center of the circle in the sensor coordinate system. By collecting coordinate data in different postures for many times, and combining the robot kinematic relationship, the least squares method is used to solve the hand-eye matrix, and the accurate calibration between the visual sensor and the robot end flange coordinate system is achieved. Experiments show that this method has achieved good results in reducing the positioning error of the three-dimensional measurement system, and because of the use of a plane target, it does not require the use of high-cost calibration tools, and has high economy and practicality. However, this method also has certain limitations, especially the high requirements for the field of view and angle of the camera. Any slight angle deviation or incomplete field of view will lead to a reduction in the accuracy of feature point recognition, which in turn affects the overall calibration effect. Therefore, ensuring accurate alignment between the camera and the target is the key to achieving high-precision calibration. Summary of the invention

[0006] The purpose of the present invention is to provide a line laser camera hand-eye calibration method for improving the stability and precise operation of a robot under complex working conditions.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A line laser camera hand-eye calibration method comprises the following steps:

[0009] S1. Based on the measurement range of the line laser camera and the space limitation of the robot end, determine the triangular pyramid calibration block and select fixed calibration points, where the line laser camera and the end tool are set at the robot end;

[0010] S2. Based on the triangular pyramid calibration block and the fixed calibration point, the TCP position calibration and the TCP posture calibration of the center point of the end tool are performed to obtain a calibrated TCP point;

[0011] S3, selecting the calibration parameters of the line laser camera, and verifying the intrinsic parameter matrix of the line laser camera in combination with the triangular pyramid calibration block;

[0012] S4, based on the verified calibration parameters and the calibrated TCP points, measuring the fixed calibration points multiple times in different postures, extracting eigenvalues ​​of the fixed calibration points, and then calculating the hand-eye calibration transformation matrix;

[0013] S5. Perform calibration error check on the hand-eye calibration transformation matrix to determine whether it meets the accuracy requirement. If so, return to step S2 to recalibrate until the accuracy requirement is met. If not, do nothing.

[0014] Furthermore, the fixed calibration point is an arbitrarily selected point on three edges of the triangular pyramid calibration block.

[0015] Further, in step S2, the step of calibrating the TCP position includes:

[0016] Move the end tool to any point in space as a reference point, wherein the selected reference point is a fixed calibration point;

[0017] The position data of the end tool in the end coordinate system after the end tool moves to the reference point in different postures for four times is recorded, and the calibrated TCP position is obtained by solving and calculating, wherein the calculation process of the calibrated TCP position includes:

[0018] Determine the relationship of the reference point to the robot base at different positions:

[0019]

[0020] In the formula, is the rotation transformation matrix of the end point of the end tool relative to the robot flange position, is the translation transformation matrix of the robot flange relative to the base position during the i-th posture movement, is the rotation transformation matrix of the robot flange relative to the base posture during the i-th posture movement, is the translation transformation matrix of the end point of the end tool relative to the flange position;

[0021] Arrange and merge the relations to obtain the overdetermined system of equations:

[0022]

[0023] The singular value decomposition method is used to solve the overdetermined equations:

[0024]

[0025] Where V is The eigenvector orthogonal matrix of The eigenvectors of an orthogonal matrix are Σ, which is a diagonal matrix with non-negative singular values. The singular values ​​are denoted by σ1, σ2, ..., σ n ,Σ + is the pseudo-inverse of Σ, that is

[0026] Solve using the least squares method The eigenvalue of and square root are taken to obtain the calibrated TCP position.

[0027] Furthermore, after the TCP position is calibrated, the error of the calibrated TCP position is measured, wherein the calculation expression of the error is:

[0028]

[0029] In the formula, δ is the calibration error, is the translation transformation matrix of the robot flange relative to the base position during the i-th posture movement, is the rotation transformation matrix of the robot flange relative to the base posture during the i-th posture movement, It is the translation transformation matrix of the end point of the end tool relative to the flange position.

[0030] Furthermore, in step S2, the step of performing TCP posture calibration includes:

[0031] Determine a fifth calibration point and a sixth calibration point, wherein the four calibration points are determined according to the end tool moving to the reference point four times in different postures, the fifth calibration point is obtained by translating a certain distance in the X+ direction based on the fourth calibration point, and the sixth calibration point is obtained by translating a certain distance in the Z+ direction based on the fourth calibration point;

[0032] Calculate the relationship between the fourth calibration point and the fifth calibration point:

[0033]

[0034] Where X is the axis vector of the X-axis of the tool coordinate system, is the translation transformation matrix of the robot flange relative to the base during the sixth posture movement, which serves as the fifth calibration point position;

[0035] Calculate the relationship between the fourth calibration point and the sixth calibration point:

[0036]

[0037] Where Z is the axial vector of the Z axis of the tool coordinate system, is the translation transformation matrix of the robot flange relative to the base position during the sixth posture movement, which serves as the sixth calibration point position;

[0038] According to the right-hand rule, the axial vector of the Y axis is obtained:

[0039] Y=Z×X

[0040] Where, Y is the axial vector of the Y axis of the tool coordinate system;

[0041] The calibrated TCP posture is obtained by combining the X, Y and Z.

[0042] Furthermore, the method further includes a step of performing a calibration error check on the calibrated TCP point, the specific steps of which include:

[0043] 1) Position verification:

[0044] Using the fixed calibration point as a position error verification point;

[0045] Record the position information when contacting the position error verification point with different postures, and calculate the average position error and the maximum position error, and judge whether the average position error and the maximum position error are within the allowable range. If so, execute the next step, if not, return to step S2 and recalibrate, wherein the process of calculating the average position error and the maximum position error includes:

[0046] Calculate the average point of the position information

[0047]

[0048] In the formula, is the average value in the x, y, and z directions, i=1, 2, ...., N represents the position information under different postures, and N is the total number of measurements;

[0049] Calculate the position point (x i ,yi ,z i ) and the average point The distance between them, as the Euclidean space distance of the position error Said The distance calculation expression is:

[0050]

[0051] Based on the Euclidean space distance Calculate the average position error and maximum position error of the TCP calibration point:

[0052]

[0053]

[0054] In the formula, is the average position error, e max is the maximum position error;

[0055] 2) Posture verification:

[0056] Using the fixed calibration points as posture error verification points;

[0057] Record the actual posture after each rotation contacts the position error verification point;

[0058] Calculate the theoretical posture according to the angle after each rotation;

[0059] The average posture error and the maximum posture error are calculated according to the theoretical posture and the actual posture, and it is determined whether the average posture error and the maximum posture error are within the allowable range. If so, the next step is executed; if not, the calibration is repeated in step S2.

[0060] Furthermore, the calibration parameters of the line laser camera include trigger mode, laser projection intensity, exposure time, acquisition mode, point cloud format and downsampling settings.

[0061] Furthermore, in step S3, the step of verifying the intrinsic parameter matrix of the line laser camera includes:

[0062] Select a line segment of known length or any line segment from the triangular pyramid calibration block as the test line segment L actual ;

[0063] Acquiring point cloud data of the inspection line segment, wherein the laser line and the inspection line segment completely overlap during the process of collecting the point cloud data;

[0064] Based on the point cloud data, extracting feature points using a feature point extraction method;

[0065] Fit the feature points and calculate the point cloud data length L of the verification line segment pcd , where the calculation expression is:

[0066]

[0067] Where, L pcd is the length of the point cloud data for verifying the line segment, (x1, y1, z1) and (x2, y2, z2) are the jump points of the point cloud data;

[0068] Calculate the measurement error between the length of the point cloud data and the length of the inspection line segment:

[0069] e ip =|L actual -L pcd |

[0070] In the formula, e ip is the measurement error, L actual To test the length of the line segment;

[0071] It is determined whether the measurement error caused by the internal parameter matrix is ​​within the allowable range. If so, the verification is passed and the next step is executed. If not, the verification fails and the internal parameter matrix is ​​readjusted.

[0072] Furthermore, in step S4, the calculation process of the hand-eye calibration transformation matrix includes:

[0073] Selecting the fixed calibration point as a calibration reference point;

[0074] Define relevant parameters and establish coordinate systems, including the robot base coordinate system O B -X B Y B Z B , end tool coordinate system O T -X T Y T Z T , Line laser camera coordinate system O L -X L Y L Z L , the camera coordinate system O of the line laser camera C -X C Y C Z C , the spatial position of the fixed calibration point P (xp, yp, zp), the image plane coordinates of the fixed calibration point in the camera (u, v), the coordinates of the fixed calibration point in the camera coordinate system (x C ,y C ,z C ), the coordinates of the fixed calibration point in the online laser camera coordinate system (xL ,y L ,z L ), the coordinates of the fixed calibration point in the robot base coordinate system (x B ,y B ,z B );

[0075] From camera theory we know that:

[0076]

[0077]

[0078] Where s is the scale factor, H is the internal parameter matrix of the camera, and T L C is the homogeneous transformation matrix between the line laser camera and the shooting camera coordinate system;

[0079] According to the line laser plane and the line laser camera coordinate system O L -X L Y L Z L X in L O L Z L The planes overlap to get y L =0, according to the relationship between 1) and 2), we have the following expression:

[0080]

[0081] The second transformation matrix that defines the line laser camera coordinate system relative to the end tool coordinate system is: The homogeneous transformation matrix of the end tool coordinate system relative to the robot base coordinate system is: And directly obtain the current pose based on the calibrated TCP point, and get the following relationship:

[0082]

[0083] In the formula, is the second transformation matrix of the line laser camera coordinate system relative to the robot end tool coordinate system, is the homogeneous transformation matrix of the robot end tool coordinate system relative to the robot base coordinate system;

[0084] Modify formula 4) to the following:

[0085]

[0086] In the formula, is the homogeneous transformation matrix of the end tool relative to the base when measuring with different postures for the i-th time, is the position coordinate of the fixed calibration point relative to the line laser camera coordinate system during the i-th measurement;

[0087] Collect 3D point cloud data of fixed calibration points camera ={(x i ,y i ,z i )|i=1,2,....,N}, and use formula 6) to extract the coordinates of the fixed calibration points;

[0088]

[0089] Where P fix The point cloud coordinates of the fixed calibration point in the online laser camera coordinate system;

[0090] According to the calibration method, equation 5) is listed as the calibration equation group:

[0091]

[0092] Formula 7) can be rearranged to obtain:

[0093]

[0094]

[0095] In the formula, Represents the rotation transformation matrix of the robot end tool relative to the robot base, represents the translation matrix of the robot end tool relative to the robot base, r1, r2, r3 represent the rotation transformation matrix of the hand-eye calibration matrix line laser camera coordinate system relative to the robot welding gun end expressed in the form of rotation components, Represents the translation matrix of the line laser camera relative to the robot end tool;

[0096] Modify equations 8) and 9) to:

[0097]

[0098] Set y L The value of is:

[0099]

[0100] Simplify equation 11):

[0101]

[0102] The singular value decomposition method is used to solve equation (12) to obtain the hand-eye calibration transformation matrix, where the solution process includes:

[0103]

[0104]

[0105] Where V is D T The eigenvector orthogonal matrix of D, U is DD T The eigenvectors of an orthogonal matrix are Σ, which is a diagonal matrix with non-negative singular values. The singular values ​​are denoted by σ1, σ2, ..., σ n ,Σ + is the pseudo-inverse of Σ, D is the hand-eye calibration transformation matrix.

[0106] Furthermore, in step S5, the least squares fitting error is used to perform calibration error check on the hand-eye calibration transformation matrix, and the expression of the least squares fitting error is:

[0107]

[0108] Where δ is the calibration error, which reflects the accuracy of the hand-eye calibration matrix, D is the hand-eye calibration transformation matrix, r1, r2, r3 represent the rotation transformation matrix of the hand-eye calibration matrix line laser camera coordinate system relative to the end of the robot welding gun expressed in the form of rotation components, Represents the translation matrix of the line laser camera relative to the robot end tool, Represents the translation matrix of the robot end tooling relative to the robot base.

[0109] Compared with the prior art, the present invention has the following beneficial effects:

[0110] (1) The triangular pyramid calibration block proposed in the present invention provides multiple clear edges and corner points, which are easy to identify and locate, and help the laser line quickly capture feature points to achieve high-precision feature extraction. By combining the triangular pyramid calibration block and the calibrated TCP points to calculate the hand-eye calibration transformation matrix, the calibration accuracy under complex working conditions can be improved to a certain extent, thereby improving the stability performance and precise operation of the robot under complex working conditions.

[0111] (2) The triangular pyramid calibration block proposed in the present invention has a simple design structure and can provide different calibration reference points in multiple directions, meet the calibration requirements under multiple postures, and reduce system errors. By selecting fixed points on the triangular pyramid calibration block, the repeatability and accuracy of the calibration process can be effectively improved, and the superposition of errors can be reduced.

[0112] (3) The present invention adds verification methods to each key step of calibration to effectively prevent the accumulation and superposition of errors. For example, the error verification of the TCP point ensures the accuracy of the TCP point by collecting fixed calibration points in multiple postures; the error verification of the line laser camera verifies the accuracy of the camera measurement by matching the laser point cloud data with the actual calibration points. In addition, the error verification of the hand-eye calibration matrix ensures the accuracy of the matrix solution through multiple measurements and least squares optimization calculations. These additional verification steps can effectively avoid potential error propagation in each link and ensure the high accuracy of the calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0114] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0115] The present embodiment provides a line laser camera hand-eye calibration method, which is applied to the end of a welding robot and is specifically designed for the needs of the robot in practical application scenarios such as weld tracking and weld three-dimensional point cloud stitching. In these applications, the key is to accurately convert the point cloud data collected by the line laser camera to the robot coordinate system, usually through a homogeneous transformation matrix to complete this conversion. Therefore, accurately solving the homogeneous transformation matrix between the line laser camera and the robot end during the calibration process is crucial to the overall performance and accuracy of the welding robot system. However, traditional calibration methods have certain shortcomings in speed and accuracy. In order to solve these problems, the method in this embodiment proposes to use a triangular pyramid calibration block for hand-eye calibration of the line laser camera. This method significantly improves the speed and accuracy of the calibration, thereby optimizing the overall performance of the welding robot system and ensuring its efficient application in complex industrial environments. The parameter specifications of the line laser camera selected in this embodiment are shown in Table 1:

[0116] Table 1 Main specifications of line laser cameras

[0117]

[0118]

[0119] like Figure 1 As shown, the method comprises the following steps:

[0120] Step 1): Select the appropriate welding gun and line laser camera models to ensure that the measurement range and accuracy of the line laser camera meet the process requirements, and also ensure that the size and weight of the welding gun match the robot load. Connect the welding gun through the flange at the end of the robot and install the connection fixture to the line laser camera in the form of eye-in-hand on the side of the flange. Adjust the appropriate position to ensure that the field of view of the line laser camera is not blocked by the welding gun or other structures, and ensure that the camera can accurately collect the required 3D information during the hand-eye calibration process or the welding process, so as to achieve accurate weld tracking and point cloud data acquisition.

[0121] Step 2): After determining the model and specifications of the welding gun and line laser camera according to step 1), design the appropriate triangular pyramid calibration block size. When designing a suitable triangular pyramid calibration block, first determine the size of the calibration block based on the measurement range of the line laser camera and the space limitation of the robot end. Too large a size may exceed the camera's field of view and affect the measurement accuracy, while too small a size may result in too few detectable feature points on the calibration block, affecting the accuracy of the calibration. At the same time, the material selection of the calibration block is also crucial. Black materials absorb the line laser, causing the laser to be unable to reflect back to the camera, and thus unable to capture valid point cloud data, so black materials must be avoided. On the contrary, the calibration block should be made of light-colored or highly reflective materials (such as 3D printing materials, 45 steel, and 6061 aluminum alloy, etc.) to ensure that the laser can be smoothly reflected and detected by the camera. However, overly reflective materials, such as polished metal or mirror materials, will also cause excessive reflection, making the capture effect of the laser point cloud unsatisfactory. Therefore, the calibration block material should avoid highly reflective surfaces to ensure that the camera can accurately obtain the point cloud data reflected by the laser and ensure the accuracy and stability of the calibration.

[0122] Select any point on the three edges of the designed triangular pyramid calibration block as a fixed calibration point and mark it. It is mainly used for subsequent hand-eye calibration measurement, hand-eye calibration matrix verification and TCP point verification.

[0123] Step 3): According to step 2), determine the triangular pyramid calibration block to calibrate the welding gun TCP point. The specific principle is to move the robot's end welding gun to any point in space as a reference point (such as a fixed calibration point), and measure it in four different postures to determine the tool center point (TCP). Keep the tool tip in contact with the reference point, and move the end tool to the reference point in different postures (that is, different angles and directions). After each movement, record the position data of the tool in the end coordinate system. And solve and calculate it using the least squares method. Here we define the fixed calibration point spatial position P (x p ,y p ,z p ), the homogeneous transformation matrix of the robot flange relative to the base is defined as The homogeneous transformation matrix of the robot welding gun end point relative to the robot flange is defined as Since the robot welding gun is rigidly connected to the flange, the transformation matrix of each measurement is a constant matrix. The transformation matrix of the flange relative to the base changes with the change of posture during each measurement. Here, the four different measurement matrices are recorded as The reference point remains unchanged relative to the robot base in four different positions, so the following relationship can be obtained:

[0124]

[0125] Here is the rotation transformation matrix of the welding gun end point relative to the flange position, is the translation transformation matrix of the welding gun end point relative to the flange position, is the rotation transformation matrix of the robot flange relative to the base posture, is the translation transformation matrix of the robot flange relative to the base.

[0126] By arranging and combining the above relationship equations, we can get the following relationship equation:

[0127]

[0128] The resulting set of equations is an overdetermined set of equations and does not have an exact solution. Therefore, singular value decomposition (SVD) can be used to solve the overdetermined set of equations. It can obtain an approximate solution by minimizing the residual. This method can not only solve the overdetermined set of equations, but also control the stability of the solution by adjusting the size of the singular value, which has a significant effect on systems with measurement noise. The translation matrix Q of the end point of the welding gun relative to the flange in the matrix equation is T E The specific solution process is as follows:

[0129]

[0130] Where V is The eigenvector orthogonal matrix of The eigenvectors of an orthogonal matrix are Σ, which is a diagonal matrix with non-negative singular values. The singular values ​​are denoted by σ1, σ2, ..., σ n . We can solve The eigenvalue of and taking the square root to get, Σ + is the pseudo-inverse of Σ, specifically the reciprocal of all non-zero singular values ​​in Σ

[0131]

[0132] After solving, you can To measure the size of the error value. Among them, δ is the calibration error, and the size of δ reflects the accuracy of the hand-eye calibration matrix. The larger δ is, the larger the hand-eye calibration error is, and vice versa.

[0133] After the position of the TCP point is calibrated and confirmed, the TCP point posture calibration is started. Based on the fourth calibration point of the previous position calibration, it is translated a certain distance in the X+ direction as the fifth calibration point. Based on the above four calibration points, it is translated a certain distance in the Z+ direction as the sixth calibration point. Because the postures of the three calibration points remain unchanged, the rotation matrix of the flange of these three points relative to the base does not change. The relationship between the fourth calibration point and the fifth calibration point is the vector of the tool coordinate system along the X+ direction, so the axial vector of the X-axis of the tool coordinate system can be obtained. Similarly, the relationship between the fourth and sixth calibration points can be deduced to obtain the axial vector of the Z axis. According to the right-hand rule, the Y-axis axial vector can be obtained as Y=Z×X

[0134] Step 4): Verify the calibrated TCP point according to step 3). Use a fixed calibration point as the error verification point of the TCP point. Touch the fixed calibration point with different postures (5-10 times), and record its position information and posture information (x, y, z, r x ,r y ,r z ). The average error and maximum error are solved. Assuming that the position information of each measurement point is (x i ,y i ,z i ), where i = 1, 2, ...., N represents the position information under different postures and N is the total number of measurements. First, the average value of all positions is calculated, which is defined here as The calculation method is as follows:

[0135]

[0136] Euclidean distance of position error Defined as the measurement position point (x i ,y i ,z i ) and the average point Distance between:

[0137]

[0138] It can be concluded that the average position error of the TCP calibration point is defined as The maximum error is defined as

[0139]

[0140] The end-point posture verification process is similar to the position verification process. Specifically, the fixed calibration point is used as a reference point to determine the known coordinates of the calibration point in the robot base coordinate system. Through the teach pendant or API interface, the robot end tool is moved to the fixed calibration point and the end point is kept in contact with the fixed calibration point. Under the premise of keeping the end point of the welding gun in contact with the fixed point, adjust the different postures (rotations) of the robot end tool, for example: rotate a certain angle around the X axis of the current end point, rotate a certain angle around the Y axis of the current end point, and rotate a certain angle around the Z axis of the current end point. After each rotation, record the current posture (rotation matrix or quaternion) of the robot end tool. According to the angle of each rotation, calculate the theoretical posture data under ideal conditions. Obtain the actual posture data of the end tool after each rotation through the robot controller or API interface. Compare the theoretical rotation matrix with the actual rotation matrix to calculate the posture error of each rotation. Using the difference in the rotation matrix (posture data), finally use a similar error measurement method as mentioned above to perform weighted average and define the maximum error. Determine whether the error parameter size meets the accuracy level of the current engineering application.

[0141] Step 5): According to step 4), after the TCP error is verified, if the error is within the allowable range in the actual application scenario, you can proceed to the next step, correctly connect the communication cable and power cable of the line laser camera, and make reasonable wiring to prevent the robot arm from being entangled and pulled. After completion, connect the line laser camera to select and debug its parameters. The parameter selection of the line laser camera is crucial to the accuracy of the calibration. Parameters such as camera resolution, projection intensity, and exposure time will directly affect the imaging effect of the laser line in the camera image. If the camera parameters are not set properly, the laser line may be blurred, distorted, or completely unable to be captured in some cases. The correct camera parameters should ensure that the laser line is clear and has sufficient contrast so that the algorithm can accurately extract the feature points of the laser line.

[0142] In the specific example, the calibration parameters of the selected line laser camera are shown in Table 2:

[0143] Table 2 Main calibration parameters of line laser camera

[0144]

[0145] Step 6): Check the intrinsic parameter matrix of the line laser camera. The intrinsic parameter matrix of the camera determines how the imaging system maps points in the three-dimensional world to the image plane. Its main parameters include focal length, principal point offset (optical center offset), and distortion coefficient. If there are errors in the intrinsic parameter matrix, especially in the laser line camera calibration process, it may cause serious deviations in the measurement results. Since the calibration is based on the image data captured by the camera, any intrinsic parameter error will directly affect the extraction accuracy of the feature points in the image. If the intrinsic parameter matrix is ​​not calibrated or the calibration error is large, the accumulated errors in the measurement process will gradually amplify. Once these errors are superimposed on the extraction of laser feature points and the calculation of camera pose, they will eventually cause the robot hand-eye calibration results to deviate from the true value, affecting the accuracy of the measurement. Therefore, reasonable error verification of the internal parameters can effectively prevent the accumulation of measurement errors and ensure the reliability and accuracy of the calibration results.

[0146] Select an edge or an arbitrary line segment L of known length from the triangular pyramid calibration block actual As a verification line segment, the actual length of the line segment is L actual The unit definition can be obtained through precise measurement, which is consistent with the distance unit of the point cloud data. actual The laser line and the calibration line segment should be completely overlapped in the field of view. At this time, the laser line and the calibration line segment should be displayed as a continuous line in the image. The laser line camera performs a single point cloud data acquisition on the calibration line segment to obtain the point cloud data of the calibration line segment. Save the point cloud file as a PCD format file and extract the collected point cloud data P cloud In the point cloud data, the verification line segment L is obtained by extracting feature points. pcd The corresponding feature point set. By fitting these feature points, the length L of the verification line segment in the point cloud data is calculated. pcd This process can be achieved by fitting a straight line using the least squares method. The calculation formula for the length of the point cloud data of the verification line segment is as follows:

[0147]

[0148] Here (x1, y1, z1) and (x2, y2, z2) are the point cloud coordinates at both ends of the verification line segment, which can be extracted through the characteristics of the line segment or can be understood as the jump point of the point cloud data. actual The length of the calibration line segment L obtained by fitting the collected point cloud data pcd Compare and calculate the measurement error e ip =|L actual -L pcd |. Determine whether the measurement error caused by the internal parameter matrix is ​​within the allowable range. If it is within the allowable range, the next step can be performed. If the measurement error caused by the internal parameter matrix is ​​too large, the internal parameter matrix needs to be readjusted.

[0149] Step 7): After the above steps are correct or meet certain requirements, the line laser camera hand-eye calibration calculation can be performed, and the fixed calibration point is selected as any point on the three edges of the triangular pyramid as the calibration reference point. For ease of understanding, define the relevant parameters and establish the corresponding coordinate system. The robot base coordinate system O B -X B Y B Z B ; Robot welding gun end tool coordinate system O T -X T Y T Z T ; Line laser camera coordinate system O L -X L Y L Z L ; Line laser camera captures camera coordinate system O C -X C Y C Z C .Π L Defined as the laser line plane and in the line laser camera coordinate system O L -X L Y L Z L X in L O L Z L The planes coincide and the spatial position of the fixed calibration point P(x p ,y p ,z p The coordinates of the fixed point in the image plane of the camera are (u, v) and the coordinates of the fixed point in the coordinate system of the camera are (x C ,y C ,z C ), the coordinates in the online laser camera coordinate system are (x L ,y L ,z L ), the coordinates in the robot base coordinate system are (x B ,y B ,z B ) From the camera imaging theory, we know that:

[0150]

[0151] Where: s is the scale factor, H is the internal parameter matrix of the shooting camera. The homogeneous transformation matrix between the line laser camera and the shooting camera coordinate system is defined as

[0152]

[0153] And because the line laser plane and the line laser camera coordinate system OL -X L Y L Z L X in L O L Z L The planes overlap to get y L =0. According to the above relationship, the following expression is obtained:

[0154]

[0155] The second transformation matrix that defines the line laser camera coordinate system relative to the robot welding gun end tool coordinate system is: The homogeneous transformation matrix of the robot welding gun end tool coordinate system relative to the robot base coordinate system is: Since the TCP point has been calibrated in step 3, the current position and posture can be read directly on the robot teaching pendant or using the API interface. From this, the following relationship can be obtained:

[0156]

[0157] The second transformation matrix of the centerline laser camera coordinate system relative to the robot welding gun end tool coordinate system is The hand-eye matrix that needs to be solved for hand-eye calibration. Since the line laser camera is rigidly connected to the welding robot, the hand-eye matrix does not change to a constant matrix regardless of the robot posture. Since the fixed point is fixed, the position relative to the robot coordinate system [x B y B z B 1] T There will be no change. Therefore, the above formula can be changed as follows:

[0158]

[0159] For ease of description, It is defined as the homogeneous transformation matrix of the robot welding gun end relative to the base each time a measurement is made, Defined as the position coordinates of the fixed calibration relative to the line laser camera coordinate system at each measurement. Adjust the robot so that the laser line of the line laser camera passes through the fixed calibration point. This calibration point is a sharp point on the edge of the triangular pyramid, and its geometric feature is a local protrusion relative to other parts. The 3D point cloud data of this point is extracted by the laser camera. Let the coordinates of this point cloud data in the camera coordinate system be P camera ={(x i ,y i ,z i)}, where i = 1, 2, ...., N represents the coordinates of each collected point cloud, and N is the total number of collected point clouds. The point cloud data is stored in the PCD file. This calibration point has an obvious feature in the 3D point cloud: the Z value is the smallest. This is because when the laser line passes through the calibration point, it is located at the lowest point in the laser irradiation area (the position closest to the laser camera in the Z direction). Let the collected point cloud data be P camera ={(x i ,y i ,z i )|i=1,2,....,N} The coordinates of the calibration points are extracted by the following formula:

[0160]

[0161] The purpose of extracting the coordinates of the calibration point here is to extract the identified feature points. Because a laser line is shot through the point cloud image of the fixed calibration point, the feature point has a significant feature that the z coordinate value is the smallest. Therefore, in order to extract the feature point, only the minimum coordinate of the Z axis is identified as the coordinate of the actual fixed calibration point acquired by the line laser camera. Only by obtaining the coordinates of the calibration point in the camera coordinate system can this feature point be converted to the end of the welding gun and then to the coordinates under the robot base through the homogeneous matrix. After conversion to the robot base coordinates, the relationship between the line laser camera coordinate system and the robot end coordinate system can be solved through multiple comparisons (because the fixed calibration point is fixed and the multiple measurement results can be combined to be equal).

[0162] Through the above calibration equations, the calibration equations are listed according to the calibration method:

[0163]

[0164] The summary is as follows:

[0165]

[0166]

[0167] Where i=1,2,....,N. Represents the rotation transformation matrix of the robot welding gun end relative to the robot base, Represents the translation matrix of the robot welding gun end relative to the robot base, r1, r2, r3 represent the hand-eye calibration matrix, and the rotation transformation matrix of the line laser camera coordinate system relative to the robot welding gun end is expressed in the form of rotation components. represents the translation matrix of the line laser camera relative to the end of the robot welding gun. It can be written as:

[0168]

[0169] Here, due to the linear laser plane and the linear laser camera coordinate system O L -X L Y L Z L X in L O L Z L The planes overlap to get y L =0 if y L = 0, the coefficients of the rotation component r2 are all 0, and the system of equations has no solution and cannot be solved. Therefore, it is set to a number close to 0 here. In the specific example, y L =0.25. The following conclusions can be drawn:

[0170]

[0171] Simplified to:

[0172]

[0173] The calibration equation set formed by calibrating it according to the above method is an overdetermined equation set. If the calibration times are less than four times, the equation set has infinite solutions. If the measurement times are greater than four times, the overdetermined equation set has solutions but no exact solutions. Singular value decomposition (SVD) can be used to solve it, which provides the optimal approximate solution by minimizing the residual error, and has the advantage of controlling the stability of the solution. It can effectively handle noisy systems, suppress the influence of noise by filtering out smaller singular values ​​or introducing regularization, avoid the instability caused by ill-conditioned matrices, and make the solution more robust and reliable. The solution of the homogeneous transformation matrix of the line laser camera coordinate system relative to the end point of the robot welding gun is as follows:

[0174]

[0175]

[0176] Where V is D T The eigenvector orthogonal matrix of D, U is DD T The eigenvectors of an orthogonal matrix are Σ, which is a diagonal matrix with non-negative singular values. The singular values ​​are denoted by σ1, σ2, ..., σ n . We can solve D T The eigenvalue of D is obtained by taking the square root, Σ + is the pseudo-inverse of Σ, specifically the reciprocal of all non-zero singular values ​​in Σ

[0177]

[0178] Step 8): According to the hand-eye calibration matrix obtained in step 7), that is, the homogeneous transformation matrix of the line laser camera coordinate system relative to the end point of the robot welding gun, the calibration error can be checked, that is, the least squares fitting error, and its expression is as follows:

[0179]

[0180] Where δ is the calibration error, and the size of δ reflects the accuracy of the hand-eye calibration matrix. The larger δ is, the larger the hand-eye calibration error is, and vice versa. After the hand-eye calibration process is completed, the hand-eye matrix can also be imported, and the robot can be moved to a fixed point in space, such as a test calibration point, to record its spatial position. The point can be identified by a line laser camera to extract the eigenvalue of the point, and the feature point can be transformed into the robot coordinate system in combination with the solved hand-eye calibration matrix. The robot can be driven to move to the point through the robot API interface to intuitively verify the accuracy of the hand-eye calibration matrix. If there is a large error, return to step 4) to re-calibrate the hand-eye.

[0181] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0182] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A line laser camera hand-eye calibration method, characterized in that: The following steps are involved: S1. Based on the measurement range of the line laser camera and the space limitation of the robot end, determine the triangular pyramid calibration block and select fixed calibration points, where the line laser camera and the end tool are set at the robot end; S2. Based on the triangular pyramid calibration block and the fixed calibration point, the TCP position calibration and the TCP posture calibration of the center point of the end tool are performed to obtain a calibrated TCP point; S3, selecting the calibration parameters of the line laser camera, and verifying the intrinsic parameter matrix of the line laser camera in combination with the triangular pyramid calibration block; S4, based on the verified calibration parameters and the calibrated TCP points, measuring the fixed calibration points multiple times in different postures, extracting eigenvalues ​​of the fixed calibration points, and then calculating the hand-eye calibration transformation matrix; S5. Perform calibration error check on the hand-eye calibration transformation matrix to determine whether it meets the accuracy requirement. If so, return to step S2 to recalibrate until the accuracy requirement is met. If not, do nothing.

2. A line laser camera hand-eye calibration method according to claim 1, characterized in that: The fixed calibration point is an arbitrarily selected point on the three edges of the triangular pyramid calibration block.

3. The hand-eye calibration method for a line laser camera according to claim 1, characterized in that: In step S2, the step of calibrating the TCP position includes: Move the end tool to any point in space as a reference point, wherein the selected reference point is a fixed calibration point; The position data of the end tool in the end coordinate system after the end tool moves to the reference point in different postures for four times is recorded, and the calibrated TCP position is obtained by solving and calculating, wherein the calculation process of the calibrated TCP position includes: Determine the relationship of the reference point to the robot base at different positions: In the formula, is the rotation transformation matrix of the end point of the end tool relative to the robot flange position, is the translation transformation matrix of the robot flange relative to the base position during the i-th posture movement, is the rotation transformation matrix of the robot flange relative to the base posture during the i-th posture movement, is the translation transformation matrix of the end point of the end tool relative to the flange position; Arrange and merge the relations to obtain the overdetermined system of equations: The singular value decomposition method is used to solve the overdetermined equations: Where V is The eigenvector orthogonal matrix of The eigenvectors of an orthogonal matrix are Σ, which is a diagonal matrix with non-negative singular values. The singular values ​​are denoted by σ1, σ2, ..., σ n ,Σ + is the pseudo-inverse of Σ, that is Using solution The eigenvalue of and square root are taken to obtain the calibrated TCP position.

4. The hand-eye calibration method for a line laser camera according to claim 3, characterized in that: It also includes measuring the error of the calibrated TCP position after the TCP position is calibrated, wherein the calculation expression of the error is: In the formula, δ is the calibration error, is the translation transformation matrix of the robot flange relative to the base position during the i-th posture movement, is the rotation transformation matrix of the robot flange relative to the base posture during the i-th posture movement, It is the translation transformation matrix of the end point of the end tool relative to the flange position.

5. The hand-eye calibration method for a line laser camera according to claim 3, characterized in that: In step S2, the steps of performing TCP posture calibration include: Determine a fifth calibration point and a sixth calibration point, wherein the four calibration points are determined according to the end tool moving to the reference point four times in different postures, the fifth calibration point is obtained by translating a certain distance in the X+ direction based on the fourth calibration point, and the sixth calibration point is obtained by translating a certain distance in the Z+ direction based on the fourth calibration point; Calculate the relationship between the fourth calibration point and the fifth calibration point: Where X is the axis vector of the X-axis of the tool coordinate system, is the translation transformation matrix of the robot flange relative to the base during the sixth posture movement, which serves as the fifth calibration point position; Calculate the relationship between the fourth calibration point and the sixth calibration point: Where Z is the axial vector of the Z axis of the tool coordinate system, is the translation transformation matrix of the robot flange relative to the base position during the sixth posture movement, which serves as the sixth calibration point position; According to the right-hand rule, the axial vector of the Y axis is obtained: Y=Z×X Where, Y is the axial vector of the Y axis of the tool coordinate system; Combine X, Y and Z to get the calibrated TCP posture.

6. The hand-eye calibration method for a line laser camera according to claim 1, characterized in that: The step of performing calibration error checking on the calibrated TCP point is also included, and the specific steps include: 1) Position verification: Using the fixed calibration point as a position error verification point; Record the position information when contacting the position error verification point with different postures, and calculate the average position error and the maximum position error, and judge whether the average position error and the maximum position error are within the allowable range. If so, execute the next step, if not, return to step S2 and recalibrate, wherein the process of calculating the average position error and the maximum position error includes: Calculate the average point of the position information In the formula, is the average value in the x, y, and z directions, i = 1, 2, ...., N represents the position information under different postures, and N is the total number of measurements; Calculate the position point (x i ,y i ,z i ) and the average point The distance between them, as the Euclidean space distance of the position error Said The distance calculation expression is: Based on the Euclidean space distance Calculate the average position error and maximum position error of the TCP calibration point: In the formula, is the average position error, e max is the maximum position error; 2) Posture verification: Using the fixed calibration points as posture error verification points; Record the actual posture after each rotation contacts the position error verification point; Calculate the theoretical posture according to the angle after each rotation; The average posture error and the maximum posture error are calculated according to the theoretical posture and the actual posture, and it is determined whether the average posture error and the maximum posture error are within the allowable range. If so, the next step is executed; if not, the calibration is repeated in step S2.

7. The hand-eye calibration method for a line laser camera according to claim 1, characterized in that: The calibration parameters of the line laser camera include trigger mode, laser projection intensity, exposure time, acquisition mode, point cloud format and downsampling settings.

8. The hand-eye calibration method for a line laser camera according to claim 1, characterized in that: In step S3, the step of verifying the intrinsic parameter matrix of the line laser camera includes: Select a line segment of known length or any line segment from the triangular pyramid calibration block as the test line segment L actual ; Acquiring point cloud data of the inspection line segment, wherein the laser line and the inspection line segment completely overlap during the process of collecting the point cloud data; Based on the point cloud data, extracting feature points using a feature point extraction method; Fit the feature points and calculate the point cloud data length L of the verification line segment pcd , where the calculation expression is: Where, L pcd is the length of the point cloud data for verifying the line segment, (x1, y1, z1) and (x2, y2, z2) are the jump points of the point cloud data; Calculate the measurement error between the length of the point cloud data and the length of the inspection line segment: And ip =|L actual -THE pcd | In the formula, e ip is the measurement error, L actual To test the length of the line segment; It is determined whether the measurement error caused by the internal parameter matrix is ​​within the allowable range. If so, the verification is passed and the next step is executed. If not, the verification fails and the internal parameter matrix is ​​readjusted.

9. The hand-eye calibration method for a line laser camera according to claim 1, characterized in that: In step S4, the calculation process of the hand-eye calibration transformation matrix includes: Selecting the fixed calibration point as a calibration reference point; Define relevant parameters and establish coordinate systems, including the robot base coordinate system O B -X B Y B Z B , end tool coordinate system O T -X T Y T Z T , Line laser camera coordinate system O L -X L Y L Z L , the camera coordinate system O of the line laser camera C -X C Y C Z C , fixed calibration point spatial position P(x p ,y p ,z p ), the coordinates of the fixed calibration point in the image plane of the camera (u, v), the coordinates of the fixed calibration point in the coordinate system of the camera (x C ,y C ,z C ), the coordinates of the fixed calibration point in the online laser camera coordinate system (x L ,y L ,z L ), the coordinates of the fixed calibration point in the robot base coordinate system (x B ,y B ,z B ); From camera theory we know that: Where s is the scale factor, H is the intrinsic parameter matrix of the shooting camera, is the homogeneous transformation matrix between the line laser camera and the shooting camera coordinate system; According to the line laser plane and the line laser camera coordinate system O L -X L Y L Z L X in L O L Z L The planes overlap to get y L =0, according to the relationship between 1) and 2), we have the following expression: The second transformation matrix that defines the line laser camera coordinate system relative to the end tool coordinate system is: The homogeneous transformation matrix of the end tool coordinate system relative to the robot base coordinate system is: And directly obtain the current pose based on the calibrated TCP point, and get the following relationship: In the formula, is the second transformation matrix of the line laser camera coordinate system relative to the robot end tool coordinate system, is the homogeneous transformation matrix of the robot end tool coordinate system relative to the robot base coordinate system; Modify formula 4) to the following: In the formula, is the homogeneous transformation matrix of the end tool relative to the base when measuring with different postures for the i-th time, is the position coordinate of the fixed calibration point relative to the line laser camera coordinate system during the i-th measurement; Collect 3D point cloud data of fixed calibration points camera ={(x i ,y i ,z i )|i=1,2,....,N}, and use formula 6) to extract the point cloud coordinates of the fixed calibration points; Where P fix The point cloud coordinates of the fixed calibration point in the online laser camera coordinate system; According to the calibration method, equation 5) is listed as the calibration equation group: Formula 7) can be rearranged to obtain: In the formula, Represents the rotation transformation matrix of the robot end tool relative to the robot base, represents the translation matrix of the robot end tool relative to the robot base, r1, r2, r3 represent the rotation transformation matrix of the hand-eye calibration matrix line laser camera coordinate system relative to the robot welding gun end expressed in the form of rotation components, Represents the translation matrix of the line laser camera relative to the robot end tool; Modify equations 8) and 9) to: Set y L The value of is: Simplify equation 11): The singular value decomposition method is used to solve equation (12) to obtain the hand-eye calibration transformation matrix, where the solution process includes: Where V is D T The eigenvector orthogonal matrix of D, U is DD T The eigenvectors of an orthogonal matrix are Σ, which is a diagonal matrix with non-negative singular values. The singular values ​​are denoted by σ1, σ2, ..., σ n ,Σ + is the pseudo-inverse of Σ, D is the hand-eye calibration transformation matrix.

10. The hand-eye calibration method for a line laser camera according to claim 1, characterized in that: In step S5, the least squares fitting error is used to perform calibration error check on the hand-eye calibration transformation matrix. The expression of the least squares fitting error is: Where δ is the calibration error, which reflects the accuracy of the hand-eye calibration matrix, D is the hand-eye calibration transformation matrix, r1, r2, r3 represent the rotation transformation matrix of the hand-eye calibration matrix line laser camera coordinate system relative to the end of the robot welding gun expressed in the form of rotation components, Represents the translation matrix of the line laser camera relative to the robot end tool, Represents the translation matrix of the robot end tooling relative to the robot base.

Images