Robot hand-eye calibration method based on spatial point method

Abstract

The invention discloses a robot hand-eye calibration method based on a spatial point method, and the method comprises the following steps: S1, fixing an observation calibration point O, and selecting four points # imgabs0 #, # imgabs1 #, # imgabs2 # and # imgabs3 #, which are not in the same plane, on a spherical surface with the calibration point O as the center of sphere and the radius R; s2, the mechanical arm is moved, the position of the center point of the tail end of the mechanical arm reaches # imgabs4, the angle of the tail end of the mechanical arm is adjusted, and the calibration point O is located in the middle of a depth camera picture on the mechanical arm; step S3, solving a calibration matrix # imgabs5 #; and S4, after solving of the calibration matrix # imgabs6 # is completed, validity of the matrix is verified. According to the method, the hand-eye calibration of the tail end of the mechanical arm with the eye on the hand constructed by the depth camera can be completed only by acquiring the data of the same calibration point at four different angles, the internal reference of the camera does not need to be concerned in the whole calibration process, the calibration process is greatly simplified, and the possibility of accumulated errors is reduced.

Description

Technical Field

[0001] The present invention relates to robot hand-eye calibration technology, and specifically to a robot hand-eye calibration method based on the spatial point method. Background Art

[0002] The existing hand-eye calibration checkerboard calibration method usually relies on the precise measurement of the calibration board, camera internal parameters, and a large amount of sampling data. There are multiple precisely positioned calibration points on the calibration board, and the robot sequentially touches these calibration points through its end tool (TCP). Each time of calibration, the positions of the calibration points need to be accurately measured in the camera coordinate system. To calculate the calibration matrix between the camera and the end-effector coordinate system of the robotic arm, the existing methods rely on the internal parameters of the camera (such as focal length, principal point, distortion coefficient, etc.), and these internal parameters are usually measured through special tools or calibration boards. The measurement of the camera internal parameters not only requires additional calibration steps, but also in practical applications, due to changes in camera equipment and different environmental lighting conditions, the accuracy of the internal parameters is often difficult to maintain stable, thus affecting the accuracy and stability of the entire calibration process.

[0003] Traditional hand-eye calibration methods usually adopt non-linear optimization algorithms to solve the coordinate calibration matrix. By continuously adjusting parameters to minimize the position and rotation errors, this process is usually complex and vulnerable to local optimal solutions, resulting in inaccurate calibration results. The computational complexity of non-linear optimization is relatively high, especially when multiple sampling points and image data are required, and the solution process may become slower.

[0004] To obtain an accurate calibration matrix, the existing technology also requires collecting a large amount of data at multiple different positions and in multiple directions, which increases the time cost of calibration, and the steps involved in this process are cumbersome and vulnerable to errors. Especially in complex environments, the stability of the calibration results is difficult to guarantee, increasing the complexity of operation and the maintenance cost of equipment.

[0005] Another limitation of the existing technology is that the calibration method usually has a strong dependence on specific robot platforms and vision devices. For example, certain robotic arm and camera configurations may require specific calibration boards or customized calibration steps. The generality of this method is poor and it cannot be widely applied to different robot systems and application scenarios.

[0006] These disadvantages of the existing technologies make the traditional hand-eye calibration methods have certain limitations in practical applications:

[0007] 1. Rely on precise camera internal parameters, the calibration process is complex and vulnerable to errors

[0008] Currently, the common "eye-in-hand" calibration methods usually require accurate camera internal parameters (such as focal length, principal point, distortion parameters, etc.) as a basis. Obtaining these internal parameters requires an additional calibration process. In practical applications, due to factors such as changes in camera lenses and ambient light, the stability and accuracy of camera internal parameters are often difficult to guarantee. This makes the calibration process in the existing technology cumbersome and vulnerable to errors, resulting in unstable calibration results and affecting the long-term performance of the robot system.

[0009] 2. Large amount of data collection, time-consuming and difficult-to-operate calibration process

[0010] In traditional hand-eye calibration methods, it is usually necessary to collect a large amount of calibration data at different angles and multiple positions, and perform complex optimization calculations. This not only places high requirements on the calibration environment (such as requiring a dedicated calibration board, multiple calibration board images, etc.), but also has high requirements for the technical level and experience of operators. In some dynamic application scenarios, the complex calibration process increases the deployment time and reduces the calibration efficiency. Especially in robot applications with rapid deployment or frequent repositioning, the cumbersome calibration and long waiting time may lead to a decrease in work efficiency.

[0011] 3. Computational complexity and local optimum problems caused by non-linear optimization

[0012] Traditional hand-eye calibration methods often rely on non-linear optimization techniques to solve the transformation matrix by continuously adjusting parameters and minimizing the error function. However, this method is prone to falling into local optimum solutions, resulting in poor stability and accuracy of the calibration results. Summary of the Invention

[0013] The object of the present invention is to provide a robot hand-eye calibration method based on the spatial point method for the deficiencies existing in the prior art. This method only needs to obtain the data of the same calibration point at four different angles to complete the hand-eye calibration of the end of the robotic arm constructed by the depth camera with the eye-in-hand. During the entire calibration process, there is no need to pay attention to the internal parameters of the camera, which greatly simplifies the calibration process and reduces the possibility of cumulative errors.

[0014] The technical solution to achieve the object of the present invention is as follows:

[0015] A robot hand-eye calibration method based on the spatial point method includes the following steps:

[0016] Step S1, select an object with a rough surface and a length, width, and height not exceeding 1 cm. In the working range space of the robotic arm of the robot, manually fix an observed calibration point O, and select 4 points that are not in the same plane on the spherical surface with the calibration point O as the center of the sphere and a radius of R , , , To read the attitude data of the TCP coordinates at the end of the robotic arm;

[0017] Step S2, move the robotic arm so that the position of the center point at the end of the robotic arm reaches , adjust the angle of the end of the robotic arm so that the calibration point O is at the middle position of the depth camera screen on the robotic arm, and record the displacement coordinates of the TCP coordinates at the end of the robotic arm at the corresponding position , as well as the three-dimensional coordinates of the calibration point in the depth camera coordinate system The coordinates are in the coordinates of the robotic arm base. At this time, let the end of the robotic arm only make translational movements so that the center point at the end of the robotic arm reaches the calibration point O, and record the displacement data part of the TCP coordinates at the end of the robotic arm at this time , repeat the action position , , , to obtain a set of matrix data, , , , where is to move the robotic arm so that when the end of the robotic arm reaches S1, S2, S3, S4, the 4X3 matrix composed of the displacement coordinates of the TCP coordinates at these 4 positions; is the 4X3 matrix composed of the coordinates (x, y, z) of the corresponding calibration point in the depth camera; is the 4X3 matrix composed of the data (x, y, z) of the displacement of the TCP coordinates at the end of the robotic arm when only the translational movement of the robotic arm is allowed to make the center point at the end of the robotic arm reach the calibration point O for these 4 points;

[0018] Step S3, calculate the 4x4 transformation matrix from the depth camera coordinate system to the end coordinate system of the robotic arm, that is, obtain the calibrated matrix after solution : First, obtain the coordinates of the calibration point O in the TCP coordinate system at the end of the robotic arm when the end of the robotic arm is at the positions , , , ; , then The augmented matrix of is:

[0019]

[0020] The augmented matrix of C is :

[0021]

[0022] Let the calibrated matrix to be solved be :

[0023]

[0024] Among them, , , represent three representations of the X-axis of the end-effector coordinate system of the robotic arm in the depth camera coordinate system C; , , respectively represent three representations of the Y-axis of the end-effector coordinate system of the robotic arm in the depth camera coordinate system C; , , respectively represent three representations of the Z-axis of the end-effector coordinate system of the robotic arm in the depth camera coordinate system C; then there is , and the following system of equations is obtained:

[0025] System of equations 1:

[0026]

[0027] System of equations 2:

[0028]

[0029] System of equations 3:

[0030]

[0031] The system of equations 1, the system of equations 2, and the system of equations 3 are all systems of four linear equations with four unknowns. Therefore, the problem of solving the matrix becomes the problem of solving three systems of four linear equations with four unknowns;

[0032] Step S4, after completing the solution of the calibration matrix , to verify the effectiveness of this matrix, a set of verifications are carried out: First, re-select multiple observation points at different spatial positions on the sphere with the calibration point O as the center and radius R in step S1 as verification points. Move the end-effector of the robotic arm to the vicinity of these verification points in turn, and by adjusting the pose of the end-effector of the robotic arm, make the calibration point located at the center of the depth camera image, so as to record the corresponding pose of the end-effector of the robotic arm and the three-dimensional coordinates of the calibration point in the depth camera coordinate system ; Subsequently, let the end-effector of the robotic arm only perform a translation operation to accurately align the center point of the end-effector with the calibration point without changing the end-effector pose, and record the pose of the end-effector of the robotic arm at this time as the true reference pose; Substitute the three-dimensional coordinates of the verification point into the calibration matrix that has been solved in step 3, that is, the theoretical position of the verification point in the end-effector coordinate system of the robotic arm is obtained ; Then compare the theoretical position with the actually measured pose Calculate the Euclidean distance error, remove the maximum and minimum values of the error, and calculate the arithmetic mean of the remaining values to obtain the distance error d. If the errors of all verification points are less than the preset threshold D, it means that the calibration matrix can be used for actual tasks; otherwise, re-execute steps S1 to S4 and iterate and optimize until the error meets the accuracy requirements.

[0033] The technical key points of this technical solution:

[0034] 1. Complete the "eye-in-hand" calibration by observing the same calibration point from 4 different angles:

[0035] This technical solution proposes a simplified hand-eye calibration method. By selecting 4 positions on the sphere with a fixed calibration point as the center and a radius of R that are not in the same plane, this calibration point is observed. This method effectively avoids the complex requirements for multiple calibration plates or multiple image acquisitions in traditional hand-eye calibration. By adjusting the end of the robotic arm to reach these positions and obtaining the corresponding calibration data, this technical solution realizes fast and simple calibration. The core of this technology lies in the selected calibration point configuration, which ensures the simplicity and accuracy of the calibration process, enabling the calibration process to be completed in a relatively short time and meeting the requirements for efficient deployment in practical applications.

[0036] 2. Without relying on camera internal parameters, directly calculate the calibration matrix depending on the pose difference:

[0037] Different from traditional hand-eye calibration methods, this technical solution does not rely on the precise measurement of camera internal parameters. Instead, through the position differences of the calibration point in the camera coordinate system and the end coordinate system at different positions of the end of the robotic arm, a linear mapping relationship for directly calculating the transformation matrix is constructed. This method omits the measurement and calibration of camera internal parameters (such as focal length, distortion, etc.), greatly simplifying the calibration process. By calculating the position differences, the coordinate transformation matrix between the camera and the end of the robotic arm can be accurately obtained, reducing the impact of inaccurate or changing internal parameters on the calibration results and improving the practicality and accuracy.

[0038] The beneficial effects of this technical solution:

[0039] 1. Avoid relying on camera internal parameters and simplify the calibration process;

[0040] Based on the relative poses of the calibration point in the geometric space under different observation perspectives, this technical solution directly solves the external parameter calibration matrix by constructing a linear mapping relationship between the end coordinate system of the robotic arm and the camera coordinate system . During the whole process, there is no need to obtain or rely on camera internal parameters, avoiding the interference of camera model errors on the calibration accuracy, greatly simplifying the operation process, and improving the convenience and stability of practical applications.

[0041] 2. Only a small number of poses are required to complete the calibration, with low data acquisition cost and simple implementation.

[0042] Different from the traditional method that requires multiple poses and multiple calibration board images for multiple fitting optimizations, this technical solution can estimate the mapping matrix between the end and the camera coordinate system only through the observation data at 4 different spatial positions. These 4 sets of data are from 4 non-coplanar positions on the spherical surface constructed with a fixed point as the center, ensuring the uniqueness and stability of geometric calculation.

[0043] By simplifying the layout of calibration points and the pose acquisition process, the data acquisition and calibration time are greatly saved, which is especially suitable for scenarios of rapid deployment and repeated use, such as industrial assembly, agricultural picking, mobile service robots, etc.

[0044] 3. Adopt the method of solving linear equations to avoid the local optimum problem introduced by non-linear optimization.

[0045] This technical solution constructs an augmented matrix of calibration points in the camera coordinate system and the end coordinate system, and derives three groups of linear equations with four unknowns, and then uses standard linear algebra methods such as the least squares method to solve the mapping matrix. This process does not require iterative solution, has no initial value dependence, and does not involve non-linear optimization models, avoiding common problems in traditional calibration such as getting stuck in local optimum and poor convergence.

[0046] Therefore, this technical solution has stronger numerical stability and convergence reliability in theory, and also shows faster operation speed and higher result repeatability in practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the structural diagram of the robotic arm of the robot in the embodiment;

[0048] Figure 2 It is for the calibration points and 、 、 、 in the embodiment;

[0049] Figure 3 It is the flowchart of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0050] The following further elaborates on the content of the present invention in conjunction with the drawings and embodiments, but does not limit the present invention.

[0051] Embodiment:

[0052] Referring to Figure 3 , the robot hand-eye calibration method based on the spatial point method includes the following steps:

[0053] Step S1, select an object with a rough surface and dimensions of no more than 1 cm in length, width, and height. Within the working range space of the robot's robotic arm, as shown in Figure 1 , manually fix an observation calibration point O, and select 4 points that are not in the same plane on the spherical surface with the calibration point O as the center of the sphere and a radius of R. , , , , as shown in Figure 2 , to read the pose data of the TCP coordinates at the end of the robotic arm;

[0054] Step S2, move the robotic arm so that the position of the center point at the end of the robotic arm reaches . Adjust the angle at the end of the robotic arm so that the calibration point O is in the middle position of the depth camera image of the robotic arm, and record the displacement coordinates of the TCP coordinates at the end of the robotic arm at the corresponding position , as well as the three-dimensional coordinates of the calibration point in the depth camera coordinate system The coordinates are in the coordinates of the robotic arm base. At this time, only let the end of the robotic arm make translational movements so that the center point at the end of the robotic arm reaches the calibration point O, and record the displacement data part of the TCP coordinates at the end of the robotic arm at this time . Repeat the action positions , , to obtain a set of matrix data, , , ; where is the 4x3 matrix composed of the displacement coordinates of the TCP coordinates at the 4 positions when moving the robotic arm so that the end of the robotic arm reaches S1, S2, S3, and S4; is the 4x3 matrix composed of the coordinates (x, y, z) of the corresponding calibration point in the depth camera; is the 4x3 matrix composed of the data (x, y, z) of the displacement of the TCP coordinates at the end of the robotic arm when only making translational movements of the robotic arm to make the center point at the end of the robotic arm reach the calibration point O for these 4 points;

[0055] Step S3, calculate the 4x4 transformation matrix from the depth camera coordinate system to the end coordinate system of the robotic arm, that is, obtain the calibrated matrix : First, obtain the coordinates of the calibration point O in the TCP coordinate system at the end of the robotic arm when the end of the robotic arm is at the positions , , , . Then , the augmented matrix of is:

[0056]

[0057] The augmented matrix of C is :

[0058]

[0059] Let the calibration matrix to be solved be :

[0060]

[0061] where , , represent the three representations of the X-axis of the end effector coordinate system in the depth camera coordinate system C; , , respectively represent the three representations of the Y-axis of the end effector coordinate system in the depth camera coordinate system C; , , respectively represent the three representations of the Z-axis of the end effector coordinate system in the depth camera coordinate system C; then there is , and the following system of equations is obtained:

[0062] System of equations 1:

[0063]

[0064] System of equations 2:

[0065]

[0066] System of equations 3:

[0067]

[0068] The system of equations 1, the system of equations 2, and the system of equations 3 are all systems of four linear equations with one variable. Therefore, the problem of solving the matrix becomes the problem of solving three systems of four linear equations with one variable;

[0069] Step S4, after completing the solution of the calibration matrix , to verify the effectiveness of this matrix, a set of verifications are carried out: First, reselect multiple observation points at different spatial positions on the sphere with the calibration point O as the center and radius R in step S1 as verification points. Move the end effector of the robotic arm to the vicinity of these verification points in turn, and by adjusting the pose of the end effector of the robotic arm, make the calibration point located at the center of the depth camera image, so as to record the corresponding pose of the end effector of the robotic arm and the three-dimensional coordinates of the calibration point in the depth camera coordinate system ; Subsequently, without changing the attitude of the end effector, only perform translational operations to accurately align the center point of the end effector of the robotic arm with the calibration point, and record the pose of the end effector of the robotic arm at this time as the true reference pose; Substitute the three-dimensional coordinates of the verification point into the calibration matrix that has been solved in Step 3 , and the theoretical position of the verification point in the coordinate system of the end effector of the robotic arm can be obtained ; Then calculate the Euclidean distance error between the theoretical position and the actually measured pose , remove the maximum and minimum values of the error, and calculate the arithmetic mean of the remaining values to obtain the distance error d. If the errors of all verification points are less than the preset threshold D, it means that the calibration matrix can be used for actual tasks; Otherwise, re-execute Step S1 to Step S4, and iterate and optimize until the error meets the accuracy requirements

[0070] In this example, in order to verify the advantages of the proposed "spatial point method" calibration method in terms of accuracy and efficiency compared with the traditional "checkerboard calibration plate method", a set of detailed comparative experiments was designed. The experimental process uniformly uses the combination system of the UR robotic arm and the depth camera as the platform, and uses the two calibration methods respectively to complete the calibration of the coordinate transformation matrix between the end effector of the robotic arm and the depth camera. Subsequently, a unified verification process is used to evaluate the accuracy of the calibration matrices obtained by the two methods, and the time consumption of the entire calibration process is recorded

[0071] First, in the spatial point method experiment, select an object with a rough surface and dimensions not exceeding 1 cm in length, width, and height. Select a visual calibration point O within the working range space of the robotic arm of the robot. On the sphere with this point as the center and a given radius R, select 4 non-coplanar spatial points , , , . The end effector of the robotic arm is sequentially moved to these points, and through the rotation adjustment of the end effector of the robotic arm, the calibration point O is always kept at the center of the image captured by the depth camera in real time, and the position of the end effector of the robotic arm and the position of the calibration point in the visual coordinate system are recorded at this time. Subsequently, accurately align the end effector of the robotic arm with the calibration point through translational operations, and record the position of the end effector of the robotic arm. Solve the calibration matrix between the end effector of the robotic arm and the depth camera through these data. After completing the calibration matrix , select 5 new spatial points as verification points, repeat the aforementioned acquisition and alignment processes, and substitute the verification points in the visual coordinate system into the transformation matrix to obtain the predicted end position, and compare the error with the true end alignment position to evaluate the calibration accuracy. The entire process does not depend on the collection and feature extraction of checkerboard images, has fewer data points, and takes less time

[0072] Subsequently, the checkerboard calibration plate method experiment was carried out. Prepare a standard nine-square checkerboard calibration plate. According to the traditional hand-eye calibration process, collect more than 10 checkerboard images at different poses. After each shot, perform sub-pixel corner extraction and image-world coordinate matching steps, and combine the end pose of the robotic arm to calculate the hand-eye transformation matrix by the least squares method. During the whole process, it is necessary to manually adjust the position and angle of the calibration plate repeatedly to ensure uniform image quality and pose distribution and sufficient data collection. After calibration, the error is also calculated using the aforementioned 5 verification points, and the process is the same as that of the space point method.

[0073] Through this comparative experiment, under the same robotic arm platform and the same verification benchmark, the time efficiency and accuracy of the two methods are fairly evaluated. In the experiment, it is also ensured that the verification points are distributed in different regions of the working space to ensure the universality and objectivity of the evaluation results. The final comparison data are summarized in Table 1, which further clearly shows the performance advantages of the space point method in practical engineering applications.

[0074] Table 1 。

[0076] The comprehensive comparison results show that the space point method is superior to the traditional calibration plate method in terms of calibration efficiency, accuracy and stability.

Claims

1. A robot hand-eye calibration method based on the spatial point method, characterized in that Including the following steps: Step S1, select an object with a rough surface and dimensions of no more than 1 cm in length, width, and height. Fix an observation calibration point O manually within the working range space of the robot's robotic arm, and select 4 points that are not in the same plane on the spherical surface with the calibration point O as the center of the sphere and a radius of R to read the pose data of the TCP coordinates at the end of the robotic arm. , , , to read the pose data of the TCP coordinates at the end of the robotic arm; Step S2, move the robotic arm so that the position of the center point at the end of the robotic arm reaches , adjust the angle of the end of the robotic arm so that the calibration point O is in the middle position of the depth camera image on the robotic arm, and record the displacement coordinates of the TCP coordinates of the end of the robotic arm at the corresponding position , as well as the three-dimensional coordinates of the calibration point in the depth camera coordinate system The coordinates are the coordinates in the base coordinate system of the robotic arm; Step S3, calculate the 4x4 transformation matrix from the depth camera coordinate system to the end-effector coordinate system of the robotic arm, i.e., obtain the calibrated matrix after solution ; Step S4, complete the solution of the calibration matrix After the solution of is obtained, to verify the effectiveness of this matrix, a set of verifications are carried out: First, reselect multiple observation points at different spatial positions on the spherical surface with the calibration point O as the center of the sphere and radius R in step S1 as verification points, and move the end of the robotic arm to the vicinity of these verification points in turn. By adjusting the pose of the end of the robotic arm, make the calibration point located at the center of the depth camera image, and thus record the corresponding pose of the end of the robotic arm and the three-dimensional coordinates of the calibration point in the depth camera coordinate system ; Subsequently, without changing the end pose of the robotic arm, only perform a translation operation to make the center point of the end of the robotic arm accurately align with the calibration point, and record the pose of the end of the robotic arm at this time as the true reference pose; Substitute the three-dimensional coordinates of the verification point into the calibration matrix that has been solved in step 3 , that is, the theoretical position of the verification point in the end coordinate system of the robotic arm can be obtained ; Then calculate the Euclidean distance error between the theoretical position and the actually measured pose . Remove the maximum and minimum values of the error, and calculate the arithmetic mean of the remaining values to obtain the distance error d. If the errors of all verification points are less than the preset threshold D, it means that the calibration matrix can be used for actual tasks; otherwise, re-execute steps S1 to S4, and iterate and optimize until the error meets the accuracy requirements.

2. The robot hand-eye calibration method based on the spatial point method according to claim 1, characterized in that In step S2, record the displacement coordinates of the TCP at the end of the robotic arm at the corresponding position , as well as the three-dimensional coordinates of the calibration point in the depth camera coordinate system . At this time, only let the end of the robotic arm perform translational movement so that the center point of the end of the robotic arm reaches the calibration point O, and record the displacement data part of the TCP coordinate at the end of the robotic arm at this time . Repeat the action position , , . Obtain a set of matrix data , , . Among them is the 4X3 matrix composed of the TCP coordinate displacement coordinates at the end of the robotic arm at these 4 positions when moving the robotic arm to make the end of the robotic arm reach S1, S2, S3, and S4; is the 4X3 matrix composed of the coordinates (x, y, z) of the corresponding calibration points in the depth camera; is the 4X3 matrix composed of the data (x, y, z) of the TCP coordinate displacement at the end of the robotic arm when only translating the robotic arm to make the center point of the end of the robotic arm reach the calibration point O for these 4 points.

3. The robot hand-eye calibration method based on the spatial point method according to claim 1, wherein, In step S3, calculate the 4x4 transformation matrix from the depth camera coordinate system to the end-effector coordinate system of the robotic arm, i.e., obtain the calibrated matrix after solution. The specific process is as follows: First, obtain the coordinates of the end point of the robotic arm at positions , , , of the calibration point O in the TCP coordinate system of the robotic arm end. If , then the augmented matrix of is as follows: ； The augmented matrix of C is : ； Let the calibration matrix to be solved be :[[-END]] ； Among them, , , represent three representations of the X-axis of the end-effector coordinate system in the depth camera coordinate system C; , , respectively represent three representations of the Y-axis of the end-effector coordinate system in the depth camera coordinate system C; , , respectively represent three representations of the Z-axis of the end-effector coordinate system in the depth camera coordinate system C; then there is , and the following system of equations is obtained: Equation set 1: ； Equation set 2: ； Equation set 3: ； Equation sets 1, 2, and 3 are all four-variable linear equation sets. Therefore, the problem of solving the matrix becomes the problem of solving three four-variable linear equation sets.

Images