Eye-in-hand rapid hand-eye calibration method for accurate clamping at tail end of robot

Abstract

The invention belongs to the technical field of robots, and discloses a quick'eye-on-hand 'hand-eye calibration method for accurate clamping at the tail end of a robot, which is used for independently calibrating two relative relations of rotation and translation of a coordinate system in order to reduce the solving difficulty of a nonlinear optimization problem. The method comprises the following steps: firstly, obtaining a CCS rotation matrix which is the same as an ECS through a three-step rotation calibration process; a translation vector between the CCS and the ECS is then calculated using the coordinate transformation. And finally, the relative relation between the center of a clamping jaw at the tail end of the robot and the binocular camera is calibrated, and real-time, in-place and accurate clamping of visual guidance is achieved. According to the method, high-precision positioning of the tail end of the robot for accurate clamping can be achieved, after calibration is completed, the center of a clamping jaw at the tail end of the robot can be accurately guided to any space target point in real time through a binocular camera, and extra LT guiding is not needed.

Description

Technical Field

[0001] The present invention belongs to the field of robot technology, and in particular relates to a hand-eye calibration technology, and specifically is an "eye-in-hand" rapid hand-eye calibration method for accurate gripping at the end of a robot. Background Art

[0002] In the field of aerospace, the transfer robot has been widely used in the intelligent manufacturing and automatic measurement process of large components. The accurate positioning of the end of the transfer robot is the key to completing tasks such as processing, grasping, and measurement. However, due to the inherent low absolute positioning accuracy of the robot (1mm), additional positioning systems such as laser tracking systems and visual measurement systems are required to assist in end positioning. "Eye-in-hand" and "Eye-to-hand" are two typical visual positioning system architectures. "Eye-to-hand" means that the visual sensor is installed at a fixed position outside the robot arm (usually above or on the side of the workspace), and the camera observes the entire workspace from a fixed perspective; "Eye-to-hand" means that the visual sensor is installed on the end effector of the robot, and moves with the movement of the robot arm. The camera can directly observe the work object and environment, provide real-time visual feedback consistent with the hand movement, and has better dynamic tracking capabilities, more accurate positioning effects, and a larger working range.

[0003] Hand-eye calibration is the key to the precise positioning of the vision system. Through hand-eye calibration, the relative position relationship between the camera coordinate system (CCS) and the robot end coordinate system (ECS) can be determined. After calibration, the target position obtained by the vision system can be converted to the robot coordinate system to facilitate the control of the robot's posture. However, existing calibration methods for "eye-in-hand" vision systems generally require special calibration equipment (calibration balls, etc.) or complex calibration operations (multiple sets of complex robot arm postures), with average calibration accuracy. At the same time, it is time-consuming, labor-intensive, and difficult to be widely used in industrial sites.

[0004] "A hand-eye calibration method and device" (patent number CN202211548085.8) uses a target image instead of a calibration block, reducing the error caused by the calibration block, and solves the target rigid body transformation relationship through a three-dimensional point set matching algorithm, which can effectively reduce the time consumption compared to the point cloud registration algorithm; in the linear solution of the hand-eye standard equation, the rotation axis method is used to solve the rotation matrix, which reduces the difficulty of solving the algorithm and effectively avoids the solution anomalies and failures caused by the matrix representation form such as trigonometric functions or rotation angles. While ensuring the calibration accuracy, the calibration process is simplified and easy to maintain, solving the problems of insufficient accuracy and inconvenient operation of the current hand-eye calibration method, but it still cannot solve the problem of displacement of the robotic arm on a moving automated guided vehicle (AGV).

[0005] In the paper "A robotic spindle end high-accuracy positioning method based on eye-in-hand vision active correction" published in 2023, B. Yu and other scholars further improved the "eye-in-hand" calibration method based on the laser tracker (LT), realized hand-eye calibration without the common field of view constraint of binocular cameras, and achieved high-precision robot end positioning through vision-based active correction. However, this method requires multiple sets of feature images and multiple pairs of complex robot motions to solve coordinate system transformations, which is time-consuming, complex, and unfriendly to workers.

[0006] Therefore, the present invention proposes a "eye in hand" fast hand-eye calibration method for accurate gripping of the robot end. By controlling the axial movement of the robot end for rotation calibration and using LT for coordinate transformation to complete translation calibration, the relationship between CCS and ECS is quickly, accurately and efficiently calibrated. Summary of the invention

[0007] The technical problem to be solved by the present invention is that in the existing "eye on hand" hand-eye calibration process, special calibration objects, multiple sets of robot movements and multiple sets of images captured during the movement are usually required to solve nonlinear optimization problems, resulting in complex calibration operations, large amounts of data to be solved, slow calculations and other problems. A fast calibration method for accurate robot grasping is proposed. The method uses a laser tracker and common feature points, and according to the basic definition of coordinate system transformation, three controlled end motions are used to calibrate the rotation matrix; then, through coordinate transformation, a single image is used to calibrate the translation vector, and high-precision calibration of the relative position between the binocular camera and the robot end coordinate system is achieved.

[0008] The technical solution of the present invention:

[0009] A "eyes in the hand" fast hand-eye calibration method for accurate gripping at the end of a robot. To reduce the difficulty of solving nonlinear optimization problems, the two relative relationships of coordinate system rotation and translation are calibrated independently. First, the CCS rotation matrix identical to the ECS is obtained through a three-step rotation calibration process. Then, the translation vector between the CCS and ECS is calculated using coordinate transformation. Finally, the relative relationship between the center of the robot's end gripper and the binocular camera is calibrated to achieve real-time, in-situ, and accurate gripping guided by vision. The specific steps are as follows: 1. A "eyes in the hand" fast hand-eye calibration method for accurate gripping at the end of a robot, characterized in that the steps are as follows:

[0010] Step 1: Build a hand-eye calibration platform with “eyes on hands”;

[0011] Build a hand-eye calibration platform, including a laser tracker 1, a target ball 2, an electric gripper 3, a binocular camera 4, a robotic arm 5, and an AGV trolley 6; the electric gripper 3 and the binocular camera 4 are fixedly installed at the end of the robotic arm 5 through a fixture, and the robotic arm 5 is fixed on the AGV trolley 6, forming a visual inspection system with "eyes on hands";

[0012] The second step is the rotation matrix calibration based on the robot's directional displacement;

[0013] According to kinematics theory, the rotation matrix between the camera coordinate system and the robot end coordinate system is defined as follows:

[0014]

[0015] in, and are the unit vectors of the camera coordinate system {C} and the robot end coordinate system {E} respectively;

[0016] Assume that the origin of the camera coordinate system {C} and the robot end coordinate system {E} coincide, and the unit vectors of each axis of the robot end coordinate system {E} have fixed angles α1, β1, and γ1 with the unit vector of the camera coordinate system {C} respectively; let a point P be along Direction of a displacement along a straight line E Δx, the motion is decomposed into [ C Δx1, C Δy1, C Δz1]; According to the geometric relationship and formula (1), the angles [α1, β1, γ1] are calculated by projection between unit vectors, that is, [ C Δx1 / E Δx, C Δy1 / E Δx, C Δz1 / EΔx]; Similarly, the angle relationship between the three axes of the camera coordinate system {C} and the robot end coordinate system {E} is determined by the controlled motion along the axis of the robot end coordinate system {E}, so the rotation matrix is ​​calculated by the following formula:

[0017]

[0018] in,[ C Δx2, C Δy2, C Δz2] and [ C Δx3, C Δy3, C Δz3] are E Δy and E The motion decomposition of the controlled motion of Δz in the camera coordinate system {C};

[0019] The third step is the calibration of translation vector based on coordinate transformation;

[0020] Through the rotation matrix calibration, the rotated camera coordinate system {CR} is obtained, which has the same axis direction as the robot end coordinate system {E}, but the two have different origins, so the translation vector between the coordinates needs to be calibrated. This process is completed by shooting the common point and combining the coordinate transformation: The binocular camera 4 shoots the common point and generates the point coordinates in the camera coordinate system C P, after the rotation matrix calibration, obtain the point coordinates P1 in the camera coordinate system {CR}; C P through the transformation matrix Convert to the coordinate system {L} of laser tracker 1, and then use the transformation matrix Convert to the robot base coordinate system {R}, and finally use the transformation matrix Convert to the robot end coordinate system {E}, and obtain the coordinate P2 in the robot end coordinate system {E}; finally, the translation vector between the robot end coordinate system {E} and the rotated camera coordinate system {CR} It is calculated by the difference between the two, namely:

[0021]

[0022] in, is the rotation matrix obtained in the second step using formula (1), Obtained through the DH table of the robot arm, and The coordinate values ​​of the same set of fixed common point target balls are measured by binocular camera 4 and laser tracker 1, and then the transformation matrix least squares fitting method based on SVD solution is used to obtain:

[0023]

[0024] Where i is the number of public point targets, p i is the coordinate of the ith common point measured by the binocular camera 4, q i The coordinates of the i-th point measured by LT, are the rotation matrix and translation vector of the camera coordinate system and the laser tracker coordinate system respectively;

[0025]

[0026] in, The rotation matrix and translation vector of the laser tracker coordinate system and the base coordinate system are obtained by formula (5). Therefore, (2), (3), (4) and (5) are used to obtain the transformation matrix that transforms the camera coordinate system {C} into the robot end coordinate system {E}

[0027] The fourth step is to determine the end position of the robot arm that can accurately grasp the target object;

[0028] During the robot's movement and gripping process, the relative position between the center of the electric gripper 3 and the binocular camera 4 is always constant. To achieve accurate gripping, the relationship needs to be calibrated and measured first. First, a target ball 2 is placed at the center of the electric gripper 3, and the target ball 2 is measured with LT to obtain the coordinates of the center of the electric gripper 3 in the laser tracker coordinate system. Then, the coordinates of the center of the electric gripper 3 in the camera coordinate system are obtained through coordinate transformation. The coordinate value is fixed and can be obtained by the following formula:

[0029]

[0030] in, C P g is the coordinate of the center of the electric gripper 3 in the camera coordinate system {C}, is the transformation matrix between the laser tracker coordinate system and the camera coordinate system, L P g(j) is the coordinate of the center of the electric gripper 3 in the coordinate system {L}, and j represents different postures of the mobile robot;

[0031] In order to facilitate the position control of the robot arm 5 during the gripping process, the target detection object and the center point of the electric gripper 3 need to be represented in the robot arm base coordinate system {R}. The coordinates of the center point of the electric gripper 3 are R P g(i) The formula (6) obtained C P g Perform coordinate transformation to obtain:

[0032]

[0033] in,R P g(i) is the coordinate of the center point of the electric gripper 3, is the transformation matrix between the camera coordinate system and the terminal coordinate system, is the transformation matrix between the terminal coordinate system and the base coordinate system in each posture, C P g is the coordinate of the center of the electric gripper 3 in the camera coordinate system {C};

[0034] At the same time, in the same way, the position coordinates of the target point in the robot base coordinate system {R} are obtained by:

[0035]

[0036] in, R P obj(j) is the coordinate of the target position measured by the binocular camera in the base frame under any posture of the robot, C(j) P obj is the coordinate of the target position measured by the binocular camera in the camera coordinate system under any posture of the robot;

[0037] Therefore, to accurately locate the end position of the robot, it is only necessary to convert the robot end coordinates into the robot base coordinate system {R} R P g(j) Coordinates of the target object R P obj(j) Just align them.

[0038] The target sphere 2 includes a laser tracker target sphere (Spherically Mounted Retro-reflectors, SMR) and a visual marker target sphere (Split Bearing Retro-reflectors, SBR).

[0039] Beneficial effects of the present invention: The "eye in hand" rapid hand-eye calibration method for accurate gripping of a robot end proposed in the present invention has a simple calibration process. During the rotation calibration process, only three controlled end movements and four images are required; during the translation calibration process, only the image of one control point and the coordinates of some common points for coordinate conversion are required, eliminating the dependence of the original calibration process on complex robot calibration postures and additional calibration objects, and the operation is simple and fast; in addition, the method can achieve high-precision positioning of the robot end for accurate gripping. After the calibration is completed, the center of the gripper at the robot end can be guided to any spatial target point in real time and accurately through a binocular camera without the need for additional LT guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the hand-eye calibration platform.

[0041] Figure 2 Schematic diagram of rotation calibration.

[0042] Figure 3 Schematic diagram of translation calibration.

[0043] In the figure: 1- laser tracker, 2- target ball, 3- electric gripper, 4- binocular camera, 5- robotic arm, 6- AGV car. DETAILED DESCRIPTION

[0044] The specific implementation of the present invention is described in detail below in conjunction with the technical scheme and the accompanying drawings.

[0045] Figure 1 : is a schematic diagram of the hand-eye calibration platform. The calibration platform of this embodiment consists of a laser tracker (Leica ATS600), a wide-angle binocular camera (Stereolabs, ZED 2i, resolution: 2560×1440, working distance: 0.3m~20m, pixel size: 2μm×2μm), a collaborative robot (AUBO i5H), SMR and SBR, an electric gripper and a workstation. The binocular camera and the electric gripper are installed at the end of the robot through a fixture (3D printed model). After assembly, their relative positions remain unchanged. 10 common points are placed on the target ball seat (Sophisticated Magnetic Nest, SMN) for calibration, and are arranged in a non-collinear and non-coplanar manner.

[0046] The present invention uses three single-axis controlled robot end motions and four images taken by a binocular camera to calibrate the rotation matrix between CCS and ECS. The translation vector is calibrated by coordinate transformation using one LT and 10 non-coplanar common points. Finally, the relative position between the center of the robot end gripper and the binocular camera is determined to achieve accurate online positioning of the robot.

[0047] The specific steps of the method are as follows:

[0048] The first step is the hand-eye calibration of "eyes on hands";

[0049] An SMR is placed at the bottom of the gripper to accurately measure the movement of the end point. After each movement, the LT measures the coordinates of the SMR and records the displacement [ E Δx, E Δy, E Δz]; To measure the decomposed motion in {C}, the binocular camera takes a total of 4 images: before and after the motion E Δx, after exercise E Δy and after exercise E Δz, the rotation matrix is ​​obtained by formula (2)

[0050] A set of 10 common points are measured by LT and a stereo camera and calibrated by the least squares method. Place the SMR on the robot end, change the robot posture and measure the coordinates of the SMR through LT, and calibrate it using the least squares method Based on the coordinates of the fixed common point measured by LT and the binocular camera, the translation vector is obtained by formula (3):

[0051] Place an SMR at the center of the gripper and measure the position of this point using LT. The formula (5) gives C P g .

[0052] The second step is to verify the gripping accuracy;

[0053] After calibration using the above method, a verification experiment was carried out to demonstrate the effectiveness of this method. Several guiding target points were set, and SMR and SBR were placed on the target points. The LT was used to measure the position of the target point SMR, and then the SBR was measured by the binocular camera to obtain the position of the target point. The end center was guided to the target point position using the hand-eye calibration result obtained in the previous step. After the guidance was completed, the LT was used to measure the end center SMR position and compared it with the target point position to obtain the absolute coordinate guidance accuracy of this method.

[0054] The results show that the average absolute positioning error of this method is 0.321mm in the X direction, 0.222mm in the Y direction, 0.387mm in the Z direction, and the average error of the guide length is 0.273mm. The above results show that this method can meet the robot positioning accuracy requirements for gripping scenarios.

Claims

1. A "eye-in-hand" rapid hand-eye calibration method for accurate gripping at the end of a robot, characterized in that: Here are the steps: Step 1: Build a hand-eye calibration platform with "eyes on hands"; A hand-eye calibration platform is constructed, including a laser tracker (1), a target ball (2), an electric gripper (3), a binocular camera (4), a mechanical arm (5), and an AGV (6); wherein the electric gripper (3) and the binocular camera (4) are fixedly mounted on the end of the mechanical arm (5) through a fixture, and the mechanical arm (5) is fixed on the AGV (6), forming an "eye-in-hand" visual inspection system; The second step is the rotation matrix calibration based on the robot's directional displacement; According to kinematics theory, the rotation matrix between the camera coordinate system and the robot end coordinate system is defined as follows: in, and are the unit vectors of the camera coordinate system {C} and the robot end coordinate system {E} respectively; Assume that the origin of the camera coordinate system {C} and the robot end coordinate system {E} coincide, and the unit vectors of each axis of the robot end coordinate system {E} have fixed angles α1, β1, and γ1 with the unit vector of the camera coordinate system {C} respectively; let a point P be along Direction of a displacement along a straight line E Δx, the motion is decomposed into [ C Δx1, C Δy1, C Δz1]; According to the geometric relationship and formula (1), the angles [α1, β1, γ1] are calculated by projection between unit vectors, that is, [ C Δx1 / E Δx, C Δy1 / E Δx, C Δz1 / E Δx]; Similarly, the angle relationship between the three axes of the camera coordinate system {C} and the robot end coordinate system {E} is determined by the controlled motion along the axis of the robot end coordinate system {E}, so the rotation matrix is ​​calculated by the following formula: in,[ C Δx2, C Δy2, C Δz2] and [ C Δx3, C Δy3, C Δz3] are E Δy and E The motion decomposition of the controlled motion of Δz in the camera coordinate system {C}; The third step is the calibration of translation vector based on coordinate transformation; Through rotation matrix calibration, the rotated camera coordinate system {CR} is obtained, which has the same axis direction as the robot end coordinate system {E}, but the two have different origins. Therefore, it is necessary to calibrate the translation vector between the coordinates. This process is completed by shooting the common point and combining the coordinate transformation: The binocular camera (4) shoots the common point and generates the point coordinates in the camera coordinate system C P, after the rotation matrix calibration, obtain the point coordinates P1 in the camera coordinate system {CR}; C P through the transformation matrix Convert to the coordinate system {L} of the laser tracker (1) and then use the transformation matrix Convert to the robot base coordinate system {R}, and finally use the transformation matrix Convert to the robot end coordinate system {E}, and obtain the coordinate P2 in the robot end coordinate system {E}; finally, the translation vector between the robot end coordinate system {E} and the rotated camera coordinate system {CR} It is calculated by the difference between the two, namely: in, is the rotation matrix obtained in the second step using formula (1), Obtained through the DH table of the robot arm, and The coordinate values ​​of the same set of fixed common point target balls are measured by binocular camera (4) and laser tracker (1), and then the transformation matrix least square fitting method based on SVD solution is used to obtain: Where i is the number of public point targets, p i is the coordinate of the ith common point measured by the binocular camera (4), q i The coordinates of the i-th point measured by LT, are the rotation matrix and translation vector of the camera coordinate system and the laser tracker coordinate system respectively; in, The rotation matrix and translation vector of the laser tracker coordinate system and the base coordinate system are obtained by formula (5). Therefore, (2), (3), (4) and (5) are used to obtain the transformation matrix that transforms the camera coordinate system {C} into the robot end coordinate system {E} The fourth step is to determine the end position of the robot arm that can accurately grasp the target object; During the robot's movement and gripping process, the relative position between the center of the electric gripper (3) and the binocular camera (4) is always constant. To achieve accurate gripping, the relationship must first be calibrated and measured. First, a target ball (2) is placed at the center of the electric gripper (3), and the target ball (2) is measured using LT to obtain the coordinates of the center of the electric gripper (3) in the laser tracker coordinate system. Then, the coordinates of the center of the electric gripper (3) in the camera coordinate system are obtained through coordinate transformation. The coordinate value is fixed and is obtained through the following formula: in, C P g is the coordinate of the center of the electric gripper (3) in the camera coordinate system {C}, is the transformation matrix between the laser tracker coordinate system and the camera coordinate system, L P g(j) is the coordinate of the center of the electric gripper (3) in the coordinate system {L}, and j represents different positions of the mobile robot; In order to facilitate the position control of the robot arm (5) during the gripping process, the target detection object and the center point of the electric gripper (3) need to be represented in the robot arm base coordinate system {R}. The coordinates of the center point of the electric gripper (3) are R P g(i) The formula (6) obtained C P g Perform coordinate transformation to obtain: in, R P g(i) is the coordinate of the center point of the electric gripper (3), is the transformation matrix between the camera coordinate system and the terminal coordinate system, is the transformation matrix between the terminal coordinate system and the base coordinate system in each posture, C P g is the coordinate of the center of the electric gripper (3) in the camera coordinate system {C}; At the same time, in the same way, the position coordinates of the target point in the robot base coordinate system {R} are obtained by: in, R P obj(j) is the coordinate of the target position measured by the binocular camera in the base frame under any posture of the robot, C(j) P obj is the coordinate of the target position measured by the binocular camera in the camera coordinate system under any posture of the robot; Therefore, to accurately locate the end position of the robot, it is only necessary to convert the robot end coordinates into the robot base coordinate system {R} R P g(j) Coordinates of the target object R P obj(j) Just align them.

Images