A robot hand-eye calibration method using an optical three-dimensional motion capture system

Abstract

The application discloses a mechanical arm hand-eye calibration method using an optical three-dimensional motion capture system and belongs to the technical field of robot vision calibration; first, the positions of a camera and target points are arranged, so that each target point is at least in the field of view of three cameras, a teaching pendant of a mechanical arm is used to set a required pose for calibration, and three-dimensional position data of the target points under corresponding poses is collected; then, according to the measurement data, a point set matching algorithm is used to fit rigid body motion of the end of the mechanical arm, and a mathematical model of calibration is converted into AX = XB; finally, a least square calculation problem is converted into an equation solving problem by using Lie algebra, and a transformation matrix of a base coordinate system of the mechanical arm-world coordinate system is obtained based on singular value decomposition; the application proposes a calibration method based on three-dimensional position data, can realize kinematic fusion of a motion capture system and a mechanical arm system, and can be directly applied to mechanical arm measurement.

Description

Technical Field

[0001] This invention belongs to the field of robot vision calibration technology, specifically relating to a method for calibrating the hand and eye of a robotic arm using an optical three-dimensional motion capture system. Background Technology

[0002] When robotic arms are used for inspection, processing, minimally invasive surgery, and measurement, they are typically combined with cameras and machine vision technology to perform motion navigation. Robotic arm systems that integrate vision are called hand-eye systems. There are two classic configurations for hand-eye systems: one where the camera is mounted at the end of the robotic arm, called an eye-in-hand system; and the other where the camera is fixed in a specific position, called an eye-to-hand system.

[0003] In a broad sense, hand-eye systems encompass all systems that use a robotic arm as the actuator and integrate visual sensors. In recent years, motion capture systems based on infrared measurement technology have been widely used in multi-body system measurement due to their absolute advantage of a large field of view. Before use, hand-eye systems require the fusion of the vision system and the robotic arm system; this process is called hand-eye calibration. However, hand-eye system calibration based on motion capture systems faces two challenges: first, the measurement data consists of three-dimensional point position information, which is insufficient for a mathematical model based on rigid body motion data; second, the field of view is affected by the movement of the robotic arm itself, with the measured object itself causing occlusion. This manifests in the measurement data as less movement in the robotic arm's wrist and primarily movement in the first three joints. Simultaneously, the field of view is affected by the camera's placement; how to acquire poses that conform to optimal pose criteria is another technical challenge.

[0004] The prior art discloses a hand-eye calibration method based on a motion capture system, which aims to establish a hand-eye system based on a motion capture system and realize the integration of a robotic arm system and a vision system. Summary of the Invention

[0005] This invention provides a hand-eye calibration method based on a motion capture system, which aims to integrate the robotic arm system and the vision system by calibrating the base coordinate system, thereby constructing a high-precision hand-eye system based on the motion capture system.

[0006] To achieve the above objectives, the present invention employs the following technical solution: On the one hand, this invention provides a method for calibrating the hand and eye of a robotic arm using an optical three-dimensional motion capture system, comprising the following steps: S1: Set the initial configuration of the robotic arm using the teach pendant; S2: Deploy the positions of 6 cameras and target points to ensure that all target points in the initial configuration are within the measurement field of view; S3: Calibrate camera internal parameters; S4: Move the robotic arm using the teach pendant to set the next pose, ensuring that at least 6 target points are within the measurement field of view; S5: Use the pose verification model to determine whether the pose selection is correct; S6: Repeat S4-S5 to acquire no less than 10 poses. If the poses acquired in three consecutive attempts fail, return to S2.

[0007] Preferably, the overall calibration process includes two modules: An eye-to-hand measurement system was constructed based on a 3D motion capture system. Specifically, six cameras and target points were deployed, and the teach pendant of the robotic arm was controlled to collect the poses required for calibration. The positions of the cameras were adjusted to ensure that sufficient poses for the pose detection model could be collected. Identify the coordinate transformation parameters between the robot arm's base coordinate system and the world coordinate system; specifically, fit the rigid body motion of the acquired pose relative to the initial pose based on the point set matching algorithm, and construct a computational model of the form AX=XB based on the rigid body motion described by the DH model computer base coordinate system, and solve the equation based on Lie algebra and the point set matching algorithm.

[0008] Preferably, the pose detection model is used to control the acquired pose to be sufficiently discrete to ensure the accuracy of the least squares-based identification algorithm; specifically, the joint angles of the initial pose and the acquired pose are first read through the teach pendant, and the initial pose and the acquired pose are calculated based on the DH model as follows: (2); in, and They are all homogeneous matrices that include rotations and translations. This represents the end-effector pose of the robotic arm in its initial configuration. The k-th pose is captured; the rigid body motion of the k-th pose relative to the initial pose is: (3); Where N represents the total number of N+1 poses collected, with the initial pose as a reference, and N rigid body motions are calculated. Then calculate the rotation angle: (4); in, It is a rotation matrix, representing the rotational component of rigid body motion; for any rotation matrix, the rotation angle is... Corresponding rotation axis Then there is another set of solutions, namely and Normalization only considers If the calculated rotation angle is greater than And less than If the pose is valid, it is considered a feasible pose; otherwise, it is considered an invalid pose.

[0009] Preferably, the point set matching algorithm is described as follows: (5); in, It is the rotational motion of the k-th pose relative to the initial configuration; It is the translational motion of the k-th pose relative to the initial configuration; and These are the initial pose and the 3D position data of the i-th target point in the k-th pose, respectively, containing a total of n target points; Further decoupling the rotation and translation in formula (4), we can transform it into: (6); and, (7); in, (8); First calculate the rotation, then formula (5) can be converted to (9); remember: (10); in, (11); right Singular value decomposition yields: (12); so: (13); Finally, Substituting into formula (6), the translation is fitted; the rotation and translation can be written in the form of the following homogeneous matrix: (14).

[0010] Preferably, the hand-eye calibration model is as follows: (15); in, and These are all rigid body movements at the end effector of the robotic arm, represented in different coordinate systems. It is the transformation matrix between the base coordinate system and the world coordinate system. , and All are homogeneous transformation matrices, elements of the SE(3) group; Each homogeneous matrix in formula (14) contains a rotation matrix and a translation vector, which can be written as: (16); Furthermore, we can decouple the rotation and translation to obtain the rotation equation: (17); Translation equations: (18); In formula (16) , and All are elements of the SO(3) group. The Lie algebra so(3) and the Lie group SO(3) satisfy an exponential mapping relationship. Therefore, the Lie algebraic form of formula (16) is: (19); in, It is a three-dimensional column vector. It is the antisymmetric matrix corresponding to the column vectors, represented as: (20); From the properties of the Lie algebra so(3): Formula (18) is transformed into the Lie algebra representation: (twenty one); Further reduction yields: (twenty two); Solve for formula (21) This becomes a point set matching problem. Considering the motion of N rigid bodies, the equations are transformed into the following problem: (twenty three); Based on the aforementioned point set matching algorithm, rotation can be achieved. Solve for; Finally, solve the translation equation (Equation (17)). Substituting the translation equations, the N equations form an overdetermined system of equations as follows: (twenty four); Therefore, the solution for translation is: (25).

[0011] On the other hand, the present invention provides an apparatus for the above-mentioned method, including a base coordinate system, a world coordinate system, an optical target, an infrared camera, and a robotic arm. The apparatus is characterized in that six infrared cameras are provided, the optical target is arranged at the end of the robotic arm, and the cameras continuously capture the three-dimensional position information of the optical target points through a set sampling frequency. The movement of the target points represents the movement of points on the rigid body at the end of the robotic arm, described in the world coordinate system. Simultaneously, the movement of the rigid body at the end of the robotic arm can be read by the robot control system and described in the base coordinate system.

[0012] Compared with the prior art, the present invention has the following beneficial effects: This invention proposes a calibration method based on three-dimensional position data, which can realize the kinematic fusion of motion capture system and robotic arm system, and can be directly applied to robotic arm measurement. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the hand-eye system of the present invention; Figure 2 This is a flowchart illustrating the measurement process of the method of the present invention; Figure 3 A schematic diagram of rigid body motion fitting for any pose relative to the initial pose; Figure 4 A flowchart of the hand-eye calibration algorithm; Figure 1 In the diagram: 1. Base coordinate system; 2. World coordinate system; 3. Optical target; 4. Infrared camera; 5. Robotic arm. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0016] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0017] The present invention will be further illustrated below with examples. These examples are only for illustrating the method of the present invention and do not limit the scope of application of the present invention in any way.

[0018] The method provided in this embodiment of the invention comprises two parts: First, an eye-to-hand measurement system was constructed based on a 3D motion capture system. Specifically, six cameras and target points were deployed, and the teach pendant of the robotic arm was controlled to collect the poses required for calibration. The positions of the cameras were adjusted to ensure that sufficient poses for the pose detection model could be collected. Second, identify the coordinate transformation parameters between the robot arm's base coordinate system and the world coordinate system. Specifically, based on the point set matching algorithm, fit the rigid body motion of the acquired pose relative to the initial pose. At the same time, based on the rigid body motion described by the DH model computer base coordinate system, construct a computational model of the form AX=XB, and solve the equation based on Lie algebra and the point set matching algorithm.

[0019] In this embodiment of the invention, the pose detection model is used to ensure that the acquired pose is sufficiently discrete to guarantee the accuracy of the least squares-based identification algorithm. Specifically, the joint angles of the initial pose and the acquired pose are first read through a teach pendant, and the initial pose and the acquired pose are calculated based on the DH model as follows: (26) in, and They are all homogeneous matrices that include rotations and translations. This represents the end-effector pose of the robotic arm in its initial configuration. Let k be the acquired pose. The rigid body motion of the k-th pose relative to the initial pose is: (27) Where N represents the total number of N+1 poses collected, with the initial pose as a reference, and N rigid body motions calculated.

[0020] Then calculate the rotation angle: (28) in, It is a rotation matrix, representing the rotational component of rigid body motion; for any rotation matrix, the rotation angle is... Corresponding rotation axis Then there is another set of solutions, namely and Normalization only considers If the calculated rotation angle is greater than And less than If the pose is valid, it is considered a feasible pose; otherwise, it is considered an invalid pose.

[0021] In embodiments of the present invention, such as Figure 2 As shown, the measurement in the first part includes the following steps: S1: Set the initial configuration of the robotic arm using the teach pendant; S2: Deploy the positions of 6 cameras and target points to ensure that all target points in the initial configuration are within the measurement field of view; S3: Calibrate camera internal parameters; S4: Move the robotic arm using the teach pendant to set the next pose, ensuring that at least 6 target points are within the measurement field of view; S5: Use the pose verification model to determine whether the pose selection is correct; S6: Repeat S4-S5 to acquire no less than 10 poses. If the poses acquired in three consecutive attempts fail, return to S2.

[0022] In this embodiment of the invention, the hand-eye calibration algorithm is as follows: Figure 4 As shown. The point set matching algorithm is described as follows: (29) in, It is the rotational motion of the k-th pose relative to the initial configuration; It is the translational motion of the k-th pose relative to the initial configuration; and These represent the 3D position data of the initial pose and the i-th target point in the k-th pose, respectively, for a total of n target points. Rigid body motion relative to the initial pose in any pose is shown below. Figure 3 As shown.

[0023] Further decoupling the rotation and translation in formula (4), we can transform it into: (30) and, (31) in, (32) First calculate the rotation, then formula (5) can be converted to (33) remember, (34) in, (35) right Singular value decomposition yields: (36) so, (37) Finally, Substituting into formula (6), the translation is fitted. The rotation and translation can be written in the form of a homogeneous matrix as follows: (38) Furthermore, the aforementioned robotic arm hand-eye calibration method is characterized in that the hand-eye calibration model is: (39) in, and These are all rigid body movements at the end effector of the robotic arm, represented in different coordinate systems. It is the transformation matrix between the base coordinate system and the world coordinate system. , and All are homogeneous transformation matrices, elements of the SE(3) group.

[0024] Each homogeneous matrix in formula (14) contains a rotation matrix and a translation vector, which can be specifically written as: (40) Furthermore, we can decouple the rotation and translation to obtain the rotation equation: (41) Translation equations: (42) In formula (16) , and All are elements of the SO(3) group. The Lie algebra so(3) and the Lie group SO(3) satisfy an exponential mapping relationship. Therefore, the Lie algebraic form of formula (16) is: (43) in, It is a three-dimensional column vector. It is the antisymmetric matrix corresponding to the column vectors, represented as: (44) From the properties of the Lie algebra so(3): Formula (18) is transformed into the Lie algebra representation: (45) Further reduction yields: (46) Solve for formula (21) This becomes a point set matching problem. Considering the motion of N rigid bodies, the equations are transformed into the following problem: (47) Based on the point set matching algorithm described above, rotation can be achieved. Solve for it.

[0025] Finally, solve the translation equation (Equation (17)). Substituting the translation equations, the N equations form an overdetermined system of equations as follows: (48) Therefore, the solution for translation is: (49) In the specific implementation process, Matlab is used for calculation. The calculation results are as follows: (50) The corresponding rotation angle can be calculated using formula (3): (50)

[0026] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for calibrating the hand and eye of a robotic arm using an optical three-dimensional motion capture system, characterized in that, Includes the following steps: S1: Set the initial configuration of the robotic arm using the teach pendant; S2: Deploy the positions of 6 cameras and target points to ensure that all target points in the initial configuration are within the measurement field of view; S3: Calibrate camera internal parameters; S4: Move the robotic arm using the teach pendant to set the next pose, ensuring that at least 6 target points are within the measurement field of view; S5: Use the pose verification model to determine whether the pose selection is correct; S6: Repeat S4-S5 to collect no less than 10 poses. If the poses collected for three consecutive times fail, return to S2. The overall calibration process consists of two modules: An eye-to-hand measurement system was constructed based on a 3D motion capture system. Specifically, six cameras and target points were deployed, and the teach pendant of the robotic arm was controlled to collect the poses required for calibration. The positions of the cameras were adjusted to ensure that sufficient poses for the pose detection model could be collected. Identify the coordinate transformation parameters between the robot arm's base coordinate system and the world coordinate system; specifically, fit the rigid body motion of the acquired pose relative to the initial pose based on the point set matching algorithm, and construct a computational model of the form AX=XB based on the rigid body motion described by the DH model computer base coordinate system, and solve the equation based on Lie algebra and the point set matching algorithm. The aforementioned pose detection model is used to control the acquired pose to be sufficiently discrete, in order to ensure the accuracy of the least squares-based identification algorithm. Specifically, the joint angles of the initial pose and the acquired pose are first read through a teach pendant, and the initial pose and the acquired pose are calculated based on the DH model as follows: (2)； in, and They are all homogeneous matrices that include rotations and translations. This represents the end-effector pose of the robotic arm in its initial configuration. The k-th pose is captured; the rigid body motion of the k-th pose relative to the initial pose is: (3)； Where N represents the total number of N+1 poses collected, with the initial pose as a reference, and N rigid body motions are calculated. Then calculate the rotation angle: (4)； in, It is a rotation matrix, representing the rotational component of rigid body motion; for any rotation matrix, the rotation angle is... Corresponding rotation axis Then there is another set of solutions, namely and Normalization only considers If the calculated rotation angle is greater than And less than If the pose is valid, it is considered a feasible pose; otherwise, it is considered an invalid pose. The point set matching algorithm is described as follows: (5)； in, It is the rotational motion of the k-th pose relative to the initial configuration; It is the translational motion of the k-th pose relative to the initial configuration; and These are the initial pose and the 3D position data of the i-th target point in the k-th pose, respectively, containing a total of n target points; Further decoupling the rotation and translation in formula (4), we can transform it into: (6)； and, (7)； in, (8)； First calculate the rotation, then formula (5) can be converted to (9)； remember: (10)； in, (11)； right Singular value decomposition yields: (12)； so: (13)； Finally, Substituting into formula (6), the translation is fitted; the rotation and translation can be written in the form of the following homogeneous matrix: (14)； The hand-eye calibration model is as follows: (15)； in, and These are all rigid body movements at the end effector of the robotic arm, represented in different coordinate systems. It is the transformation matrix between the base coordinate system and the world coordinate system. , and All are homogeneous transformation matrices, elements of the SE(3) group; Each homogeneous matrix in formula (14) contains a rotation matrix and a translation vector, which can be written as: (16)； Furthermore, we can decouple the rotation and translation to obtain the rotation equation: (17)； Translation equations: (18)； In formula (16) , and All are elements of the SO(3) group. The Lie algebra so(3) and the Lie group SO(3) satisfy an exponential mapping relationship. Therefore, the Lie algebraic form of formula (16) is: (19)； in, It is a three-dimensional column vector. It is the antisymmetric matrix corresponding to the column vectors, represented as: (20)； From the properties of the Lie algebra so(3): Formula (18) is transformed into the Lie algebra representation: (21)； Further reduction yields: (22)； Solve for formula (21) This becomes a point set matching problem. Considering the motion of N rigid bodies, the equations are transformed into the following problem: (23)； Based on the aforementioned point set matching algorithm, rotation can be achieved. Solve for; Finally, solve the translation equation (Equation (17)). Substituting the translation equations, the N equations form an overdetermined system of equations as follows: (24)； Therefore, the solution for translation is: (24)。 2. An apparatus for implementing the method of claim 1, comprising a base coordinate system (1), a world coordinate system (2), an optical target (3), an infrared camera (4), and a robotic arm (5), characterized in that, The infrared camera (4) is provided with 6 units. The optical target (3) is arranged at the end of the robotic arm (5). The camera (4) continuously captures the three-dimensional position information of the optical target point through the set sampling frequency. The movement of the target point represents the movement of the point on the rigid body at the end of the robotic arm, which is described in the world coordinate system (2). At the same time, the movement of the rigid body at the end of the robotic arm can be read by the robot control system and described in the base coordinate system (1).

Images