A pose graph-based method for correcting pose errors and calibration errors of hand-eye systems

Abstract

The present invention provides a pose graph-based method for correcting pose errors and calibration errors in a hand-eye system, relating to the fields of robotics and machine vision. The method comprises step S1: establishing a hand-eye system consisting of a robotic arm and a camera system, and placing a target object with geometric or color features at a target location; step S2: moving the robotic arm to a preset position and recording the pose measurements of the robotic arm end; step S3: photographing the target object at the preset position in step S2 and calculating the pose measurements of the target object in the camera coordinate system; step S4: repeating steps S2 and S3 to obtain a data set containing multiple pairs of data, each corresponding to an edge in the pose graph; and step S5: obtaining an error function expression for each edge based on the pose logical relationship in the hand-eye system, and adding a perturbation term to the corresponding quantity to be optimized. This method can reduce the impact of measurement errors between the robotic arm and the camera on calibration accuracy.

Description

Technical Field

[0001] The present invention relates to the field of robotics and machine vision technology, and in particular to a method for correcting posture errors and calibration errors of a hand-eye system based on a posture graph. Background Art

[0002] In intelligent robots, the hand-eye system, comprised of a robotic arm and a camera, is a widely used machine perception system. The robotic arm possesses the ability to grasp and move objects, while the camera perceives surrounding objects in two- or three-dimensional images. This combined system can perceive specific information about the surrounding environment and provide feedback to the surrounding environment. In practice, to facilitate independent debugging of the robotic arm and camera, the coordinate system of the former is often based on the base of the robotic arm, while the coordinate system of the latter has its origin near the camera lens. To achieve this perception-feedback link, the perceived object's pose in the camera coordinate system must be converted to the robotic arm's coordinate system using a six-degree-of-freedom transformation matrix consisting of rotations and translations. This matrix is ​​often referred to as the hand-eye calibration matrix.

[0003] In order to solve the hand-eye calibration matrix, it is necessary to introduce an additional target with obvious geometric or color features into the hand-eye system. By controlling the robotic arm to a certain suitable posture and using the camera to capture the target position in this posture, this process is repeated multiple times until the number of times required to calculate the hand-eye calibration matrix is ​​reached. During the entire calibration calculation process, the measured robotic arm posture information refers to the flange end posture matrix derived from the joint angle. Its accuracy is limited by the spatial position of the flange, the robotic arm forward solution model, and the installation accuracy. The measured target position information is the posture matrix calculated based on the pixel-level feature points of the camera image. Its accuracy is restricted by the camera image resolution and distortion, the shooting distance angle, and the feature point extraction algorithm.

[0004] The current method for solving hand-eye calibration completely ignores the impact of the above-mentioned factors on the accuracy of the robot arm pose matrix and the pose of the target object in the image. It directly sets the two poses to constant quantities and then obtains the global optimal solution. As a result, the calculated value of the calibration result is significantly different from the actual value, which cannot meet the requirements of high-precision task operations. Summary of the Invention

[0005] The problem solved by the present invention is how to reduce the influence of measurement errors of the robotic arm and the camera on calibration accuracy.

[0006] To solve the above problems, the present invention provides a method for correcting posture error and calibration error of a hand-eye system based on a posture graph, comprising the steps of:

[0007] S1: Build a hand-eye system consisting of a robotic arm and a camera system to place objects with geometric or color features at the target location;

[0008] S2: Move the robotic arm to the preset position and record the posture of the end of the robotic arm;

[0009] S3: photographing the target object at the preset position in step S2, and calculating the pose of the target object in the camera coordinate system;

[0010] S4: By repeating steps S2 and S3, a data set containing multiple pairs of data is obtained, and each set of data corresponds to an edge in the pose graph;

[0011] S5: Based on the pose logic relationship in the hand-eye system, the error function expression of each edge is obtained, and a disturbance term is added to the corresponding quantity to be optimized;

[0012] S6: Using the adjoint property and BCH approximation, calculate the Jacobian matrix of the error function of each edge;

[0013] S7: Optimize the global error function by combining the Jacobian matrix of step S6 with the Levenberg-Marquardt method to obtain the error-corrected robot and target object poses, as well as the corresponding hand-eye calibration matrix error.

[0014] In this method, pose graph optimization is applied to the calibration calculation process to correct the hand-eye system pose measurement errors, namely the robot arm pose error and the target object pose error, thereby reducing the hand-eye calibration matrix calculation error. The pose graph uses a graphical representation to describe the robot and target poses, as well as their logical constraints. The hand-eye system calibration matrix and the robot and target pose variables to be optimized can be considered nodes in the graph, while the logical constraints between them can be considered edges. By reducing the error function of the entire system while satisfying the logical constraints, the hand-eye system pose error can be corrected and the optimized hand-eye calibration matrix can be directly obtained.

[0015] Furthermore, the hand-eye system in step S1 includes two modes: eye-to-hand mode and eye-in-hand mode.

[0016] Furthermore, in step S4, the calibration matrix of the hand-eye system, the robot posture quantity, and the variables to be optimized of the target object posture in different coordinate systems are used as nodes to be optimized.

[0017] Furthermore, the error function expression in step S5 is:

[0018] error i =ln(A i HT i O -1 ) ∨ ;

[0019] Among them, Ai Expressed as the robot posture; T i It is represented as the pose of the target object in the camera coordinate system; O is represented as the pose of the target object in the robot coordinate system; H is represented as the hand-eye calibration matrix; ∨ It is expressed as the conversion from Lie group to Lie algebra operation symbols;

[0020] Amount to be optimized A i ,H,T i ,O add disturbances respectively The error after disturbance is:

[0021]

[0022] Furthermore, step S6 includes:

[0023] S61: Separation of variables and disturbance terms based on adjoint properties;

[0024] S62: Calculate the Jacobian matrix of each edge error function for the robot and target object poses to be optimized based on the BCH approximation.

[0025] Furthermore, based on the adjoint properties of Lie groups and Lie algebras, we can obtain:

[0026]

[0027] The perturbation error is transformed into:

[0028]

[0029] Furthermore, based on the BCH linear approximation, we can obtain:

[0030]

[0031] The transformed perturbation error is transformed again into:

[0032]

[0033] in, They are respectively expressed as the Jacobian matrices corresponding to the optimization variables:

[0034]

[0035]

[0036]

[0037]

[0038] Among them, J r is represented as a right Jacobian matrix.

[0039] Furthermore, the global error function is optimized to obtain the objective function:

[0040]

[0041] Using the Jacobian matrix and the Levenberg-Marquardt method, we can get the modified A i , T i and O, and the hand-eye calibration matrix error H.

[0042] The present invention adopts the above technical solution to achieve the following beneficial effects:

[0043] By applying pose graph optimization methods to the calibration calculation process, the present invention can correct the hand-eye system pose measurement errors, the robot arm pose errors, and the target object pose errors, thereby reducing the hand-eye calibration matrix calculation errors. This method can be used to correct the hand-eye system pose errors and directly obtain the corrected hand-eye calibration matrix. It can also be used to improve the accuracy of calibration based on the robot arm and camera measurement errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The present invention provides a method for correcting the pose error and calibration error of the hand-eye system based on the pose graph. Figure 1 ;

[0045] Figure 2 The present invention provides a method for correcting the pose error and calibration error of the hand-eye system based on the pose graph. Figure 2 ;

[0046] Figure 3 The present invention provides a method for correcting the pose error and calibration error of the hand-eye system based on the pose graph. Figure 3 ;

[0047] Figure 4 A diagram of the environment setup for the eye-to-hand mode and eye-in-hand mode in the pose graph-based hand-eye system pose error and calibration error correction method provided in an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of the structure of the optimization graph in the pose error and calibration error correction method of the hand-eye system based on the pose graph provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0050] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solutions of the present invention, but the present invention is not limited to these embodiments.

[0051] Example

[0052] This embodiment provides a method for correcting the pose error and calibration error of a hand-eye system based on a pose graph. Figure 1 and Figure 3 As shown, the method includes the steps of:

[0053] S1: Build a hand-eye system consisting of a robotic arm and a camera system to place objects with geometric or color features at the target location;

[0054] S2: Move the robotic arm to the preset position and record the posture of the end of the robotic arm;

[0055] S3: photographing the target object at the preset position in step S2, and calculating the pose of the target object in the camera coordinate system;

[0056] S4: By repeating steps S2 and S3, a data set containing multiple pairs of data is obtained, and each set of data corresponds to an edge in the pose graph;

[0057] S5: Based on the pose logic relationship in the hand-eye system, the error function expression of each edge is obtained, and a disturbance term is added to the corresponding quantity to be optimized;

[0058] S6: Using the adjoint property and BCH approximation, calculate the Jacobian matrix of the error function of each edge;

[0059] S7: Optimize the global error function by combining the Jacobian matrix of step S6 with the Levenberg-Marquardt method to obtain the error-corrected robot and target object poses, as well as the corresponding hand-eye calibration matrix error.

[0060] See Figure 3 Specifically, a hand-eye system based on a robotic arm and a camera is built, the robotic arm is moved to a suitable position, the end-face pose matrix at this time is recorded, the pose of the target object in the camera coordinate system is photographed and calculated, multiple sets of data are obtained in a loop, and it is judged whether the amount of data meets the user's needs, that is, whether it is sufficient. If not, continue to photograph and calculate the pose quantity; if so, it is converted into a graph model connected by edges and vertices, the error function expression of each edge is listed, and a disturbance term is added to the vertex to be optimized. The adjoint property and the BCH approximation are used to calculate the corresponding Jacobian matrix, optimize the objective function, and obtain the various values ​​of the error correction.

[0061] See Figure 4Specifically, a hand-eye system consisting of a robotic arm and a camera is built. Based on user requirements, an object with distinct geometric or color features is placed in a specific location using either eye-to-hand or eye-in-hand mode. Since the differences between the two modes do not significantly affect subsequent steps, the following steps use the eye-in-hand mode as an example. Here, the pose of the object in the robotic arm coordinate system is assumed to be O, and the hand-eye calibration matrix is ​​H.

[0062] Specifically, the robotic arm is moved to a suitable position, and the posture A1 of the end of the robotic arm at this time is recorded.

[0063] Among them, the calibration matrix of the hand-eye system in step S4, the robot posture quantity and the variables to be optimized of the target object posture in different coordinate systems are used as nodes to be optimized.

[0064] See Figure 5 Specifically, the target object is photographed at the position of step S2, and the pose of the target object in the camera coordinate system at this time is calculated. Repeat steps S2-S3, record the pose of the target object in the robot arm and camera coordinate system, and record them as A and i With T i , each set of data corresponds to an edge in the pose graph; the calibration matrix H of the hand-eye system, the robot pose A i and the target object pose O, T in different coordinate systems i The variables to be optimized can be regarded as nodes to be optimized in the graph.

[0065] The error function expression in step S5 is:

[0066] error i =ln(A i HT i O -1 ) ∨ ;

[0067] Among them, A i Expressed as the robot posture; T i It is represented as the pose of the target object in the camera coordinate system; O is represented as the pose of the target object in the robot coordinate system; H is represented as the hand-eye calibration matrix; ∨ It is expressed as the conversion from Lie group to Lie algebra operation symbols;

[0068] Amount to be optimized A i ,H,T i ,O add disturbances respectively The error after disturbance is:

[0069]

[0070] See Figure 2 , wherein step S6 includes:

[0071] S61: Separation of variables and disturbance terms based on adjoint properties;

[0072] S62: Calculate the Jacobian matrix of each edge error function for the robot and target object poses to be optimized based on the BCH approximation.

[0073] Among them, based on the adjoint properties of Lie groups and Lie algebras, we get:

[0074] e ξ∧ T=Te (Ad(T-1)ξ)∧ ;

[0075] The perturbation error is transformed into:

[0076]

[0077] Among them, based on the BCH linear approximation:

[0078]

[0079] The transformed perturbation error is transformed again into:

[0080]

[0081] in, They are respectively expressed as the Jacobian matrices corresponding to the optimization variables:

[0082]

[0083]

[0084]

[0085]

[0086] Among them, J r is represented as a right Jacobian matrix.

[0087] Specifically, the Jacobian matrix obtained in step S6 is combined with the Levenberg-Marquardt method to calculate the global optimal solution, and the error-corrected postures of the robot and the target object, as well as the corresponding hand-eye calibration matrix, are obtained.

[0088] Among them, the objective function is obtained by optimizing the global error function:

[0089]

[0090] Using the Jacobian matrix and the Levenberg-Marquardt method, we can get the modified Ai , T i and O, and the hand-eye calibration matrix error H.

[0091] Specifically, A i With T i is the recorded arm pose and the object pose in the camera coordinate system; H and O are the hand-eye calibration matrix and the target pose in the arm coordinate system. An initial value can be randomly or estimated, or a rough value can be pre-calculated using traditional methods such as Tsai or Park. Using the Jacobian matrix and Levenberg-Marquardt method, the objective function is optimized, and then the modified A can be obtained. i , T i and O, and the hand-eye calibration matrix error H.

[0092] This method applies pose graph optimization to the calibration calculation process, thereby correcting the hand-eye system pose measurement errors, the robot arm pose errors, and the target object pose errors, thereby reducing the error in the hand-eye calibration matrix calculation. This method can be used to correct the hand-eye system pose errors and directly obtain the corrected hand-eye calibration matrix. It can also be used to improve the accuracy of calibration based on the measurement errors of the robot arm and camera.

[0093] Although the present disclosure is disclosed as above, the protection scope of the present disclosure is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A method for correcting pose error and calibration error of a hand-eye system based on a pose graph, characterized in that: Including steps: S1: Build a hand-eye system consisting of a robotic arm and a camera system, place a target with geometric or color features at the target location, and set the hand-eye calibration matrix H of the target in the robotic arm coordinate system; S2: Move the robotic arm to the preset position and record the posture of the end of the robotic arm; S3: photographing the target object at the preset position in step S2, and calculating the pose of the target object in the camera coordinate system; S4: By repeating steps S2 and S3, a data set containing multiple pairs of data is obtained, and the data set corresponds to the edges in the pose graph, wherein the data set includes the pose of the end of the robot arm and the pose of the target object in the camera coordinate system; S5: Based on the pose logic relationship in the hand-eye system, the error function expression of each edge is obtained, and a disturbance term is added to the corresponding quantity to be optimized; S6: Using the adjoint property and BCH approximation, calculate the Jacobian matrix of the error function of each edge; S7: Optimize the global error function by combining the Jacobian matrix of step S6 with the Levenberg-Marquardt method to obtain the error-corrected robot and target object poses, as well as the corresponding hand-eye calibration matrix error.

2. The method for correcting pose error and calibration error of a hand-eye system based on a pose graph according to claim 1, characterized in that: The hand-eye system in step S1 includes two modes: eye-to-hand mode and eye-in-hand mode.

3. The method for correcting pose error and calibration error of a hand-eye system based on a pose graph according to claim 1, characterized in that: The hand-eye calibration matrix, the robot posture quantity and the variables to be optimized of the target object posture in different coordinate systems are used as nodes to be optimized.

4. The method for correcting pose error and calibration error of a hand-eye system based on a pose graph according to claim 1, characterized in that: The error function expression in step S5 is: error i =ln(A i HT i O -1 ) ∨ ; Among them, A i Expressed as the robot posture; T i It is represented as the pose of the target object in the camera coordinate system; O is represented as the pose of the target object in the robot coordinate system; H is represented as the hand-eye calibration matrix; ∨ Represented as the operator symbol for converting from Lie algebra matrix to vector; Amount to be optimized A i ,H,T i ,O add disturbances respectively The error after disturbance is:

5. The method for correcting pose error and calibration error of a hand-eye system based on a pose graph according to claim 4, characterized in that: The step S6 comprises: S61: Separation of variables and disturbance terms based on adjoint properties; S62: Calculate the Jacobian matrix of each edge error function for the robot and target object poses to be optimized based on the BCH approximation.

6. The method for correcting pose error and calibration error of a hand-eye system based on a pose graph according to claim 5, characterized in that: Based on the adjoint properties of Lie groups and Lie algebras, we get: The perturbation error is transformed into:

7. The method for correcting pose error and calibration error of a hand-eye system based on a pose graph according to claim 6, characterized in that: Based on the BCH linear approximation, we get: The transformed perturbation error is transformed again into: in, They are respectively expressed as the Jacobian matrices corresponding to the optimization variables: Here, Jr represents the right Jacobian matrix.

8. The method for correcting pose error and calibration error of a hand-eye system based on a pose graph according to claim 7, characterized in that: Optimize the global error function to obtain the objective function: Using the Jacobian matrix and the Levenberg-Marquardt method, we can get the modified A i , T i and O, and the hand-eye calibration matrix error H.

Images