Rapid and efficient hand-eye calibration system and method in narrow scene

Abstract

The invention provides a rapid and efficient hand-eye calibration system and method in a narrow scene, and relates to the technical field of hand-eye calibration, and the method comprises the steps: shooting a fixed calibration ball from a plurality of different poses, and obtaining the data of the calibration ball; processing the calibration ball data, and obtaining the coordinates of the center of the calibration ball in the camera coordinate system under each pose through ball fitting; constructing a constraint equation based on a hand-eye transformation matrix by using the obtained coordinates; solving the constraint equation to obtain an initial estimation value of the hand-eye transformation matrix; based on the initial estimation value, defining an error model and constructing an error equation set; and iteratively solving the error equation set, updating the estimation of the hand-eye transformation matrix until a convergence condition is met, and obtaining a final optimized hand-eye transformation matrix. According to the method, the problem that a traditional calibration target is difficult to arrange in a narrow space is solved, the iteration process is rapid, and the problem that the iteration process is complex and consumes long time is solved while the precision is considered.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hand-eye calibration, in particular to a fast and efficient hand-eye calibration system and method in a narrow scene. BACKGROUND

[0002] Hand-eye calibration is a key technology to determine the spatial transformation relationship between the robot end effector (hand) and the vision sensor (eye), and is widely used in robot vision guidance, assembly and detection, etc. The accuracy of hand-eye calibration determines the operation accuracy of the robot arm. The existing hand-eye calibration methods usually need to be carried out in a spacious environment, using standard calibration boards and multi-view image acquisition. They are mainly divided into traditional methods based on mathematical analysis and optimization and methods based on deep learning. The traditional method usually relies on robot kinematics and vision measurement information, and solves the transformation relationship between the camera coordinate system and the robot end coordinate system through mathematical modeling; the method based on deep learning usually uses large-scale robot image data, and learns the mapping relationship between the camera and the robot end through the neural network, so as to automatically extract features and solve the hand-eye transformation matrix in a data-driven manner.

[0003] Therefore, it is necessary to design a fast and efficient hand-eye calibration system and method in a narrow scene. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a fast and efficient hand-eye calibration system and method in a narrow scene.

[0005] To achieve the above purpose, the present application provides the following solutions: The present application provides a fast and efficient hand-eye calibration system in a narrow scene, comprising: a robot unit, a vision sensing unit, a calibration target unit and a processing and computing unit; The robot unit comprises a robot base and a robot arm, the robot arm is arranged on the robot base, and an end effector is arranged on the end of the robot arm through an end effector mounting flange; The vision sensing unit comprises at least one camera fixedly mounted on the end effector mounting flange, for collecting image or point cloud data; The calibration target unit comprises at least one calibration ball fixedly placed in the robot workspace; The processing and computing unit is in communication connection with the robot unit and the vision sensing unit, for receiving the robot end pose data and the data collected by the camera, and performing hand-eye calibration calculation.

[0006] Preferably, the calibration ball is a single standard sphere.

[0007] Preferably, the camera is a structured light three-dimensional measurement binocular camera.

[0008] This invention also provides a rapid and efficient hand-eye calibration method for confined spaces, applicable to the aforementioned rapid and efficient hand-eye calibration system for confined spaces, comprising: The robot unit controls the camera to take pictures of the stationary calibration ball from multiple different poses, and obtains a series of robot end poses and corresponding calibration ball images or point cloud data; Process calibration sphere images or point cloud data, and obtain the coordinates of the calibration sphere center in the camera coordinate system at each pose by sphere fitting; Using the robot unit end-effector poses and sphere center camera coordinates corresponding to the obtained multiple poses, constraint equations based on the hand-eye transformation matrix are constructed. Solve the constraint equations to obtain an initial estimate of the hand-eye transformation matrix; Based on the initial estimate, define the error model and construct a system of error equations; The error equations are solved iteratively to update the estimate of the hand-eye transformation matrix until the convergence condition is met, thus obtaining the final optimized hand-eye transformation matrix.

[0009] Preferably, the constraint equations based on the hand-eye transformation matrix are constructed as follows: For any two poses i and j, based on the fact that the position of the calibration sphere's center remains unchanged in the robot's base coordinate system, the constraint equations are established as follows: (1) In the formula, For the robot hand-eye transformation matrix that needs to be calibrated, , .

[0010] Preferably, the constraint equations are solved to obtain an initial estimate of the hand-eye transformation matrix, specifically: The equations are transformed into a system of linear equations using the Kronecker product. Singular value decomposition is performed on the linear equation system to solve for the initial rotation matrix and initial translation vector of the hand-eye transformation matrix, thus obtaining the initial estimate. Preferably, based on the initial estimate, an error model is defined and a system of error equations is constructed as follows: Define the relationship between the true value and the initial estimate of the hand-eye transformation matrix, that is, define the error model as follows: (2) The rotation error is approximated by a first-order Taylor expansion: ,in It is the identity matrix. For vectors The antisymmetric matrix; The error model relationship is substituted into the constraint equation, first-order approximation is carried out, and a linear equation group about the error vector, i.e., an error equation group, is derived.

[0011] Preferably, the error equation group is iteratively solved, the estimation of the hand-eye transformation matrix is updated until a convergence condition is met, and a final optimized hand-eye transformation matrix is obtained, specifically as follows: The current error equation group is solved to obtain the rotation error estimation and the translation error estimation of the current iteration; The rotation error is converted into an incremental rotation matrix by using an exponential mapping, and the rotation matrix and the translation vector are updated; It is judged whether the error converges or not, if the error converges, the current estimation value is output as a final result, otherwise, the current estimation value is used as a new initial value to repeat the iteration process until a convergence condition is met, and a final optimized hand-eye transformation matrix is obtained.

[0012] According to the specific embodiments provided by the present application, the following technical effects are disclosed: The present application provides a fast and efficient hand-eye calibration system and method in a narrow scene, the system comprising a robot unit, a visual sensing unit, a calibration target unit and a processing and computing unit, the method comprising controlling the robot unit to drive the camera to capture a fixed calibration ball from multiple different poses, obtaining a series of robot end poses and corresponding calibration ball images or point cloud data; processing the calibration ball images or point cloud data, obtaining the coordinates of the calibration ball centers in the camera coordinate system at each pose through sphere fitting; using the obtained robot unit end poses and ball center camera coordinates corresponding to multiple poses, constructing a constraint equation based on the hand-eye transformation matrix; solving the constraint equation to obtain an initial estimation value of the hand-eye transformation matrix; defining an error model and constructing an error equation group based on the initial estimation value; iteratively solving the error equation group, updating the estimation of the hand-eye transformation matrix until a convergence condition is met, and obtaining a final optimized hand-eye transformation matrix. The present application uses the sphere center coordinates of the calibration ball point cloud data after sphere fitting and the robot end poses as inputs, and uses the hand-eye transformation matrix as the output, which is more accurate and more efficient than the prior art. The original equation set is constructed in the form of "the rotation matrix and the translation vector", unknowns are separated by using the properties of Kronecker product, the final solving equation set is constructed, then, the SVD singular value decomposition method is adopted to realize the synchronous solving of rotation and translation, so that the initial calibration result is obtained; in the optimization process of the initial calibration result, the error is approximated by the first-order Taylor expansion, the true value is represented by the approximate representation of the error and the initial calibration result, so that the error equation set is constructed and the error is solved, the calibration result is updated by mapping the error to the Lie group Lie algebra, and the iterative optimization is performed until the error converges, so that the final relatively accurate calibration result is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor under the premise of the drawings.

[0014] Figure 1 The structure schematic diagram of the fast and efficient hand-eye calibration system in the narrow scene of the present application; Figure 2 The schematic diagram of sphere fitting; Figure 3 The process schematic diagram of the fast and efficient hand-eye calibration method in the narrow scene of the present application.

[0015] The figure legend: 1, robot base; 2, end effector; 3, camera; 4, calibration ball. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0017] The application aims to provide a fast and efficient hand-eye calibration system and method in a narrow scene, using a calibration ball as a calibration target, and using only one calibration ball at multiple positions to perform initial value calculation and iterative optimization, solving the problem of difficulty in arranging a traditional calibration target in a narrow space, and the method is fast in the iterative process, balancing accuracy while solving the problem of complex and time-consuming iterative process.

[0018] In order to make the above-mentioned purposes, characteristics and advantages of the application more obvious and easy to understand, the application will be further described in detail below in combination with the drawings and specific embodiments.

[0019] As shown in Figure 1 , the application provides a fast and efficient hand-eye calibration system in a narrow scene, comprising a robot unit, a visual sensing unit, a calibration target unit and a processing and calculation unit. The robot unit comprises a robot base 1 and a robot arm, the robot arm is arranged on the robot base 1, and an end effector 2 is arranged on the end of the robot arm through an end effector mounting flange. The visual sensing unit comprises at least one camera 3 fixedly mounted on the end effector mounting flange, for collecting image or point cloud data. The calibration target unit comprises at least one calibration ball 4 fixedly placed in the robot workspace. The processing and calculation unit is in communication connection with the robot unit and the visual sensing unit, for receiving robot end pose data and data collected by the camera 3, and performing hand-eye calibration calculation.

[0020] The calibration ball 4 is a single standard ball.

[0021] The camera 3 is a structured light three-dimensional measurement binocular camera.

[0022] As shown in Figure 3 , the application also provides a fast and efficient hand-eye calibration method in a narrow scene, applied to the fast and efficient hand-eye calibration system in a narrow scene, using the coordinates of the center of the standard ball in the camera and the pose of the robot end as input, solving the initial calibration result of the robot hand-eye transformation and iteratively optimizing to obtain the accurate calibration result, the specific implementation steps are as follows: Control the robot unit to drive the camera to shoot the fixed calibration ball from multiple different poses, and obtain a series of robot end poses and corresponding calibration ball images or point cloud data, as shown in Figure 2 ; Process the calibration ball image or point cloud data, and obtain the coordinates of the center of the calibration ball in the camera coordinate system at each pose through ball fitting; The robot unit end position and the camera coordinates of the sphere center corresponding to the obtained multiple poses are used to construct a constraint equation based on a hand-eye transformation matrix. An initial estimate of the hand-eye transformation matrix is obtained by solving the constraint equation. Based on the initial estimate, an error model is defined and an error equation set is constructed. The error equation set is iteratively solved to update the estimate of the hand-eye transformation matrix until the convergence condition is met, and the final optimized hand-eye transformation matrix is obtained.

[0023] The calibration sphere image or point cloud data is processed to obtain the coordinates of the calibration sphere center in the camera coordinate system at each pose by sphere fitting, specifically: The calibration sphere data collected by the camera is processed, as shown in Figure 2 , sphere fitting is performed to obtain the coordinates of the sphere center in the camera, and the robot end effector poses at multiple positions are changed , ,… , and the coordinates of the calibration sphere center in the camera , ,… .

[0024] A constraint equation based on the hand-eye transformation matrix is constructed, specifically: For any two poses i and j, based on the fact that the position of the calibration sphere center in the robot base coordinate system is constant, the constraint equation is established as: (1) In the formula, is the robot hand-eye transformation matrix to be calibrated, , ; Ai represents the robot end effector pose, RAi represents the rotation part, tAi represents the translation part, Pi and Pj represent the coordinates of the calibration sphere center in the camera coordinate system when the robot end effector is in the i-th and j-th pose; Its matrix representation is: (2) Where , RX represents the rotation part, tX represents the translation part, and the expansion is: (3) An initial estimate of the hand-eye transformation matrix is obtained by solving the constraint equation, specifically: The equation is converted into a linear equation set using Kronecker product, specifically: Using the properties of Kronecker product: , where Represents the Kronecker product between two matrices. Indicates the matrix The column vector obtained by expanding in column-major order: , This represents the transpose of a matrix. Specifically: (4) in, , To express the transpose of point coordinates, extract the common factor: (5) The matrix is ​​represented as: (6) Construct an overdetermined system of equations matrix by adding data from multiple locations, and let the matrix be: , , ; The overdetermined system of equations is as follows: (7) Singular value decomposition is performed on the linear equation system to solve for the initial rotation matrix and initial translation vector of the hand-eye transformation matrix, thus obtaining the initial estimate, specifically: For the submatrix of equation (7) Perform SVD singular value decomposition: Solution vector From vector Extracting the initial rotation matrix and the initial translation vector ,in This indicates taking the m-th to n-th elements from the vector.

[0025] Based on the initial estimate, an error model is defined and a system of error equations is constructed as follows: Define the relationship between the true value and the initial estimate of the hand-eye transformation matrix, that is, define the error model as follows: It satisfies the following with the initial value: (8) In the formula, , These represent the rotation and translation components of the truth value of the hand-eye calibration matrix, respectively. , These represent the rotation and translation components of the initial hand value, respectively. , These represent the rotation error and translation error between the true value and the initial value, respectively. The rotation error is approximately a first-order Taylor expansion: ,in It is the identity matrix. For vectors The antisymmetric matrix; definition: ; definition: , ; in, Let x, y, and z represent the components of translation error and rotation error in the three directions, respectively. Then, in equation (8)... and Replace the part in equation (3) and have: (9) Expand and move items: (10) make have: (11) Substituting the error model relationship into the constraint equations and performing a first-order approximation, we derive a system of linear equations about the error vector, namely the error equation system, which is as follows: Using the properties of antisymmetric matrix and vector operations have: (12) Let matrix ,have: (13) The error equations are solved iteratively to update the estimate of the hand-eye transformation matrix until the convergence condition is met, thus obtaining the final optimized hand-eye transformation matrix, specifically: Solve the current set of error equations to obtain the rotation error estimate and translation error estimate for the current iteration; The rotation error is converted into an incremental rotation matrix using an exponential mapping, and the rotation matrix and translation vector are then updated. Determine whether the error has converged. If it has converged, output the current estimated value as the final result. Otherwise, use the current estimated value as the new initial value and repeat the iterative process until the convergence condition is met to obtain the final optimized hand-eye transformation matrix. The above process will be described in detail: (13) Solution vector Extract rotation and translation errors , , Logarithmic mapping to Lie algebra of three-dimensional vectors After adding the error, the exponent is mapped to a matrix to update the initial value: (14) replace the (8) with and replace the (8) with and , and keep updating until and the change converges (the change relative to the last iteration is within a threshold), and the accurate solution is obtained.

[0026] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be mutually referred to.

[0027] The principles and implementation manners of the present application are described by using specific examples in the specification. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A rapid and efficient hand-eye calibration system for confined spaces, characterized in that, include: Robot unit, vision sensing unit, calibration target unit, and processing and computing unit; The robot unit includes a robot base and a robot arm. The robot arm is mounted on the robot base, and an end effector is mounted on the end effector end of the robot arm via an end effector mounting flange. The visual sensing unit includes at least one camera fixedly mounted on the end effector mounting flange for acquiring images or point cloud data. The calibration target unit includes at least one calibration ball fixedly placed within the robot's workspace; The processing and computing unit is communicatively connected to the robot unit and the vision sensing unit, and is used to receive robot end pose data and data collected by the camera, and to perform hand-eye calibration calculations.

2. The system according to claim 1, characterized in that, The calibration ball is a single standard sphere.

3. The system according to claim 2, characterized in that, The camera is a structured light 3D measurement binocular camera.

4. A rapid and efficient hand-eye calibration method for confined spaces, applied to the rapid and efficient hand-eye calibration system for confined spaces as described in any one of claims 1-3, characterized in that, include: The robot unit controls the camera to take pictures of the stationary calibration ball from multiple different poses, and obtains a series of robot end poses and corresponding calibration ball images or point cloud data; Process calibration sphere images or point cloud data, and obtain the coordinates of the calibration sphere center in the camera coordinate system at each pose by sphere fitting; Using the robot unit end-effector poses and sphere center camera coordinates corresponding to the obtained multiple poses, constraint equations based on the hand-eye transformation matrix are constructed. Solve the constraint equations to obtain an initial estimate of the hand-eye transformation matrix; Based on the initial estimate, define the error model and construct a system of error equations; The error equations are solved iteratively to update the estimate of the hand-eye transformation matrix until the convergence condition is met, thus obtaining the final optimized hand-eye transformation matrix.

5. The method according to claim 4, characterized in that, The constraint equations based on the hand-eye transformation matrix are constructed as follows: For any two poses i and j, based on the fact that the position of the calibration sphere's center remains unchanged in the robot's base coordinate system, the constraint equations are established as follows: （1） In the formula, For the robot hand-eye transformation matrix that needs to be calibrated, , .

6. The method according to claim 5, characterized in that, Solve the constraint equations to obtain an initial estimate of the hand-eye transformation matrix, specifically: The equations are transformed into a system of linear equations using the Kronecker product. Singular value decomposition is performed on the linear equation system to solve for the initial rotation matrix and initial translation vector of the hand-eye transformation matrix, thus obtaining the initial estimate.

7. The method according to claim 6, characterized in that, Based on the initial estimate, an error model is defined and a system of error equations is constructed as follows: Define the relationship between the true value and the initial estimate of the hand-eye transformation matrix, that is, define the error model as follows: （2） The rotation error is approximated by a first-order Taylor expansion: ,in It is the identity matrix. For vectors The antisymmetric matrix; Substituting the error model relationship into the constraint equations and performing a first-order approximation, we derive a system of linear equations about the error vector, i.e., the error equation system.

8. The method according to claim 7, characterized in that, The error equations are solved iteratively to update the estimate of the hand-eye transformation matrix until the convergence condition is met, thus obtaining the final optimized hand-eye transformation matrix, specifically: Solve the current set of error equations to obtain the rotation error estimate and translation error estimate for the current iteration; The rotation error is converted into an incremental rotation matrix using an exponential mapping, and the rotation matrix and translation vector are then updated. Determine whether the error has converged. If it has converged, output the current estimated value as the final result. Otherwise, use the current estimated value as the new initial value and repeat the iterative process until the convergence condition is met to obtain the final optimized hand-eye transformation matrix.

Images