Hand-eye calibration method and device

Abstract

Some embodiments of the present application provide a hand-eye calibration method and device, which simplifies the installation of the calibration block, uses a two-dimensional graphical target map on the manipulator, and uses a three-dimensional point set composed of the graph center to solve the pose, thereby reducing the processing and installation precision requirements of the calibration block, and the difficulty of solving the conversion relationship of the calibration block. The method uses a target map instead of a calibration block, reduces the error generated by the calibration block, and solves the target rigid body transformation relationship through a three-dimensional point set matching algorithm, which can effectively reduce the time consumption compared with the point cloud registration algorithm. In the linear solution of the hand-eye standard equation, the rotation matrix is solved by using the rotation axis method, which reduces the algorithm solving difficulty and effectively avoids the solving abnormality and failure problem caused by the matrix form represented by the trigonometric function or the rotation angle. Under the premise of ensuring the calibration accuracy, the calibration process is simplified and convenient to maintain, solving the problem that the current hand-eye calibration method is not accurate enough and not convenient to operate.

Description

Technical Field

[0001] Some embodiments of this application relate to the field of hand-eye calibration technology, and in particular to a hand-eye calibration method and apparatus. Background Technology

[0002] With the continuous iteration of artificial intelligence and intelligent manufacturing technologies, the demand for high-precision automation is increasing, such as in applications like picking, palletizing, welding, dispensing, and assembly. Among these applications, high-precision vision guidance technology is a key link in the layout of intelligent systems, and hand-eye calibration is of paramount importance in vision guidance.

[0003] Currently, the commonly used hand-eye calibration method with the eye outside the hand involves using a 3D camera to photograph the calibration blocks on the manipulator in different poses, obtaining the conversion relationship between the calibration blocks and the camera through 3D feature point calibration, and then using the manipulator controller to directly obtain the conversion relationship between the manipulator and the base world reference, thereby obtaining the conversion relationship between the manipulator and the camera, thus achieving the purpose of hand-eye calibration.

[0004] However, the methods described above require high precision in the processing and installation of calibration blocks, and necessitate the design of a device for loading and unloading calibration blocks on the robotic arm, making them inconvenient to use and maintain. Furthermore, the conversion relationship between the calibration block and the camera is generally obtained through point cloud matching, which is computationally time-consuming and imposes limitations on the orientation of the calibration block. For example, point cloud matching algorithms have restrictions on the initial orientation, which affects the calibration results. Therefore, current hand-eye calibration methods are neither accurate enough nor convenient to operate. Summary of the Invention

[0005] Some embodiments of this application provide a hand-eye calibration method and apparatus that simplify the calibration process and facilitate maintenance while ensuring calibration accuracy, thereby solving the problems of insufficient accuracy and inconvenience in operation of current hand-eye calibration methods.

[0006] In a first aspect, some embodiments of this application provide a hand-eye calibration method, including:

[0007] Establish the camera coordinate system, the robot arm coordinate system, the base world reference coordinate system, and the target map coordinate system;

[0008] A first rigid body transformation relationship is established based on the manipulator coordinate system and the base world reference coordinate system;

[0009] A second rigid body transformation relationship is established based on the target image coordinate system and the camera coordinate system;

[0010] Obtain the third rigid body transformation relationship between the target image coordinate system and the robot arm coordinate system;

[0011] Establish rigid body transformation formulas based on the first rigid body transformation relationship, the second rigid body transformation relationship, and the third rigid body transformation relationship;

[0012] The target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system is solved based on the rigid body transformation formula.

[0013] In some embodiments, before performing the steps of establishing the camera coordinate system, the robot arm coordinate system, the base world reference coordinate system, and the target image coordinate system, the method further includes: mounting the camera in the base world reference coordinate system, and attaching the target image to the robot arm; the target image includes at least one two-dimensional graphic.

[0014] In some embodiments, the step of establishing a first rigid body transformation relationship based on the manipulator coordinate system and the base world reference coordinate system includes: controlling a camera to capture a target image mounted on the manipulator and storing the captured posture image;

[0015] Traverse the posture images; read and record the first rigid body transformation relationship between the manipulator coordinate system and the base world reference coordinate system based on the posture images.

[0016] In some embodiments, the step of controlling a camera to capture a target image mounted on a robotic arm and storing the captured posture image includes: controlling the robotic arm to move in different spatial postures; capturing the target image attached to the robotic arm and controlling the camera to capture the target image to generate a posture image; and storing the posture image.

[0017] In some embodiments, the step of establishing a second rigid body transformation relationship based on the target image coordinate system and the camera coordinate system includes: obtaining the center position of each two-dimensional graphic in the target image; constructing a reference feature point set in the camera coordinate system based on the center position; constructing a three-dimensional point set based on the reference feature point set; and calculating the second rigid body transformation relationship between the target image coordinate system and the three-dimensional point set in the camera coordinate system based on a three-dimensional point set matching algorithm.

[0018] In some embodiments, the step of obtaining the center position of each two-dimensional graphic in the target image includes: obtaining each two-dimensional graphic in the target image; parsing each two-dimensional graphic; and calculating the center position of each two-dimensional graphic based on the parsing results of each two-dimensional graphic.

[0019] In some embodiments, the step of constructing a reference feature point set in the camera coordinate system based on the center position of the circle includes: obtaining a plane with Z=0 in the camera coordinate system; and constructing a reference feature point set in the camera coordinate system based on the center position of the circle and the plane.

[0020] In some embodiments, the step of constructing a three-dimensional point set based on the reference feature point set includes: traversing the target images captured by the manipulator during different spatial posture movements; acquiring three-dimensional imaging information based on the plane and the target images; and constructing the three-dimensional point set based on the three-dimensional imaging information.

[0021] In some embodiments, the step of solving the target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system based on the rigid body transformation relationship includes: reading the rigid body transformation relationship to obtain the rotation axis and rotation angle corresponding to the rotation matrix based on the rigid body transformation relationship; calculating the correspondence between the rotation matrix and the rotation axis based on the rotation axis and the rotation angle; calculating the rotation matrix based on the covariance matrix, the least squares method, and the correspondence; and solving the target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system based on the rotation matrix and the identity matrix.

[0022] Secondly, some embodiments of this application provide a hand-eye calibration device applied to the hand-eye calibration method of the first aspect. The device includes a coordinate system establishment unit, a first rigid body transformation relationship establishment unit, a first rigid body transformation relationship establishment unit, a first rigid body transformation relationship establishment unit, a relationship establishment unit, and a target solving unit, comprising:

[0023] The coordinate system establishment unit is used to establish the camera coordinate system, the robot arm coordinate system, the base world reference coordinate system, and the target image coordinate system;

[0024] The first rigid body transformation relationship establishment unit is used to establish a first rigid body transformation relationship based on the manipulator coordinate system and the base world reference coordinate system;

[0025] The second rigid body transformation relationship establishment unit is used to establish a second rigid body transformation relationship based on the target map coordinate system and the camera coordinate system;

[0026] The third rigid body transformation relationship establishment unit is used to obtain the third rigid body transformation relationship between the target image coordinate system and the manipulator coordinate system;

[0027] The relation establishment unit is used to establish rigid body transformation relation based on the first rigid body transformation relation, the second rigid body transformation relation, and the third rigid body transformation relation;

[0028] The target solution unit is used to solve the target rigid body transformation relationship according to the rigid body transformation relationship.

[0029] As can be seen from the above technical solutions, some embodiments of this application provide an eye calibration method and apparatus. This method simplifies the installation of calibration blocks by using a two-dimensional target image attached to the robotic arm and solving the attitude problem using a three-dimensional point set formed by the image centers. This reduces the requirements for the processing and installation accuracy of the calibration blocks, as well as the difficulty of solving the transformation relationship of the calibration blocks. This method uses a target image instead of calibration blocks, reducing the errors generated by the calibration blocks. The target rigid body transformation relationship is solved by a three-dimensional point set matching algorithm, which can effectively reduce the time consumption compared with the point cloud registration algorithm. In the linear solution of the hand-eye standard equation, the rotation matrix is ​​solved by using a rotation axis method, which reduces the difficulty of the algorithm solution and effectively avoids the solution anomalies and failures caused by matrix representations such as trigonometric functions or rotation angles. While ensuring calibration accuracy, the calibration process is simplified and easy to maintain, solving the problems of insufficient accuracy and inconvenience of operation in current hand-eye calibration methods. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in some embodiments of this application or in the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This application provides hand-eye calibration diagrams for some embodiments.

[0032] Figure 2 This is a schematic flowchart of a hand-eye calibration method provided in some embodiments of this application;

[0033] Figure 3 A flowchart illustrating the process of establishing a first rigid body transformation relationship based on the manipulator coordinate system and the base world reference coordinate system for some embodiments of this application;

[0034] Figure 4 A flowchart illustrating the process of establishing a second rigid body transformation relationship based on the target map coordinate system and the camera coordinate system, provided for some embodiments of this application;

[0035] Figure 5 A flowchart illustrating the process of solving the target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system based on the rigid body transformation relationship for some embodiments of this application;

[0036] Figure 6 This is a schematic diagram of the hand-eye calibration device provided in some embodiments of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. The technical solutions provided by each embodiment of this application will be described in detail below with reference to the accompanying drawings.

[0038] In hand-eye calibration methods, "eye outside the hand" means the camera is fixed in one position to take pictures, and the robotic arm or moving parts do not move the camera. In this case, calibration requires capturing the entire field of view at once, and then the robotic parts make correspondences with the points in the image one by one before calibration. For example, a commonly used hand-eye calibration method with "eye outside the hand" involves using a 3D camera to photograph the calibration blocks on the robotic arm in different poses, obtaining the conversion relationship between the calibration blocks and the camera through 3D feature point calibration, and then using the robotic arm controller to directly obtain the conversion relationship between the robotic arm and the base world reference, thereby obtaining the conversion relationship between the robotic arm and the camera, thus achieving the purpose of hand-eye calibration.

[0039] To facilitate a further understanding of the hand-eye calibration principle, the hand-eye calibration process will now be described in detail with reference to specific diagrams and actual operation procedures. Figure 1 This application provides hand-eye alignment diagrams for some embodiments, such as... Figure 1 As shown, 01 is the calibration block, 02 is the robot arm, and 03 is the camera. The calibration block 01 is mounted on the robot arm 02. The pose transformation relationship between the robot arm's actuator coordinate system and the base (world reference coordinate system) can be directly read through the robot arm's control panel. Simultaneously, the point cloud of the calibration block is captured by the camera 03 (e.g., a 3D camera), and registered with the standard model point cloud to obtain the pose transformation relationship between the calibration block 01 and the camera 03. Based on the two transformation relationships under multiple movements of the robot arm 02 (as shown by the dotted lines in the figure, the robot arm 02 can move along the direction of the dotted lines), a standard hand-eye equation AX = XB is established. By solving this equation, the rigid body transformation relationship between the camera coordinate system and the base's world reference coordinate system, i.e., the rigid body transformation matrix, is obtained, thus achieving hand-eye calibration.

[0040] However, the aforementioned hand-eye calibration methods require high precision in the processing and installation of the calibration block 01, and necessitate the design of a device for loading and unloading the calibration block 01 on the robotic arm 02, making them inconvenient to use and maintain. Furthermore, the conversion relationship between the calibration block 01 and the camera 03 is generally obtained through point cloud matching, which is computationally time-consuming and imposes limitations on the orientation of the calibration block 01. For example, the point cloud matching algorithm has limitations on the initial orientation, which also affects the calibration results. Therefore, it is evident that current hand-eye calibration methods are not accurate enough and are not convenient to operate.

[0041] To address the issues of insufficient accuracy and inconvenience in current hand-eye calibration methods, this application provides a hand-eye calibration method in some embodiments. In this technical solution, the installation of the calibration block 01 is simplified. A target image composed of simple two-dimensional graphics such as circles, rings, triangles, polygons, and regular polygons is affixed to the robotic arm 02. The attitude is solved using a three-dimensional point set composed of the center of the image, such as the center of a circle, the center of a ring, the center of a triangle, or the center of a polygon. This reduces the accuracy requirements for the processing and installation of the calibration block, as well as the difficulty in solving the conversion relationship of the calibration block.

[0042] To facilitate understanding of the technical solutions in some embodiments of this application, the steps are described in detail below with reference to some specific embodiments and accompanying drawings. Figure 2 This is a schematic flowchart of a hand-eye calibration method provided in some embodiments of this application, such as... Figure 2 As shown, the hand-eye calibration method provided in this application may include the following steps S1-S6, specifically including:

[0043] S1: Establish the camera coordinate system, robot arm coordinate system, base world reference coordinate system, and target image coordinate system.

[0044] In some embodiments, before establishing the camera coordinate system, the robot arm coordinate system, the base world reference coordinate system, and the target image coordinate system, it is necessary to mount the camera in the base world reference coordinate system and attach the target image to the robot arm, wherein the target image includes at least one two-dimensional graphic.

[0045] For example, see [link to previous article] Figure 1 The camera can be a 3D camera, which is mounted in the base world reference coordinate system. The target image is fixed to the robot arm, and the robot arm coordinate system A, the base world reference coordinate system B, the camera coordinate system C, and the target image coordinate system T are established. After the above coordinate systems are established, the following step S2 can be executed.

[0046] S2: Establish the first rigid body transformation relationship based on the robot's coordinate system and the base's world reference coordinate system.

[0047] Figure 3 A flowchart illustrating the process of establishing a first rigid body transformation relationship based on the robot's coordinate system and the base's world reference coordinate system, as provided in some embodiments of this application, is shown below. Figure 3As shown, in some embodiments, the first rigid body transformation relationship can be established in the following manner: controlling a camera to capture a target image mounted on a robotic arm and storing the captured posture image. For example, the robotic arm can be controlled to move in different spatial postures to capture the target image attached to the robotic arm, and the camera can be controlled to capture the target image to generate a posture image, which is then stored. After storing the posture image, the posture images can be traversed, and the first rigid body transformation relationship between the robotic arm coordinate system and the base world reference coordinate system can be read and recorded based on the posture image.

[0048] For ease of description, the first rigid body transformation relationship between the manipulator coordinate system and the base world reference coordinate system is denoted as... In some embodiments, the first rigid body transformation relationship between the manipulator coordinate system and the base world reference coordinate system... This information can be read from the robot arm's control panel. For example, a 3D camera can be used to capture an image of the target mounted on the robot arm. During each movement, the first rigid body transformation relationship between the base's world reference coordinate system and the robot arm's coordinate system can be read and recorded via the robot arm's control panel. After step S2 is completed, step S3 can be executed.

[0049] S3: Establish the second rigid body transformation relationship based on the target image coordinate system and the camera coordinate system.

[0050] Figure 4 This application provides a flowchart illustrating the process of establishing a second rigid body transformation relationship based on the target map coordinate system and the camera coordinate system in some embodiments, such as... Figure 4 As shown, establishing the second rigid body transformation relationship based on the target image coordinate system and the camera coordinate system can include the following steps: First, obtain the center positions of each two-dimensional graphic in the target image. Then, construct a set of reference feature points in the camera coordinate system based on the center positions. Next, construct a three-dimensional point set based on the reference feature point set. Finally, calculate the second rigid body transformation relationship between the three-dimensional point set in the target image coordinate system and the camera coordinate system based on the three-dimensional point set matching algorithm.

[0051] The step of obtaining the center position of each two-dimensional graphic in the target image can be achieved through the following steps: first, obtain each two-dimensional graphic in the target image; then, analyze each two-dimensional graphic; and finally, calculate the center position of each two-dimensional graphic based on the analysis results. In the step of constructing the reference feature point set in the camera coordinate system based on the center position, first, obtain the plane with Z=0 in the camera coordinate system; then, construct the reference feature point set in the camera coordinate system based on the center position and the plane. In the step of constructing the three-dimensional point set based on the reference feature point set, first, traverse the target images captured by the robot during different spatial posture movements; then, obtain three-dimensional imaging information based on the plane and the target image; finally, construct the three-dimensional point set based on the three-dimensional imaging information. The following section further explains the above steps in conjunction with the actual operation steps and process principles.

[0052] For ease of description, in some embodiments, the second rigid body transformation relationship established between the target image coordinate system and the camera coordinate system can be denoted as Let the target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system be denoted as: The hand-eye relationship to be determined is the rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system. In the embodiments of this application, since the geometric figures on the target image are all two-dimensional figures, for example, they can be two-dimensional figures of certain fixed shapes pre-defined according to actual needs. For example, they can be circles, rings, triangles, polygons, regular polygons, etc. It is understood that the two-dimensional figures in the embodiments of this application are not limited to the figures listed above, and can also be other shapes, etc., which are not specifically limited in this application.

[0053] After obtaining the various two-dimensional graphics in the target image, the center position of each graphic can be determined by analyzing the graphics. For example, when the target image is circular, its center position is the center of the circle; when the target image is an annulus, its center position is the center of the annulus; when the target image is a triangle, its center position is the center of the triangle; when the target image is a polygon, its center position is the center of the polygon, and so on. When constructing the reference feature point set, the three-dimensional point set corresponding to the target image features on the Z=0 plane of the 3D camera can be used as the reference. This is equivalent to adding Z=0 to the center position of each two-dimensional graphic to form a three-dimensional point set. In this way, since the center of the two-dimensional graphics has already been calculated, the process of capturing the reference target image by the 3D camera can be reduced. Secondly, in the target image captured by the robot during different spatial posture movements, three-dimensional imaging information can be obtained based on the Z=0 plane and the target image, and then a three-dimensional point set can be constructed based on the three-dimensional imaging information. For example, the center of a circle or polygon and the set of three-dimensional points can be obtained using the three-dimensional circle and three-dimensional line segment features in the target image. Finally, the second rigid body transformation relationship of the three-dimensional point set in the target image coordinate system and the camera coordinate system can be calculated based on the three-dimensional point set matching algorithm.

[0054] In some embodiments, the 3D point set matching algorithm can be implemented as follows. Assume there is a 3D point set {x}. i},{y i}, where i = 1, 2, ..., n, n is a positive integer, and i represents the i-th point in the point set, {x i},{y i Let} represent the source point set and the target point set, respectively. The optimal matching objective function is:

[0055] Then, the optimal solution can be obtained by performing singular value decomposition on the covariance matrix of the two pairs of point sets:

[0056]

[0057]

[0058] Where, μ x ,μ y Let each be a centroid of a two-point set. Let Σ be the variance of the two point sets. xy Let be the covariance matrix of the source point set relative to the target point set.

[0059] The rotation matrix and translation vector of the source point set relative to the target point set can be obtained from the following formula:

[0060] R = USV T ,t=μ y -Rμ x ;

[0061] Where U,V is Σ xy The SVD decomposition result, S, can be selected according to the following formula:

[0062]

[0063] The three-dimensional point set matching algorithm can be implemented through the method described in the above embodiments. After step S3 is completed, step S4 can be executed.

[0064] S4: Obtain the third rigid body transformation relationship between the target image coordinate system and the robot arm coordinate system.

[0065] In some embodiments, since the target image is fixed to the manipulator, the third rigid body transformation relationship between the target image coordinate system and the manipulator coordinate system is constant. For ease of description, the attitude information under multiple movements and the constant third rigid body transformation relationship between the target image coordinate system and the manipulator coordinate system can be denoted as... After step S4 is completed, step S5 can be executed.

[0066] S5: Establish rigid body transformation formulas based on the first rigid body transformation relationship, the second rigid body transformation relationship, and the third rigid body transformation relationship.

[0067] In the above steps, the first rigid body transformation relationship between the manipulator coordinate system and the base world reference coordinate system has been obtained respectively. Second rigid body transformation relationship between target image coordinate system and camera coordinate system The third rigid body transformation relationship between the target coordinate system and the robot arm coordinate system is denoted as: The target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system is to be determined. The following rigid body transformation relationship can then be established:

[0068]

[0069] After the rigid body transformation relationship is established, the following step S6 can be executed.

[0070] S6: Solve the target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system based on the rigid body transformation relationship.

[0071] Figure 5 This application provides flowcharts illustrating the process of solving the target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system based on rigid body transformation relations in some embodiments, such as... Figure 5As shown, the target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system can be solved using the rigid body transformation formula as follows: First, read the rigid body transformation formula to obtain the rotation axis and rotation angle corresponding to the rotation matrix; then, calculate the correspondence between the rotation matrix and the rotation axis based on the rotation axis and rotation angle; after the correspondence is calculated, calculate the rotation matrix based on the covariance matrix, the least squares method, and the correspondence; finally, solve the target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system based on the rotation matrix and the identity matrix.

[0072] In some embodiments, the robotic arm can be controlled to move n times in different postures, where n is a positive integer. For example, to make the results more accurate, n ≥ 15 can be set, resulting in the following formula (2):

[0073]

[0074] By transforming the above formula (2), we can obtain the following formula (3);

[0075]

[0076] Formula (3) can be used to construct the standard hand-eye calibration equation system AX = XB, where i = 1, 2, ..., n-1:

[0077]

[0078] In some embodiments, during the process of solving the standard hand-eye calibration equations, the linear equations of the hand-eye equations can be solved, for example, including but not limited to the two-step method based on Park or Horaud, and used as the initial values ​​for nonlinear optimization methods, such as LM optimization, quasi-Newton iteration, etc. Finally, a high-precision rigid body transformation matrix is ​​obtained through nonlinear optimization.

[0079] After reading the rigid body transformation equation, the rotation axis and rotation angle corresponding to the rotation matrix can be obtained from the rigid body transformation equation. In some embodiments, this can be obtained from the homogeneous hand-eye calibration matrix equation AX = XB.

[0080]

[0081] The rotation matrix R corresponds to the rotation axis n and rotation angle θ as follows:

[0082]

[0083]

[0084] Where tr(R) represents the rank of the matrix, R(i,j) represents the value of the rotation matrix R at position (i,j), and nx, ny, and nz are the x, y, and z components of the rotation axis n, respectively.

[0085] Let the rotation matrix R A and rotation matrix R B The corresponding axis of rotation is n A and n B The relationship between Rx and the axis of rotation is (Equation (6)):

[0086] R x n B =n A

[0087] (6) Let the covariance matrix be:

[0088]

[0089] The rotation matrix obtained by least squares is as follows:

[0090]

[0091] From formula (5), we can see that,

[0092] (R A -I)t X =R X t B -t A

[0093] Formula (8)

[0094] Here, I is the identity matrix. Therefore, substituting into the rotation matrix R... X sum matrix R A t B The translation vector t can then be obtained using least squares. X Rotation matrix R X Translation vector t X Construct the target rigid transformation matrix:

[0095]

[0096] It should be noted that, in this embodiment, since a target image is used instead of a calibration block, the errors generated by the calibration block are greatly reduced, such as those related to machining accuracy, installation, and calibration block reference imaging. The target rigid body transformation relationship is solved by a three-dimensional point set matching algorithm, which can effectively reduce the time consumption compared to the point cloud registration algorithm. Secondly, in the linear solution of the hand-eye standard equation, the rotation matrix is ​​solved by using a rotation axis method, which reduces the difficulty of the algorithm solution and effectively avoids the solution anomalies and failures caused by matrix representations such as trigonometric functions or rotation angles. In this way, the calibration process is simplified and easy to maintain while ensuring calibration accuracy, solving the problems of insufficient accuracy and inconvenience of operation in the current hand-eye calibration methods.

[0097] As can be seen from the above technical solutions, the hand-eye calibration method provided in the above embodiments uses a target image instead of a calibration block, reducing the error generated by the calibration block. The target rigid body transformation relationship is solved by a three-dimensional point set matching algorithm, which can effectively reduce the time consumption compared with the point cloud registration algorithm. In the linear solution of the standard hand-eye equation, the rotation matrix is ​​solved by the rotation axis method, which reduces the difficulty of the algorithm solution and effectively avoids the solution anomalies and failures caused by the matrix form of trigonometric functions or rotation angles. Under the premise of ensuring calibration accuracy, the calibration process is simplified and easy to maintain, solving the problems of insufficient accuracy and inconvenience of operation of the current hand-eye calibration method.

[0098] In some embodiments, alternative solutions can be implemented for certain steps in the above-described hand-eye calibration method. For example, during the construction of the baseline feature point set and the 3D point set, the 2D imaging function of the 3D camera can be utilized to first locate the circle and line features in the 2D image, obtain the corresponding 2D center, and then directly obtain the 3D center coordinates using the mapping relationship between the 2D and 3D images in the 3D camera. This reduces the errors caused by 3D circle and 3D line segment features (generated by 3D camera imaging and 3D geometric fitting algorithms, mainly affected by noisy data), and the algorithms for finding circles and lines in 2D images are more accurate and less time-consuming than 3D algorithms, thereby improving calibration accuracy.

[0099] In some embodiments, for example, when solving the target rigid body transformation relationship, nonlinear optimization, such as Levenberg-Marquarelt (LM), gradient descent, quasi-Newton method, quadratic cone optimization, etc., can be added to compare with linear solutions to obtain results with smaller errors and to obtain more accurate hand-eye calibration results.

[0100] Based on the above-described hand-eye calibration method, some embodiments of this application also provide a hand-eye calibration device. Figure 6 Here are some schematic diagrams of the hand-eye calibration device provided in some embodiments of this application, such as... Figure 6As shown, the hand-eye calibration device includes a coordinate system establishment unit 01, a first rigid body transformation relationship establishment unit 02, a second rigid body transformation relationship establishment unit 03, a third rigid body transformation relationship establishment unit 04, a relationship establishment unit 05, and a target solution unit 06, wherein:

[0101] Coordinate system establishment unit 01 is used to establish the camera coordinate system, robot arm coordinate system, base world reference coordinate system, and target image coordinate system;

[0102] The first rigid body transformation relationship establishment unit 02 is used to establish the first rigid body transformation relationship based on the manipulator coordinate system and the base world reference coordinate system;

[0103] The second rigid body transformation relationship establishment unit 03 is used to establish the second rigid body transformation relationship based on the target map coordinate system and the camera coordinate system;

[0104] The third rigid body transformation relationship establishment unit 04 is used to obtain the third rigid body transformation relationship between the target image coordinate system and the robot arm coordinate system;

[0105] The relation establishment unit 05 is used to establish rigid body transformation relations based on the first rigid body transformation relation, the second rigid body transformation relation, and the third rigid body transformation relation;

[0106] The target solution unit 06 is used to solve the target rigid body transformation relationship based on the rigid body transformation relationship.

[0107] As can be seen from the above technical solutions, the hand-eye calibration device provided in the above embodiments uses a target image instead of a calibration block, reducing the error generated by the calibration block. The target rigid body transformation relationship is solved by a three-dimensional point set matching algorithm, which can effectively reduce the time consumption compared with the point cloud registration algorithm. In the linear solution of the standard hand-eye equation, the rotation matrix is ​​solved by using a rotation axis method, which reduces the difficulty of the algorithm solution and effectively avoids the solution anomalies and failures caused by matrix representations such as trigonometric functions or rotation angles. Under the premise of ensuring calibration accuracy, the calibration process is simplified and easy to maintain, solving the problems of insufficient accuracy and inconvenience of operation in the current hand-eye calibration methods.

[0108] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A hand-eye calibration method, characterized in that, The method comprises the following steps: establishing a camera coordinate system, a robot coordinate system, a base world reference coordinate system and a target map coordinate system; establishing a first rigid body transformation relationship according to the robot coordinate system and the base world reference coordinate system; establishing a second rigid body transformation relationship according to the target map coordinate system and the camera coordinate system; obtaining a third rigid body transformation relationship of the target map coordinate system and the robot coordinate system; establishing a rigid body transformation relationship formula according to the first rigid body transformation relationship, the second rigid body transformation relationship and the third rigid body transformation relationship; solving a target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system according to the rigid body transformation relationship formula; Before the step of establishing a camera coordinate system, a robot coordinate system, a base world reference coordinate system and a target map coordinate system, the method further comprises the following steps: installing a camera under the base world reference coordinate system, and pasting a target map on a robot; the target map comprises at least one two-dimensional pattern; The step of establishing a second rigid body transformation relationship according to the target map coordinate system and the camera coordinate system comprises the following steps: obtaining the center positions of each two-dimensional pattern in the target map; constructing a reference feature point set under the camera coordinate system according to the center positions; constructing a three-dimensional point set according to the reference feature point set; calculating a second rigid body transformation relationship of the three-dimensional point set under the target map coordinate system and the camera coordinate system based on a three-dimensional point set matching algorithm; When constructing the reference feature point set, the three-dimensional point set is constructed by taking the three-dimensional point set corresponding to the target map features on the Z=0 plane of the 3D camera as the reference, and adding Z=0 on the basis of the center positions of each two-dimensional pattern; three-dimensional imaging information is obtained based on the plane of Z=0 and the target map in the target map photographed by the robot in different spatial postures, and a three-dimensional point set is constructed according to the three-dimensional imaging information.

2. The hand-eye calibration method of claim 1, wherein, The step of establishing a first rigid body transformation relationship according to the robot coordinate system and the base world reference coordinate system comprises the following steps: controlling the camera to photograph the target map installed on the robot and storing the photographed posture pictures; traversing the posture pictures; reading and recording the first rigid body transformation relationship of the robot coordinate system and the base world reference coordinate system according to the posture pictures.

3. The hand-eye calibration method of claim 2, wherein, The step of controlling the camera to photograph the target map installed on the robot and storing the photographed posture pictures comprises the following steps: controlling the robot to move in different spatial postures; capturing the target map pasted on the robot, and controlling the camera to photograph the target map to generate posture pictures; storing the posture pictures.

4. The hand-eye calibration method of claim 1, wherein, The step of obtaining the center positions of each two-dimensional pattern in the target map comprises the following steps: obtaining each two-dimensional pattern in the target map; analyzing each two-dimensional pattern; calculating the center positions of each two-dimensional pattern according to the analysis results of each two-dimensional pattern.

5. The hand-eye calibration method of claim 1, wherein, The step of solving a target rigid body transformation relationship between the camera coordinate system and the base world reference coordinate system according to the rigid body transformation relationship formula comprises the following steps: reading the rigid body transformation relationship formula to obtain a rotation axis and a rotation angle corresponding to a rotation matrix according to the rigid body transformation relationship formula; calculating the corresponding relationship between the rotation matrix and the rotation axis according to the rotation axis and the rotation angle; calculating the rotation matrix based on a covariance matrix, a least square method and the correspondence; solving a target rigid transformation relationship between the camera coordinate system and the base world reference coordinate system according to the rotation matrix and a unit matrix.

6. A hand-eye calibration device, the device being suitable for use in the method of any one of claims 1 to 5, the device comprising a coordinate system establishing unit, a first rigid body transformation relationship establishing unit, a second rigid body transformation relationship establishing unit, a third rigid body transformation relationship establishing unit, a relationship establishing unit and a target solving unit, characterized in that, comprising: the coordinate system establishing unit is configured to establish a camera coordinate system, a manipulator coordinate system, a base world reference coordinate system and a target map coordinate system; the first rigid transformation relationship establishing unit is configured to establish a first rigid transformation relationship according to the manipulator coordinate system and the base world reference coordinate system; the second rigid transformation relationship establishing unit is configured to establish a second rigid transformation relationship according to the target map coordinate system and the camera coordinate system; the third rigid transformation relationship establishing unit is configured to obtain a third rigid transformation relationship between the target map coordinate system and the manipulator coordinate system; the relationship establishing unit is configured to establish a rigid transformation relationship according to the first rigid transformation relationship, the second rigid transformation relationship and the third rigid transformation relationship; the target solving unit is configured to solve a target rigid transformation relationship according to the rigid transformation relationship.

Images