An automatic hand-eye calibration method for Delta robots

Abstract

The present invention relates to an automatic hand-eye calibration method for a Delta robot. The calibration plate is located on a mobile platform of the Delta robot and has a color dot matrix. The method comprises: (1) positioning the center of the mobile platform above the starting point of a working plane; (2) obtaining a vertical distance from the calibration plate to the working plane and a vertical distance from a Delta robot coordinate system to the mobile platform; (3) automatically moving the center of the mobile platform of the Delta robot along a certain path and stopping at each key point, and recording the corresponding image and space coordinates respectively; (4) a. sequentially identifying the color dot matrix in a plurality of images and calculating the image coordinates of the center of the color dot matrix; b. converting the space coordinates of the center of the mobile platform when it moves to each key point into the coordinates of the center of the color dot matrix; (5) using the obtained space coordinates and image coordinates of the center of the color dot matrix to form a plurality of point pairs, and estimating a mapping matrix H. The operation process of hand-eye calibration is simplified, thereby improving work efficiency.

Description

Technical Field

[0001] The present application belongs to the technical field of robot vision calibration, and specifically relates to an automatic hand-eye calibration method for a Delta robot. Background Art

[0002] Robot hand-eye calibration has always been a hot research direction. Now there are many calibration methods. In the field of industrial applications, the traditional calibration method based on the calibration plate and the nine-point calibration method are the most widely used calibration methods. For Delta robots, usually only the nine-point calibration method can be used. First, take a picture of the scene with a camera to obtain the coordinates of the target object in the image, and then transform the xy coordinates of the corresponding robot coordinate system through the mapping matrix H. The mapping matrix here is the value that needs to be calibrated by the hand-eye calibration method. The usual method is to set multiple calibration points on the working plane to obtain the coordinates of its image, then move the center of the mobile platform of the Delta robot, align the center of the robot mobile platform with the calibration point and extract the corresponding robot coordinate xy component to form a matching pair of <image coordinate point, robot coordinate xy component>, and finally estimate the mapping matrix H through the corresponding algorithm. This method requires manual control and alignment, so the hand-eye calibration time is long, the efficiency is relatively low, and the accuracy cannot be guaranteed; it also requires relatively complex preparation work and manual teaching. Summary of the invention

[0003] Purpose of the present invention: In order to overcome the defects of the prior art, the present invention provides an automatic hand-eye calibration method for a Delta robot, which simplifies the operation process of hand-eye calibration and thus improves work efficiency.

[0004] The technical solution of the present invention is: an automatic hand-eye calibration method for a Delta robot, which is implemented based on a camera and a calibration plate. The calibration plate is located on a mobile platform of the Delta robot, has a color dot matrix, and a gripper is provided at the center of the mobile platform. The offset from the center of the mobile platform to the center of the color dot matrix in the Delta robot coordinate system is , the method comprising:

[0005] (1) Move the Delta robot so that the center of the mobile platform is above the starting point of the working plane;

[0006] (2) The vertical distance from the calibration plate to the working plane is measured to be h cw At the same time, the vertical distance from the Delta robot coordinate system to the mobile platform is obtained through the robot control program as h bc ;

[0007] (3) The center of the Delta robot's mobile platform automatically moves from the starting point of the working plane to the top of its end point along a certain path. The Delta robot stops at each key point during the movement and records the corresponding images. and the spatial coordinates of the center of the robot mobile platform are , where N corresponds to the number of key points;

[0008] (4) a. For multiple images in sequence The color dot matrix in the image is identified by first identifying the image coordinates of the centers of multiple individual color dots, and then calculating the image coordinates of the center of the color dot matrix. ;

[0009] b. Convert the spatial coordinates of the center of the mobile platform when it moves to each key point into the spatial coordinates of the center of the color dot matrix in the Delta robot coordinate system ;

[0010] (5) Use the spatial coordinates and image coordinates of the center of the color dot matrix to form multiple point pairs , and estimate the mapping matrix H;

[0011] (6) Detect the coordinates of the target object on the working plane in the Delta robot coordinate system:

[0012] 6.1 Define a coordinate system with the camera as the origin, that is, the camera coordinate system. This coordinate system is only a position offset of the Delta robot coordinate system. ;

[0013] 6.2 Detecting the coordinates of any given point of the target image on the working plane , under camera projection geometry, the given point is the projection of the center of the target object and the projection of the center of a color matrix;

[0014] 6.3 Through the mapping matrix H Transformed into the corresponding spatial coordinates of the center of the color matrix in the Delta robot coordinate system , and then convert to the coordinates in the camera coordinate system ;

[0015] 6.4 In camera projection geometry, starting from the origin of the camera coordinate system and passing through the center of the color matrix (Delta robot coordinate system ) intersects the working plane at one point , the intersection point and the center of the color dot matrix are the same point on the camera image. Since the coordinates of the target on the working plane in the camera coordinate system are and At the same location, the projection ray passes through Restore the coordinates of the target object on the working plane in the Delta robot coordinate system.

[0016] By adopting the above technical solution, the mapping matrix H is estimated by using the key point image and the spatial coordinate point pair, and the mapping relationship between the image coordinates of the center of the color dot matrix and the coordinates of the center of the robot mobile platform is established; the image coordinates obtained by detecting the target object on the working plane can be used to infer its position in the robot coordinate system through geometric relationships. The automated method proposed by the present invention for the characteristics of the Delta robot simplifies the operation process of hand-eye calibration and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a structural diagram of a Delta robot mobile platform with a calibration plate in a specific embodiment of the present invention;

[0018] Figure 2 It is a geometric relationship diagram on the YZ coordinate plane of the Delta robot in a specific embodiment of the present invention;

[0019] Figure 3 This is a flow chart of hand-eye calibration in a specific embodiment of the present invention;

[0020] Figure 4 It is a flow chart of target object detection on a working plane in a specific embodiment of the present invention.

[0021] In the figure: mobile platform 1, calibration plate 2, grabbing part 3. DETAILED DESCRIPTION

[0022] The present invention discloses an automatic hand-eye calibration method for a Delta robot. The method is implemented based on a camera and a calibration plate. The calibration plate is fixed or integrally arranged on a mobile platform 1 of the Delta robot. The calibration plate 2 has a color dot matrix. The color dot matrix has multiple color dots and is distributed in a 3*3 dot matrix. The color dots include but are not limited to red, green and yellow. A gripper 3 is arranged at the center of the mobile platform 1. The gripper is a suction cup or a clamp. The offset from the center of the mobile platform to the center of the color dot matrix is ​​precisely fixed in the Delta robot coordinate system. , so knowing the position of the center of the mobile platform, we can know the position of the center of the color matrix.

[0023] The method comprises:

[0024] (1) Adjust the Delta robot or the working plane so that the Delta robot is parallel to the working plane and the distance from the center of the mobile platform to the working plane remains constant when the Delta robot moves horizontally.

[0025] (2) The camera is installed at the same height as the top of the Delta robot. Its position is such that the image it captures just covers the working area of ​​the Delta robot. The positional relationship between the camera and the Delta robot is precisely fixed. The camera is calibrated, that is, the image captured by it has no distortion or has minimized distortion.

[0026] (3) The Delta robot moves so that the center of the mobile platform is above the starting point (upper left corner) of the working plane, and the gripper moves to a position very close to the working plane, with the distance controlled to less than 1 cm;

[0027] (4) Measure the vertical distance h from the calibration plate to the working plane using a vernier caliper or ruler. cw At the same time, the vertical distance from the Delta robot coordinate system to the mobile platform is obtained through the robot control program as h bc ;

[0028] (5) The center of the Delta robot's mobile platform automatically moves from the starting point of the working plane to the end point (lower right corner) along a certain path. The Delta robot stops at each key point during the movement and records the corresponding images. and the spatial coordinates of the center of the robot mobile platform are , where N corresponds to the number of key points;

[0029] Among them, multiple key points are distributed in a 3*3 dot matrix, denoted as

[0030]

[0031] The moving path of the center of the mobile platform of the Delta robot is P11, P12, P13, P21, P22, P23, P31, P32, P33 or P11, P12, P13, P23, P22, P21, P31, P32, P33;

[0032] (6) a. For each of the 9 images The color dot matrix in the image is identified by first identifying the image coordinates of the centers of the 9 individual color dots, and then calculating the image coordinates of the center of the color dot matrix. , the calculation formula is as follows:

[0033] ;

[0034] b. Convert the spatial coordinates of the center of the mobile platform when it moves to each key point into the spatial coordinates of the center of the color dot matrix in the Delta robot coordinate system , the conversion formula is as follows: ;

[0035] (7) Use the obtained spatial coordinates and image coordinates of the center of the color dot matrix to form multiple point pairs , use OpenCV's findHomography function to estimate the mapping matrix H, satisfying

[0036] , where the definition , is the calculated homogeneous coordinate term, so that

[0037] ;

[0038] (8) Detect the coordinates of the target object on the working plane in the Delta robot coordinate system:

[0039] 8.1 Define a coordinate system with the camera as the origin, namely the camera coordinate system. This coordinate system is only a position offset of the Delta robot coordinate system. ;

[0040] 8.2 Homogeneous coordinates of any given point of the target image detected on the working plane , under camera projection geometry, the given point is the projection of the center of the target object and the projection of the center of a color matrix;

[0041] 8.3 Through the mapping matrix H Transformed into XY axis homogeneous coordinates in the Delta robot coordinate system , the corresponding color dot center space coordinates ,in , and then convert to the coordinates in the camera coordinate system , is the position of the center of the color matrix in the camera coordinate system;

[0042] 8.4 In camera projection geometry, starting from the origin of the camera coordinate system and passing through the center of the color matrix (Delta robot coordinate system ) intersects the working plane at one point , the intersection point and the center of the color dot matrix are the same point on the camera image. Since the coordinates of the target on the working plane in the camera coordinate system are and At the same location, the projection ray, , , calculated through geometric proportion relationship:

[0043]

[0044]

[0045] , where the sign function takes the positive or negative sign of the value,

[0046] Afterwards through Restore the coordinates of the target object on the working plane in the Delta robot coordinate system.

[0047] The present invention accurately fixes the calibration point on the mobile platform of the Delta robot in the form of a color dot matrix (the center of the color dot matrix is ​​the calibration point). When performing hand-eye calibration, the mobile platform is moved as close to the working plane as possible. By moving the mobile platform, the image coordinates and robot coordinates of the calibration point can be obtained at the same time, and then the mapping matrix H is calculated;

[0048] The mapping matrix H calculated in this way is not essentially the correspondence between the image points and the calibration points on the working plane, but the relationship between the image points and the calibration points on the plane slightly offset from the working plane. Therefore, when using it, the resulting robot coordinates need to be further converted to the working plane through geometric relationships.

[0049] Since the color dot matrix is ​​fixed on the robot's mobile platform, you only need to set the starting and ending points of the center coordinates of the robot's mobile platform within the camera's visible range, and the robot can automatically complete the calibration work.

Claims

1. An automatic hand-eye calibration method for a Delta robot, characterized in that: The method is implemented based on a camera and a calibration plate. The calibration plate is located on the mobile platform of the Delta robot. The calibration plate has a color dot matrix. A gripper is provided at the center of the mobile platform. The offset from the center of the mobile platform to the center of the color dot matrix in the Delta robot coordinate system is R t c =[ R x c , R y c , R z c ], the method comprising: (1) The Delta robot moves so that the center of the mobile platform is above the starting point of the working plane; (2) The vertical distance from the calibration plate to the working plane is measured to be h cw At the same time, the vertical distance from the Delta robot coordinate system to the mobile platform is obtained through the robot control program as h bc ; (3) The center of the mobile platform of the Delta robot is automatically moved from the starting point of the working plane to the top of the end point according to a certain path. The Delta robot stops at each key point during the movement and records the corresponding images as I i , i = 1, ..., N and the spatial coordinates of the center of the robot mobile platform are R p i =[ R x i , R y i , R z i ] T , i = 1, ..., N, where N corresponds to the number of key points; (4)a. Sequentially process multiple images I i , i = 1, ..., N, the color dot matrix is ​​identified, the image coordinates of the center of multiple individual color dots are first identified, and then the image coordinates of the center of the color dot matrix are calculated I p i ; b. Convert the spatial coordinates of the center of the mobile platform when it moves to each key point into the spatial coordinates of the center of the color dot matrix in the Delta robot coordinate system R p′ i ; (5) Use the obtained spatial coordinates and image coordinates of the center of the color dot matrix to form multiple point pairs ( I p i , R p′ i ), i = 1, ..., N, and estimate the mapping matrix H.

2. The automatic hand-eye calibration method for a Delta robot according to claim 1, characterized in that: Multiple key points are distributed in a 3*3 dot matrix, denoted as The starting point on the working plane is the upper left corner, the end point is the lower right corner, and the moving path of the center of the mobile platform of the Delta robot is p 11 , p 12 , p 13 , p 21 , p 22 , p 23 , p 31 , p 32 , p 33 or p 11 , p 12 , p 13 , p 23 , p 22 , p 21 , p 31 , p 32 , p 33 .

3. The automatic hand-eye calibration method for a Delta robot according to claim 1, characterized in that: The calculation formula in step 4a is as follows:

4. The automatic hand-eye calibration method for a Delta robot according to claim 1, characterized in that: The conversion formula in step 4b is as follows: R p′ i =[ R x′ i , R y′ i , R z′ i ] T = R p i + R t c 。 5. The automatic hand-eye calibration method for a Delta robot according to claim 1, characterized in that: In step 5, OpenCV's findHomography function is used to estimate the mapping matrix H, satisfying where the definition w i is the calculated homogeneous coordinate term, so that 6. The automatic hand-eye calibration method for a Delta robot according to claim 1, characterized in that: It also includes step 6 to detect the coordinates of the target object on the working plane in the Delta robot coordinate system: 6.1 Define a coordinate system with the camera as the origin, namely the camera coordinate system, which is only a position offset t of the Delta robot coordinate system cam =[x c ,y c , z c ]; 6.2 Homogeneous coordinates of any given point of the target image detected on the working plane Under camera projection geometry, the given point is the projection of the center of the target object and the projection of the center of a certain color matrix; 6.3 Through the mapping matrix H Transformed into the corresponding spatial coordinates of the center of the color matrix in the Delta robot coordinate system R p, and then convert to the coordinates in the camera coordinate system c p; 6.4 In camera projection geometry, the center coordinates of the color dot matrix starting from the origin of the camera coordinate system and passing through the Delta robot coordinate system are R The ray of p intersects the working plane at one point R p′, the intersection point and the center of the color dot matrix are the same point on the camera image. Since the coordinates of the target on the working plane in the camera coordinate system c p′ and c p at the same location as the projection ray, through R p′= c p′+t cam Restore the coordinates of the target object on the working plane in the Delta robot coordinate system.

7. The automatic hand-eye calibration method for a Delta robot according to claim 6, characterized in that: Homogeneous coordinates in step 6.2 Transformed into XY axis homogeneous coordinates in the Delta robot coordinate system through the mapping matrix H Its corresponding spatial coordinates R p=[ R x, R y, R z] = [x / w, y / w, R z], where R z=-h bc .

8. The automatic hand-eye calibration method for a Delta robot according to claim 6, characterized in that: Step 6.3 c p= R pt cam .

9. The automatic hand-eye calibration method for a Delta robot according to claim 6, characterized in that: In step 6.4 c p′=[ c x′, c y′, c z′], c p=[ c x, c y, c z] are the positions of the target object and the center of the color dot matrix on the working plane in the camera coordinate system, which are calculated through the geometric proportional relationship as follows: c z′=-(h bc +h cw ), where the sign function takes the sign of the value.

Images