Robot tool system calibration method based on image recognition

Abstract

The invention discloses a robot tool system calibration method based on image recognition, belongs to the technical field of image data processing, and aims to solve the problem of low tool system calibration precision caused by an unknown space pose relationship between an auxiliary tool and a tool system in an existing tool system calibration method. S2, feature points under a # imgabs0 # system are extracted; s3, solving a relative pose matrix of the # imgabs 1 # system relative to the # imgabs 2 # system; s4, solving the pose relation of the tool system relative to the tail end of the robot; according to the method, an industrial camera is adopted to shoot a tool system image, feature points are extracted through Hough transformation, external parameters of the camera are determined through scene calibration, a hand-eye equation is established, and a hand-eye calibration Tsai two-step method is adopted to solve the space pose relation between the tool system and the tail end of the robot. The problem that the tool system calibration precision is low due to the fact that the space pose relation between the auxiliary tool and the tool system is unknown is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image data processing, and in particular relates to a robot tool system calibration method based on image recognition. Background Art

[0002] In the actual application of robots, whether it is precision assembly and welding on industrial production lines or cargo handling in logistics warehousing, accurate calibration of the robot tool system is a key prerequisite for ensuring the quality of the operation. The calibration accuracy of the tool system directly affects the interaction accuracy between the robot's end effector and the target object.

[0003] At present, traditional robot tool system calibration methods mainly include mechanical measurement methods and sensor-based methods. Mechanical measurement methods usually require the use of high-precision measuring tools, such as micrometers, laser interferometers, etc., to measure and adjust each parameter of the robot tool system one by one. Although this method has high accuracy, it is cumbersome to operate, time-consuming and labor-intensive, and requires high professional skills from the operator. Moreover, in the actual production environment, frequent use of mechanical measurement tools for calibration will have a significant impact on the production schedule and increase the production costs of the enterprise.

[0004] The Chinese patent publication number is "CN114227678A", and the patent name is "A robot arm TCP tool coordinate system calibration method and device". The method provides a robot arm TCP tool coordinate system calibration method and device, including: clamping the feature recognition sphere on the welding wire of the robot arm, and rotating the feature recognition sphere until the second white mark on the feature recognition sphere points to the preset x-axis of the robot arm TCP coordinate system; obtaining the laser point cloud data on the surface of the feature recognition sphere, and determining the robot arm TCP coordinate system based on the laser point cloud data; obtaining the conversion matrix between the robot coordinate system and the robot arm TCP coordinate system, and using the conversion matrix as the robot arm TCP tool coordinate system. The shortcomings of this method are: the feature sphere is clamped on the welding wire of the robot arm, and the feature sphere is used as an auxiliary tool. There is no method for determining the spatial posture relationship between the auxiliary tool and the robot arm, so the spatial posture error will be brought into the tool system calibration, reducing the tool system calibration accuracy. Summary of the invention

[0005] In order to solve the problem of low tool system calibration accuracy caused by unknown spatial posture relationship between auxiliary tools and tool system in existing tool system calibration methods, the present invention proposes a robot tool system calibration method based on image recognition.

[0006] The technical solution of the present invention to solve the technical problem is as follows:

[0007] A robot tool system calibration method based on image recognition, the method comprising the following steps:

[0008] Step S1: Establishing a measurement system;

[0009] Build on the robot System, established on the end of the robot The tool system is fixed to the end of the robot and a System, set up on the camera Coordinate System and Pixel Coordinate System System, camera shooting tool system Relative The system is fixed and does not move. For the Second shooting time Relative to The relative pose matrix of the system, for The inverse matrix of For the Second shooting time Relative to The relative pose matrix of the system, for The inverse matrix of Indicates Shooting, .

[0010] Step S2: Extraction of feature points under the system;

[0011] S21: Define feature points The coordinates of the system;

[0012] definition The center point of the circle is taken as the feature point , set the feature point for System origin, characteristic circle radius is a known quantity, define 0° on the characteristic circle, and The points at the positions are respectively used as feature points , and ,angle and If the angle is known, then , , , exist Coordinates of the system , , , ;

[0013] S22: image and robot end position data acquisition;

[0014] Control the robot to move the tool system into the camera's shooting range. Shot contains , , , The image obtained when the feature point is , in The corresponding robot end position data is collected simultaneously by shooting , Represents the robot end position coordinates, Represents the robot's terminal posture angle, repeat the above steps n times to obtain n sets of images and robot terminal posture data;

[0015] S23: Extract feature points The pixel coordinate value in the system;

[0016] point is the optical center of the camera, point For point exist The projection of the system, As a known quantity, Hough transform is used to extract the image The feature points in , , and , , , and They are , , , The pixel feature points corresponding to the feature points are then obtained using the OpenCV image processing algorithm. , , and exist Pixel coordinate values ​​in the system , , and ;

[0017] Step S3: Solving Relative to The relative pose matrix of the system;

[0018] Calculate the first Second shooting time Relative to The relative pose matrix of the system ;

[0019] Step S4: solving the position and posture relationship between the tool system and the robot end;

[0020] The relative position relationship between the tool system and the robot end is obtained using the Tsai two-step method of hand-eye calibration .

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The present invention adopts an industrial camera to shoot the tool system image, extracts feature points through Hough transform, determines the external parameters of the camera through scene calibration, establishes the hand-eye equation and adopts the hand-eye calibration Tsai two-step method to solve the spatial posture relationship between the tool system and the robot end. This method does not use auxiliary tools, thus avoiding the problem of low calibration accuracy of the tool system due to the unknown spatial posture relationship between the auxiliary tools and the tool system. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a robot tool system calibration method based on image recognition according to the present invention;

[0024] Figure 2 A schematic diagram of the principle of a robot tool system calibration method based on image recognition according to the present invention;

[0025] Figure 3 A schematic diagram of feature point extraction of a robot tool system calibration method based on image recognition according to the present invention;

[0026] Figure 4 A schematic diagram of feature point coordinate solution of a robot tool system calibration method based on image recognition according to the present invention;

[0027] Figure 5 Schematic diagram of feature point extraction in an embodiment of the present invention;

[0028] In the figure: 1. Robot; 2. Robot end; 3. Tool system; 4. Camera; 5. Feature circle. DETAILED DESCRIPTION

[0029] The present invention is further described in detail below with reference to the accompanying drawings.

[0030] like Figure 1 As shown, a robot tool system calibration method based on image recognition, the specific steps of the method are as follows:

[0031] Step S1: Establishing a measurement system;

[0032] like Figure 2 As shown, establish on robot 1 System, established on the robot end 2 The tool system 3 is fixed to the robot end 2, and the tool system 3 is established on the tool system 3. System, established on camera 4 Coordinate System and Pixel Coordinate System System, Camera 4 Shooting Tool System 3 Relative The system is fixed and does not move. For the Second shooting time Relative to The relative pose matrix of the system, for The inverse matrix of For the Second shooting time Relative to The relative pose matrix of the system, for The inverse matrix of Indicates Shooting, ;

[0033] Step S2: Extraction of feature points under the system;

[0034] S21: Define feature points The coordinates of the system;

[0035] like Figure 3 As shown, the definition The center point of the circle is taken as the feature point , set the feature point for Origin, characteristic circle 5 radius As a known quantity, define 0° on characteristic circle 5, and The points at the positions are respectively used as feature points , and ,angle and If the angle is known, then , , , exist Coordinates of the system , , , Determined by formulas (1), (2), (3), and (4) respectively:

[0036] (1)

[0037] (2)

[0038] (3)

[0039] (4);

[0040] S22: image and robot end position data acquisition;

[0041] Control robot 1 to move tool system 3 into the shooting range of camera 4. Shot contains , , , The image obtained when the feature point is , in The corresponding robot end position data is collected simultaneously by shooting , Represents the robot end position coordinates, Represents the robot's terminal posture angle. Repeat the above steps n times to obtain n sets of images and robot terminal posture data.

[0042] S22: Extract feature points The pixel coordinate value in the system;

[0043] like Figure 4 As shown, point is the optical center of camera 4, point For point exist The projection under the system, Coordinates of the system As a known quantity, Hough transform is used to extract the image The feature points in , , and , , , and They are , , , The pixel feature points corresponding to the feature points are then obtained using the OpenCV image processing algorithm. , , and exist Pixel coordinate values ​​in the system , , and .

[0044] Step S3: Solving Relative to The relative pose matrix of the system;

[0045] Use formula (5) to calculate the Second shooting time Relative to The relative pose matrix of the system .

[0046] (5)

[0047] In formula (5): is the camera's internal parameter, is a known quantity, , for The X-axis component of the camera focal length, for The Y-axis component of the camera's focal length, and is a known quantity, For point exist The X-axis coordinate value of the system, For point exist The Y-axis coordinate value of the system, and is a known quantity, , , .

[0048] Step S4: solving the position and posture relationship between the tool system and the robot end;

[0049] Use the hand-eye calibration Tsai two-step method to solve formula (6) and get The relative position relationship between the tool system 3 and the robot end 2 during the first shooting .

[0050] (6)

[0051] In formula (6): , , , , , For the The rotation matrix of the robot end position during the first shot, is the rotation matrix around the X axis, is the rotation angle around the X axis, is the rotation matrix around the Y axis, is the rotation angle around the Y axis, is the rotation matrix around the Z axis, is the rotation angle around the Z axis, For the The translation matrix of the robot end position during the first shot.

[0052] Example:

[0053] A robot tool system calibration method based on image recognition, such as Figure 1 As shown, the specific steps of this method are as follows:

[0054] Step S1: Establishing a measurement system;

[0055] like Figure 2 As shown, establish on robot 1 System, established on the robot end 2 The tool system 3 is fixed to the robot end 2, and the tool system 3 is established on the tool system 3. System, established on camera 4 Coordinate System and Pixel Coordinate System System, Camera 4 Shooting Tool System 3 Relative The system is fixed and does not move. For the Second shooting time Relative to The relative pose matrix of the system, for The inverse matrix of For the Second shooting time Relative to The relative pose matrix of the system, for The inverse matrix of Indicates Shooting, ;

[0056] The robot 1 in this embodiment is the EFORT GR680 robot, the tool system 3 is a spray gun, and the camera 4 is the Huarui Technology industrial camera 12CG-E.

[0057] Step S2: Extraction of feature points under the system;

[0058] S21: Define feature points The coordinates of the system;

[0059] like Figure 3 As shown, the definition The center point of the circle is taken as the feature point , set the feature point for Origin, characteristic circle 5 radius , define 0° on characteristic circle 5, and The points at the positions are respectively used as feature points , and , , but , , , exist Coordinates of the system , , , Determined by formulas (1), (2), (3), and (4) respectively.

[0060] (1)

[0061] (2)

[0062] (3)

[0063] (4)

[0064] Table 1 Coordinates of feature points in the world coordinate system

[0065] frequency Feature point A (mm) Feature point B (mm) Feature point C (mm) Feature point D (mm) 1 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 2 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 3 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 4 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 5 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 6 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 7 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 8 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 9 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0) 10 (0,0,0) (2,0,0) (1.41,0.5,0) (-1.41,0.5,0)

[0066] S22: Extract feature points The pixel coordinate value in the system;

[0067] Control robot 1 to move tool system 3 into the shooting range of camera 4. Shot contains , , , The image obtained when the feature point is , in The corresponding robot end position data is collected simultaneously by shooting , Represents the robot end position coordinates, Represents the robot terminal posture angle. Repeat the above steps 10 times to obtain 10 sets of images and robot terminal posture data as shown in Table 2.

[0068] Table 2 Robot terminal position data

[0069] Number of test groups <![CDATA[x i (mm)]]> <![CDATA[y i (mm)]]> <![CDATA[z i (mm)]]> <![CDATA[α i (°)]]> <![CDATA[β i (°)]]> <![CDATA[λ i (°)]]> 1 1795.96 87.30 193.19 -1.35 -2.91 -0.27 2 1787.25 80.64 189.11 -1.95 -3.18 -0.28 3 1807.02 85.87 197.80 -1.33 -2.59 0.28 4 1810.33 89.60 200.42 -0.95 -1.95 0.29 5 1819.87 99.34 203.06 4.56 -0.89 0.18 6 1826.63 104.83 204.79 5.28 0.05 1.02 7 1831.62 110.00 208.97 5.87 0.38 1.46 8 1836.47 117.93 215.09 0.47 1.48 1.44 9 1825.41 115.66 207.54 -2.85 1.85 -2.58 10 1803.56 100.79 185.47 2.84 1.75 -3.20

[0070] S23: Extract feature points The pixel coordinate value in the system;

[0071] like Figure 4 As shown, point is the optical center of camera 4, point For point exist The projection under the system, Coordinates of the system As a known quantity, Hough transform is used to extract the image The feature points in , , and , , , and They are , , , The pixel feature points corresponding to the feature points are then obtained using the OpenCV image processing algorithm. , , and exist Pixel coordinate values ​​in the system , , and , as shown in Table 3:

[0072] Table 3 Coordinates of feature points in pixel coordinates

[0073] frequency <![CDATA[Pixel point A i ’ (pix)]]> <![CDATA[Pixel point B i ’ (pix)]]> <![CDATA[Pixel point C i ’ (pix)]]> <![CDATA[Pixel D i ’ (pix)]]> 1 (53.32,69.97) (59.71,69.96) (53.31,63.81) (46.93,69.96) 2 (53.34,69.92) (59.23,69.92) (59.35,69.77) (46.83,69.46) 3 (53.26,69.90) (59.41,69.89) (53.26,63.75) (46.79,69.57) 4 (53.18,69.84) (59.36,69.85) (53.17,69.70) (46.64,69.66) 5 (53.56,69.99) (59.47,69.99) (53.56,69.84) (46.75,69.84) 6 (53.34,69.81 (59.26,69.80) (53.33,69.35) (46.56,69.78) 7 (53.28,69.75) (59.16,69.74) (53.28,69.60) (46.63,69.47) 8 (53.46,69.63) (59.12,69.61) (53.45,69.48) (46.59,69.33) 9 (53.51,69.52) (59.09,69.50) (53.50,69.43) (46.69,69.54) 10 (53.13,69.71) (59.01,69.72) (53.13,69.66) (46.47,69.86)

[0074] Step S3: Solving Relative to The relative pose matrix of the system;

[0075] Use formula (5) to calculate the Second shooting time Relative to The relative pose matrix of the system , the results are shown in Table 5.

[0076] (5)

[0077] In formula (5): , As shown in Table 4, , .

[0078] Table 4λ iCalculated Value

[0079] frequency 1 2 3 4 5 6 7 8 9 10 <![CDATA[λ i ]]> 0.00125 0.00123 0.00118 0.00124 0.00127 0.00119 0.00126 0.00121 0.00114 0.00116

[0080] Table 5 Relative to The relative pose matrix of the system

[0081] frequency Rotation Matrix Translation Vector frequency Rotation Matrix Translation Vector 1 #timg# #timg# 6 #timg# #timg# 2 #timg# #timg# 7 #timg# #timg# 3 #timg# #timg# 8 #timg# #timg# 4 #timg# #timg# 9 #timg# #timg# 5 #timg# #timg# 10 #timg# #timg#

[0082] Step S4: solving the position and posture relationship between the tool system and the robot end;

[0083] Use the hand-eye calibration Tsai two-step method to solve formula (6) and get The relative position relationship between the tool system 3 and the robot end 2 during the first shooting .

[0084] (6)

[0085] In formula (6): , , , , , For the The rotation matrix of the robot end position during the first shot, is the rotation matrix around the X axis, is the rotation angle around the X axis, is the rotation matrix around the Y axis, is the rotation angle around the Y axis, is the rotation matrix around the Z axis, is the rotation angle around the Z axis, For the The translation matrix of the robot end position during the first shot.

[0086] The result obtained is .

Claims

1. A robot tool system calibration method based on image recognition, characterized in that: The method comprises the following steps: Step S1: Establishing a measurement system; Build on the robot System, established on the end of the robot The tool system is fixed to the end of the robot and a System, create a Coordinate System and Pixel Coordinate System System, camera shooting tool system Relative The system is fixed and does not move. For the Second shooting time Relative to The relative pose matrix of the system, for The inverse matrix of For the Second shooting time Relative to The relative pose matrix of the system, for The inverse matrix of Indicates Shooting, ; Step S2: Extraction of feature points under the system; S21: Extraction of feature points under the system; definition The center point of the circle is taken as the feature point , set the feature point for System origin, characteristic circle radius is a known quantity, define 0° on the characteristic circle, and The points at the positions are respectively used as feature points , and ,angle and If the angle is known, then , , , exist Coordinates of the system , , , Determined by formulas (1), (2), (3), and (4) respectively: (1) (2) (3) (4)； S22: image and robot end position data acquisition; Control the robot to move the tool system into the camera's shooting range. Shot contains , , and The image obtained when the feature point is , in The corresponding robot end position data is collected simultaneously by shooting , Represents the robot end position coordinates, Represents the robot's terminal posture angle, repeat the above steps n times to obtain n sets of images and robot terminal posture data; S23: Extract feature points The pixel coordinate value in the system; point is the optical center of the camera, point For point exist The projection of the system, As a known quantity, Hough transform is used to extract the image The feature points in , , and , , , and They are , , , The pixel feature points corresponding to the feature points are then obtained using the OpenCV image processing algorithm. , , and exist Pixel coordinate values ​​in the system , , and ; Step S3: Solving Relative to The relative pose matrix of the system; Calculate the first Second shooting time Relative to The relative pose matrix of the system ; Step S4: solving the position and posture relationship between the tool system and the robot end; The relative position relationship between the tool system and the robot end is obtained using the Tsai two-step method of hand-eye calibration .

2. The robot tool system calibration method based on image recognition according to claim 1, characterized in that: The specific steps of step S3 are: Use formula (5) to calculate the Second shooting time Relative to The relative pose matrix of the system ; (5) In formula (5): is the camera’s internal parameter, is a known quantity, , for The X-axis component of the camera focal length, for The Y-axis component of the camera's focal length, and is a known quantity, For point exist The X-axis coordinate value of the system, For point exist The Y-axis coordinate value of the system, and is a known quantity, , ， 。 3. The robot tool system calibration method based on image recognition according to claim 2, characterized in that: The specific steps of step S4 are: Use the hand-eye calibration Tsai two-step method to solve formula (6) and get The relative position relationship between the tool system and the robot end during the first shooting ; (6) In formula (6): , , , , , For the The rotation matrix of the robot end position during the first shot, is the rotation matrix around the X axis, is the rotation angle around the X axis, is the rotation matrix around the Y axis, is the rotation angle around the Y axis, is the rotation matrix around the Z axis, is the rotation angle around the Z axis, For the The translation matrix of the robot end position during the first shot.

Images