A Calibration Method for a Robot Tool System Based on Image Recognition

Abstract

A calibration method for a robot tool system based on image recognition, belonging to the technical field of image data processing. To solve the problem of low calibration accuracy of the tool system caused by the unknown spatial pose relationship between the auxiliary tool and the tool system in the existing tool system calibration methods, the method includes the following steps: Step S1: Establish a measurement system; Step S2: Extract feature points in the #imgabs0# system; Step S3: Solve the relative pose matrix of the #imgabs1# system relative to the #imgabs2# system; Step S4: Solve the pose relationship between the tool system and the robot end. The present invention uses an industrial camera to capture images of the tool system, extracts feature points through the Hough transform, determines the external parameters of the camera through scene calibration, establishes a hand-eye equation and uses the Tsai two-step method of hand-eye calibration to solve the spatial pose relationship between the tool system and the robot end. This method does not use an auxiliary tool, avoiding the problem of low calibration accuracy of the tool system caused by the unknown spatial pose relationship between the auxiliary tool and the tool system.

Description

Technical Field

[0001] The present invention belongs to the technical field of image data processing, and particularly relates to a calibration method for a robot tool system based on image recognition. Background Technique

[0002] In the actual application of robots, whether it is precision assembly and welding on industrial production lines or goods handling in logistics warehousing, accurate calibration of the robot tool system is a key prerequisite for ensuring the quality of operations. The calibration accuracy of the tool system directly affects the interaction accuracy between the end effector of the robot and the target object.

[0003] Currently, traditional robot tool system calibration methods mainly include mechanical measurement methods and sensor-based methods. Mechanical measurement methods usually require the use of high-precision measurement tools, such as micrometers, laser interferometers, etc., to measure and adjust each parameter of the robot tool system one by one. Although this method has high precision, it is cumbersome to operate, time-consuming and laborious, and also requires high professional skills from operators. Moreover, in the actual production environment, frequent use of mechanical measurement tools for calibration will have a greater impact on production progress and increase the production cost of enterprises.

[0004] The Chinese patent publication number is "CN114227678A", and the patent name is "A Calibration Method and Device for the TCP Tool Coordinate System of a Robot Arm". This method provides a calibration method and device for the TCP tool coordinate system of a robot arm, including: clamping a feature recognition sphere on the welding wire of the robot arm and rotating the feature recognition sphere until the second white mark on the feature recognition sphere points to the preset x-axis of the TCP coordinate system of the robot arm; obtaining the laser point cloud data on the surface of the feature recognition sphere and determining the TCP coordinate system of the robot arm based on the laser point cloud data; obtaining the transformation matrix between the robot coordinate system and the TCP coordinate system of the robot arm and using this transformation matrix as the TCP tool coordinate system of the robot arm. The deficiencies of this method are as follows: The feature sphere is clamped on the welding wire of the robot arm. As an auxiliary tool, the method for determining the spatial pose relationship between the auxiliary tool and the robot arm is not given. Therefore, this spatial pose error will be introduced into the calibration of the tool system, reducing the calibration accuracy of the tool system. Summary of the Invention

[0005] In order to solve the problem of low calibration accuracy of the tool system caused by the unknown spatial pose relationship between the auxiliary tool and the tool system in the existing tool system calibration method, the present invention proposes a calibration method for a robot tool system based on image recognition.

[0006] The technical solution for the present invention to solve the technical problem is as follows:

[0007] A calibration method for a robot tool system based on image recognition, the method comprising the following steps:

[0008] Step S1: Establish a measurement system;

[0009] Establish a B system on the robot, establish an E system at the end of the robot, fix the tool system at the end of the robot, establish a W system on the tool system, establish a C system and a pixel coordinate system P on the camera. When the camera takes pictures of the tool system, the C system is fixed relative to the B system. is the relative pose matrix of the E system with respect to the B system at the i-th shot. is the inverse matrix of is the relative pose matrix of the C system with respect to the W system at the i-th shot. is the inverse matrix of, where i represents the i-th shot, and i = 1, 2, 3... n.

[0010] Step S2: Feature point extraction in the P system;

[0011] S21: Define the coordinates of the feature points in the W system;

[0012] Define the center point of the W system as the feature point A. Let the feature point A be the origin of the W system, and the radius r of the feature circle be a known quantity. Define the points at the 0°, θ, and δ positions on the feature circle as the feature points B, C, and D respectively. The angles θ and δ are known quantities. Then the coordinates of A, B, C, and D in the W system are A W ( W X A , W Y A , W Z A ), B W ( W X B , W Y B , W Z B ), C W ( W X C , W Y C , W Z C ), D W ( W X D , W Y D , W Z D );

[0013] S22: Image and robot end pose data acquisition;

[0014] Control the robot to move the tool system into the shooting range of the camera. The image obtained when the camera takes the i-th shot containing feature points A, B, C, and D is P i , and collect the corresponding robot end pose data Z at the i-th shot i ={x i ,y i ,z i ,α i ,β i ,λ i}, where x i ,y i ,z i represent the robot end position coordinates, and α i ,β i ,λ i represent the robot end attitude angles. Repeat the above steps n times to obtain n sets of images and robot end pose data;

[0015] S23: Extract the pixel coordinate values of the feature points in the P system;

[0016] Point O is the optical center of the camera, and point O' is the projection of point O in the P system. O'=(c x ,c y ) is a known quantity. Use the Hough transform to extract the feature points A' i ,B' i ,C' i and D' i in the image P i ,A' i ,B' i ,C' i and D' i are the pixel feature points corresponding to the feature points A, B, C, and D respectively. Then use the OpenCV image processing algorithm to obtain the pixel coordinate values of the feature points A' i ,B' i ,C' i and D' W in the P system and

[0017] Step S3: Solve the relative pose matrix of the C system with respect to the W system;

[0018] Calculate the relative pose matrix of the C system with respect to the W system at the i-th shot

[0019] Step S4: Solve the relative pose relationship between the tool system and the robot end;

[0020] Use the Tsai two-step method of hand-eye calibration to obtain the relative pose relationship between the tool system and the robot end

[0021] The present invention has the following beneficial effects compared with the prior art:

[0022] The present invention uses an industrial camera to capture the image of the tool system, extracts feature points through Hough transformation, determines the external parameters of the camera through scene calibration, establishes the hand-eye equation, and uses the Tsai two-step method for hand-eye calibration to solve the spatial pose relationship between the tool system and the robot end. This method does not use auxiliary tools, avoiding the problem of low calibration accuracy of the tool system caused by the unknown spatial pose relationship between the auxiliary tool and the tool system. Description of the Drawings

[0023] Figure 1 is a flow chart of a method for calibrating a robot tool system based on image recognition according to the present invention;

[0024] Figure 2 is a schematic principle diagram of a method for calibrating a robot tool system based on image recognition according to the present invention;

[0025] Figure 3 is a schematic diagram of feature point extraction of a method for calibrating a robot tool system based on image recognition according to the present invention;

[0026] Figure 4 is a schematic diagram of solving the coordinates of feature points of a method for calibrating a robot tool system based on image recognition according to the present invention;

[0027] Figure 5 is a schematic diagram of feature point extraction in an embodiment of the present invention;

[0028] In the figure: 1, robot; 2, robot end; 3, tool system; 4, camera; 5, feature circle. Detailed Embodiment

[0029] The present invention will be further described in detail below with reference to the accompanying drawings.

[0030] As Figure 1 shown, a method for calibrating a robot tool system based on image recognition, the specific steps of the method are as follows:

[0031] Step S1: Establish a measurement system;

[0032] As Figure 2 shown, establish a B system on the robot 1, establish an E system on the robot end 2, the tool system 3 is fixedly connected to the robot end 2, establish a W system on the tool system 3, establish a C system and a pixel coordinate system P system on the camera 4. When the camera 4 captures the tool system 3, the C system is fixed relative to the B system, is the relative pose matrix of the E system relative to the B system at the i-th capture, is the inverse matrix of, is the relative pose matrix of the C system with respect to the W system at the i-th shooting, is the inverse matrix of, where i represents the i-th shooting, i = 1, 2, 3... n;

[0033] Step S2: Feature point extraction in the P system;

[0034] S21: Define the coordinates of the feature points in the W system;

[0035] As shown in Figure 3 , define the center point of the W system as the feature point A. Let the feature point A be the origin of the W system, and the radius r of the feature circle 5 be a known quantity. Define the points at the 0°, θ, and δ positions on the feature circle 5 as the feature points B, C, and D respectively, and the angles θ and δ be known quantities. Then the coordinates of A, B, C, and D in the W system A W ( W X A , W Y A , W Z A ), B W ( W X B , W Y B , W Z B ), C W ( W X C , W Y C , W Z C ), D W ( W X D , W Y D , W Z D ) are determined by formulas (1), (2), (3), and (4) respectively:

[0036]

[0037] S22: Image and robot end pose data acquisition;

[0038] Control the robot 1 to move the tool system 3 into the shooting range of the camera 4. The image obtained when the camera 4 shoots for the i-th time and contains the feature points A, B, C, and D is P i , and collect the corresponding robot end pose data Z i = {x i , y i , z i , α i, β i , λ i}, x i , y i , z i represent the end - effector position coordinates of the robot, α i , β i , λ i represent the end - effector attitude angles of the robot. Repeat the above steps n times to obtain n sets of image and end - effector pose data.

[0039] S22: Extract the pixel coordinate values of the feature points in the P coordinate system;

[0040] As Figure 4 shown, point O is the optical center of camera 4, and point O' is the projection of point O in the P coordinate system. Its coordinates O'(c x , c y ) in the P coordinate system are known quantities. Use the Hough transform to extract the feature points A' i , B' i , C' i , and D' i in image P i . A' i , B' i , C' i , and D' i are the pixel feature points corresponding to the feature points A, B, C, and D respectively. Then use the OpenCV image - processing algorithm to obtain the pixel coordinate values of the feature points A' i , B' i , C' i , and D' i in the P coordinate system and

[0041] Step S3: Solve the relative pose matrix of the C coordinate system with respect to the W coordinate system;

[0042] Use formula (5) to calculate the relative pose matrix of the C coordinate system with respect to the W coordinate system at the i - th shooting

[0043]

[0044] In formula (5): K is the internal parameter of the camera, which is a known quantity, f x is the component of the camera focal length in the X - axis direction in the P coordinate system, f y is the component of the camera focal length in the Y - axis direction in the P coordinate system, f x and f y are known quantities, c x is the X - axis coordinate value of point O' in the P coordinate system, c y is the Y - axis coordinate value of point O' in the P coordinate system, cx and c y are known quantities,

[0045] Step S4: Solve the relative pose relationship between the tool system and the robot end;

[0046] Use the Tsai two-step method for hand-eye calibration to solve formula (6) to obtain the relative pose relationship between the tool system 3 and the robot end 2 at the i-th shot

[0047] AX = XB (6)

[0048] In formula (6): Let R i = R z (γ i )·R y (β i )·R x (α i ), R i is the rotation matrix of the robot end pose at the i-th shot, R x (α i ) is the rotation matrix around the X-axis, α i is the rotation angle around the X-axis, R y (β i ) is the rotation matrix around the Y-axis, β i is the rotation angle around the Y-axis, R z (γ i ) is the rotation matrix around the Z-axis, γ i is the rotation angle around the Z-axis, is the translation matrix of the robot end pose at the i-th shot.

[0049] Example:

[0050] A robot tool system calibration method based on image recognition, as Figure 1 shown, the specific steps of this method are as follows:

[0051] Step S1: Establish a measurement system;

[0052] As Figure 2 shown, establish a B system on the robot 1, establish an E system on the robot end 2, the tool system 3 is fixedly connected to the robot end 2, establish a W system on the tool system 3, establish a C system and a pixel coordinate system P on the camera 4. When the camera 4 shoots the tool system 3, the C system is fixed relative to the B system, is the relative pose matrix of the E system relative to the B system at the i-th shot, is 's inverse matrix, is the relative pose matrix of the C system with respect to the W system during the i-th shooting, is 's inverse matrix, where i represents the i-th shooting, i = 1, 2, 3…10;

[0053] In this embodiment, the robot 1 selects the Effort GR680 robot, the tool system 3 selects a spray gun, and the camera 4 selects the Huari Technology industrial camera 12CG-E.

[0054] Step S2: Feature point extraction in the P system;

[0055] S21: Define the coordinates of the feature points in the W system;

[0056] As shown in Figure 3 , define the center point of the W system as the feature point A. Let the feature point A be the origin of the W system, the radius r of the feature circle 5 is 2 mm, and define the points at the 0°, θ, and δ positions on the feature circle 5 as the feature points B, C, and D respectively, where θ = 60° and δ = 120°. Then the coordinates of A, B, C, and D in the W system, A W ( W X A , W Y A , W Z A ), B W ( W X B , W Y B , W Z B ), C W ( W X C , W Y C , W Z C ), D W ( W X D , W Y D , W Z D ) are determined by formulas (1), (2), (3), and (4) respectively.

[0057]

[0058] Table 1 Coordinates of Feature Points in the World Coordinate System

[0059]

[0060]

[0061] S22: Extract the pixel coordinate values of the feature points in the P coordinate system;

[0062] Control the robot 1 to move the tool system 3 into the shooting range of the camera 4. When the camera 4 takes the i-th picture containing feature points A, B, C, and D, the obtained image is P i , and collect the corresponding robot end pose data Z at the i-th shooting i = {x i , y i , z i , α i , β i , λ i}, where x i , y i , z i represent the robot end position coordinates, and α i , β i , λ i represent the robot end pose angles. Repeat the above steps 10 times to obtain 10 groups of images and robot end pose data as shown in Table 2.

[0063] Table 2 Robot end pose data

[0064]

[0065] S23: Extract the pixel coordinate values of the feature points in the P coordinate system;

[0066] As Figure 4 shown, point O is the optical center of the camera 4, and point O' is the projection of point O in the P coordinate system. Its coordinates O'(c x , c y ) in the P coordinate system are known quantities. Use the Hough transform to extract the feature points A' i , B' i , C' i and D' i in the image P i . A' i , B' i , C' i and D' i are the pixel feature points corresponding to the feature points A, B, C, and D respectively. Then use the OpenCV image processing algorithm to obtain the pixel coordinate values i , B' i , C' i and D' i of A', B', C', and D' in the P coordinate system and as shown in Table 3:

[0067] Table 3 Feature point coordinates in pixel coordinates

[0068]

[0069] Step S3: Solve the relative pose matrix of the C system with respect to the W system;

[0070] Use formula (5) to calculate the relative pose matrix of the C system with respect to the W system at the i-th shot The results are shown in Table 5.

[0071]

[0072] In formula (5): As shown in Table 4,

[0073]

[0074] Table 4 λ i Calculated value

[0075]

[0076] Table 5 Relative pose matrix of the C system with respect to the W system

[0077]

[0078]

[0079] Step S4: Solve the pose relationship of the tool system relative to the robot end;

[0080] Use the Tsai two-step method of hand-eye calibration to solve formula (6) to obtain the relative pose relationship of the tool system 3 with respect to the robot end 2 at the i-th shot

[0081] AX = XB (6)

[0082] In formula (6): Let R i = R z (γ i )·R y (β i )·R x (α i ), R i is the rotation matrix of the robot end pose at the i-th shot, R x (α i ) is the rotation matrix around the X axis, α i is the rotation angle around the X axis, R y (β i ) is the rotation matrix around the Y axis, β i is the rotation angle around the Y axis, R z (γi ) is the rotation matrix for rotation about the Z-axis, γ i is the rotation angle for rotation about the Z-axis, is the translation matrix of the end pose of the robot at the i-th shot.

[0083] The result obtained is

Claims

1. A calibration method for a robot tool system based on image recognition, characterized in that, The method includes the following steps: Step S1: Establish a measurement system; Establish the B coordinate system on the robot, establish the E coordinate system at the end of the robot, fix the tool coordinate system to the end of the robot, establish the W coordinate system on the tool coordinate system, establish the C coordinate system and the pixel coordinate system P on the camera. When the camera captures the tool coordinate system, the C coordinate system is fixed relative to the B coordinate system. It is the relative pose matrix of the E coordinate system with respect to the B coordinate system at the i-th capture. is the inverse matrix of It is the relative pose matrix of the C coordinate system with respect to the W coordinate system at the i-th capture. is the inverse matrix of, where i represents the i-th capture, and i = 1, 2, 3... n. Step S2: Feature point extraction in the P system; S21: Feature point extraction in the P system; Define the center point of the W system as the feature point A, and set the feature point A as the origin of the W system. The radius r of the feature circle is a known quantity. Define the points at the 0°, θ, and δ positions on the feature circle as the feature points B, C, and D respectively, and the angles θ and δ are known quantities. Then the coordinates of A, B, C, and D in the W system, A W ( W X A , W Y A , W Z A ) are determined by formulas (1), (2), (3), and (4) respectively: W ( W X B , W Y B , W Z B ) are determined by formulas (1), (2), (3), and (4) respectively: W ( W X C , W Y C , W Z C ) are determined by formulas (1), (2), (3), and (4) respectively: D W ( W X D , W Y D , W Z D ) are determined by formulas (1), (2), (3), and (4) respectively: S22: Acquisition of image and robot end pose data; Control the robot to move the tool system into the shooting range of the camera. The image obtained when the camera takes the i-th shot containing feature points A, B, C, and D is P i , and collect the corresponding robot end pose data Z at the i-th shot i ={x i , y i , z i , α i , β i , λ i}, where x i , y i , z i represent the robot end position coordinates, and α i , β i , λ i represent the robot end attitude angles. Repeat the above steps n times to obtain n sets of images and robot end pose data; S23: Extract the pixel coordinate values of the feature points in the P system; Point O is the optical center of the camera, and point O′ is the projection of point O in the P coordinate system. O′ = (c x , c y ) is a known quantity. The Hough transform is used to extract the feature points A′ i in the image P i , B′ i , C′ i and D′ i . A′ i , B′ i , C′ i and D′ i are the pixel feature points corresponding to the feature points A, B, C, and D respectively. Then, the OpenCV image processing algorithm is used to obtain the pixel coordinate values of the feature points A′ i , B′ i , C′ i and D′ i in the P coordinate system and Step S3: Solve the relative pose matrix of the C system with respect to the W system; Calculate the relative pose matrix of the C system with respect to the W system at the i-th shooting Step S4: Solve the relative pose relationship between the tool system and the robot end; The relative pose relationship between the tool system and the robot end is obtained by using the Tsai two-step method for hand-eye calibration 2. The method for calibrating a robot tool system based on image recognition according to claim 1, wherein The specific steps of the said step S3 are: The relative pose matrix of the C system with respect to the W system at the i-th shooting is calculated using Equation (5). In formula (5): K is the internal parameter of the camera, which is a known quantity. f x is the component of the camera focal length in the X-axis direction in the P coordinate system, f y is the component of the camera focal length in the Y-axis direction in the P coordinate system, f x and f y are known quantities, c x is the X-axis coordinate value of point O' in the P coordinate system, c y is the Y-axis coordinate value of point O' in the P coordinate system, c x and c y are known quantities.

3. The method for calibrating a robot tool system based on image recognition according to claim 2, wherein The specific steps of the said step S4 are: Using the Tsai two-step method for hand-eye calibration to solve formula (6), the relative pose relationship between the tool system and the robot end-effector at the i-th shot is obtained AX = XB (6) In formula (6): Let R i = R z (γ i ) · R y (β i ) · R x (α i ), R i is the rotation matrix of the end - effector pose of the robot at the i - th shot. R x (α i ) is the rotation matrix about the X - axis, and α i is the rotation angle about the X - axis. R y (β i ) is the rotation matrix about the Y - axis, and β i is the rotation angle about the Y - axis. R z (γ i ) is the rotation matrix about the Z - axis, and γ i is the rotation angle about the Z - axis. is the translation matrix of the end - effector pose of the robot at the i - th shot.

Images