Pose relationship calibration method for spraying robot and positioner based on machine vision

Abstract

This paper presents a machine vision-based method for calibrating the pose relationship between a painting robot and a positioner. Belonging to the field of robot and positioner pose relationship calibration technology, this method addresses the low level of automation in existing calibration processes. The method includes the following steps: Step 1: Acquiring calibration data; First, a checkerboard calibration plate is installed at the end of a two-axis positioner, and a camera is fixed to the end of the robot; Second, the rotation of the two-axis positioner is controlled to move the calibration plate, and the camera captures an image of the calibration plate in the pixel coordinate system {H}. Step 2: Calculating A... j (u, v) are the coordinates of the corner points of the calibration plate in the camera coordinate system {C}; Step 3: Calculate the corner point coordinates of the calibration plate. C P j ( C x, C y, C z) Coordinates in the robot's base coordinate system {B} B P j ( B x, B y, B z); Step 4: Calculation of the axis direction vector of the positioner; Step 5: Calculation of the positioner position; Step 6: Calculation of the spatial pose of the positioner; This realizes non-contact acquisition of calibration data, eliminating the need for manual robot positioning operations and improving the automation level of calibration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot and positioner pose relationship calibration, and particularly relates to a spraying robot and positioner pose relationship calibration method based on machine vision. BACKGROUND

[0002] Robot and positioner pose relationship calibration is a key problem in the field of robots. For a target workpiece with large size or complex appearance, the spraying robot is often limited by kinematic constraints, and cannot achieve spraying outside the effective working space of the robot, so certain measures need to be taken to solve the problem of insufficient effective working space of the robot. The target workpiece is positioned by the positioner, so that the workpiece reaches a suitable pose, and the requirement for the effective working space of the robot is reduced, thereby solving the above problem. When the spraying robot performs a spraying task, the positioner and the robot need to be coordinated to complete the task, so the robot and the positioner pose relationship calibration needs to be performed.

[0003] Chinese patent with publication number CN 103063213 A and title "Calibration method for pose relationship between welding robot and positioner" is disclosed. The calibration data is collected by an operator through visual observation and manual teaching to complete the coincidence of the tip of the conical tool. The process needs manual participation, and the automation level is low. SUMMARY

[0004] The present application provides a spraying robot and positioner pose relationship calibration method based on machine vision to solve the problem of low automation level in the calibration process in the prior art.

[0005] A spraying robot and positioner pose relationship calibration method based on machine vision includes the following steps:

[0006] Step 1: Collecting calibration data;

[0007] Let the robot base coordinate system be {B}; the robot end coordinate system be {E}; the camera coordinate system be {C}; the calibration board coordinate system be {W}; the two-axis positioner base coordinate system be {TB} composed of a rotation axis and an inclination axis; and the pixel coordinate system be {H} with a certain point in the upper left corner of the image as the origin of {H} H O;

[0008] First, install the checkerboard calibration board at the end of the two-axis positioner, and fix the camera at the end of the robot;

[0009] Second, control the rotation axis of the two-axis positioner to move, drive the calibration board to move, and the camera to shoot the calibration board to obtain the calibration board image in the pixel coordinate system {H}, as shown in formula (1), and record the robot end pose matrix

[0010]

[0011] wherein: a j (u, v) is the pixel value of the jth shot calibration board image, j = 1, 2, …, n; u and v are pixel position variables of A j (u, v); M and N are the number of rows and columns of A j (u, v);

[0012] Step 2: Calculate A j (u, v) in the camera coordinate system {C} of the calibration board corner point C P j ( C x, C y, C z), j = 1, 2, …, n;

[0013] Using the Harris corner detection algorithm in Opencv3, put A j (u, v) into formula (2) to get the coordinates of all corner points of the jth shot in the camera coordinate system {C} C P j ( C x, C y, C z);

[0014]

[0015] Step 3: Calculate the coordinates of the corner points of the calibration board C P j ( C x, C y, C z) in the robot base coordinate system {B} B P j ( B x, B y, B z);

[0016] Put the corner coordinates of the jth shot calibration board C P j ( C x, C y, C z) into formula (3) to find B P j ( B x, B y, B z),

[0017]

[0018] wherein: is the transformation matrix of {B} relative to {E}; is the transformation matrix of {E} relative to {C};

[0019] Step 4: Calculation of the direction vector of the axis of the positioner;

[0020] Calibration board corner points from the 1st to the nth shot B Q j ( B x j , B y j , B z j ), j = 1, 2,..., n Substitute Q B Q j ( B x j , B y j , B z j ) into equation (4) to find the values of a1, a2, and a3,

[0021]

[0022] Assume that the direction vector of the rotation axis of the positioner is where: are the unit vectors of x, y, and z in the robot base coordinate system {B}, respectively; n B x, B y, B z 1x , n 1y , n 1z ,

[0023]

[0024] Similarly, the direction vector of the tilt axis of the positioner can also be calculated as

[0025] Step 5: Calculation of the position of the positioner;

[0026] Calibration board corner point coordinates from the 1st to the nth shot B Q j ( B x j , B y j , B z j ), j = 1, 2,..., n Substitute Q 2 -y 2-z 2 The point B Q j ( B x j , B y j , B z j ) is substituted into formula (6), and the values of b1, b2, b3 and b4 are obtained,

[0027]

[0028] Suppose that the coordinates of the center of the circular arc B O1(O 1x , O 1y , O 1z ) are points on the rotating shaft of the positioner, and formula (7) can be used to obtain O 1x , O 1y , O 1z ,

[0029]

[0030] Similarly, the coordinates of the point B O2(O 2x , O 2y , O 2z ) on the tilting shaft of the positioner can also be obtained;

[0031] Step 6: calculation of the spatial pose of the positioner;

[0032] The direction vectors and are used to obtain the pose matrix of the positioner base coordinate system {TB} relative to the robot base coordinate system {B} as

[0033] The position vector of the positioner base coordinate system {TB} relative to the robot base coordinate system {B} is which can be obtained by formula (8)

[0034] Wherein:

[0035] The conversion matrix of the positioner base coordinate system {TB} relative to the robot base coordinate system {B} is obtained Thus, the pose relationship between the spraying robot and the positioner based on machine vision is calibrated.

[0036] Advantages of the present application:

[0037] This method uses a camera to collect calibration data to calibrate the spatial pose of the positioner, replacing the manual judgment operation of aligning the center point of the robot tool with the cusp on the positioner. This achieves non-contact data collection and eliminates the need for manual robot positioning, thus improving the automation level of calibration. Attached Figure Description

[0038] Figure 1 This is a flowchart of the machine vision-based method for calibrating the pose relationship between a painting robot and a positioner according to the present invention.

[0039] Figure 2 A schematic diagram illustrating the relationship between the coordinate systems in the pose calibration of the painting robot and the positioner.

[0040] Figure 3 This is a schematic diagram of the extraction of corner points of the calibration plate according to the present invention. Detailed Implementation

[0041] The present invention will now be described in further detail with reference to the accompanying drawings.

[0042] like Figure 1 As shown, the method for calibrating the pose relationship between a painting robot and a positioner based on machine vision includes the following steps:

[0043] Step 1: Collect calibration data

[0044] like Figure 2 As shown, let the base coordinate system of the painting robot be {B}; the coordinate system of the painting robot end effector be {E}; the coordinate system of the 3D camera be {C}; the coordinate system of the checkerboard calibration board be {W}; {TB} represents the two-axis positioner base coordinate system composed of the rotation axis and the tilt axis; and the pixel coordinate system be {H}, with a point at the top left corner of the image as the origin of {H}. H O.

[0045] First, the checkerboard calibration plate is installed at the end of the two-axis positioner, and the 3D camera is fixed to the end of the robot.

[0046] Secondly, the rotation of the two-axis positioner is controlled to drive the movement of the checkerboard calibration plate. The 3D camera captures the checkerboard calibration plate image in the pixel coordinate system {H}, as shown in formula (1). Figure 3 As shown. Simultaneously, the robot's end-effector pose matrix is ​​recorded.

[0047]

[0048] Where: a j (u, v) represent the pixel values ​​of the checkerboard calibration board image captured in the j-th photograph, j = 1, 2, ..., n; u and v are the pixel values ​​of A. j The pixel position variables (u, v) are A; M and N are the pixel position variables of A.j Row and column number of (u, v).

[0049] Step 2: Calculate A j (u, v) in the camera coordinate system {C} C P j ( C x, C y, C z)

[0050] Use the Harris corner detection algorithm in Opencv3 to put A j (u, v) into formula (2) to get the coordinates of all corner points of the jth shot checkerboard calibration board in the camera coordinate system {C} C P j ( C x, C y, C z), j = 1, 2,..., n.

[0051]

[0052] Take a j (0, 0) in A j (u, v) as an example, and calculate a j (0, 0) formula (2):

[0053] The coordinates of the first shot checkerboard calibration board corner point a1(0, 0) in the camera coordinate system {C} C P1( C x, C y, C z) = (-274.369, 291.953, 1309.392);

[0054] The coordinates of the second shot checkerboard calibration board corner point a2(0, 0) in the camera coordinate system {C} C P2( C x, C y, C z) = (-342.849, 149.579, 1357.32);

[0055] The coordinates of the third shot checkerboard calibration board corner point a3(0, 0) in the camera coordinate system {C} C P3( C x, C y, C z) = (-354.83, 88.5679, 1362.29);

[0056] The coordinates of the corner point a4(0, 0) of the chessboard calibration board in the camera coordinate system {C} in the 4th shooting C P4(x, y, z) = (-112.168, 410.179, 1305.98); C x, C y, C z) = (-112.168, 410.179, 1305.98);

[0057] Step 3: Calculate the coordinates of the corner points of the chessboard calibration board C P j ( C x, C y, C z) in the robot base coordinate system {B} B P j ( B x, B y, B z)

[0058] Substitute the coordinates of the corner points of the calibration board in the jth shooting C P j ( C x, C y, C z) into formula (3) to obtain B P j ( B x, B y, B z)

[0059]

[0060] Where: is the transformation matrix of {B} relative to {E}, is the transformation matrix of {E} relative to {C}

[0061] The first shooting, the coordinates of the corner points of the chessboard calibration board C P1( C x, C y, C z) = (-274.369, 291.953, 1309.392) in the robot base coordinate system {B} B P1( B x, B y, B z) = (2376.48, 103.03, 815.37);

[0062] The second shooting, the coordinates of the corner points of the chessboard calibration board C P2( C x, C y,C The coordinates of z) = (-342.849, 149.579, 1357.32) in the robot's base coordinate system {B} B P2( B x, B y, B z)=(2431.06,-8.51,815.14);

[0063] The third shoot Corner coordinates of the chessboard calibration board C P3( C x, C y, C The coordinates of z) = (-354.83, 88.5679, 1362.29) in the robot's base coordinate system {B} B P3( B x, B y, B z)=(2412.99,-155.76,820.64)

[0064] The fourth shoot, Corner coordinates of the chessboard calibration board C P4( C x, C y, C The coordinates of z) = (-112.168, 410.179, 1305.98) in the robot's base coordinate system {B} B P4( B x, B y, B z)=(2311.80,-264.27,820.25);

[0065] Step 4: Calculation of the positioner axis direction vector

[0066] The calibration board corner points from the first to the fourth shooting. B Q j ( B x j , B y j , B z j ), j = 1, 2, 3, 4 are on the plane a1x + a2y + a3 = z, will B Q j ( B x j , B y j , B z j Substitute these values ​​into formula (4) to find the values ​​of a1, a2, and a3.

[0067]

[0068] Suppose the direction vector of the rotating shaft of the positioner is Where: are the unit vectors of x, y, z in the robot base coordinate system {B} respectively. By calculation from formula (5), we can get B x, B y, B z 1x , n 1y , n 1z

[0069]

[0070] Then the direction vector of the rotating shaft of the positioner is

[0071]

[0072] Similarly, the direction vector of the tilting shaft of the positioner can also be obtained

[0073]

[0074] Step 5: Positioner position calculation

[0075] From the first to the fourth shot of the calibration board corner point coordinates B Q j ( B x j , B y j , B z j ), j = 1, 2, 3, 4, in the space circular arc b1x + b2y + b3z + b4 = -x 2 -y 2 -z 2 , substitute the point B Q j ( B x j , B y j , B z j ) into formula (6) to obtain the values of b1, b2, b3, b4

[0076]

[0077] Suppose the coordinates of the circular arc center B O1(O 1x , O 1y , O 1z ) are points on the rotating shaft of the positioner, and from formula (7) we can get 1x O 1y, O 1z

[0078]

[0079] Then the coordinates of the point on the rotating axis of the positioner

[0080] B O1(O 1x , O 1y , O 1z ) = (2195.597863, -54.537243, 814.983392)

[0081] Similarly, the coordinates of the point on the tilting axis of the positioner are obtained

[0082] B O2(O 2x , O 2y , O 2z ) = (-0.015131, 0.010648, 0.999829)

[0083] Step 6: Calculation of the spatial pose of the positioner

[0084] Using the direction vectors and , the pose matrix of the positioner base coordinate system {TB} relative to the robot base coordinate system {B} is obtained as

[0085]

[0086] The position vector of the positioner base coordinate system {TB} relative to the robot base coordinate system {B} is which can be calculated by formula (8)

[0087] Where:

[0088]

[0089] The transformation matrix of the positioner base coordinate system {TB} relative to the robot base coordinate system {B} is obtained Thus, the pose relationship between the spraying robot and the positioner based on machine vision is calibrated.

Claims

1. A method for calibrating the pose relationship between a machine vision-based spraying robot and a positioner, characterized in that, The method comprises the following steps: Step 1: collecting calibration data; Let the robot base coordinate system be {B}; the robot end coordinate system be {E}; the camera coordinate system be {C}; the calibration board coordinate system be {W}; {TB} represents the two-axis displacement machine base coordinate system composed of a rotation axis and an inclination axis; the pixel coordinate system {H} takes a certain point in the upper left corner of the image as the origin of {H} H O; Firstly, install the chessboard calibration plate at the end of the two-axis displacement machine, and fix the camera at the end of the robot; Secondly, the rotation axis of the two-axis displacement machine is controlled to drive the calibration board to move, and the camera captures the calibration board image in the pixel coordinate system {H}, as shown in equation (1), while recording the end pose matrix of the robot wherein: a j (u, v) is the pixel value of the jth shot calibration board image, j = 1, 2, …, n; u and v are pixel position variables of A j (u, v); M and N are the number of rows and columns of A j (u, v). Step 2: Compute A j (u, v) are the coordinates of the corner points of the calibration board in the camera coordinate system {C} C P j ( C x, C y, C z), j = 1, 2,..., n; The Harris corner detection algorithm in Opencv3 is used to detect the corner points of the A j (u, v) in formula (2) to obtain the coordinates of all corner points of the jth shot in the camera coordinate system {C} C P j ( C x, C y, C z) Step 3: Calculate the corner point coordinates of the calibration board C P j ( C x, C y, C z) in the robot base coordinate system {B} B P j ( B x, B y, B z) in the robot base coordinate system {B} The corner point coordinates of the calibration plate photographed for the jth time C P j ( C x, C y, C z) are substituted into Equation (3) to find B P j ( B x, B y, B z). wherein: is a transformation matrix of {B} with respect to {E}; is a transformation matrix of {E} with respect to {C}; Step 4: calculation of the displacement machine shaft direction vector; corner points of the calibration plate from the 1st to the nth photographing B Q j ( B x j , B y j , B z j ), j = 1, 2,..., n, on the plane a1x + a2y + a3 = z, substituting B Q j ( B x j , B y j , B z j ) into formula (4), the values of a1, a2, and a3 are solved, Suppose the direction vector of the rotating shaft of the positioner is wherein: are the unit vectors of x, y, z in the robot base coordinate system {B} respectively; n B x, B y, B z can be calculated by formula (5) 1x n 1y n 1z , By the same token, the direction vector of the tilt axis of the positioner can be obtained as Step 5: displacement machine position calculation; The coordinates of the calibration board corner points from the first to the nth shooting. B Q j ( B x j , B y j , B z j ), j = 1, 2, ..., n in the spatial circular arc b1x + b2y + b3z + b4 = -x 2 -y 2 -z 2 Up, point B Q j ( B x j , B y j , B z j Substitute these values ​​into formula (6) to find the values ​​of b1, b2, b3, and b4. Assume the coordinate of the center of the circular arc B O1(O 1x , O 1y , O 1z ) is a point on the rotating shaft of the positioner, and O 1x , O 1y , O 1z , Similarly, the coordinates of the point on the tilt axis of the positioner can also be obtained B O2(O 2x , O 2y , O 2z ); Step 6: calculation of the spatial pose of the displacement machine; Utilizing direction vectors and The pose matrix of the positioner base coordinate system {TB} relative to the robot base coordinate system {B} is obtained as Position vector of the base coordinate system of the positioner {TB} with respect to the base coordinate system of the robot {B} is obtainable by formula (8), wherein: A transformation matrix of the positioner base coordinate system {TB} relative to the robot base coordinate system {B} is obtained Thus, the pose relationship between the spraying robot and the positioner based on machine vision is calibrated.

Images