A robot kinematic parameter calibration method and system based on monocular mobile vision

Abstract

The present invention discloses a robot kinematic parameter calibration method and system based on monocular mobile vision. The present invention uses a monocular camera to avoid the shortcomings of large measurement volume and limited measurement range of binocular or multi-eye equipment at the end; by converting the measurement information into rays of points on the camera normalized plane transformed from the origin of the camera coordinate system and the pixel coordinates of the center of the circle of the collected image, it avoids the measurement of high-precision depth information that requires high costs to achieve, avoids the complex equipment installation and debugging required for traditional calibration methods such as laser trackers, and the inconvenience caused by the large volume of the measuring equipment, reduces the cost required for robot kinematic parameter calibration and simplifies the operation steps; by applying the error model to identify parameters, the absolute positioning accuracy of the robot is effectively improved.

Description

Technical Field

[0001] The invention relates to a robot kinematic parameter calibration method and system based on monocular mobile vision, belonging to the technical field of robots. Background Art

[0002] With the need for transformation and upgrading of the manufacturing industry, intelligent industrial robots with integrated vision systems are increasingly used in modern factories, and the application requirements for offline programming technology of robots are also getting higher and higher. The accuracy of robots is generally divided into absolute positioning accuracy and repeat positioning accuracy. Repeat positioning accuracy is the ability of industrial robots to return to the same position many times. Absolute positioning accuracy is the ability of robots to accurately move to the desired position in three-dimensional space. However, at present, robots at home and abroad can only achieve a high level of repeat positioning accuracy, but absolute positioning accuracy is difficult to meet the needs of high-precision machining. Therefore, the absolute positioning accuracy of robots has become one of the important indicators to measure their working performance, and the absolute positioning accuracy of robots can be effectively improved through robot kinematic calibration, and the effect is significant.

[0003] Existing vision-based kinematic calibration of industrial robots generally uses two or more cameras installed on the end of the robot when using vision for calibration. This results in the end measurement equipment occupying a large volume and requiring high costs. On the one hand, it places high requirements on the control and position of the robot end; on the other hand, during the calibration process, it is necessary to observe the standard ball from all angles. The appropriate measurement range of each camera is very limited, resulting in high requirements on the range of robot movement, which is prone to collisions and the measured object being outside the measurement range. In addition, the robot position needs to be adjusted according to the data obtained after the camera obtains the data, which seriously affects the calibration efficiency.

[0004] Therefore, the prior art requires a method to reduce costs to improve the efficiency of vision-based kinematic calibration methods. Summary of the invention

[0005] The present invention provides a robot kinematic parameter calibration method and system based on monocular mobile vision, through which the robot kinematic parameter calibration can be realized under monocular mobile vision.

[0006] The technical solution of the present invention is: a robot kinematic parameter calibration method based on monocular mobile vision, comprising:

[0007] The acquisition step controls the robot to move to different specified positions, and the camera installed at the end of the robot acquires the image of the sphere fixed in the robot base coordinate system;

[0008] The fitting step is to transform the origin of the camera coordinate system and the pixel coordinates of the center of the collected image into points on the camera normalized plane in the robot base coordinate system to establish a ray, and fit the center point of the sphere through the ray;

[0009] An acquisition step is to establish a kinematic parameter error and a distance error vector model between the ray and the center point of the sphere to obtain the identified kinematic parameter error;

[0010] In the calibration step, the identified kinematic parameter errors are compensated into the model, and the coordinates of the sphere center point and the identified kinematic parameters are updated iteratively. When the number of iterations reaches the set condition or the error obtained by the iteration is less than the threshold, the iteration is stopped, and the kinematic parameters calculated at this time are compensated into the robot controller to complete the calibration.

[0011] The method also includes a step of determining a clear and complete imaging range of the sphere in the camera. The specific steps of the method are as follows:

[0012] Calibrate the camera's internal parameters to obtain the camera's internal parameter K;

[0013] The camera is used to acquire complete standard sphere images with clear edges at n different positions;

[0014] The pixel coordinates of the pixels constituting the circular edge portion on the image are obtained by an image processing algorithm;

[0015] A circle is fitted according to the pixel coordinates of the pixels at the edge of the circle by a circle fitting algorithm, thereby obtaining the pixel coordinates of the center of the circle;

[0016] Select a pixel at any point on the circular edge and take its pixel coordinates; obtain the coordinates of the center of the standard sphere in the camera coordinate system based on the pixel coordinates of the center of the circle, the pixel coordinates of any point, the coordinates of the camera normalized plane, the camera intrinsic parameter K, and the radius R of the standard sphere;

[0017] The coordinates of the center of the standard ball of n standard ball images in the camera coordinate system are obtained; and a clear and complete imaging range of the center of the standard ball in the camera coordinate system is obtained through the obtained n sphere center coordinate values.

[0018] The relationship formula for obtaining the coordinates of the center of the standard ball in the camera coordinate system is:

[0019]

[0020]

[0021]

[0022] Among them, p cx 、p cy Respectively represent the x-direction and y-direction coordinate values ​​of the center of the circle in the pixel coordinate system, prx 、p ry Respectively represent the x-direction and y-direction coordinate values ​​of a pixel of an arbitrary circular edge in the pixel coordinate system; s ,y s 、z s They represent the coordinate values ​​of the center of the standard ball in the x, y, and z directions in the camera coordinate system, respectively. r ,y r Respectively represent the pixel coordinates selected in the pixel coordinate system (p rx ,p ry ) is transformed to the coordinate values ​​of the point in the camera normalized plane in the x and y directions in the camera coordinate system through the camera intrinsic parameter K; R is the radius of the standard sphere; T represents transpose.

[0023] When the center of the standard ball (x s ,y s ,z s )When the following relationship is satisfied in the camera coordinate system, the center of the standard sphere can be considered to be within the clear and complete imaging range of the camera:

[0024] x min m1 <x s <x max u1

[0025] y min m2 <y s <y max u2

[0026] z min m3 <z s <z max ·u3

[0027] Among them, x min 、x max is the minimum and maximum value of the x-direction coordinate of the center of the standard ball in the camera coordinate system of the n standard ball images; y min ,y max is the minimum and maximum value of the y-direction coordinate of the center of the standard ball in the camera coordinate system of the n standard ball images; min 、z max are the minimum and maximum values ​​of the z-direction coordinates of the center of the standard ball of the n standard ball images in the camera coordinate system; the values ​​of the weight coefficients m1, m2, and m3 are greater than or equal to 1, and the values ​​of the weight coefficients u1, u2, and u3 are less than or equal to 1.

[0028] The method comprises:

[0029] The robot's kinematic model is established based on the robot's nominal kinematic parameters, and the hand-eye relationship between the camera and the robot is obtained through hand-eye calibration;

[0030] Fix the standard ball, control the robot to move so that the camera can clearly image the standard ball, calculate the coordinates of the center of the standard ball in the camera coordinate system, and combine the kinematic model and the hand-eye relationship to obtain the coordinates of the center of the standard ball in the robot base coordinate system;

[0031] According to the position of the center of the standard ball in the robot base coordinate system, M camera poses in the robot base coordinate system are randomly generated with the clear and complete imaging range of the camera, so that the standard ball is in the clear and complete imaging range; the generated M poses are based on the established kinematic model and hand-eye relationship, and the robot kinematics inverse solution is used to calculate M groups of robot joint angles that can enable the camera to clearly image the standard ball, so that a group of joint angles corresponds to one pose;

[0032] Use a camera to shoot a standard ball at each set of angles;

[0033] Extract the pixel coordinates of the center of the circle on the captured image. A pixel coordinate of the center of the circle can be extracted from the captured image corresponding to each set of joint angles.

[0034] In the robot base coordinate system, a ray is established by transforming the origin of the camera coordinate system and the pixel coordinates of the center of the collected image into points on the camera normalized plane;

[0035] All rays are calculated based on the collected data, and the point with the shortest sum of distances to all rays is obtained as the center point of the fitted standard sphere;

[0036] Establish an equation for the error between the theoretical value coordinate transformation matrix and the actual value coordinate transformation matrix;

[0037] Establish the relationship between the robot end camera pose error and kinematic parameter error;

[0038] Establish an error model between the robot end camera posture error and the converted ray error;

[0039] Define the distance error and distance error vector, and obtain the Jacobian matrix of the relationship between the point-to-ray distance error vector, the ray starting point and the ray midpoint;

[0040] Get the kinematic parameter errors.

[0041] The ray is defined as:

[0042]

[0043] [x cj y cj 1] T =K -1 [p cjx pcjy 1] T

[0044]

[0045]

[0046] Among them, P cj is the coordinate of the origin of the camera coordinate system in the robot base coordinate system at the jth pose, that is, the starting point of the ray; Represents the theoretical coordinate transformation matrix from the camera coordinate system to the robot base coordinate system at the jth pose x cj ,y cj is the center of the circle in the pixel coordinate system (p cjx ,p cjy ) is transformed into the value of the point in the camera normalized plane in the x direction and y direction in the camera coordinate system; K is the camera internal parameter; P nj The pixel coordinates of the circle center at the jth pose are converted into the coordinates of the point in the camera normalized plane in the robot base coordinate system, that is, the intersection point of the ray and the camera normalized plane; v j Represents the unit direction vector of the ray in the robot's base coordinate system at the jth pose of the robot.

[0047] The equation for establishing the error between the theoretical value coordinate transformation matrix and the actual value coordinate transformation matrix is ​​expressed as:

[0048] T i R =T i +T i δ i

[0049] Among them, T i R Represents the coordinate transformation matrix T i The corresponding actual value coordinate transformation matrix, δ i represents the differential operator of the i-th robot joint;

[0050] Establish the robot end camera pose error Relationship with kinematic parameter errors:

[0051]

[0052] in, The influence of the posture errors of the six robot joints on the camera posture error is shown respectively; e i represents the pose error of the i-th joint, i = 1, 2, 3, 4, 5, 6; I is the unit matrix, e c is the hand-eye matrix error; J is is the abbreviation of ; G is the conversion matrix from the kinematic parameter error to the posture errors of the six robot joints; Δx1, Δx2, Δx3, Δx4, Δx5, Δx6 are the kinematic parameter errors of the six joints respectively; ΔE is the overall kinematic parameter error.

[0053] The error model between the robot end camera posture error and the converted ray error is established:

[0054]

[0055] Among them, ΔP cj represents the P caused by the kinematic error of the robot at the jth position cj The change in ΔP nj represents the P caused by the kinematic error of the robot at the jth position nj The amount of change; represents the rotation matrix from the camera coordinate system to the jth position of the robot, O is the zero matrix; is the camera pose error at the end of the robot; x cj ,y cj is the center of the circle in the pixel coordinate system (p cjx ,p cjy ) is transformed into the x- and y-direction values ​​of a point in the camera normalized plane in the camera coordinate system.

[0056] The Jacobian matrix B of the relationship between the distance from the point to the ray, the starting point of the ray and the middle point of the ray j Specifically:

[0057] [x j y j z j ] T =(Iv j v j T )(PP cj )

[0058]

[0059]

[0060]

[0061] Among them, [x j y j z j ] T is the shortest vector from the jth ray to point P, which is called the distance error vector; v jRepresents the unit direction vector of the ray in the robot's base coordinate system at the jth position; x j ,y j 、z j is the coordinate value of the distance error vector in the x, y and z directions in the robot base coordinate system; d is the distance error; ΔP cj , ΔP nj is the ray error, ΔP cj represents the P caused by the kinematic error of the robot at the jth position cj The change in ΔP nj represents the P caused by the kinematic error of the robot at the jth position nj The amount of change.

[0062] A robot kinematic parameter calibration system based on monocular mobile vision, comprising:

[0063] The acquisition module is used to control the robot to run to different specified positions and postures. The camera installed at the end of the robot collects the image of the sphere fixed in the robot's base coordinate system; the fitting module is used to transform the origin of the camera coordinate system and the pixel coordinates of the center of the collected image into points on the camera's normalized plane in the robot's base coordinate system to establish rays, and fit the center point of the sphere through the rays; the acquisition module is used to establish a kinematic parameter error and a distance error vector model between the ray and the center point of the sphere, and obtain the identified kinematic parameter error; the calibration module is used to compensate the identified kinematic parameter error into the model, and iteratively update the coordinates of the center point of the sphere and the identified kinematic parameters; when the number of iterations reaches the set condition or the error obtained by the iteration is less than the threshold, the iteration is stopped, and the kinematic parameters calculated at this time are compensated to the robot controller to complete the calibration.

[0064] The beneficial effects of the present invention are as follows: the present invention uses a monocular camera, thereby avoiding the shortcomings of large measurement volume and limited measurement range of binocular or multi-eye equipment when the equipment is at the end; by converting the measurement information into rays of points on the normalized plane of the camera, which are transformed from the origin of the camera coordinate system and the pixel coordinates of the center of the circle of the collected image, it avoids the measurement of high-precision depth information that requires high costs, avoids the complex equipment installation and debugging required for traditional calibration methods such as laser trackers, and the inconvenience caused by the large size of the measuring equipment, reduces the cost required for calibration of the robot's kinematic parameters and simplifies the operating steps; and effectively improves the absolute positioning accuracy of the robot by applying the error model to identify parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 Schematic diagram of the relationship between various coordinate systems of the present invention;

[0066] Figure 2Comparison of distance error before and after calibration Figure 1 ;

[0067] Figure 3 Comparison of distance error before and after calibration Figure 2 . DETAILED DESCRIPTION

[0068] The invention is further described below in conjunction with the accompanying drawings and embodiments, but the content of the invention is not limited to the scope of the embodiments.

[0069] Example 1: Figure 1-2 As shown, a robot kinematic parameter calibration method based on monocular mobile vision includes:

[0070] In the acquisition step, the robot is controlled to run to different specified positions and postures, and the camera installed at the end of the robot acquires the image of the sphere fixed in the robot base coordinate system; in the fitting step, in the robot base coordinate system, a ray is established by transforming the pixel coordinates of the center of the camera coordinate system origin and the acquired image into a point on the camera normalized plane (the plane of z=1 in the camera coordinate system), and the center point of the sphere is fitted through the ray; in the acquisition step, a kinematic parameter error and a distance error vector model between the ray and the center point of the sphere are established to obtain the identified kinematic parameter error; in the calibration step, the identified kinematic parameter error is compensated into the model, and the coordinates of the center point of the sphere and the identified kinematic parameters are iteratively updated; when the number of iterations reaches the set condition or the error obtained by the iteration is less than the threshold, the iteration is stopped, and the kinematic parameters calculated at this time are compensated into the robot controller to complete the calibration.

[0071] Furthermore, the method may further include a step of determining a clear and complete imaging range of the sphere in the camera. The specific steps of the method are as follows:

[0072] Step 1: Calibrate the camera’s internal parameters to obtain the camera’s internal parameter K.

[0073] Specifically, the camera's intrinsic parameter K can be obtained by calibrating the camera's intrinsic parameters using Zhang Zhengyou's calibration method.

[0074] Step 2: Obtain complete standard ball images with clear edges at n different positions through camera acquisition;

[0075] Specifically, a standard ball is placed in front of the camera so that the camera can acquire complete standard ball images with clear edges at n different positions; taking five different positions as an example, the standard ball is moved to the position closest to the camera, and complete standard ball images with clear edges at the upper, lower, left and right positions of the nearest imaging image are acquired (taking the above example, the complete standard ball image is located at the upper position of the entire imaging image); the standard ball is moved to the farthest position from the camera, and a complete standard ball image with clear edges at any position of the farthest position in the imaging image is acquired; taking the Basler acA1600-20gm camera as an example, a complete standard ball image at the upper left position of the nearest imaging image is captured, and the edge pixels of the standard ball are checked through the image software. If the edge pixels are below 4-5 pixels, the edge can be considered clear. When other cameras are used, an appropriate edge pixel range is selected as needed; the complete image indicates that there is no missing standard ball in the entire image.

[0076] Step 3, obtain the pixel coordinates of the pixels constituting the circular edge portion on the image through an image processing algorithm;

[0077] Specifically, the pixel coordinates of the pixels constituting the circular edge portion on the image to be processed can be obtained through an image processing algorithm; wherein the image processing algorithm can specifically be a Canny edge detection algorithm or a Sobel edge detection algorithm.

[0078] Step 4, fitting a circle by a circle fitting algorithm according to the pixel coordinates of the pixels at the edge of the circle, thereby obtaining the pixel coordinates of the center of the circle;

[0079] Specifically, a circle can be fitted according to the pixel coordinates of the pixels at the edge of the circle by a circle fitting algorithm, thereby obtaining the pixel coordinates of the center of the circle (p cx ,p cy );where p cx 、p cy They respectively represent the x-direction and y-direction coordinate values ​​of the circle center in the pixel coordinate system; wherein the circle fitting algorithm may be the least squares method.

[0080] Step 5. Select a pixel at any point on the circular edge and take its pixel coordinates. According to the pixel coordinates of the center of the circle, the pixel coordinates of any point, the coordinates of the camera normalized plane, the camera internal parameter K, and the radius R of the standard ball, obtain the coordinates of the center of the standard ball in the camera coordinate system.

[0081] Specifically, you can select any pixel on the edge of the circle (the pixel point is selected from Step 3, and this selection method can obtain the pixel coordinates more quickly), and its pixel coordinates are (p rx ,p ry ); According to the pixel coordinates of the center of the circle (p cx,p cy ), pixel coordinates (p rx ,p ry ), camera normalized plane coordinates (x r ,y r ), camera internal parameter K, standard sphere radius R, establish a relationship to solve the coordinates of the center of the standard sphere in the camera coordinate system, and calculate the coordinates of the center of the standard sphere in the camera coordinate system (x s ,y s ,z s ).

[0082] Furthermore, the coordinate relationship of the center of the standard ball in the camera coordinate system can be set as:

[0083]

[0084]

[0085]

[0086] Among them, p rx 、p ry Respectively represent the x-direction and y-direction coordinate values ​​of a pixel of an arbitrary circular edge in the pixel coordinate system; s ,y s 、z s They represent the coordinate values ​​of the center of the standard ball in the x, y, and z directions in the camera coordinate system, respectively. r ,y r Respectively represent the pixel coordinates selected in the pixel coordinate system (p rx ,p ry ) is transformed to the coordinate values ​​of the point in the camera normalization plane in the x and y directions in the camera coordinate system through the camera intrinsic parameter K; T represents transpose.

[0087] Step 6. Obtain the coordinates of the center of the standard ball of n standard ball images in the camera coordinate system; obtain a clear and complete imaging range of the center of the standard ball in the camera coordinate system through the obtained n sphere center coordinate values.

[0088] Specifically, the n standard ball images collected in Step 2 can be processed by using Steps 3 to 5 to obtain the coordinates (x1, y1, z1), (x2, y2, z2), ..., (x n ,y n ,z n ); A clear and complete imaging range of the center of the standard ball in the camera coordinate system is obtained by obtaining the n sphere center coordinate values.

[0089] Furthermore, we can set the center of the standard ball (x s ,y s ,z s )When the following relationship is satisfied in the camera coordinate system, the center of the standard sphere can be considered to be within the clear and complete imaging range of the camera:

[0090] x min m1 <x s <x max u1

[0091] y min m2 <y s <y max u2

[0092] z min m3 <z s <z max ·u3

[0093] Among them, x min 、x max for x1,x2,...,x n The minimum and maximum values ​​in y min ,y max is y1,y2,...,y n The minimum and maximum values ​​in z min 、z max for z1,z2,...,z n The minimum and maximum values ​​in ; the values ​​of weight coefficients m1, m2, and m3 are greater than or equal to 1 (the coefficients can be the same or different, such as 1 and 1.1); the values ​​of weight coefficients u1, u2, and u3 are less than or equal to 1 (the coefficients can be the same or different, such as 1 and 0.9).

[0094] The calibration process of the method is specifically as follows:

[0095] Step 7, establish the robot's kinematic model according to the robot's nominal kinematic parameters, and obtain the hand-eye relationship between the camera and the robot through hand-eye calibration;

[0096] Specifically, the robot's kinematic model can be established based on the robot's nominal kinematic parameters, and the hand-eye relationship between the camera and the robot can be obtained through hand-eye calibration:

[0097] T e b =T1·T2·T3·T4·T5·T6

[0098]

[0099] Among them, Ti Represents the theoretical value coordinate transformation matrix of the robot's i-th robot joint, i = 1, 2, 3, 4, 5, 6; Represents the theoretical value coordinate transformation matrix from the robot's end coordinate system to the robot's base coordinate system, which is also the robot's kinematic model; Represents the theoretical value coordinate transformation matrix from the camera coordinate system to the robot base coordinate system; Represents the theoretical coordinate transformation matrix from the camera coordinate system to the robot end coordinate system, also known as the hand-eye relationship.

[0100] Step 8. Fix the standard ball, control the robot to run until the camera can clearly image the standard ball, calculate the coordinates of the center of the standard ball in the camera coordinate system, and obtain the coordinates of the center of the standard ball in the robot base coordinate system by combining the kinematic model and the hand-eye relationship;

[0101] Specifically, the standard ball can be fixed (that is, the standard ball is fixed relative to the robot base coordinate system), and the robot can be controlled to run so that the camera can clearly image the standard ball. According to the processing of Step 3-Step 5, the coordinates (x s ,y s ,z s ), bring it into the model established in step 7, establish a relationship to solve the coordinates of the center of the standard ball in the robot base coordinate system, and calculate the coordinates of the center of the standard ball in the robot base coordinate system (X s ,Y s ,Z s );

[0102] The relationship between the coordinates of the center of the standard ball in the robot base coordinate system is:

[0103]

[0104] Step 9. According to the position of the center of the standard ball in the robot base coordinate system, randomly generate M camera poses in the robot base coordinate system with the clear and complete imaging range of the camera, so that the standard ball is in the clear and complete imaging range; the generated M poses are based on the established kinematic model and hand-eye relationship, and the robot kinematics inverse solution is used to calculate M groups of robot joint angles that can enable the camera to clearly image the standard ball, so that one group of joint angles corresponds to one pose;

[0105] Specifically, according to the position of the center of the standard ball in the robot base coordinate system, M camera poses in the robot base coordinate system can be randomly generated with the camera's clear and complete imaging range, so that the standard ball is in the clear and complete imaging range; the generated M poses are based on the model established in step 7, and the robot kinematics are inversely solved to obtain M groups of robot joint angles that enable the camera to clearly image the standard ball, so that a group of joint angles corresponds to one pose.

[0106] Step 10, use the camera to shoot the standard ball at each angle;

[0107] Specifically, the M groups of joint angles obtained by Step 9 can be used on the robot, and the standard ball can be photographed using a camera at each group of angles.

[0108] Step 11, extract the pixel coordinates of the center of the circle on the captured image. The captured image corresponding to each set of joint angles can extract a pixel coordinate of the center of the circle;

[0109] Specifically, the pixel coordinates of the center of the circle on the image captured in Step 10 can be extracted through the steps of Step 3-Step 4. The pixel coordinates of the center of the circle (p cjx ,p cjy ), where p cjx 、p cjy Represents the x- and y-direction coordinate values ​​of the center of the image collected at the j-th pose in the pixel coordinate system.

[0110] Step 12, in the robot base coordinate system, transform the origin of the camera coordinate system and the pixel coordinates of the center of the collected image into points on the camera normalized plane to establish a ray;

[0111] Define the ray:

[0112]

[0113] [x cj y cj 1] T =K -1 [p cjx p cjy 1] T

[0114]

[0115]

[0116] Among them, P cjis the coordinate of the origin of the camera coordinate system in the robot base coordinate system at the jth pose, that is, the starting point of the ray; Represents the theoretical value coordinate transformation matrix from the camera coordinate system to the robot base coordinate system at the jth pose x cj ,y cj is the center of the circle in the pixel coordinate system (p cjx ,p cjy ) is transformed into the value of the point in the camera normalized plane in the x direction and y direction in the camera coordinate system; K is the camera intrinsic parameter; P nj The pixel coordinates of the circle center at the jth pose are transformed into the coordinates of the point in the camera normalized plane in the robot base coordinate system, that is, the intersection point of the ray and the camera normalized plane; v j Represents the unit direction vector of the ray in the robot base coordinate system when the robot is in the jth posture; that is, M postures indicate that M rays can be defined; j = 1, 2, ..., M.

[0117] Step 13, calculate all the rays according to the collected data, and obtain the point with the shortest sum of distances to all the rays as the center point of the fitted standard sphere;

[0118] Specifically, all rays can be calculated based on the collected data, and the point with the shortest sum of distances to all rays can be obtained as the center point P of the fitted standard sphere using the following formula:

[0119]

[0120] Where I is the identity matrix.

[0121] Step 14, establish an equation for the error between the theoretical value coordinate transformation matrix and the actual value coordinate transformation matrix; establish the relationship between the robot end camera posture error and the kinematic parameter error;

[0122] Specifically, since the robot kinematics model and the hand-eye relationship established in step 7 contain errors, the error relationship between the theoretical value coordinate transformation matrix and the actual value coordinate transformation matrix is ​​expressed as:

[0123] T i R =T i +T i δ i

[0124] Among them, T i R Represents the coordinate transformation matrix T i The corresponding actual value coordinate transformation matrix, represents the differential operator of the i-th robot joint;

[0125] Establish the robot end camera pose error Relationship with kinematic parameter errors:

[0126]

[0127] in, The influence of the posture errors of the six robot joints on the camera posture error is shown respectively; e i =[ i dx i dy i dz i δx i δy i δz] T represents the pose error of the i-th joint, i = 1, 2, 3, 4, 5, 6; I is the unit matrix, e c is the hand-eye matrix error; J is abbreviation of ; G is the conversion matrix from kinematic parameter error to posture error of 6 robot joints; Δx1, Δx2, Δx3, Δx4, Δx5, Δx6 are the kinematic parameter errors of 6 joints respectively; ΔE is the overall kinematic parameter error.

[0128] Step 15, establish an error model between the robot end camera posture error and the converted ray error;

[0129] Specifically, the model is as follows:

[0130]

[0131] Among them, ΔP cj represents the P caused by the kinematic error of the robot at the jth position cj The change in ΔP nj represents the P caused by the kinematic error of the robot at the jth position nj The amount of change; represents the rotation matrix from the camera coordinate system to the jth position of the robot, O is the zero matrix; ΔP cj , ΔP nj are two ray error variables; for convenience, as an intermediate matrix.

[0132] Step 16, define the distance error d and the distance error vector [x j y j z j ] T , establish the ray error ΔP cj , ΔP njThe Jacobian matrix B of the relationship between the point-to-ray distance, the ray starting point and the ray midpoint is obtained by using the distance error model between the ray and the sphere center point P. j :

[0133] [x j y j z j ] T =(Iv j v j T )(PP cj )

[0134]

[0135]

[0136]

[0137] Among them, x j ,y j 、z j are the coordinate values ​​of the distance error vector in the x, y, and z directions in the robot base coordinate system.

[0138] Step 17. For a ray, construct the error equation according to Step 7-Step 16:

[0139] [x j y j z j ] T =B j ·M j ·J·G·ΔE

[0140] Simplify the above (abbreviate the left side as bj and the right side as B j ·M j ·J·G is abbreviated as A j ),get:

[0141] b j =A j ΔE

[0142] Since there are multiple rays when shooting multiple times, we can construct an equation similar to the above:

[0143]

[0144] The kinematic parameter error ΔE is solved by least square iteration, and the center point P is updated by Step 13 according to the latest kinematic parameters during the iteration process; when the calculated distance error d no longer decreases, the iteration is stopped, the kinematic parameter error is output, and the obtained kinematic parameter error is used as the kinematic parameter compensation value for compensation.

[0145] By observing the standard ball with a monocular camera, when there is a measurement error in the camera, avoid depth information and use ray information to represent the center of the standard ball. Through steps step7-step17, the identified kinematic parameters are compensated to the kinematic model in the robot controller to complete the calibration of the robot.

[0146] A robot kinematic parameter calibration system based on monocular mobile vision, comprising:

[0147] The acquisition module is used to control the robot to run to different specified positions and postures. The camera installed at the end of the robot acquires the spherical image fixed in the robot's base coordinate system.

[0148] A fitting module is used to transform the origin of the camera coordinate system and the pixel coordinates of the center of the collected image into points on the camera normalized plane in the robot base coordinate system to establish a ray, and fit the center point of the sphere through the ray;

[0149] An acquisition module is used to establish a kinematic parameter error and a distance error vector model between the ray and the center point of the sphere to obtain the identified kinematic parameter error;

[0150] The calibration module is used to compensate the identified kinematic parameter errors into the model, and iteratively update the coordinates of the sphere center point and the identified kinematic parameters; when the number of iterations reaches the set condition or the error obtained by the iteration is less than the threshold, the iteration is stopped, and the kinematic parameters calculated at this time are compensated into the robot controller to complete the calibration.

[0151] like Figure 2 , 3 As shown, before the calibration method of the present invention is used, the existing error is as shown in the curve before calibration; after the calibration method of the present invention is used, the existing error is as shown in the curve after calibration; through the simulation experiment results under multiple postures (number of points), it can be seen that after calibration, the distance error d is significantly reduced, which shows that by applying the error model to identify parameters, the present invention effectively improves the absolute positioning accuracy of the robot. At the same time, it can be seen that before the calibration method of the present invention is used, the distance error is of different sizes and fluctuates greatly; while the error after calibration of the present invention tends to be stable and the error is small. Further, with Figure 3Taking the results in as an example, the average distance error before calibration is 3.2112 mm, and the average distance error after calibration is 0.4583 mm. From the results, it can be seen that the distance error of the present invention is reduced by 85.73%.

[0152] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A robot kinematic parameter calibration method based on monocular mobile vision, characterized in that: include: The acquisition step controls the robot to move to different specified positions, and the camera installed at the end of the robot acquires the image of the sphere fixed in the robot base coordinate system; The fitting step is to transform the origin of the camera coordinate system and the pixel coordinates of the center of the collected image into points on the camera normalized plane in the robot base coordinate system to establish a ray, and fit the center point of the sphere through the ray; An acquisition step is to establish a kinematic parameter error and a distance error vector model between the ray and the center point of the sphere to obtain the identified kinematic parameter error; In the calibration step, the identified kinematic parameter errors are compensated into the model, and the coordinates of the ball center point and the identified kinematic parameters are updated iteratively. When the number of iterations reaches the set condition or the error obtained by the iteration is less than the threshold, the iteration is stopped, and the kinematic parameters calculated at this time are compensated into the robot controller to complete the calibration. The method also includes a step of determining a clear and complete imaging range of the sphere in the camera. The specific steps of the method are as follows: Calibrate the camera's internal parameters to obtain the camera's internal parameter K; The camera is used to acquire complete standard sphere images with clear edges at n different positions; The pixel coordinates of the pixels constituting the circular edge portion on the image are obtained by an image processing algorithm; A circle is fitted according to the pixel coordinates of the pixels at the edge of the circle by a circle fitting algorithm, thereby obtaining the pixel coordinates of the center of the circle; Select a pixel at any point on the circular edge and take its pixel coordinates; obtain the coordinates of the center of the standard sphere in the camera coordinate system based on the pixel coordinates of the center of the circle, the pixel coordinates of any point, the coordinates of the camera normalized plane, the camera intrinsic parameter K, and the radius R of the standard sphere; Obtain the coordinates of the center of the standard ball of n standard ball images in the camera coordinate system; obtain the clear and complete imaging range of the center of the standard ball in the camera coordinate system through the obtained n sphere center coordinate values; The relationship formula for obtaining the coordinates of the center of the standard ball in the camera coordinate system is: Among them, p cx 、p cy Respectively represent the x-direction and y-direction coordinate values ​​of the center of the circle in the pixel coordinate system, p rx 、p ry Respectively represent the x-direction and y-direction coordinate values ​​of a pixel of an arbitrary circular edge in the pixel coordinate system; s ,y s 、z s They represent the coordinate values ​​of the center of the standard ball in the x, y, and z directions in the camera coordinate system, respectively. r ,y r Respectively represent the pixel coordinates selected in the pixel coordinate system (p rx ,p ry ) is transformed to the coordinate values ​​of the point in the camera normalized plane in the x and y directions in the camera coordinate system through the camera intrinsic parameter K; R is the radius of the standard sphere; T represents transpose.

2. The robot kinematic parameter calibration method based on monocular mobile vision according to claim 1, characterized in that: When the center of the standard ball (x s ,y s ,z s )When the following relationship is satisfied in the camera coordinate system, the center of the standard sphere can be considered to be within the clear and complete imaging range of the camera: x min ·m1<x s <x max ·u1 <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> min <h2 style=";text-align:left;direction:ltr"> m2 <y <h2 style=";text-align:left;direction:ltr"> s <h2 style=";text-align:left;direction:ltr"><y <h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> ·u2 <h2 style=";text-align:left;direction:ltr">z<h2 style=";text-align:left;direction:ltr"> min <h2 style=";text-align:left;direction:ltr"> m3 <z <h2 style=";text-align:left;direction:ltr"> s <h2 style=";text-align:left;direction:ltr"><z <h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> ·u3 Among them, x min 、x max is the minimum and maximum value of the x-direction coordinate of the center of the standard ball in the n standard ball images in the camera coordinate system; y min ,y max is the minimum and maximum value of the y-direction coordinate of the center of the standard ball in the n standard ball images in the camera coordinate system; z min 、z max are the minimum and maximum values ​​of the z-direction coordinates of the center of the standard ball in the camera coordinate system of the n standard ball images; the values ​​of the weight coefficients m1, m2, and m3 are greater than or equal to 1, and the values ​​of the weight coefficients u1, u2, and u3 are less than or equal to 1.

3. The robot kinematic parameter calibration method based on monocular mobile vision according to claim 1, characterized in that: The method comprises: The robot's kinematic model is established based on the robot's nominal kinematic parameters, and the hand-eye relationship between the camera and the robot is obtained through hand-eye calibration; Fix the standard ball, control the robot to move so that the camera can clearly image the standard ball, calculate the coordinates of the center of the standard ball in the camera coordinate system, and combine the kinematic model and the hand-eye relationship to obtain the coordinates of the center of the standard ball in the robot base coordinate system; According to the position of the center of the standard ball in the robot base coordinate system, M camera poses in the robot base coordinate system are randomly generated with the clear and complete imaging range of the camera, so that the standard ball is in the clear and complete imaging range; the generated M poses are based on the established kinematic model and hand-eye relationship, and the robot kinematics inverse solution is used to calculate M groups of robot joint angles that can enable the camera to clearly image the standard ball, so that a group of joint angles corresponds to one pose; Use a camera to shoot a standard ball at each set of angles; Extract the pixel coordinates of the center of the circle on the captured image. A pixel coordinate of the center of the circle can be extracted from the captured image corresponding to each set of joint angles. In the robot base coordinate system, a ray is established by transforming the origin of the camera coordinate system and the pixel coordinates of the center of the collected image into points on the camera normalized plane; All rays are calculated based on the collected data, and the point with the shortest sum of distances to all rays is obtained as the center point of the fitted standard sphere; Establish an equation for the error between the theoretical value coordinate transformation matrix and the actual value coordinate transformation matrix; Establish the relationship between the robot end camera pose error and kinematic parameter error; Establish an error model between the robot end camera posture error and the converted ray error; Define the distance error and distance error vector, and obtain the Jacobian matrix of the relationship between the point-to-ray distance error vector, the ray starting point and the ray midpoint; Get the kinematic parameter errors.

4. The robot kinematic parameter calibration method based on monocular mobile vision according to claim 3 is characterized in that: The ray is defined as: [x cj y cj 1] T =K -1 [p cjx p cjy 1] T Among them, P cj is the coordinate of the origin of the camera coordinate system in the robot base coordinate system at the jth pose, that is, the starting point of the ray; Represents the theoretical value coordinate transformation matrix from the camera coordinate system to the robot base coordinate system at the jth pose x cj ,y cj is the center of the circle in the pixel coordinate system (p cjx ,p cjy ) is transformed into the value of the point in the camera normalized plane in the x direction and y direction in the camera coordinate system; K is the camera internal parameter; P nj The pixel coordinates of the circle center at the jth pose are transformed into the coordinates of the point in the camera normalized plane in the robot base coordinate system, that is, the intersection point of the ray and the camera normalized plane; v j Represents the unit direction vector of the ray in the robot's base coordinate system at the jth pose of the robot.

5. The robot kinematic parameter calibration method based on monocular mobile vision according to claim 3 is characterized in that: The equation for establishing the error between the theoretical value coordinate transformation matrix and the actual value coordinate transformation matrix is ​​expressed as: in, Represents the coordinate transformation matrix T i The corresponding actual value coordinate transformation matrix, δ i represents the differential operator of the i-th robot joint; Establish the robot end camera pose error Relationship with kinematic parameter errors: in, The influence of the posture errors of the six robot joints on the camera posture error is shown respectively; e i represents the pose error of the i-th joint, i = 1, 2, 3, 4, 5, 6; I is the unit matrix, e c is the hand-eye matrix error; J is abbreviation of ; G is the conversion matrix from kinematic parameter error to posture error of 6 robot joints; Δx1, Δx2, Δx3, Δx4, Δx5, Δx6 are the kinematic parameter errors of 6 joints respectively; ΔE is the overall kinematic parameter error.

6. The robot kinematic parameter calibration method based on monocular mobile vision according to claim 3 is characterized in that: The error model between the robot end camera posture error and the converted ray error is established: Among them, ΔP cj represents the P caused by the kinematic error of the robot at the jth position cj The change in ΔP nj represents the P caused by the kinematic error of the robot at the jth position nj The amount of change; represents the rotation matrix from the camera coordinate system to the jth position of the robot, O is the zero matrix; is the camera pose error at the end of the robot; x cj ,y cj is the center of the circle in the pixel coordinate system (p cjx ,p cjy ) is transformed into the x- and y-direction values ​​of the point in the camera normalized plane in the camera coordinate system.

7. The robot kinematic parameter calibration method based on monocular mobile vision according to claim 3 is characterized in that: The Jacobian matrix B of the relationship between the point-to-ray distance error vector, the ray starting point and the ray midpoint j Specifically: [x j y j z j ] T =(I-v j v j T )(P-P cj ) Among them, [x j y j z j ] T is the shortest vector from the jth ray to point P, which is called the distance error vector; v j Represents the unit direction vector of the ray in the robot's base coordinate system at the jth position; x j ,y j 、z j is the coordinate value of the distance error vector in the x, y and z directions in the robot base coordinate system; d is the distance error; ΔP cj , ΔP nj is the ray error, ΔP cj represents the P caused by the kinematic error of the robot at the jth position cj The change in ΔP nj represents the P caused by the kinematic error of the robot at the jth position nj The amount of change.

8. A robot kinematic parameter calibration system based on monocular mobile vision, used in the robot kinematic parameter calibration method based on monocular mobile vision as claimed in claim 1, characterized in that: include: The acquisition module is used to control the robot to run to different specified positions and postures. The camera installed at the end of the robot acquires the spherical image fixed in the robot's base coordinate system. A fitting module is used to transform the origin of the camera coordinate system and the pixel coordinates of the center of the collected image into points on the camera normalized plane in the robot base coordinate system to establish a ray, and fit the center point of the sphere through the ray; An acquisition module is used to establish a kinematic parameter error and a distance error vector model between the ray and the center point of the sphere to obtain the identified kinematic parameter error; The calibration module is used to compensate the identified kinematic parameter errors into the model, and iteratively update the coordinates of the sphere center point and the identified kinematic parameters; when the number of iterations reaches the set condition or the error obtained by the iteration is less than the threshold, the iteration is stopped, and the kinematic parameters calculated at this time are compensated into the robot controller to complete the calibration.

Images