A hand-eye calibration device and calibration method for a camera outside a robotic arm

Abstract

The present invention discloses a hand-eye calibration device for a camera outside a robot arm and a calibration method thereof. The hand-eye calibration device includes a robot body, a calibration plate, and an image coordinate acquisition device. A robot arm is arranged on the robot body, and the robot arm and the image coordinate acquisition device are arranged above the calibration plate; the calibration plate is a three-dimensional calibration plate, and a plurality of calibration points are arranged on the upper surface of the calibration plate, and the spatial coordinates of the plurality of calibration points are all different. The calibration method includes the following steps: S1, establishing a coordinate system U1 through the point coordinates of the calibration points in the image coordinate system, and establishing a coordinate system U2 through the world coordinates of the robot end at the calibration point position; S2, setting a calibration equation; S3, solving the calibration equation to obtain a conversion matrix; S4, using verification points to perform error verification on the conversion matrix.

Description

Technical Field

[0001] The present invention relates to the field of robots, and in particular to a hand-eye calibration device for a camera outside a robot arm and a calibration method thereof. Background Art

[0002] With the development of science and technology, robots have been widely used and gradually become the core productivity. In the promotion and application of robots, users have higher and higher requirements for the absolute accuracy of robots. Before leaving the factory, manufacturers will calibrate the robots to improve their absolute accuracy. Before the autonomous mobile robot is put into use, it is necessary to calibrate the robot's visual module and other components to ensure that the robot is normal and meets the use standards. In addition, as the robot is used for a longer time, it may cause wheel wear, changes in mechanical structure, etc., which will cause the initial calibration accuracy to decrease. When the accuracy drops to a certain level, the relevant functions of the robot will be seriously affected. Therefore, the calibration of the robot is very important.

[0003] In the field of robotics, the use of visual methods for pose estimation to help the robot achieve accurate target grasping has important applications. Among them, hand-eye calibration is a very basic and critical issue. Simply put, the purpose of hand-eye calibration is to obtain the relationship between the robot coordinate system and the camera coordinate system, and finally transfer the results of visual recognition to the robot coordinate system. The accuracy of the calibration result often depends to a large extent on the accuracy of the test data. Therefore, accurate acquisition of the calibration plate data is the key to the accuracy of the calibration. The current calibration method mostly uses high-precision ceramic balls or chessboards to obtain feature values ​​for hand-eye calibration. This single-feature method is difficult to ensure the accuracy of feature data, thereby reducing the accuracy of the hand-eye calibration results and affecting the overall work quality of industrial robots. Hand-eye calibration can be divided into two categories according to the installation method of the camera: eye-to-hand and eye-in-hand. The former refers to the camera being installed on a fixed bracket, and its position is fixed relative to the base of the robot arm. The latter refers to the camera being installed on the end effector of the robot arm, and its position is fixed relative to the end effector of the robot arm. Most existing hand-eye calibration methods use a calibration board to perform calibration by iteratively solving two types of high-dimensional nonlinear matrix equations: AX=XB or AX=YB.

[0004] The existing calibration method generally involves setting up high-precision equipment such as laser trackers on the measuring equipment for calibration. This method is not only expensive, but also the processor processes too much data and is prone to lag. Although it can improve the trajectory accuracy of the robot, the operation is complicated and the cost is high, which is not conducive to the implementation of robot calibration operations. In addition, when using this method to calibrate the robot, both the site preparation and the calibration test process require professional personnel, resulting in a high threshold and cost for robot calibration; therefore, it is necessary to improve it. Summary of the invention

[0005] The object of the present invention is to provide a hand-eye calibration device and a calibration method for a camera outside a robotic arm that is simple to operate and easy to use in order to solve the above-mentioned problems.

[0006] In order to achieve the above object, the technical solution of the present invention is:

[0007] A hand-eye calibration device for a camera outside a robotic arm comprises a robot body, a calibration plate, and an image coordinate acquisition device, wherein the robot body, the calibration plate, and the image coordinate acquisition device are all fixed on the ground, a robotic arm is arranged on the robot body, and the robotic arm and the image coordinate acquisition device are both arranged above the calibration plate; the calibration plate is a three-dimensional calibration plate, and a plurality of calibration points are arranged on the upper surface of the calibration plate, and the spatial coordinates of the plurality of calibration points are all different.

[0008] Furthermore, the robot body includes a fixed slide rail, a movable plate, a first rotary servo, a second rotary servo, and a robotic arm. The fixed slide rail is fixed on the ground, the movable plate is arranged at the upper end of the fixed slide rail and the movable plate is slidably connected to the fixed slide rail, a push-pull servo is arranged between the movable plate and the fixed slide rail, and the push-pull servo controls the movable plate to slide on the fixed slide rail; the first rotary servo is fixedly connected to the movable plate, the output shaft of the first rotary servo is fixedly connected to the second rotary servo, and the output shaft of the second rotary servo is fixedly connected to the robotic arm.

[0009] Furthermore, the calibration plate includes a fixed connecting plate, a first trapezoidal block, and a second trapezoidal block. The fixed connecting plate, the first trapezoidal block, and the second trapezoidal block are integrally formed. The first trapezoidal block and the second trapezoidal block are both located at the upper end of the fixed connecting plate, and the bottom end of the first trapezoidal block and the bottom end of the second trapezoidal block are respectively located at two ends of the fixed connecting plate. The height of the first trapezoidal block is lower than the height of the second trapezoidal block. The calibration point is set on the upper end surface of the fixed connecting plate or the inclined surface of the first trapezoidal block and the second trapezoidal block.

[0010] Furthermore, the hand-eye calibration device also includes a robot controller and a calibration data processor. The robot controller is connected to the push-pull servo, the first rotary servo, and the second rotary servo. The calibration data processor is respectively connected to the robot controller and the image coordinate acquisition device for communication.

[0011] Furthermore, the calibration data processor includes a data transmission module, a calibration execution module, and an error verification module. The signal output end of the data transmission module is connected to the signal input end of the calibration execution module, and the signal output end of the calibration execution module is connected to the signal input end of the error verification module.

[0012] A method for calibrating a hand-eye calibration device with a camera outside a robotic arm comprises the following steps:

[0013] S1, read the coordinates L1, L2, L3 of the calibration point in the image coordinate system, establish the coordinate system U1 according to the coordinates L1, L2, L3, and read the world coordinate points P1, P2, P3 of the robot end at the calibration point position, and establish the coordinate system U2 according to the world coordinate points P1, P2, P3;

[0014] S2, setting calibration equation;

[0015] S3. Solve the calibration equation to obtain the conversion matrix

[0016] S4. Use the verification point pair transformation matrix Perform error verification.

[0017] Furthermore, the step S1 specifically includes the following steps:

[0018] S11, use the image coordinate reading device to read the image coordinates of the three calibration points B1, B2, and B3 in the calibration plate, which are: L 1 (a 1 , b 1 , c 1 ), L 2 (a 2 , b 2 , c 2 ), L 3 (a 3 , b 3 , c 3 );

[0019] S12, use the robot controller to control the end of the robot arm to move to the calibration points B1, B2, and B3 in sequence, and record the coordinates of the end of the robot arm in the world coordinate system at this time, which are P 1 (x 1 ,y 1 , z 1 ), P 2 (x 2 ,y 2 , z 2 ), P 3 (x 3 ,y 3, z 3 ) and the rotation angle Γ at each coordinate point 1 (α 1 , β 1 , γ 1 ), Γ 2 (α 2 , β 2 , γ 2 ), Γ 3 (α 3 , β 3 , γ 3 );

[0020] S13. Use L 1 (a 1 , b 1 , c 1 ), L 2 (a 2 , b 2 , c 2 ), L 3 (a 3 , b 3 , c 3 ) Establish coordinate system U1;

[0021] L 2 (a 2 , b 2 , c 2 ) is used as the origin to establish the coordinate system U1 and find the vector V 1 ,V 2 , and set V 1 The direction is the positive direction of the X-axis of the coordinate system U1; then:

[0022]

[0023] V 2 =L 3 -L 2 =(a 3 -a 2 ,b 3 -b 2 ,c 3 -c 2 );

[0024] Then the positive direction of the Z axis of the coordinate system U1 is the vector V 1 ,V 2 The positive direction of the Y axis of the coordinate system U1 is the vector product of OX and OZ;

[0025]

[0026]

[0027] The rotation matrix R from the world coordinate W to the coordinate system U1 can be obtained;

[0028]

[0029] Obtain the transformation matrix from world coordinates W to coordinate system U1;

[0030]

[0031] With point coordinates P 1 (x 1 ,y 1 , z 1 ), P 2 (x 2 ,y 2 , z 2 ), P 3 (x 3 ,y 3 , z 3 ) is multiplied by the transformation matrix to obtain the fixed point P in the world coordinate system 1 (x 1 ,y 1 , z 1 ), P 2 (x 2 ,y 2 , z 2 ), P 3 (x 3 ,y 3 , z 3 ) points B1, B2, B3 in coordinate system U1;

[0032]

[0033]

[0034]

[0035] S13, use point coordinates P 1 (x 1 ,y 1 , z 1 ), P 2 (x 2 ,y 2 , z 2 ), P 3 (x 3 ,y 3 , z 3 ) Establish coordinate system U2 and find the transformation matrix from world coordinates to coordinate system U2:

[0036]

[0037] Use the calibration point coordinates L in the image coordinate system 1 (a 1 , b 1 , c 1 ), L 2 (a 2 , b 2 , c 2 ), L 3 (a 3 , b 3 , c 3 ) is multiplied by the transformation matrix to obtain the corresponding points A1, A2, A3 in the coordinate system U2:

[0038]

[0039]

[0040]

[0041] At this time, the origins and coordinate system directions of coordinate system U1 and coordinate system U2 completely coincide.

[0042] Furthermore, the step S2 specifically includes the following steps:

[0043] S21. Assume that the conversion equation of the first motion of the end of the robot arm is:

[0044]

[0045]

[0046]

[0047] S22. Substitute the coordinate points A1, A2, A3 and B1, B2, B3 transformed by coordinate system U1 and coordinate system U2; get:

[0048]

[0049]

[0050]

[0051] Calibration plate coordinate system U g and the robot base coordinate system U w They are relatively static, so the conversion relationship between them is a constant. After eliminating it, we can get:

[0052]

[0053]

[0054]

[0055] Multiply both sides by

[0056]

[0057]

[0058]

[0059] Simplifying, we get:

[0060] X*A=B;

[0061] Therefore, the transformation matrix between the robot end coordinate system and the camera coordinate system is This is the variable X.

[0062] Furthermore, the step S3 specifically includes the following steps:

[0063] S31, converting the rotation matrix into a rotation vector;

[0064]

[0065] S32, vector normalization;

[0066]

[0067] S33, modified Rodriguez parameter indicating posture change;

[0068]

[0069] S34, calculate the initial rotation vector P ce ′;

[0070] skew(P eij +P cij ) c ' e =P cij -P eij ;

[0071] Among them, skew means taking the antisymmetric matrix of this vector;

[0072] S35, calculating the rotation vector;

[0073]

[0074] S36, calculating the rotation matrix;

[0075]

[0076] S37, calculating the translation vector;

[0077]

[0078] S38, calculating a translation matrix;

[0079]

[0080] S39, obtain the conversion matrix for:

[0081]

[0082] Furthermore, the step S4 specifically includes the following steps:

[0083] S41, select the i-th verification point in the space, and use the image coordinate reading device to read the image coordinate A of the current verification point i1 =(a i1 ,b i1 ,c i1 );

[0084] S42, control the end of the robotic arm to move to the verification point, and read the world coordinate B of the end of the robotic arm at the verification point i =(x i ,y i ,z i );

[0085] S43, the world coordinate B i =(x i ,y i ,z i ) into the solved transformation matrix The calculated value A of the image coordinates of the verification point is obtained i2 =(a i2 ,b i2 ,c i2 );

[0086] Calculate the calibration error of the verification point;

[0087]

[0088] S44, after calculating the calibration errors of n verification points, find the average value Δerror of the calibration errors of all verification points;

[0089]

[0090] S45, determine whether the average value Δerror is greater than the error threshold of 0.2mm; if it is greater than the error threshold of 0.2mm, it means that the calibration point selection is incorrect or the required transformation matrix is ​​incorrect; if it is less than the error threshold of 0.2mm, it means that the calibration error is within the allowable range and the transformation matrix is ​​correct. The solution is correct.

[0091] Compared with the prior art, the present invention has the following advantages and positive effects:

[0092] When performing the calibration operation, the present invention only needs to move the robot's mechanical arm above the calibration plate. The entire calibration process is mainly completed by the robot to be calibrated, the image coordinate acquisition device, and the calibration data processor. The operation steps are simple, which lowers the threshold for robot calibration operations; at the same time, the calibration accuracy is high, and the error can be less than 0.2mm, which effectively improves the calibration accuracy of the robot; on the other hand, the present invention does not require precise measuring instruments when performing the calibration operation, which has a huge cost advantage compared to the laser tracker with a maintenance and purchase cost of millions of dollars. In addition, it is not restricted by the environment and light when performing the calibration operation, and can be applied to most production environments, and calibration can be performed at any time. At the same time, it is simple to operate, convenient and fast, and reduces a large number of working steps of the handling robot, further improving the use effect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0094] Figure 1 It is a system schematic diagram of the calibration device;

[0095] Figure 2 It is a schematic diagram of the structure of the calibration device;

[0096] Figure 3 It is the structural diagram of the calibration board;

[0097] Figure 4 is a schematic diagram of the locations of the calibration points;

[0098] Figure 5 is a schematic cross-sectional view of the calibration plate;

[0099] Figure 6 The framework diagram of the calibration data processor;

[0100] Figure 7 The framework flow chart of the calibration method;

[0101] Figure 8 The flowchart of the error verification framework. DETAILED DESCRIPTION

[0102] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work, any modifications, equivalent substitutions, improvements, etc., should be included in the protection scope of the present invention.

[0103] like Figure 1 , Figure 2 , Figure 3 As shown, the present invention provides a robot calibration device, including a robot body 1 and a robot controller 4 communicatively connected to the robot body 1, a calibration data processor 3, a calibration board 2, and an image coordinate acquisition device 5;

[0104] The robot body 1, the calibration plate 2, and the image coordinate acquisition device 5 are all fixed on the ground. A mechanical arm 101 is arranged on the robot body 1. The mechanical arm 101 and the image coordinate acquisition device 5 are both arranged above the calibration plate 2. The calibration plate 2 is a three-dimensional calibration plate. A plurality of calibration points are arranged on the upper surface of the calibration plate 2, and the spatial coordinates of the plurality of calibration points are all different.

[0105] The robot body 1 includes a fixed slide rail 101, a movable plate 102, a first rotary servo 103, a second rotary servo 104, and a mechanical arm 105. The fixed slide rail 101 is fixed on the ground, the movable plate 102 is arranged at the upper end of the fixed slide rail 101 and the movable plate 102 is slidably connected to the fixed slide rail 101, a push-pull servo is arranged between the movable plate 102 and the fixed slide rail 101, and the push-pull servo controls the movable plate 102 to slide on the fixed slide rail 101; the first rotary servo 103 is fixedly connected to the movable plate 102, the output shaft of the first rotary servo 103 is fixedly connected to the second rotary servo 104, and the output shaft of the second rotary servo 104 is fixedly connected to the mechanical arm 105. The spatial movement of the mechanical arm is realized by the coordinated operation of the push-pull servo, the first rotary servo, and the second rotary servo, and the structure is simple, the cost is low, and the use effect of the present invention is improved.

[0106] The calibration plate 2 includes a fixed connecting plate 201, a first trapezoidal block 202, and a second trapezoidal block 203. The fixed connecting plate 201, the first trapezoidal block 202, and the second trapezoidal block 203 are integrally formed. The first trapezoidal block 202 and the second trapezoidal block 203 are both located at the upper end of the fixed connecting plate 201, and the bottom ends of the first trapezoidal block 202 and the second trapezoidal block 203 are respectively located at both ends of the fixed connecting plate 201. The height of the first trapezoidal block 202 is lower than the height of the second trapezoidal block 203. The calibration point is set on the upper end surface of the fixed connecting plate 201 or on the inclined surfaces of the first trapezoidal block 202 and the second trapezoidal block 203.

[0107] The calibration plate is different from the ordinary checkerboard calibration plate. The calibration plate is a three-dimensional structure. There is a large height difference and horizontal displacement between each calibration point in the calibration plate, which makes the spatial coordinates (x, y, z) between the calibration points have a large difference in the x, y, z directions. This ensures that the subsequent solution of the transformation matrix will not be overfitted, making the calibration result more accurate.

[0108] The robot controller 4 can be integrated into the robot body 1; the robot controller 1 can also be a device independent of the robot body 1, such as a control cabinet, etc.; it should be noted that both the controller and the control cabinet have a hardware system that can run software, including, for example, a processor (MCU or CPU, etc. or other implementations), a memory (ROM or RAM, etc. or other implementations). Various motion parameters of the robot body 1 can be read in the robot controller 4, such as the position (rotation angle and spatial coordinates) of the end of the robot arm in the world coordinate system; in addition, the robot arm of the robot body 1 is a multi-axis robot arm, which is a multi-purpose manipulator that can realize automatic control, reprogrammable, multi-degree-of-freedom, and the degrees of freedom of motion are built into a spatial right-angle relationship. Its working behavior is mainly through completing linear motion along the X, Y, and Z axes. Among them, the coordinate system of the 6-axis robot includes three translation quantities of X, Y, and Z and three rotation quantities of R (X), R (Y), and R (Z) rotating around these three axes respectively.

[0109] The robot controller 4 and the calibration data processor 3 are connected to each other through wired Ethernet, and the calibration data processor 3 and the image coordinate acquisition device 5 are connected to each other through wired Ethernet; the communication protocols include: Ethernet TCP / IP, Modbus, RS232 and any other standard protocol or custom protocol.

[0110] The image coordinate acquisition device 5 may be a laser sensor, a camera or a data acquisition device combining the two, and is communicatively connected to the calibration data processor 3 to acquire the image coordinates of the calibration points in the calibration plate 2 .

[0111] The calibration board 2 is used to provide calibration points for the robot body 1 and the image coordinate acquisition device 5 to acquire the world coordinates and image coordinates of the calibration points. Unlike ordinary checkerboard calibration boards, the calibration board 2 is a three-dimensional device. There are four calibration points in the calibration board, and there is a height difference between the calibration points, so that the spatial coordinates (x, y, z) of each calibration point are different. Since there are large distances between the calibration points in the x, y and z directions, this makes the subsequent transformation matrix calculation and calibration results more accurate; the structural diagram of the calibration board is shown in FIG. Figure 3 , Figure 4 , Figure 5 As shown;

[0112] The calibration data processor 3 is connected to the robot controller 4 and the image coordinate acquisition device 5. The calibration data processor 3 can be a computer, an industrial computer, a microcomputer or any device with data processing function, or an information upload terminal connected to these devices; the calibration data processor 3 can be independently set or additionally installed on the image coordinate acquisition device 5. Figure 6 As shown, the calibration data processor includes a data transmission module, a calibration execution module and an error verification module.

[0113] Data transmission module: used to read the world coordinate system coordinates of the robot end in the robot controller 4 and the image coordinates of the calibration points in the image coordinate acquisition device 5.

[0114] Calibration execution module: The calibration execution module is used to generate the conversion equation AX=B for the two sets of coordinates in the robot end world coordinate system and the image coordinate system, and finally solve the conversion matrix X between the image coordinate system and the robot end coordinate system.

[0115] The specific calibration process of the above calibration device is as follows: Figure 7 As shown, the relative position relationship between the camera and the robot is obtained by solving the calibration equation. The equation requires the input of multiple sets of calibration plate poses in the camera coordinate system and the corresponding robot arm end poses in the robot coordinate system. The pose of the robot arm end in the robot coordinate system can be directly obtained from the robot system, and the pose of the calibration plate in the camera coordinate system allows the camera to take pictures of the calibration plate. After fixing the calibration plate and the camera to a certain position, under the default premise that the position of the calibration plate or the camera is known, the robot is allowed to move according to the calibration position in the calibration plate. The camera takes a picture once to obtain the image coordinates of all the calibration points in the calibration plate. After the entire calibration process is completed, the obtained calibration relationship is substituted into the entire robot system, and the camera is allowed to command the robot to move to a certain point. The error size is compared between the point where the robot arm actually moves and the point input by the command.

[0116] The specific calibration steps are as follows:

[0117] Step 1: Read the coordinates of the calibration point in the image coordinate system L1, L2, L3, and establish the coordinate system U1 based on the three coordinates. At the same time, read the world coordinates P1, P2, P3 of the robot end at the calibration point position, and establish the coordinate system U2 based on these three coordinates. At this time, the origin and direction of U1 and U2 completely coincide;

[0118] Step 2: Convert L1, L2, L3 to the U2 coordinate system to obtain coordinates A1, A2, A3. Substitute A1, A2, A3 and P1, P2, P3 into the equation X*A=B;

[0119] Step 3: Solve the transformation matrix;

[0120] Step 4: Use the verification points to perform error verification on the transformation matrix.

[0121] The step 1 specifically includes:

[0122] 1.1. Fix the image coordinate reading device 5 above the calibration plate 2, and keep the position of the calibration plate 2 unchanged. Use the image coordinate reading device 5 to read the image coordinates of the three calibration points B1, B2, and B3 on the calibration plate 2: L 1 (a 1 , b 1 , c 1 ), L 2 (a 2 , b 2 , c 2 ), L 3 (a 3 , b 3 , c 3 );

[0123] 1.2. Use the robot controller 4 to control the end of the robot arm of the robot body 1 to move to the calibration points B1, B2, and B3 in turn, and record the coordinates P of the end of the robot in the world coordinate system at this time. 1 (x 1 ,y 1 , z 1 ), P 2 (x 2 ,y 2 , z 2 ), P 3 (x 3 ,y 3 , z 3 ), and the rotation angle Γ at this time 1 (α 1 , β 1 , γ 1 ), Γ 2 (α 2 , β2 , γ 2 ), Γ 3 (α 3 , β 3 , γ 3 );

[0124] 1.3 Use L 1 (a 1 , b 1 , c 1 ), L 2 (a 2 , b 2 , c 2 ), L 3 (a 3 , b 3 , c 3 ) Establish coordinate system U1;

[0125] L 2 Establish a coordinate system for the origin and find the vector V 1 ,V 2 , and set V 1 The direction is the positive direction of the X axis:

[0126]

[0127] V 2 =L 3 -L 2 =(a 3 -a 2 ,b 3 -b 2 ,c 3 -c 2 );

[0128] Then the positive direction of the Z axis is vector V 1 ,V 2 The positive direction of the Y axis is the vector product of OX and OZ;

[0129]

[0130]

[0131] The rotation matrix R and translation matrix t of the world coordinate W to the U1 coordinate system are obtained as follows:

[0132]

[0133] Finally, the transformation matrix from world coordinate W to U1 coordinate system can be obtained:

[0134]

[0135] With point coordinates P1 (x 1 ,y 1 , z 1 ), P 2 (x 2 ,y 2 , z 2 ), P 3 (x 3 ,y 3 , z 3 ) is multiplied by the transformation matrix to obtain the fixed point P in the world coordinate system 1 (x 1 ,y 1 , z 1 ), P 2 (x 2 ,y 2 , z 2 ), P 3 (x 3 ,y 3 , z 3 )Points B1, B2, B3 in the U1 coordinate system:

[0136]

[0137]

[0138]

[0139] 1.4. Similarly, use the point coordinates P 1 (x 1 ,y 1 , z 1 ), P 2 (x 2 ,y 2 , z 2 ), P 3 (x 3 ,y 3 , z 3 ) Establish coordinate system U2 and find the transformation matrix from world coordinates to U2 coordinates:

[0140]

[0141] Use the calibration point coordinates L in the image coordinate system 1 (a 1 , b 1 , c 1 ), L 2 (a 2 , b 2 , c 2 ), L 3 (a 3 , b3 , c 3 ) is multiplied by the transformation matrix to obtain the corresponding points A1, A2, A3 in the U2 coordinate system:

[0142]

[0143]

[0144]

[0145] At this time, the origin and direction of U1 and U2 completely coincide.

[0146] The step 2 specifically includes:

[0147] Derivation of eye-outside-hand calibration equations;

[0148] Parameter Description:

[0149] U g : Calibration plate coordinate system;

[0150] U w : World coordinate system (robotic arm base coordinate system);

[0151] U c : Camera coordinate system;

[0152] U e : Robot arm end coordinate system;

[0153] The conversion relationship between the camera coordinate system and the robot base coordinate system;

[0154] The transformation relationship between the robot end coordinate system and the robot base coordinate system;

[0155] The conversion relationship between the camera coordinate system and the calibration plate coordinate system;

[0156] The conversion relationship between the calibration plate coordinate system and the robot arm base coordinate system;

[0157] Problem derivation;

[0158] The camera coordinate system and the robot base coordinate system are fixed. During the calibration process, the movement of the end of the robot arm is controlled, so the coordinate system of the end of the robot arm and the calibration plate coordinate system are moving and changing.

[0159] 2.1. Assume that the conversion equation of the first motion is:

[0160]

[0161]

[0162]

[0163] 2.2. Substitute the coordinates A1, A2, A3 and B1, B2, B3 of the calibration points recorded in the first motion into the coordinate systems of U1 and U2;

[0164]

[0165]

[0166]

[0167] Calibration plate coordinate system U g and the robot base coordinate system U w are relatively static, so their conversion relationship is a constant, which will be eliminated:

[0168]

[0169]

[0170]

[0171] Multiply both sides by

[0172]

[0173]

[0174]

[0175] Simplifying, we get:

[0176] X*A=B;

[0177] Transformation Matrix Can be read directly, conversion matrix Obtained from camera calibration. The transformation matrix between the robot end coordinate system and the camera coordinate system This is the variable X we need.

[0178] The step 3 specifically includes:

[0179] Solve the calibration equation using a two-step method:

[0180] First solve the rotation matrix from the above calibration equation Substitute it into the solution to get the translation matrix The calculation steps are as follows:

[0181] 3.1. Convert the rotation matrix into a rotation vector;

[0182]

[0183] 3.2, vector normalization;

[0184]

[0185] 3.3, Modified Rodriguez parameter represents posture change;

[0186]

[0187] 3.4. Calculate the initial rotation vector P ce ′;

[0188] skew(P eij +P cij )P′ ce =P cij -P eij ;

[0189] Among them, skew means taking the antisymmetric matrix of this vector.

[0190] 3.5. Calculate the rotation vector;

[0191]

[0192] 3.6. Calculate the rotation matrix;

[0193]

[0194] 3.7. Calculate the translation vector;

[0195]

[0196] 3.8. Calculate the translation matrix;

[0197]

[0198] 3.9. Obtain the transformation matrix for:

[0199]

[0200] Among them, the translation matrix is ​​a matrix with three rows and one column, and the rotation matrix is ​​a matrix with three rows and three columns. The final transformation matrix is is a matrix with four rows and four columns.

[0201] When a calibration process is completed, the error of the calibration result must be measured to determine whether the currently calculated calibration result is basically correct and can meet the accuracy requirements of the present invention; the present invention proposes an error measurement method that can quantify both the rotation and translation parts of the calibration result. The method obtains the image coordinates of the verification point and the robot end coordinates in space, and then substitutes the obtained transformation matrix, compares the actual space coordinates of the verification point with the space coordinates obtained using the matrix equation, and obtains the error value. By setting the error threshold, it is possible to determine whether the transformation matrix is ​​correct and whether the actual coordinates of the verification point are obtained incorrectly.

[0202] Step 4 is as follows Figure 8 As shown, the specific steps are as follows:

[0203] 4.1. Select the i-th verification point in the space and use the image coordinate reading device to read the image coordinate A of the current verification point i1 =(a i1 ,b i1 ,c i1 );

[0204] 4.2. Move the robot end to the verification point and read the world coordinates B of the robot end at that point. i =(x i ,y i ,z i );

[0205] 4.3. Substitute coordinate B1 into the solved transformation matrix The calculated value A of the image coordinates of the verification point can be obtained i2 =(a i2 ,b i2 ,c i2 );

[0206] Calculate the calibration error at this point:

[0207]

[0208] 4.4. The preset error threshold is 0.2, in mm (millimeter). i If it is greater than 0.2, it is determined that the verification point is selected incorrectly and the verification point needs to be reselected.

[0209] 4.5. After selecting n verification points, calculate the average value Δerror of all errors;

[0210]

[0211] Determine whether Δerror is greater than the error threshold of 0.2 mm. If it is greater, it means that the calibration point selection is incorrect and the required transformation matrix is ​​incorrect. If it is less than , it means that the calibration error is within the allowable range and the transformation matrix solution is correct.

[0212] When performing the calibration operation, the present invention only needs to move the robot's mechanical arm above the calibration plate. The entire calibration process is mainly completed by the robot to be calibrated, the image coordinate acquisition device, and the calibration data processor. The operation steps are simple, which lowers the threshold for robot calibration operations; at the same time, the calibration accuracy is high, and the error can be less than 0.2mm, which effectively improves the calibration accuracy of the robot; on the other hand, the present invention does not require precise measuring instruments when performing the calibration operation, which has a huge cost advantage compared to the laser tracker with a maintenance and purchase cost of millions of dollars. In addition, it is not restricted by the environment and light when performing the calibration operation, and can be applied to most production environments, and calibration can be performed at any time. At the same time, it is simple to operate, convenient and fast, and reduces a large number of working steps of the handling robot, further improving the use effect of the present invention.

Claims

1. A hand-eye calibration device with a camera outside a robotic arm, characterized in that: The hand-eye calibration device comprises a robot body, a calibration plate, and an image coordinate acquisition device, wherein the robot body, the calibration plate, and the image coordinate acquisition device are all fixed on the ground, a mechanical arm is arranged on the robot body, and the mechanical arm and the image coordinate acquisition device are arranged above the calibration plate; the calibration plate is a three-dimensional calibration plate, and a plurality of calibration points are arranged on the upper surface of the calibration plate, and the spatial coordinates of the plurality of calibration points are all different; The robot body comprises a fixed slide rail, a movable plate, a first rotary servo, a second rotary servo, and a mechanical arm, wherein the fixed slide rail is fixed on the ground, the movable plate is arranged at the upper end of the fixed slide rail and the movable plate is slidably connected to the fixed slide rail, a push-pull servo is arranged between the movable plate and the fixed slide rail, and the movable plate is controlled to slide on the fixed slide rail by the push-pull servo; the first rotary servo is fixedly connected to the movable plate, the output shaft of the first rotary servo is fixedly connected to the second rotary servo, and the output shaft of the second rotary servo is fixedly connected to the mechanical arm; The calibration plate includes a fixed connecting plate, a first trapezoidal block, and a second trapezoidal block. The fixed connecting plate, the first trapezoidal block, and the second trapezoidal block are integrally formed. The first trapezoidal block and the second trapezoidal block are both located at the upper end of the fixed connecting plate, and the bottom end of the first trapezoidal block and the bottom end of the second trapezoidal block are respectively located at both ends of the fixed connecting plate. The height of the first trapezoidal block is lower than the height of the second trapezoidal block. The calibration point is set on the upper end surface of the fixed connecting plate or the inclined surface of the first trapezoidal block and the second trapezoidal block.

2. The hand-eye calibration device with a camera outside a robot arm as claimed in claim 1, characterized in that: The hand-eye calibration device also includes a robot controller and a calibration data processor. The robot controller is connected to the push-pull servo, the first rotary servo, and the second rotary servo. The calibration data processor is respectively connected to the robot controller and the image coordinate acquisition device for communication.

3. The hand-eye calibration device with a camera outside a robot arm as claimed in claim 2, characterized in that: The calibration data processor includes a data transmission module, a calibration execution module, and an error verification module. The signal output end of the data transmission module is connected to the signal input end of the calibration execution module, and the signal output end of the calibration execution module is connected to the signal input end of the error verification module.

4. A calibration method for the hand-eye calibration device of the camera outside the robot arm as claimed in claim 3, characterized in that: The following steps are involved: S1, read the coordinates L1, L2, L3 of the calibration point in the image coordinate system, establish the coordinate system U1 according to the coordinates L1, L2, L3, and read the world coordinate points P1, P2, P3 of the robot end at the calibration point position, and establish the coordinate system U2 according to the world coordinate points P1, P2, P3; S2, setting calibration equation; S3, solving the calibration equation to obtain the conversion matrix; S4. Use the verification points to perform error verification on the transformation matrix.

5. The calibration method of the hand-eye calibration device of the camera outside the robot arm as claimed in claim 4, characterized in that: The step S1 specifically includes the following steps: S11, using an image coordinate reading device to read the image coordinates of the three calibration points B1, B2, and B3 in the calibration plate, which are respectively: L1 (a1, b1, c1), L2 (a2, b2, c2), and L3 (a3, b3, c3); S12, use the robot controller to control the end of the robot arm of the robot body to move to the calibration points B1, B2, and B3 in turn, and record the coordinates of the end of the robot arm in the world coordinate system at this time, which are P1 (x1, y1, z1), P2 (x2, y2, z2), P3 (x3, y3, z3) and the rotation angles Γ1 (α1, β1, γ1), Γ2 (α2, β2, γ2), Γ3 (α3, β3, γ3) at each coordinate point; S13. Use L1 (a1, b1, c1), L2 (a2, b2, c2), and L3 (a3, b3, c3) to establish a coordinate system U1; Establish the coordinate system U1 with L2 (a2, b2, c2) as the origin, find the vectors V1 and V2, and set the direction of V1 as the positive direction of the X-axis of the coordinate system U1; then: V2=L3-L2=(a3-a2,b3-b2,c3-c2); Then the positive direction of the Z axis of the coordinate system U1 is the vector product of vectors V1 and V2, and the positive direction of the Y axis of the coordinate system U1 is the vector product of OX and OZ; The rotation matrix from world coordinate W to coordinate system U1 can be obtained Obtain the transformation matrix from world coordinate W to coordinate system U1 Use point coordinates P1(x1, y1, z1), P2(x2, y2, z2), P3(x3, y3, z3) and the transformation matrix Multiply them to get the world coordinate system subscript points P1 (x1, y1, z1), P2 (x2, y2, z2), P3 (x3, y3, z3) in the coordinate system U1 at points B1, B2, B3; S13. Use point coordinates P1 (x1, y1, z1), P2 (x2, y2, z2), P3 (x3, y3, z3) to establish coordinate system U2, and calculate the transformation matrix from world coordinates to coordinate system U2 Use the calibration point coordinates L1 (a1, b1, c1), L2 (a2, b2, c2), L3 (a3, b3, c3) in the image coordinate system and the transformation matrix Multiply them to get the corresponding points A1, A2, A3 in coordinate system U2: At this time, the origins and coordinate system directions of coordinate system U1 and coordinate system U2 completely coincide.

6. The calibration method of the hand-eye calibration device of the camera outside the robot arm as claimed in claim 5, characterized in that: The step S2 specifically includes the following steps: S21. Assume that the conversion equation of the first motion of the end of the robot arm is: S22. Substitute the coordinate points A1, A2, A3 and B1, B2, B3 transformed by coordinate system U1 and coordinate system U2; get: Calibration plate coordinate system U g and the robot base coordinate system U w They are relatively static, so the conversion relationship between them is a constant. After eliminating it, we can get: Multiply both sides by Simplifying, we get: X*A=B; Therefore, the transformation matrix between the robot end coordinate system and the camera coordinate system is This is the variable X.

7. The calibration method of the hand-eye calibration device of the camera outside the robot arm as claimed in claim 6, characterized in that: The step S3 specifically comprises the following steps: S31, converting the rotation matrix into a rotation vector; S32, vector normalization; S33, modified Rodriguez parameter indicating posture change; S34, calculate the initial rotation vector P ce ′; skew(P eij +P cij )P′ ce =P cij -P eij ; Among them, skew means taking the antisymmetric matrix of this vector; S35, calculate the rotation vector P ce ; S36. Calculate the rotation matrix S37, calculating the translation vector; S38, calculating a translation matrix; S39, obtain the conversion matrix for:

8. The calibration method of the hand-eye calibration device of the camera outside the robot arm as claimed in claim 7, characterized in that: The step S4 specifically comprises the following steps: S41, select the i-th verification point in the space, and use the image coordinate reading device to read the image coordinate A of the current verification point i1 =(a i1 ,b i1 ,c i1 ); S42, control the end of the robotic arm to move to the verification point, and read the world coordinate B of the end of the robotic arm at the verification point i =(x i ,y i ,z i ); S43, the world coordinate B i =(x i ,y i ,z i ) into the solved transformation matrix The calculated value A of the image coordinates of the verification point is obtained i2 =(a i2 ,b i2 ,c i2 ); Calculate the calibration error of the verification point; S44, after calculating the calibration errors of n verification points, find the average value Δerror of the calibration errors of all verification points; S45, determine whether the average value Δerror is greater than the error threshold of 0.2mm; if it is greater than the error threshold of 0.2mm, it means that the calibration point selection is incorrect or the required transformation matrix is ​​incorrect; if it is less than the error threshold of 0.2mm, it means that the calibration error is within the allowable range and the transformation matrix is ​​correct. The solution is correct.

Images