Industrial mechanical arm zero deviation calibration method based on vision

Abstract

The invention belongs to the field of industrial robot error calibration, and particularly relates to a vision-based industrial mechanical arm zero deviation calibration method, which comprises the following steps of: 1) fixing a vision camera at the tail end of a mechanical arm, and fixing a marker in a working area range of the mechanical arm; (2) the mechanical arm is manually operated, the spatial position of a camera fixed to the tail end of the mechanical arm is changed, all corresponding pose data are recorded, the mechanical arm is moved multiple times, and multiple sets of calibration data are collected; (3) the pose change of the marker when the spatial position of the camera fixed to the tail end of the mechanical arm is changed is obtained, and a calibration equation based on zero deviation and hand-eye pose deviation is constructed; and (4) multiple sets of collected calibration data are substituted into the calibration equation, and after optimization is conducted, the zero deviation of the mechanical arm and a hand-eye pose transformation matrix are obtained and compensated to configuration of a mechanical arm controller. According to the method, the zero deviation calibration of the mechanical arm and the calibration of the hand-eye pose transformation between the mechanical arm and the visual equipment can be completed at the same time only through one calibration process.

Description

Technical Field

[0001] The invention belongs to the field of industrial robot error calibration, and in particular is a vision-based industrial robot arm zero position deviation calibration method. Background Art

[0002] The six-degree-of-freedom industrial robot plays an important role in industrial production. Combined with the end tool, it can perform a variety of fine processes such as welding, spraying, grinding, and assembly. The rise of technologies such as offline programming and teaching-free has put forward higher requirements on the absolute positioning accuracy of the robot body, as well as the hand-eye calibration accuracy between the robot and the matching visual equipment. The existing hand-eye calibration schemes are all performed after the robot accuracy calibration. If there is a deviation in the kinematic parameters of the robot body, it will seriously affect the results of the hand-eye calibration. When the robot leaves the factory, the basic kinematic parameters are usually preliminarily calibrated. However, the joint zero position of the robot may still be lost during transportation or use.

[0003] Many robot arm manufacturers have provided zero-position retrieval or calibration solutions. Some calibration methods based on special equipment are costly and difficult to apply on a large scale; the methods mentioned in the invention patent with publication number: CN107717993A, entitled: An efficient and convenient simple robot calibration method and the invention patent with publication number: CN113211436A, entitled: Six-degree-of-freedom serial robot error calibration method based on genetic algorithm need to collect calibration data based on the judgment of the human eye, which is cumbersome and difficult to avoid introducing errors. Summary of the invention

[0004] The purpose of the present invention is to provide a calibration method for zero-position deviation calibration of an industrial robot arm based on vision, which can simultaneously complete the calibration of the zero-position deviation of the robot arm and the calibration of the posture transformation between the robot arm and the visual device through only one calibration process. The method uses a three-dimensional or two-dimensional visual camera installed on the robot to collect the position information of the marker fixed on the workbench for multiple times, so as to realize the calibration of the zero-position deviation of the robot arm and the hand-eye posture, reduce the error caused by human factors, improve the calibration efficiency and accuracy, and overcome the defects of the prior art.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a vision-based industrial robot arm zero deviation calibration method, comprising the following steps:

[0006] 1) Fix the visual camera at the end of the robotic arm and fix the marker within the working area of ​​the robotic arm;

[0007] 2) Keep the marker always within the visual range of the camera, manually operate the robotic arm, change the spatial position of the camera fixed at the end of the robotic arm, record all corresponding posture data, and move the robotic arm multiple times to collect multiple sets of calibration data;

[0008] 3) Obtain the change in the position and posture of the marker when the spatial position of the camera fixed at the end of the robotic arm is changed, and construct a calibration equation based on the zero position deviation and the hand-eye posture deviation according to the change in the position and posture of the marker;

[0009] 4) The collected multiple sets of calibration data are brought into the calibration equation, and after optimization, the robot arm zero position deviation and hand-eye posture transformation matrix are obtained, and then compensated to the configuration of the robot arm controller to complete the deviation calibration.

[0010] The visual camera is any one of a three-dimensional camera for collecting point clouds or a two-dimensional array camera for collecting image information, a line laser sensor, and a point laser sensor;

[0011] The marker is any one of a calibration sphere, a calibration cone or a calibration block of a three-dimensional calibration object, or the marker is any one of a checkerboard calibration plate or other types of calibration plates of a two-dimensional calibration object.

[0012] In the step 2), when the visual camera is selected as a three-dimensional camera for collecting point clouds, a three-dimensional calibration object is fixed within the working area of ​​the robot arm;

[0013] The calibration data is: the rotation angle value θ of each joint of the robot arm i (i=1,2,3,4,5,6), and the spatial position P(x,y,z) of the center of the calibration ball in the 3D camera coordinate system;

[0014] The rotation angle value of the robot arm is obtained in the controller of the robot arm, and the coordinate position of the center of the sphere is obtained by performing spherical surface processing on the point cloud data taken by the three-dimensional camera.

[0015] In the step 3), the change in the position and posture of the calibration ball when the spatial position of the camera fixed at the end of the robotic arm is changed is specifically:

[0016] When the visual camera is selected as the three-dimensional camera for collecting point clouds, the base coordinate system of the robot is set at point B, the end of the robot is at point E, the origin of the camera is at point C, and the center position of the calibration ball is A. Then the position transformation of the calibration ball in the base coordinate system is: T BA ; The position transformation from the robot end to the camera origin remains unchanged, which is T EC ; From the perspective of the visual camera, the pose transformation of the calibration ball is expressed as:

[0017]

[0018] in, is the position transformation of the center of the calibration ball under the camera’s perspective at the i-th posture, Indicates the position transformation of the robot base coordinates relative to the robot end at the i-th posture.

[0019] In the step 3), according to the change of the position and posture of the marker, a calibration equation based on the zero position deviation and the hand-eye position and posture deviation is constructed, which is specifically:

[0020] When the visual camera is selected as the 3D camera for collecting point clouds, the 3D camera can only detect displacement information when measuring the calibration sphere, and cannot obtain posture transformation. The zero position of the robot axis 1 can be any position, so the zero position of the 1st axis does not participate in the calibration during the zero position deviation calibration. When collecting multiple sets of data for calibration, the parameters of each link of the robot and the values ​​of each joint of the robot under different postures are set to be known, then the transformation matrix of the six-axis robot base relative to the end is a function of the zero position deviation δθ of each joint, that is:

[0021]

[0022] Substitute the function of the zero position deviation δθ of each joint into formula (1) and extract The position part of the robot is obtained by calibrating the zero position deviation of the vision-based robot as follows:

[0023]

[0024] Among them, (x CA ,y CA ,z CA ) is the position coordinate value of the calibration ball in the visual camera coordinate system, (x BA ,y BA ,z BA ) is the position coordinate of the calibration ball relative to the robot base.

[0025] In the step 2), when the visual camera is selected as any one of a two-dimensional array camera, a line laser sensor, and a point laser sensor for collecting image information;

[0026] The calibration object adopts a two-dimensional calibration object, such as a checkerboard calibration plate or other types of calibration plates;

[0027] The calibration data is as follows: fix the two-dimensional camera at the end of the robotic arm, shoot the checkerboard calibration plate in different postures, record the joint position of the robotic arm during each shooting, and the position value of the checkerboard calibration plate relative to the two-dimensional camera.

[0028] The step 4) is specifically:

[0029] 4-1) Multiple sets of calibration data obtained by selecting any combination of visual cameras and markers are put into the error calibration equation, and the calibration results are obtained after optimization;

[0030] 4-2) After calibration, the robot joint zero position deviation δθ is obtained. i (i=2,3,4,5,6), and obtain the relative position transformation relationship T between the robot end and the camera origin CE , and the position coordinates of the calibration ball relative to the robot base (x BA ,y BA ,z BA ) and compensate it into the configuration of the robot controller to complete the deviation calibration.

[0031] A vision-based industrial robot arm zero-position deviation calibration system comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the vision-based industrial robot arm zero-position deviation calibration method when executing the computer program.

[0032] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the vision-based industrial robot arm zero-position deviation calibration method is implemented.

[0033] The present invention has the following beneficial effects and advantages:

[0034] 1. The present invention proposes a vision-based industrial robot arm zero-position deviation calibration method, which uses a three-dimensional or two-dimensional visual camera installed on the robot to repeatedly collect the position information of the marker fixed on the workbench, so as to realize the calibration of the robot arm zero-position deviation and hand-eye posture, and use vision to collect marker information to reduce errors caused by human factors and improve calibration efficiency and accuracy.

[0035] 2. The method proposed in the present invention can simultaneously complete the calibration of the robot arm zero-position deviation and the calibration of the hand-eye posture transformation between the robot arm and the visual device through only one calibration process, which effectively improves the calibration efficiency and calibration accuracy.

[0036] 3. The method proposed in the present invention is applicable to the kinematic parameter calibration and hand-eye calibration of any serial robot arm. The calibration equipment is low-cost, users do not need special training, and it is easy to be applied in batches in existing robot arm products. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the working process of a method for calibrating a robot arm joint zero position deviation and a hand-eye posture deviation in one embodiment of the present invention;

[0038] Figure 2It is a schematic diagram of the principle of performing joint zero position and hand-eye posture deviation calibration in one embodiment of the present invention, where the visual device is a three-dimensional camera and the marker is a standard calibration ball;

[0039] Figure 3 It is a schematic diagram of the principle of performing joint zero position and hand-eye posture deviation calibration in one embodiment of the present invention, where the visual device is a two-dimensional camera and the marker is a standard chessboard calibration plate;

[0040] Figure 4 It is a schematic diagram of a process in which a robotic arm carries a three-dimensional vision camera and collects calibration data around a standard calibration sphere in one embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0042] like Figure 1 As shown, a calibration method for zero-position deviation calibration of an industrial robot arm based on vision proposed in the present invention comprises the following steps:

[0043] 1) Fix the visual camera at the end of the robotic arm and fix the marker within the working area of ​​the robotic arm;

[0044] 2) Keep the marker always within the visual range of the camera, manually operate the robotic arm, change the spatial position of the camera fixed at the end of the robotic arm, record all corresponding posture data, and move the robotic arm multiple times to collect multiple sets of calibration data;

[0045] a. When the visual camera is selected as a 3D camera for collecting point clouds, a 3D calibration object is fixed within the working area of ​​the robot arm; the calibration data is: the rotation angle value θ of each joint of the robot arm i (i=1,2,3,4,5,6), and the spatial position P(x,y,z) of the center of the calibration ball in the 3D camera coordinate system;

[0046] The rotation angle value of the robot arm is obtained in the controller of the robot arm, and the coordinate position of the center of the sphere is obtained by performing spherical surface processing on the point cloud data taken by the three-dimensional camera.

[0047] b. When the visual camera is selected as any one of a two-dimensional array camera, a line laser sensor, and a point laser sensor for collecting image information;

[0048] The calibration object adopts a two-dimensional calibration object, such as a checkerboard calibration plate or other types of calibration plates;

[0049] The calibration data is as follows: fix the two-dimensional camera at the end of the robotic arm, shoot the checkerboard calibration plate in different postures, record the joint position of the robotic arm during each shooting, and the position value of the checkerboard calibration plate relative to the two-dimensional camera.

[0050] 3) Obtain the change in the position and posture of the marker when the spatial position of the camera fixed at the end of the robotic arm is changed, and construct a calibration equation based on the zero position deviation and the hand-eye posture deviation according to the change in the position and posture of the marker;

[0051] In the present invention, no matter whether a three-dimensional camera or a two-dimensional camera is selected and combined with the corresponding marker, the calibration data collected is the same, that is: the coordinates of the marker relative to the camera + the joint angle value.

[0052] 4) The collected multiple sets of calibration data are brought into the calibration equation, and after optimization, the robot arm zero position deviation and hand-eye posture transformation matrix are obtained, and then compensated to the configuration of the robot arm controller to complete the deviation calibration.

[0053] The key technical points of the present invention are:

[0054] 1) In the above step 1), the visual camera fixed at the end of the robot arm can be any visual device, including but not limited to a three-dimensional camera for collecting point clouds, a two-dimensional array camera for collecting image information, a line laser sensor, a point laser sensor, etc. The fixed marker can be any object whose coordinate position can be detected by the visual camera, including but not limited to a standard calibration ball, a chessboard, a calibration block, a calibration cone, etc. By adopting a visual measurement solution, the deviation calibration result is only related to the accuracy of the visual device, thereby avoiding calibration errors caused by human factors;

[0055] 2) The error calibration model in step 3) above combines the hand-eye posture deviation of the robot arm and the visual device with the motion parameter deviation of the robot arm itself, realizing a one-time calibration of the hand-eye deviation and the body deviation, effectively improving the calibration efficiency and accuracy.

[0056] In the above steps, the steps are further described in conjunction with the embodiments. Figure 1 As shown, the vision-based industrial robot arm zero-position deviation calibration method provided in this embodiment 1 includes the following steps:

[0057] Step 1: Fix the visual device at the end of the robot arm and fix the marker at the working area of ​​the robot arm. In this embodiment, the visual device selects a 3D camera, and the marker selects a standard calibration sphere. The installation position relationship with the robot arm is as follows: Figure 4 shown.

[0058] Step 2.1: Keep the calibration ball within the visible range of the 3D camera, manually move the robotic arm, and change the position and posture of the 3D camera. The state of the robotic arm collecting data is shown in the attached figure. Figure 4 shown.

[0059] Step 2.2: Record the rotation angle values ​​of each joint of the robot arm at different positions and postures: θ i (i=1,2,3,4,5,6), and the spatial position P(x,y,z) of the center of the calibration ball in the 3D camera coordinate system. The rotation angle value of the robot arm can be read in the controller of the robot arm, and the coordinate position of the center of the ball can be obtained by performing spherical processing on the point cloud data captured by the 3D camera.

[0060] Step 2.3: Move the robotic arm multiple times. In this embodiment, 30 sets of data are collected.

[0061] Step 3: Establish an error calibration model.

[0062] like Figure 2 As shown, the robot's base coordinate system is at point B, the robot's end is at point E, the camera's origin is at point C, and the center of the calibration ball is A. Then the position transformation of the calibration ball in the base coordinate system is T BA ; The position transformation from the robot end to the camera origin remains unchanged, which is T EC When the robot's joint angles are at different positions, the robot's end will change to positions such as E1, E2, etc. Then, from the camera's perspective, the calibration ball's position transformation can be expressed as follows:

[0063]

[0064] in, is the position transformation of the center of the calibration ball under the camera’s perspective at the i-th posture, Indicates the position transformation of the robot base coordinates relative to the robot end at the i-th posture. Since the 3D camera can only detect displacement information when measuring the calibration ball, it cannot obtain the posture transformation, so Only the displacement changes, and the attitude transformation is always a unit matrix. At the same time, there is only a positional relationship between the center position A of the calibration ball and the base of the robot arm, T BA It is also a homogeneous matrix with only displacement information but no posture transformation. Based on the above reasons, the zero position of the robot axis 1 can be any position, so the zero position of the axis 1 does not participate in the calibration when calibrating the zero position deviation. When collecting multiple sets of data for calibration, it is assumed that the parameters of each link of the robot and the values ​​of each joint of the robot in different postures are known. Then the transformation matrix of the six-axis robot base relative to the end can be regarded as a function of the zero position deviation δθ of each joint, which can be written as:

[0065]

[0066] Substitute it into formula (1) and remove From the position part, we can get the basic formula for the robot zero deviation calibration based on vision:

[0067]

[0068] Step 4.1: Substitute multiple sets of calibration data into the error calibration equation (3) and obtain the calibration result after optimizing.

[0069] Step 4.2: After calibration, the robot joint zero position deviation δθ is obtained. i (i=2,3,4,5,6) at the same time, the relative position transformation relationship T between the robot end and the camera origin can also be obtained CE , and the position coordinates of the calibration ball relative to the robot base (x BA ,y BA ,z BA ). For robot applications combined with cameras, it will effectively improve the efficiency and accuracy of robot body parameter calibration and hand-eye calibration between the robot and the camera.

[0070] Embodiment 2 (Extended Embodiment): The three-dimensional camera and the measured marker in the above embodiment 1 can be replaced with other devices according to the needs of the site. The three-dimensional camera can be replaced with a two-dimensional array camera, a line laser sensor, a point laser sensor, etc. The corresponding measured marker can be a two-dimensional checkerboard calibration plate, a calibration block, etc. Figure 3 As shown, a two-dimensional camera is fixed at the end of the robotic arm, and the checkerboard calibration plate is photographed in different postures. The joint positions and the position values ​​of the checkerboard relative to the camera are recorded at each shooting. The data obtained in this way can also be used for the calibration method proposed in the present invention. The calibration effect depends on the resolution of the camera used.

[0071] To summarize, the two embodiments of the present invention utilize a three-dimensional or two-dimensional visual camera installed on a robot to repeatedly collect position information of a marker fixed on a workbench, thereby realizing the calibration of the robot arm's zero-position deviation and the hand-eye posture. Both embodiments only require one calibration process to simultaneously complete the calibration of the robot arm's zero-position deviation and the calibration of the hand-eye posture transformation between the robot arm and the visual device, thereby effectively improving the calibration efficiency and calibration accuracy.

[0072] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0073] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A vision-based industrial robot arm zero deviation calibration method, characterized in that: The following steps are involved: 1) Fix the visual camera at the end of the robotic arm and fix the marker within the working area of ​​the robotic arm; 2) Keep the marker always within the visual range of the camera, manually operate the robotic arm, change the spatial position of the camera fixed at the end of the robotic arm, record all corresponding posture data, and move the robotic arm multiple times to collect multiple sets of calibration data; 3) Obtain the change in the position and posture of the marker when the spatial position of the camera fixed at the end of the robotic arm is changed, and construct a calibration equation based on the zero position deviation and the hand-eye posture deviation according to the change in the position and posture of the marker; 4) The collected multiple sets of calibration data are brought into the calibration equation, and after optimization, the robot arm zero position deviation and hand-eye posture transformation matrix are obtained, and then compensated to the configuration of the robot arm controller to complete the deviation calibration.

2. The method for zero deviation calibration of an industrial robot arm based on vision according to claim 1, characterized in that: The visual camera is any one of a three-dimensional camera for collecting point clouds or a two-dimensional array camera for collecting image information, a line laser sensor, and a point laser sensor; The marker is any one of a calibration sphere, a calibration cone or a calibration block of a three-dimensional calibration object, or the marker is any one of a checkerboard calibration plate or other types of calibration plates of a two-dimensional calibration object.

3. The method for zero deviation calibration of an industrial robot arm based on vision according to claim 1, characterized in that: In the step 2), when the visual camera is selected as a three-dimensional camera for collecting point clouds, a three-dimensional calibration object is fixed within the working area of ​​the robot arm; The calibration data is: the rotation angle value θ of each joint of the robot arm i (i=1,2,3,4,5,6), and the spatial position P(x,y,z) of the center of the calibration ball in the 3D camera coordinate system; The rotation angle value of the robot arm is obtained in the controller of the robot arm, and the coordinate position of the center of the sphere is obtained by performing spherical surface processing on the point cloud data taken by the three-dimensional camera.

4. The method for zero deviation calibration of an industrial robot arm based on vision according to claim 1, characterized in that: In the step 3), the change in the position and posture of the calibration ball when the spatial position of the camera fixed at the end of the robotic arm is changed is specifically: When the visual camera is selected as the three-dimensional camera for collecting point clouds, the base coordinate system of the robot is set at point B, the end of the robot is at point E, the origin of the camera is at point C, and the center position of the calibration ball is A. Then the position transformation of the calibration ball in the base coordinate system is: T BA ; The position transformation from the robot end to the camera origin remains unchanged, which is T EC ; From the perspective of the visual camera, the pose transformation of the calibration ball is expressed as: in, is the position transformation of the center of the calibration ball under the camera’s perspective at the i-th posture, Indicates the position transformation of the robot base coordinates relative to the robot end at the i-th posture.

5. The method for zero deviation calibration of an industrial robot arm based on vision according to claim 1, characterized in that: In the step 3), according to the change of the position and posture of the marker, a calibration equation based on the zero position deviation and the hand-eye position and posture deviation is constructed, which is specifically: When the visual camera is selected as the 3D camera for collecting point clouds, the 3D camera can only detect displacement information when measuring the calibration sphere, and cannot obtain posture transformation. The zero position of the robot axis 1 can be any position, so the zero position of the 1st axis does not participate in the calibration during the zero position deviation calibration. When collecting multiple sets of data for calibration, the parameters of each link of the robot and the values ​​of each joint of the robot under different postures are set to be known, then the transformation matrix of the six-axis robot base relative to the end is a function of the zero position deviation δθ of each joint, that is: Substitute the function of the zero position deviation δθ of each joint into formula (1) and extract The position part of the robot is obtained by calibrating the zero position deviation of the vision-based robot as follows: Among them, (x CA ,y CA ,z CA ) is the position coordinate value of the calibration ball in the visual camera coordinate system, (x BA ,y BA ,z BA ) is the position coordinate of the calibration ball relative to the robot base.

6. The method for zero deviation calibration of an industrial robot arm based on vision according to claim 1, characterized in that: In the step 2), when the visual camera is selected as any one of a two-dimensional array camera, a line laser sensor, and a point laser sensor for collecting image information; The calibration object adopts a two-dimensional calibration object, such as a checkerboard calibration plate or other types of calibration plates; The calibration data is as follows: fix the two-dimensional camera at the end of the robotic arm, shoot the checkerboard calibration plate in different postures, record the joint position of the robotic arm during each shooting, and the position value of the checkerboard calibration plate relative to the two-dimensional camera.

7. The method for zero deviation calibration of an industrial robot arm based on vision according to claim 1, characterized in that: The step 4) is specifically: 4-1) Multiple sets of calibration data obtained by selecting any combination of visual cameras and markers are put into the error calibration equation, and the calibration results are obtained after optimization; 4-2) After calibration, the robot joint zero position deviation δθ is obtained. i (i=2,3,4,5,6), and obtain the relative position transformation relationship T between the robot end and the camera origin CE , and the position coordinates of the calibration ball relative to the robot base (x BA ,y BA ,z BA ) and compensate it into the configuration of the robot controller to complete the deviation calibration.

8. A vision-based industrial robot arm zero deviation calibration system, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement a vision-based industrial robot arm zero-position deviation calibration method as described in any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, a vision-based industrial robot arm zero-position deviation calibration method as described in any one of claims 1 to 7 is implemented.

Images