A vision-based industrial robot zero deviation calibration method

Abstract

The present application belongs to the field of industrial robot error calibration, and specifically relates to a vision-based industrial robot zero deviation calibration method, which comprises the following steps: 1) fixing a vision camera at the end of a robot arm and fixing a marker in the working area of the robot arm; 2) manually operating the robot arm to change the spatial position of the camera fixed at the end of the robot arm, recording all corresponding pose data, and moving the robot arm multiple times to collect multiple sets of calibration data; 3) obtaining the pose change of the marker when the spatial position of the camera fixed at the end of the robot arm is changed, and constructing a calibration equation based on zero deviation and hand-eye pose deviation; 4) bringing the collected multiple sets of calibration data into the calibration equation, and after optimization, obtaining the robot zero deviation and the hand-eye pose transformation matrix, and compensating into the configuration of the robot controller. The present application can simultaneously complete the calibration of the robot zero deviation and the calibration of the hand-eye pose transformation between the robot arm and the vision device through only one calibration process.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of industrial robot error calibration, in particular to a vision-based industrial robot zero deviation calibration method. BACKGROUND

[0002] Six-degree-of-freedom industrial robots play an important role in industrial production. Combined with end tools, they can perform various fine processes such as welding, spraying, grinding, and assembly. The rise of offline programming and demonstration-free technology has put forward higher requirements for the absolute positioning accuracy of the robot body and the hand-eye calibration accuracy between the robot and the visual equipment. Existing hand-eye calibration schemes are performed after robot accuracy calibration. If the kinematic parameters of the robot body have deviations, it will seriously affect the results of hand-eye calibration. When the robot is shipped, it usually performs preliminary calibration of basic kinematic parameters. However, the loss of joint zero position of the robot may still occur during transportation or use.

[0003] Many robot manufacturers provide zero position finding or calibration schemes. Some calibration methods based on special equipment are costly and difficult to apply widely. The methods mentioned in the invention patent with the publication number CN107717993A and the name A high-efficiency and convenient simple robot calibration method and the invention patent with the publication number CN113211436A and the name Six-degree-of-freedom serial robot error calibration method based on genetic algorithm require manual visual judgment to collect calibration data, which is tedious and difficult to avoid errors. SUMMARY

[0004] The purpose of the present application is to provide a vision-based industrial robot zero deviation calibration method, which simultaneously completes robot zero deviation calibration and robot and vision device pose transformation calibration through a calibration process. This method uses a three-dimensional or two-dimensional vision camera installed on the robot to collect position information of a marker fixed on the workbench multiple times to achieve robot zero deviation and hand-eye pose calibration, reduce errors caused by human factors, improve calibration efficiency and accuracy, and overcome the defects of the prior art.

[0005] The technical solution adopted by the present application to achieve the above purpose is: a vision-based industrial robot zero deviation calibration method, comprising the following steps:

[0006] 1) Fix the vision camera at the end of the robot and fix the marker in the robot work area;

[0007] 2) Keep the marker in the visual range of the camera all the time, manually operate the mechanical arm, change the spatial position of the camera fixed at the end of the mechanical arm, record all the pose data, and move the mechanical arm multiple times to collect multiple sets of calibration data;

[0008] 3) Obtain the pose change of the marker when the spatial position of the camera fixed at the end of the mechanical arm is changed, and construct a calibration equation based on zero deviation and hand-eye pose deviation according to the pose change of the marker;

[0009] 4) Bring the collected multiple sets of calibration data into the calibration equation, and after optimization, obtain the mechanical arm zero deviation and hand-eye pose transformation matrix, and compensate it into the configuration of the mechanical arm controller to complete the deviation calibration.

[0010] The visual camera is any one of a three-dimensional camera for collecting point clouds, a two-dimensional area array camera for collecting image information, a line laser sensor, and a point laser sensor;

[0011] The marker is any one of a calibration ball, a calibration cone, or a calibration block of a three-dimensional calibration object, or any one of a checkerboard calibration board or other type of calibration board of a two-dimensional calibration object.

[0012] In the step 2), when the visual camera is a three-dimensional camera for collecting point clouds, a three-dimensional calibration object is fixed in the working area range of the mechanical arm;

[0013] The calibration data are the rotation angle values θ i of each joint of the mechanical arm (i=1, 2, 3, 4, 5, 6), and the spatial position P(x, y, z) of the center of the calibration ball in the three-dimensional camera coordinate system. i

[0014] The rotation angle values of the mechanical arm are obtained in the controller of the mechanical arm, and the center coordinate position is obtained by performing spherical processing on the point cloud data photographed by the three-dimensional camera.

[0015] In the step 3), the pose change of the calibration ball when the spatial position of the camera fixed at the end of the mechanical arm is changed is obtained, specifically:

[0016] When the visual camera is a three-dimensional camera for collecting point clouds, the base coordinate system of the robot is at point B, the end of the robot is at point E, the origin of the camera is at point C, and the center of the calibration ball is at point A. The pose transformation of the calibration ball in the base coordinate system is T BA ; the pose transformation from the end of the robot to the origin of the camera is always unchanged, which is T EC ; and the pose transformation of the calibration ball in the visual camera view is:

[0017]

[0018] wherein,​ a position transformation of a ball center of the calibration ball in a camera perspective at the i-th pose, a pose transformation of a robot base coordinate relative to a robot end at the i-th pose.

[0019] In the step 3), the calibration equation based on the zero position deviation and the hand-eye pose deviation is constructed according to the pose change of the marker, specifically:

[0020] When the vision camera is selected as a three-dimensional camera for collecting point clouds, since the three-dimensional camera can only detect displacement information when measuring the calibration ball and cannot obtain the pose transformation, the zero position of the robot arm 1 axis can be any position, and therefore the zero position of the 1 axis does not participate in the calibration during the zero position deviation calibration; in the case of collecting multiple groups of data for calibration, the parameters of each link of the robot arm and the values of each joint of the robot at different poses are set as known, and therefore the transformation matrix of the six-axis robot base relative to the end is a function of the zero position deviation δθ of each joint, that is:

[0021]

[0022] The function of the zero position deviation δθ of each joint is brought into formula (1), and the position part of is taken out, and therefore the calibration equation of the robot zero position deviation based on vision is:

[0023]

[0024] where (x CA , y CA , z CA ) is the position coordinate value of the calibration ball in the perspective of the vision camera coordinate system, and (x BA , y BA , z BA ) is the position coordinate of the calibration ball relative to the robot base.

[0025] In the step 2), when the vision camera is selected as any one of a two-dimensional area array camera, a line laser sensor, and a point laser sensor for collecting image information;

[0026] The calibration object adopts a chessboard calibration plate or other types of calibration plates of a two-dimensional calibration object;

[0027] Therefore, the calibration data is that the two-dimensional camera is fixed at the end of the robot arm, and the chessboard calibration plate is shot at different poses, and the position value of the chessboard calibration plate relative to the two-dimensional camera is recorded at each shooting time.

[0028] The step 4) is specifically:

[0029] ​4-1) Substitute multiple sets of calibration data obtained by selecting any combination of vision cameras and markers into the error calibration equation, and obtain the calibration result after optimization;

[0030] 4-2) After calibration, the zero-position deviation δθ of the robot joint is obtained. i While obtaining the relative pose transformation relationship T between the robot end effector and the camera origin (i = 2, 3, 4, 5, 6), the relative pose transformation relationship T between the robot end effector and the camera origin is also obtained. CE And the position coordinates (x, y) of the calibrated ball relative to the robot base. BA ,y BA ,z BA The error is then compensated and incorporated into the configuration of the robotic arm controller to complete the deviation calibration.

[0031] A vision-based industrial robotic arm zero-position deviation calibration system includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the vision-based industrial robotic arm zero-position deviation calibration method when the computer program is executed.

[0032] A computer-readable storage medium storing a computer program that, when executed by a processor, implements a vision-based industrial robotic arm zero-position deviation calibration method.

[0033] The present invention has the following beneficial effects and advantages:

[0034] 1. The present invention proposes a vision-based industrial robotic arm zero-position deviation calibration method, which uses a three-dimensional or two-dimensional vision camera installed on the robot to repeatedly collect position information of a marker fixed on the worktable, thereby calibrating the zero-position deviation of the robotic arm and the hand-eye posture. By using vision to collect marker information, errors caused by human factors are reduced, and calibration efficiency and accuracy are improved.

[0035] 2. The method proposed in this invention can simultaneously complete the zero-position deviation calibration of the robotic arm and the calibration of the hand-eye pose transformation between the robotic arm and the vision device through only one calibration process, effectively improving calibration efficiency and calibration accuracy.

[0036] 3. The method proposed in this invention is applicable to the kinematic parameter calibration and hand-eye calibration of any serial robotic arm. The calibration equipment is inexpensive, requires no special training from users, and is easy to apply in batches to existing robotic arm products. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the workflow of the method for calibrating the zero-position deviation of the robotic arm joint and the hand-eye posture deviation in one embodiment of the present invention.

[0038] Figure 2is an embodiment of the present application, the visual device is a three-dimensional camera, the marker is a standard calibration ball, and the principle diagram of executing joint zero position and hand-eye pose deviation calibration is shown in the figure.

[0039] Figure 3 is an embodiment of the present application, the visual device is a two-dimensional camera, the marker is a standard chessboard calibration plate, and the principle diagram of executing joint zero position and hand-eye pose deviation calibration is shown in the figure.

[0040] Figure 4 is an embodiment of the present application, the mechanical arm carries a three-dimensional visual camera, surrounds a standard calibration ball, and the process diagram of collecting calibration data is shown in the figure. DETAILED DESCRIPTION

[0041] The embodiments of the present application are described in detail below: the embodiments are implemented on the premise of the technical solutions of the present application, and detailed implementation modes and specific operation processes are given. It should be noted that, for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application.

[0042] As shown in Figure 1 , a calibration method for industrial robot zero deviation calibration based on vision proposed by the present application includes the following steps:

[0043] 1) Fix the visual camera at the end of the mechanical arm, and fix the marker in the working area range of the mechanical arm;

[0044] 2) Keep the marker always in the visual range of the camera, manually operate the mechanical arm to change the spatial position of the camera fixed at the end of the mechanical arm, record all the pose data, and move the mechanical arm multiple times to collect multiple sets of calibration data;

[0045] a. When the visual camera is selected as a three-dimensional camera for collecting point cloud data, a three-dimensional marker is fixed in the working area range of the mechanical arm; then the calibration data is: the rotation angle value θ of each joint of the mechanical arm (i = 1, 2, 3, 4, 5, 6), and the spatial position P (x, y, z) of the center of the calibration ball in the three-dimensional camera coordinate system; i

[0046] Among them, the rotation angle value of the mechanical arm is obtained in the controller of the mechanical arm, and the center coordinate position is obtained by performing spherical processing on the point cloud data collected by the three-dimensional camera.

[0047] b. When the visual camera is selected as any one of a two-dimensional area array camera, a line laser sensor, and a point laser sensor for collecting image information;

[0048] The calibration object adopts a two-dimensional calibration object, such as a chessboard calibration plate or other types of calibration plates. ​

[0049] The calibration data is that a two-dimensional camera is fixed at the end of the mechanical arm, the checkerboard calibration board is photographed in different postures, and the joint position of the mechanical arm and the position value of the checkerboard calibration board relative to the two-dimensional camera are recorded each time.

[0050] 3) The pose change of the marker when the spatial position of the camera fixed at the end of the mechanical arm is changed is obtained, and a calibration equation based on zero deviation and hand-eye pose deviation is constructed according to the pose change of the marker;

[0051] In the present application, no matter whether a three-dimensional camera or a two-dimensional camera is selected, the calibration data collected after combination with the corresponding marker is the same, that is, the coordinate of the marker relative to the camera + joint angle value.

[0052] 4) After the collected multiple sets of calibration data are brought into the calibration equation and optimization is performed, the mechanical arm zero deviation and the hand-eye pose transformation matrix are obtained, and are compensated into the configuration of the mechanical arm controller, and the deviation calibration is completed.

[0053] The technical key points of the present application are:

[0054] 1) In the above step 1), the vision camera fixed at the end of the mechanical arm can be any kind of vision device, including but not limited to a three-dimensional camera for collecting point clouds, a two-dimensional area array camera for collecting image information, a line laser sensor, a point laser sensor, etc., and the fixed marker can be any object that can be detected by the vision camera, including but not limited to a standard calibration ball, a checkerboard, a calibration block, a calibration cone, etc. The deviation calibration result is only related to the precision of the vision device by using the vision measurement scheme, and the calibration error caused by human factors is avoided.

[0055] 2) The error calibration model in the above step 3) combines the hand-eye pose deviation of the mechanical arm and the vision device and the motion parameter deviation of the mechanical arm itself together, realizes the one-time calibration of the hand-eye deviation and the body deviation, and effectively improves the calibration efficiency and the calibration precision.

[0056] In the above steps, the steps are further illustrated in combination with the embodiments, as shown in the following Figure 1 The vision-based industrial mechanical arm zero deviation calibration method provided in the present embodiment 1 comprises the following steps:

[0057] Step 1: Fix the vision device at the end of the mechanical arm, and fix the marker in the working area of the mechanical arm. In the present embodiment, the vision device is a three-dimensional camera, and the marker is a standard calibration ball. The installation position relationship with the mechanical arm is as shown in Figure 4 .

[0058] Step 2.1: Keep the calibration ball within the 3D camera's field of view, manually move the robotic arm to change the position and orientation of the 3D camera. The status of the robotic arm's data acquisition is shown in the attached figure. Figure 4 As shown.

[0059] Step 2.2: Record the rotation angle values ​​of each joint of the robotic arm under different positions and orientations: θ i (i = 1, 2, 3, 4, 5, 6), and the spatial position P(x, y, z) of the calibrated sphere's center in the 3D camera coordinate system. The rotation angle of the robotic arm can be read from the robotic arm's controller, and the sphere's center coordinate position can be obtained by performing spherical processing on the point cloud data captured by the 3D camera.

[0060] Step 2.3: Move the robotic arm multiple times. In this embodiment, 30 sets of data are collected.

[0061] Step 3: Establish an error calibration model.

[0062] like Figure 2 As shown, let the robot's base coordinate system be at point B, the robot's end effector be at point E, the camera's origin be at point C, and the center of the calibration sphere be at point A. Then, the pose transformation of the calibration sphere in the base coordinate system is T. BA The pose transformation from the robot's end effector to the camera origin remains constant, and is T. EC When the robot's joints are at different angles, the robot's end effector will change to positions E1, E2, etc. Therefore, from the camera's perspective, the pose transformation of the calibration ball can be represented as follows:

[0063]

[0064] in, The position change of the calibrated ball's center from the camera's perspective during the i-th pose. This represents the pose transformation of the robot's base coordinates relative to the robot's end effector at the i-th pose. Since the 3D camera can only detect displacement information when measuring the calibration sphere and cannot acquire pose transformation data, therefore... Only displacement changes; the attitude transformation remains a unit matrix. Simultaneously, there is only a positional relationship between the calibration sphere's center position A and the robot arm's base, T. BA It is also a homogeneous matrix containing only displacement information and no attitude transformation. For the above reasons, the zero position of axis 1 of the robotic arm can be arbitrary, so the zero position of axis 1 is not included in the calibration during zero-position deviation calibration. When collecting multiple sets of data for calibration, assuming that the parameters of each link of the robotic arm and the values ​​of each joint of the robot under different attitudes are known, the transformation matrix of the six-axis robotic arm base relative to the end effector can be regarded as a function of the zero-position deviation δθ of each joint, and can be written as:

[0065]

[0066] Bring it into formula (1), and remove the position part of , the basic formula of visual-based robot zero deviation calibration is obtained as:

[0067]

[0068] Step 4.1: Bring multiple sets of calibration data into the error calibration equation formula (3), and obtain the calibration result after optimization.

[0069] Step 4.2: After calibration, the robot joint zero deviation δθ i (i = 2, 3, 4, 5, 6) can be obtained, and the relative pose transformation relationship T CE between the robot end and the camera origin can also be obtained, as well as the position coordinates (x BA , y BA , z BA ) of the calibration ball relative to the robot base. For robots combined with cameras, the efficiency and accuracy of robot body parameter calibration and robot-camera hand-eye calibration will be effectively improved.

[0070] Embodiment 2 (extended embodiment): The three-dimensional camera and the measured marker in embodiment 1 described above can be replaced by other devices according to the needs of the site. The three-dimensional camera can be replaced by a two-dimensional area array camera, a line laser sensor, a point laser sensor, etc., and the corresponding measured marker can be a two-dimensional checkerboard calibration board, a calibration block, etc. As shown in Figure 3 , fix the two-dimensional camera at the end of the mechanical arm, and shoot the checkerboard calibration board at different poses, record the joint position and the position value of the checkerboard relative to the camera at each shooting time. The data obtained in this way can also be used for the calibration method proposed in the present application, and the calibration effect depends on the resolution of the camera used.

[0071] In summary, the two embodiments in the present application use a three-dimensional or two-dimensional vision camera installed on a robot to collect position information of a marker fixed on a workbench multiple times, to realize calibration of mechanical arm zero deviation and hand-eye pose. Both embodiments only need to complete mechanical arm zero deviation calibration and hand-eye pose transformation calibration between the mechanical arm and the vision device through one calibration process, effectively improving the calibration efficiency and calibration accuracy.

[0072] Those skilled in the art can understand that the above description is only the preferred embodiment of the present application, and the features described in various embodiments of the present disclosure and / or claims can be combined or combined, even if such combination or combination is not explicitly described in the present disclosure. It is not intended to limit the present application, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacement for part of the technical features, any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0073] Although preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to these embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application. Obviously, those skilled in the art can make various modifications and changes to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and changes of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and changes.

Claims

1. A vision-based zero bias calibration method for an industrial robot arm, characterized in that, Includes the following steps: 1) Fix the vision camera to the end of the robotic arm and fix the marker within the working area of ​​the robotic arm; 2) Keep the marker within the camera's field of view, manually operate the robotic arm to change the spatial position of the camera fixed at the end of the robotic arm, record all corresponding pose data, and move the robotic arm multiple times to collect multiple sets of calibration data. In step 2), when the vision camera is selected as a 3D camera for acquiring point clouds, a 3D calibration object is used to fix it within the working area of ​​the robotic arm. The calibration data are: rotation angle values of each joint of the mechanical arm , and a spatial position of the ball center in a three-dimensional camera coordinate system ​ The rotation angle of the robotic arm is obtained in the controller of the robotic arm, and the coordinate position of the sphere center is obtained by performing spherical processing on the point cloud data captured by the 3D camera. 3) Obtain the pose change of the marker when the spatial position of the camera fixed at the end of the robotic arm is changed, and construct a calibration equation based on the zero position deviation and hand-eye pose deviation based on the pose change of the marker. The specific steps for obtaining the pose change of the calibration ball when the spatial position of the camera fixed at the end of the robotic arm is changed are as follows: When the visual camera is selected as a three-dimensional camera for collecting point clouds, let the base coordinate system of the robot be at point , the end of the robot be at point , the origin of the camera be at point , and the position of the center of the calibration ball be , then the pose transformation of the calibration ball in the base coordinate system is: ; the pose transformation from the end of the robot to the origin of the camera is always unchanged, and is ; in the visual camera view, the pose transformation of the calibration ball is represented as: （1） wherein, is a position transformation of the center of the calibration sphere in the camera view at the i-th pose, represents a pose transformation of the robot base coordinate relative to the robot end at the i-th pose; The calibration equation based on the pose change of the marker and the hand-eye pose deviation is constructed as follows: When the vision camera is selected as a 3D camera for acquiring point clouds, since the 3D camera can only detect displacement information and cannot acquire posture transformation when measuring the calibration sphere, the zero position of axis 1 of the robotic arm can be any position. Therefore, the zero position of axis 1 is not included in the calibration during zero-position deviation calibration. When multiple sets of data are collected for calibration, and the parameters of each link of the robotic arm and the values ​​of each joint of the robot under different postures are known, the transformation matrix of the six-axis robotic arm base relative to the end effector is the zero-position deviation of each joint. The function, that is: （2） Zero position deviation of each joint Substitute the function into formula (1) and extract... For the position part, the calibration equation for the robot's zero-position deviation based on vision is obtained as follows: ; in, To calibrate the position coordinates of the sphere from the perspective of the visual camera coordinate system, To calibrate the position coordinates of the ball relative to the robot base; 4) Substitute the collected calibration data into the calibration equation, perform optimization, obtain the zero-position deviation of the robotic arm and the hand-eye pose transformation matrix, and compensate it into the configuration of the robotic arm controller to complete the deviation calibration.

2. The vision-based industrial robotic arm zero-position deviation calibration method according to claim 1, characterized in that, The visual camera is any one of a 3D camera for acquiring point clouds or a 2D area array camera, a line laser sensor, or a point laser sensor for acquiring image information. The marker is any one of the calibration ball, calibration cone, or calibration block of a three-dimensional calibration object, or the marker is a checkerboard calibration board of a two-dimensional calibration object.

3. The vision-based industrial robotic arm zero-position deviation calibration method according to claim 1, characterized in that, In step 2), the vision camera is selected as any one of a two-dimensional area array camera, a line laser sensor, or a point laser sensor for acquiring image information; The calibration material uses a checkerboard calibration plate of a two-dimensional calibration material; The calibration data is as follows: fix a two-dimensional camera to the end of the robotic arm, take pictures of the checkerboard calibration board in different postures, and record the position of the robotic arm joints and the position value of the checkerboard calibration board relative to the two-dimensional camera at each shooting.

4. The vision-based industrial robotic arm zero-position deviation calibration method according to claim 1, characterized in that, Step 4) specifically involves: 4-1) Substitute multiple sets of calibration data obtained by selecting any combination of vision cameras and markers into the error calibration equation, and obtain the calibration result after optimization; 4-2) After calibration, the zero-position deviation of the robot joints is obtained. Simultaneously, the relative pose transformation relationship between the robot's end effector and the camera origin is obtained. And the position coordinates of the calibration ball relative to the robot base. The error is then compensated and incorporated into the configuration of the robotic arm controller to complete the deviation calibration.

5. A vision-based zero-position deviation calibration system for industrial robotic arms, characterized in that, It includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement, when executing the computer program, a vision-based industrial robotic arm zero-position deviation calibration method as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements a vision-based industrial robotic arm zero-position deviation calibration method as described in any one of claims 1-4.

Images