Industrial robot hand-eye calibration plate and calibration method for single-line structured light

Abstract

The invention belongs to the field of robot vision, and particularly relates to an industrial robot hand-eye calibration plate facing single-line structured light and a calibration method, the industrial robot hand-eye calibration plate comprises a calibration plate body and four straight lines L1, L2, L3 and L4 arranged on the calibration plate body; the surface of the calibration plate body is black or white, and the surface of the calibration plate body is rectangular; the calibration plate body is arranged right below the structured light camera, the size of the calibration plate body is set to meet the imaging requirement of the structured light camera, and the size of the calibration plate body is set to meet the effective view field width of the structured light camera; the straight line L2 and the straight line L3 are diagonals on the surface of the calibration plate body respectively; the straight lines L1, L2 and L3 intersect at a point 01; the straight line L4 is parallel to the straight line L1, and the straight line intersects with the L2 and the L3; and a point 02 is also arranged on the straight line L4. The calibration method is simple in process and easy to operate, can be used across robot platforms, and is beneficial to application in actual industrial scenes.

Description

Technical Field

[0001] The invention belongs to the field of robot vision, in particular to an industrial robot hand-eye calibration plate and a calibration method facing single-line structured light. Background Art

[0002] With the development of science and technology and the improvement of the intelligence level of industrial robots, systems such as weld tracking, laser contour measurement, and three-dimensional scanning based on structured light are widely used in industrial scenarios such as robot handling, assembly, and grasping. When vision and robots are combined to achieve positioning and measurement functions, the hand-eye calibration of robots and vision is an unavoidable key link. In actual industrial applications, a set of calibration methods with simple processes, convenient operation, and cross-robot platforms is very important. Calibration methods with convenient operation and low process complexity can not only save offline operation time of industrial production, but also be easy for operators to learn and use, which can directly save costs and improve benefits for enterprises; in the past ten years, the world's robot industry has flourished, and various brands have competed. The same manufacturer may use industrial robots of multiple brands at the same time. A set of calibration methods that can be used across robot platforms is also crucial for enterprises.

[0003] The implementation of common line structured light vision hand-eye calibration methods often focuses on their own visual characteristics and relies on a customized calibration process. It has high requirements on the skill level of robot and vision system operators and even requires complex logic programming to be implemented. As a result, only a few technical workers in the enterprise can use it, which is not conducive to its use and promotion in industrial scenarios. Summary of the invention

[0004] The purpose of the present invention is to provide a hand-eye calibration method for industrial robots for single-line structured light. Two calibration plates and calibration processes are designed to assist the robot in establishing a tool coordinate system, and to overlap the tool coordinate system with the structured light vision coordinate system, that is, the coordinate values ​​based on the vision coordinate system obtained by structured light vision measurement are also the coordinate values ​​under the robot tool system. This calibration method is based on the tool coordinate system calibration method for industrial robots and can be used across brands, such as FANUC, ABB, KUKA, Siasun and other common industrial robots on the market.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: an industrial robot hand-eye calibration plate facing single-line structured light, comprising: a calibration plate body and four straight lines L1, L2, L3, and L4 arranged on the calibration plate body;

[0006] Wherein, the surface of the calibration plate body is black or white, and the surface of the calibration plate body is rectangular;

[0007] The calibration plate body is arranged directly below the structured light camera, and the size of the calibration plate body is set to meet the imaging requirements of the structured light camera, and the size of the calibration plate body is set to meet the effective field of view width of the structured light camera;

[0008] The straight line L2 and the straight line L3 are diagonal lines on the surface of the calibration plate body respectively; and the straight lines L1, L2, and L3 intersect at a point 01;

[0009] The straight line L4 is arranged parallel to the straight line L1, and the straight line intersects with L2 and L3;

[0010] A little point O2 is also provided on the straight line L4.

[0011] The size of the calibration plate is set to meet the imaging requirements of the structured light camera, specifically:

[0012] Assume that the effective field width of the structured light camera, that is, the imaging width of the laser line in the image, is w laser ; Assume the width of the calibration plate is w plate , height is h plate , the ratio satisfies the following constraints, namely: 0.2≤w plate / h plate ≤5, 0.2w plate ≤w laser ≤w plate .

[0013] The straight lines L1, L2, and L3 intersect at point 01, forming three pairs of angles. Assuming the counterclockwise angle is positive, the angles ∠L2L1 and ∠L1L3 are not less than 30 degrees respectively, and ∠L2L3=∠L2L1+∠L1L3.

[0014] The straight line L4 and point 02 are set as follows:

[0015] Set the length of line L1 to w plate , straight line L4 is parallel to straight line L1, and a perpendicular line is drawn through point 01 to straight line L4, the foot of which is point 02; and the distance between perpendicular line 02 and 01 is not less than

[0016] When the surface of the calibration plate is set to black, the straight lines L1, L2, L3, and L4 are set to white, and the points 01 and 02 are set to black;

[0017] When the laser line of the structured light camera is irradiated from the black background to the white straight line, the brightness of the observed laser line increases, and at point 0102, an easily observable dark spot appears.

[0018] When the surface of the calibration plate is set to white, the straight lines L1, L2, L3, and L4 are set to black, and the points 01 and 02 are set to white;

[0019] When the laser line of the structured light camera is irradiated from the white background to the black straight line, the brightness of the laser line is observed to decay, and at point 0102, an easily observable bright spot appears.

[0020] A calibration method for a hand-eye calibration plate of an industrial robot for single-line structured light comprises the following steps:

[0021] 1) Control the robot to carry a structured light camera and make the laser line shine vertically on the straight line L1 on the calibration plate, and obtain and record the robot's position T1 when point 01 coincides with the origin of the visual system;

[0022] 2) Based on the posture T1, move the visual laser line so that the laser line is vertically irradiated on the straight line L2 and straight line L3 on the calibration plate in turn, and obtain and record the robot's postures T2 and T3 when point 01 coincides with the origin of the visual system;

[0023] 3) Control the robot to return to position T1, control the robot to translate along the positive y direction, and keep the laser line always coincident with L1, and the translation amount is not less than 0.3w laser , record the robot's position T4 at this time;

[0024] 4) Control the robot to return to position T1, control the robot to translate along the positive x direction, and make the laser line coincide with L4, continue to translate the robot, so that the coordinate reading of point 02 in the visual system is (0,0,0), and record the robot position T5 at this time;

[0025] 5) The robot's postures T1~T5 are combined with the industrial robot's own calibration function to complete the industrial robot's hand-eye calibration.

[0026] The step 1) is specifically:

[0027] 1-1) Control the robot to carry the structured light camera and make the laser line shine vertically on the straight line L1 on the calibration plate; at the same time, control the robot to move the laser line up and down along the z direction to keep the laser always shining on L1, that is, control the robot's Rx angle close to zero;

[0028] 1-2) Control the robot to adjust its position so that the z coordinates of the laser line readings at all locations on the straight line L1 are always equal, that is, the Ry angle is close to zero;

[0029] 1-3) Control the robot to translate so that the coordinate reading of point 01 in the visual system is (0,0,0); record the robot's position T1 at this time.

[0030] The step 2) is specifically:

[0031] 2-1) Based on posture T1, move the visual laser line, rotate it around the negative direction of the z-axis to coincide with L2, then control the robot to rotate around the y-axis by no less than 30 degrees, keep the laser line always coincident with L2, translate the robot so that the coordinate reading of point 01 in the visual system is (0,0,0), reach posture T2, and record the robot posture at this time.

[0032] 2-2) Control the robot to move the visual laser line based on posture T1, rotate it in the positive direction around the z-axis to coincide with L3, and then rotate it in the negative direction around the y-axis by no less than 30 degrees to keep coinciding with L3, translate the robot so that the coordinate reading of point 01 in the visual system is (0,0,0); record the robot posture T3 at this time.

[0033] The step 5) is specifically:

[0034] 5-1) According to the robot positions T1~T3, realize the operation of aligning with the same spatial point in the visual coordinate system;

[0035] 5-2) According to the robot postures T4 and T5, the operation of calibrating the positive directions of the x-axis and y-axis of the visual coordinate system is realized;

[0036] 5-3) Use the calibration function of the industrial robot to calibrate the direction of the tool coordinate system;

[0037] 5-4) According to steps 5-1) to 5-3), complete the hand-eye calibration of the industrial robot.

[0038] The present invention has the following beneficial effects and advantages:

[0039] 1. The present invention can realize the hand-eye calibration of line structured light vision cameras and industrial robots in conjunction with a plane calibration plate.

[0040] 2. The calibration method of the present invention has a simple process, is easy to operate, and can be used across robot platforms, which is conducive to application in actual industrial scenarios.

[0041] 3. The present invention can assist the robot in establishing a tool coordinate system and overlap the tool coordinate system with the structured light vision coordinate system, that is, the coordinate values ​​based on the vision coordinate system obtained by structured light vision measurement are also the coordinate values ​​under the robot tool system.

[0042] 4. The two embodiments of the present invention can correspond to different application scenarios through the brightness of the laser line and the bright spot or dark spot conditions formed. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic structural diagram of an embodiment of a plane calibration plate (white background with black lines) of the present invention;

[0044] Figure 2A schematic structural diagram of an embodiment of a plane calibration plate (black background and white lines) of the present invention;

[0045] Figure 3 Schematic diagram of the coordinate system oxz and the coordinate system oyz of the structured light vision camera of the present invention;

[0046] Figure 4 Schematic diagram of the principle of structured light scanning line L1 in the hand-eye calibration of the present invention;

[0047] Figure 5 Schematic diagram of the principle of structured light scanning line L2 in the hand-eye calibration of the present invention;

[0048] Figure 6 Schematic diagram of the structured light scanning line L3 in the hand-eye calibration of the present invention;

[0049] Figure 7 Schematic diagram of the principle of obtaining robot posture T4 in the hand-eye calibration of the present invention;

[0050] Figure 8 Schematic diagram of the principle of obtaining robot posture T5 in the hand-eye calibration of the present invention;

[0051] Fig. 9 Schematic diagram of the principle of aligning three robot postures with the same point in space in the hand-eye calibration of the present invention;

[0052] Fig.10 Schematic diagram of determining the x- and y-axis directions of the tool coordinate system in the hand-eye calibration of the present invention. DETAILED DESCRIPTION

[0053] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0054] like Figure 1 As shown, it is a schematic diagram of the structure of the plane calibration plate of the present invention, and two plane calibration plates P1 are designed; including: a calibration plate body and four straight lines L1, L2, L3, L4 arranged on the calibration plate body;

[0055] The surface of the calibration plate is black or white, and the surface of the calibration plate is rectangular; the calibration plate is arranged directly below the structured light camera, and the size of the calibration plate meets the imaging requirements of the structured light camera; the size of the calibration plate meets the effective field of view width of the structured light camera;

[0056] Line L2 and line L3 are diagonal lines on the surface of the calibration plate respectively; and lines L1, L2, and L3 intersect at point 01;

[0057] Line L4 is arranged parallel to line L1, and the line intersects L2 and L3;

[0058] A little bit of 02 is also provided on the straight line L4.

[0059] Among them, the plane calibration plate P1, assuming that the effective field width of the visual structured light is w, that is, at the standard measurement height, the imaging width of the laser line in the image is w laser ; Assume the width of the calibration plate is w plate , height is h plate , the ratio satisfies 0.2≤w plate / h plate ≤5, 0.2w plate ≤w laser ≤w plate .

[0060] For the straight lines L1, L2, and L3, assuming that the counterclockwise angle is positive, the included angles ∠L2L1 and ∠L1L3 are not less than 30 degrees respectively, and ∠L2L3 = ∠L2L1 + ∠L1L3.

[0061] Flat calibration plate P 1， Let the length of line segment L1 be w plate , straight line L4 is parallel to L1, and a perpendicular line is drawn through point 01, with the foot of the perpendicular being point 02. The distance between point 02 and point 01 is not less than

[0062] The color of the flat calibration plate P1 is divided into two designs; one is white with black lines, such as Figure 1 The other is black background with white lines, such as Figure 2 shown.

[0063] The flat calibration plate P1 with white background and black lines has a light background, dark lines L1, L2, L3, and L4, and light points 01 and 02. For this calibration plate, when the laser line is irradiated from the light background to the dark straight line, the attenuation of the laser line brightness can be clearly seen. At points 0102, bright spots that are easy to observe will appear, making them easy to observe during subsequent calibration operations.

[0064] The flat calibration plate P1 with black background and white lines has a dark background, light lines L1, L2, L3, and L4, and dark points 01 and 02. For this calibration plate, when the laser line is irradiated from the dark background to the light straight line, the brightness of the laser line can be clearly seen to increase. At points 0102, dark spots that are easy to observe will appear, making them easy to observe during subsequent calibration operations.

[0065] A calibration method is designed according to the calibration plate of the present invention, which can be used in conjunction with the calibration plate P1 to implement the hand-eye calibration of an industrial robot based on line structured light vision. Through the calibration process, the robot can be guided to calibrate the tool coordinate system to a position that coincides with the visual camera coordinate system. The coordinate output measured by the structured light camera is based on both the visual coordinate system and the robot tool coordinate system.

[0066] Usually, the structured light vision coordinate system is constructed in two ways, such as Figure 3 As shown, one is the oxz system, the coordinate reading of the point on the laser line is (x, 0, z), and the other is the oyz system, the coordinate reading of the point on the laser line is (0, y, z). The subsequent explanation is for the oyz system; for the oxz system, only the coordinate axis is transformed, and there is no change in principle;

[0067] Specifically, there are 5 steps in total:

[0068] Step 1: Figure 4 As shown, control the robot to make the visual laser line hit the calibration plate straight line L1 vertically; at the same time, ensure that the robot is controlled to move the laser line up and down along the z direction, keeping the laser always shining on L1, that is, the Rx angle is close to zero; control the robot to adjust the posture so that the z coordinates of the laser line readings at various locations on the straight line L1 are always equal, that is, the Ry angle is close to zero. Control the robot to translate so that the coordinate reading of point 01 in the visual system is (0,0,0); record the robot posture T1 at this time.

[0069] Step 2: If Figure 5 As shown, based on posture T1, move the visual laser line, rotate it around the negative direction of the z-axis to coincide with L2, and then control the robot to rotate around the y-axis by no less than 30 degrees in a square, keeping the laser line always coincident with L2, translate the robot so that the coordinate reading of point 01 in the visual system is (0,0,0), reach posture T2, and record the robot posture at this time.

[0070] Step 3: If Figure 6 As shown, the robot is controlled to move the visual laser line based on posture T1, rotate it in the positive direction around the z-axis to coincide with L3, and then rotate it in the negative direction around the y-axis by no less than 30 degrees to keep coinciding with L3, and translate the robot so that the coordinate reading of point 01 in the visual system is (0,0,0); record the robot posture T3 at this time.

[0071] Step 4: Control the robot to return to position T1, control the robot to translate along the positive y direction, and keep the laser line always coincident with L1, and the translation amount is not less than 0.3w laser , record the robot position T4 at this time, such as Figure 7 .

[0072] Step 5: Figure 8 As shown, the robot is controlled to return to position T1, and the robot is controlled to translate along the positive x direction, and the laser line coincides with L4, and the robot continues to translate so that the coordinate reading of point 02 in the visual system is (0,0,0), reaching position T5.

[0073] In the above steps, steps 4 and 5 implement the operation of the robot determining the positive directions of the x and y axes. The direction of the tool coordinate system can be calibrated using the calibration function of the industrial robot.

[0074] Steps 1-3 implement the operation of the robot using three different postures T1, T2, and T3 to aim at the same spatial point, such as Fig. 9 As shown; Steps 4 and 5 implement the operation of the robot determining the positive direction of the x and y axes; the calibration method from steps 1-5 can be combined with the calibration function of the industrial robot to calibrate the tool coordinate system. Specifically, the FANUC six-axis industrial robot corresponds to XXX calibration steps 1-X; ABB six-axis industrial robot, Siasun six-axis industrial robot.

[0075] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0076] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An industrial robot hand-eye calibration board for single-line structured light, characterized in that: include: A calibration plate body and four straight lines L1, L2, L3, L4 arranged on the calibration plate body; Wherein, the surface of the calibration plate body is black or white, and the surface of the calibration plate body is rectangular; The calibration plate body is arranged directly below the structured light camera, and the size of the calibration plate body is set to meet the imaging requirements of the structured light camera, and the size of the calibration plate body is set to meet the effective field of view width of the structured light camera; The straight line L2 and the straight line L3 are diagonal lines on the surface of the calibration plate body respectively; and the straight lines L1, L2, and L3 intersect at a point 01; The straight line L4 is arranged parallel to the straight line L1, and the straight line intersects with L2 and L3; A little point O2 is also provided on the straight line L4.

2. The industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: The size of the calibration plate is set to meet the imaging requirements of the structured light camera, specifically: Assume that the effective field width of the structured light camera, that is, the imaging width of the laser line in the image, is w laser ; Assume the width of the calibration plate is w plate , height is h plate , the ratio satisfies the following constraints, namely: 0.2≤w plate / h plate ≤5, 0.2w plate ≤w laser ≤w plate .

3. The industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: The straight lines L1, L2, and L3 intersect at point 01, forming three pairs of angles. Assuming the counterclockwise angle is positive, the angles ∠L2L1 and ∠L1L3 are not less than 30 degrees respectively, and ∠L2L3=∠L2L1+∠L1L3.

4. The industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: The straight line L4 and point 02 are set as follows: Set the length of line L1 to w plate , straight line L4 is parallel to straight line L1, and a perpendicular line is drawn through point 01 to straight line L4, the foot of the perpendicular on straight line L4 is point 02; and the distance between perpendicular line 0201 is not less than 5. The industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: When the surface of the calibration plate is set to black, the straight lines L1, L2, L3, and L4 are set to white, and the points 01 and 02 are set to black; When the laser line of the structured light camera is irradiated from the black background to the white straight line, the brightness of the observed laser line increases, and at point 0102, an easily observable dark spot appears.

6. The industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: When the surface of the calibration plate is set to white, the straight lines L1, L2, L3, and L4 are set to black, and the points 01 and 02 are set to white; When the laser line of the structured light camera is irradiated from the white background to the black straight line, the brightness of the laser line is observed to decay, and at point 0102, an easily observable bright spot appears.

7. The calibration method of the industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: The following steps are involved: 1) Control the robot to carry a structured light camera and make the laser line shine vertically on the straight line L1 on the calibration plate, and obtain and record the robot's position T1 when point 01 coincides with the origin of the visual system; 2) Based on the posture T1, move the visual laser line so that the laser line is vertically irradiated on the straight line L2 and straight line L3 on the calibration plate in turn, and obtain and record the robot's postures T2 and T3 when point 01 coincides with the origin of the visual system; 3) Control the robot to return to position T1, control the robot to translate along the positive y direction, and keep the laser line always coincident with L1, and the translation amount is not less than 0.3w laser , record the robot's position T4 at this time; 4) Control the robot to return to position T1, control the robot to translate along the positive x direction, and make the laser line coincide with L4, continue to translate the robot, so that the coordinate reading of point 02 in the visual system is (0,0,0), and record the robot position T5 at this time; 5) The robot’s postures T1~T5 are combined with the industrial robot’s built-in calibration function to complete the industrial robot’s hand-eye calibration.

8. The calibration method of the industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: The step 1) is specifically: 1-1) Control the robot to carry the structured light camera and make the laser line shine vertically on the straight line L1 on the calibration plate; at the same time, control the robot to move the laser line up and down along the z direction to keep the laser always shining on L1, that is, control the robot's Rx angle close to zero; 1-2) Control the robot to adjust its position so that the z coordinates of the laser line readings at all locations on the straight line L1 are always equal, that is, the Ry angle is close to zero; 1-3) Control the robot to translate so that the coordinate reading of point 01 in the visual system is (0,0,0); record the robot's position T1 at this time.

9. The calibration method of the industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: The step 2) is specifically: 2-1) Based on posture T1, move the visual laser line, rotate it around the negative direction of the z-axis to coincide with L2, then control the robot to rotate around the y-axis by no less than 30 degrees, keep the laser line always coincident with L2, translate the robot so that the coordinate reading of point 01 in the visual system is (0,0,0), reach posture T2, and record the robot posture at this time. 2-2) Control the robot to move the visual laser line based on posture T1, rotate it in the positive direction around the z-axis to coincide with L3, and then rotate it in the negative direction around the y-axis by no less than 30 degrees to keep coinciding with L3, translate the robot so that the coordinate reading of point 01 in the visual system is (0,0,0); record the robot posture T3 at this time.

10. The calibration method of the industrial robot hand-eye calibration plate for single-line structured light according to claim 1, characterized in that: The step 5) is specifically: 5-1) According to the robot positions T1~T3, realize the operation of aligning with the same spatial point in the visual coordinate system; 5-2) According to the robot postures T4 and T5, the operation of calibrating the positive directions of the x-axis and y-axis of the visual coordinate system is realized; 5-3) Use the calibration function of the industrial robot to calibrate the direction of the tool coordinate system; 5-4) According to steps 5-1) to 5-3), complete the hand-eye calibration of the industrial robot.

Images