Industrial robot hand-eye calibration board and calibration method for single-line structured light

Abstract

The application belongs to the field of robot vision, and particularly relates to an industrial robot hand-eye calibration board for single-line structured light and a calibration method, which comprises a calibration board body and four straight lines L1, L2, L3 and L4 arranged on the calibration board body; the surface of the calibration board body is black or white, and the surface of the calibration board body is rectangular; the calibration board body is arranged directly below a structured light camera, and the size of the calibration board body is arranged to meet the imaging requirements of the structured light camera; the size of the calibration board body and the effective field of view width of the structured light camera meet the requirements; the straight line L2 and the straight line L3 are diagonal lines on the surface of the calibration board body; the straight lines L1, L2 and L3 intersect at a point 01; the straight line L4 is arranged in parallel to the straight line L1, and the straight line L4 intersects with the straight lines L2 and L3; and a point 02 is further arranged on the straight line L4. The calibration method of the application has a simple process, is easy to operate, can be used across robot platforms, and is beneficial to application in actual industrial scenes.

Description

Technical Field

[0001] This invention belongs to the field of robot vision, specifically a calibration board and calibration method for the hand and eye of an industrial robot oriented towards single-line structured light. Background Technology

[0002] With the development of science and technology and the improvement of the intelligence level of industrial robots, systems such as weld seam tracking, laser contour measurement, and structured light-based 3D scanning are widely used in industrial scenarios such as robot handling, assembly, and grasping. When vision and robots are combined to achieve positioning and measurement functions, hand-eye calibration of the robot and vision is an unavoidable key step. In practical industrial applications, a calibration method that is simple in process, easy to operate, and can be used across robot platforms is very important. A calibration method that is easy to operate and has low process complexity can not only save offline operation time in industrial production, but also make it easy for operators to learn and use, which can directly save costs and improve efficiency for enterprises. In the past decade or so, the global robot industry has flourished, with various brands competing fiercely. A single manufacturing company may use industrial robots from multiple brands simultaneously, making a calibration method that can be used across robot platforms crucial for the company.

[0003] Common line structured light vision hand-eye calibration methods often focus on their own visual characteristics and rely on customized calibration processes. They require a high level of skill from both the robot and the vision system operator, and may even require complex logic programming to implement. As a result, only a few skilled workers in enterprises can usually use them, which is detrimental to their use and promotion in industrial scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a hand-eye calibration method for industrial robots using single-line structured light. Two calibration boards and a calibration process are designed to assist the robot in establishing a tool coordinate system, aligning the tool coordinate system with the structured light vision coordinate system. This means that the coordinate values ​​obtained from structured light vision measurements, based on the vision coordinate system, are also coordinate values ​​within the robot's tool system. This calibration method is built upon the tool coordinate system calibration method for industrial robots and can be used across different brands, such as FANUC, ABB, KUKA, Siasun, and other commonly available industrial robots.

[0005] The technical solution adopted by the present invention to achieve the above objectives is: an industrial robot hand-eye calibration plate for single-line structured light, comprising: a calibration plate body and four straight lines L1, L2, L3, and L4 disposed on the calibration plate body;

[0006] The surface of the calibration plate is black or white, and the surface of the calibration plate is rectangular.

[0007] The calibration plate is positioned directly below the structured light camera, and the dimensions of the calibration plate are set to meet the imaging requirements of the structured light camera. The dimensions of the calibration plate are set to satisfy the effective field of view of the structured light camera.

[0008] The straight lines L2 and L3 are diagonals on the surface of the calibration plate; and the straight lines L1, L2, and L3 intersect at point O1.

[0009] The straight line L4 is set parallel to the straight line L1, and the straight line intersects with L2 and L3;

[0010] A small dot 02 is also provided on the straight line L4.

[0011] The dimensions of the calibration plate are set to meet the imaging requirements of the structured light camera, specifically as follows:

[0012] Let the effective field of view of the structured light camera, i.e., the imaging width of the laser line in the image, be... Let the width of the calibration plate be... Gao Wei The proportion satisfies the following constraint: , .

[0013] The lines L1, L2, and L3 intersect at point O1, forming three pairs of included angles. Assuming the counter-clockwise angle is positive, then the included angles... , The angles are each no less than 30 degrees. .

[0014] The line L4 and point O2 are set as follows:

[0015] Let the length of line L1 be Line L4 is parallel to line L1. A perpendicular line is drawn from point O1 to line L4, and the foot of the perpendicular is point O2. The distance between the perpendiculars O2 and O1 is not less than [missing information]. .

[0016] When the surface of the calibration plate is set to black, the straight lines L1, L2, L3, and L4 are set to white, and points 01 and 02 are set to black.

[0017] When the laser line from the structured light camera shines from a black background onto a white straight line, the observed brightness of the laser line increases, and at point 0102, an easily observable dark spot appears.

[0018] When the surface of the calibration plate is set to white, then lines L1, L2, L3, and L4 are set to black, and points 01 and 02 are set to white;

[0019] When the laser line from the structured light camera shines from a white background onto a black straight line, the brightness of the observed laser line decreases, and at point 0102, an easily observable bright spot appears.

[0020] A calibration method for an industrial robot hand-eye calibration plate oriented towards single-line structured light includes the following steps:

[0021] 1) Control the robot to carry a structured light camera and make the laser line shine perpendicularly on the straight line L1 on the calibration plate, and obtain and record the robot's pose T1 when point 01 coincides with the origin of the vision system.

[0022] 2) Based on pose T1, move the vision laser line so that the laser line illuminates the straight lines L2 and L3 on the calibration plate in sequence, and obtain and record the robot's poses T2 and T3 when point 01 coincides with the origin of the vision system.

[0023] 3) Control the robot to return to pose T1, control the robot to translate along the positive y-direction, and keep the laser line always aligned with L1, with a translation amount not less than [missing value]. Record the robot's pose T4 at this moment;

[0024] 4) Control the robot to return to pose T1, control the robot to translate along the positive x direction and make the laser line coincide with L4, continue to translate the robot, make the coordinate reading of point 02 in the vision system (0, 0, 0), and record the robot pose T5 at this time;

[0025] 5) The robot's poses T1 to T5, combined with the industrial robot's built-in calibration function, complete the hand-eye calibration of the industrial robot.

[0026] Step 1) specifically refers to:

[0027] 1-1) Control the robot to carry the structured light camera and make the laser line shine perpendicularly on the straight line L1 on the calibration plate; at the same time, control the robot to move the laser line up and down along the z direction to keep the laser always shining on L1, that is, control the robot's Rx angle to be close to zero.

[0028] 1-2) Control the robot to adjust its pose so that the z-coordinate of the laser line readings at all points on the straight line L1 is always equal, that is, the Ry angle is close to zero;

[0029] 1-3) Control the robot to translate so that the coordinate reading of point 01 in the vision system is (0, 0, 0); record the robot pose T1 at this time.

[0030] Step 2) specifically refers to:

[0031] 2-1) Based on pose T1, move the vision laser line and rotate it around the negative z-axis to coincide with L2. Then control the robot to rotate around the y-axis square by no less than 30 degrees, keeping the laser line always coincident with L2. Translate the robot so that the coordinate reading of point 01 in the vision system is (0, 0, 0), and reach pose T2. Record the robot pose at this time.

[0032] 2-2) Control the robot to move the vision laser line based on the pose T1, rotate around the z-axis in the positive direction to coincide with L3, and then rotate around the y-axis in the negative direction by no less than 30 degrees to keep it coinciding with L3. Translate the robot so that the coordinate reading of point 01 in the vision system is (0,0, 0); record the robot pose T3 at this time.

[0033] Step 5) specifically involves:

[0034] 5-1) Based on the robot's poses T1 to T3, perform the operation of aligning with the same spatial point in the visual coordinate system;

[0035] 5-2) Based on the robot's poses T4 and T5, perform operations to calibrate the positive directions of the x and y axes of the visual coordinate system;

[0036] 5-3) Use the industrial robot's built-in calibration function to calibrate the orientation of the tool coordinate system;

[0037] 5-4) Based on steps 5-1) to 5-3), complete the hand-eye calibration of the industrial robot.

[0038] The present invention has the following beneficial effects and advantages:

[0039] 1. This invention, in conjunction with a planar calibration plate, can achieve hand-eye calibration of line structured light vision cameras and industrial robots.

[0040] 2. The calibration method of the present invention has a simple process, is easy to operate, and can be used across robot platforms, which is beneficial for its application in actual industrial scenarios.

[0041] 3. This invention can assist the robot in establishing a tool coordinate system, which coincides with the structured light vision coordinate system. That is, the coordinate values ​​obtained by structured light vision measurement based on the vision coordinate system are also coordinate values ​​under the robot's tool system.

[0042] 4. The two embodiments of the present invention can be adapted to different application scenarios based on the laser line brightness and the conditions of bright or dark spots formed. Attached Figure Description

[0043] Figure 1 A schematic diagram of the structure of an embodiment of the planar calibration plate (white background with black lines) of the present invention;

[0044] Figure 2A schematic diagram of the structure of an embodiment of the planar calibration plate (black background with white lines) of the present invention;

[0045] Figure 3 Schematic diagram of the coordinate system oxz and coordinate system oyz of the structured light vision camera of the present invention;

[0046] Figure 4 A schematic diagram illustrating the principle of structured light scanning line L1 in the hand-eye calibration of this invention;

[0047] Figure 5 A schematic diagram illustrating the principle of structured light scanning line L2 in hand-eye calibration of this invention;

[0048] Figure 6 A schematic diagram illustrating the principle of structured light scanning of the straight line L3 in the hand-eye calibration of this invention;

[0049] Figure 7 A schematic diagram illustrating the principle of T4 for obtaining robot pose in hand-eye calibration according to the present invention;

[0050] Figure 8 A schematic diagram of the principle of obtaining robot pose T5 in hand-eye calibration of the present invention;

[0051] Figure 9 A schematic diagram illustrating the principle of aligning three robot poses to the same point in space during hand-eye calibration in this invention.

[0052] Figure 10 A schematic diagram illustrating the determination of the x and y axes of the tool coordinate system in the hand-eye calibration of this invention. Detailed Implementation

[0053] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0054] like Figure 1 The diagram shown is a structural schematic of the planar calibration plate of the present invention. Two planar calibration plates P1 are designed, including: a calibration plate body and four straight lines L1, L2, L3, and L4 disposed on the calibration plate body;

[0055] The calibration plate has a black or white surface and is rectangular. The calibration plate is positioned directly below the structured light camera, and its dimensions are set to meet the imaging requirements of the structured light camera. The dimensions of the calibration plate are set to satisfy the effective field of view of the structured light camera.

[0056] Lines L2 and L3 are diagonals on the surface of the calibration plate; and lines L1, L2, and L3 intersect at point O1.

[0057] Line L4 is set parallel to line L1, and the line intersects with L2 and L3;

[0058] There is also a small 02 on the straight line L4.

[0059] Among them, the planar calibration plate P1 is defined as follows: the effective field of view of the visual structured light, that is, the imaging width of the laser line in the image at the standard measurement height, is... Let the width of the calibration plate be... Gao Wei The proportion satisfies , .

[0060] Lines L1, L2, and L3, assuming counterclockwise angle is positive, the included angle... , The angles are each no less than 30 degrees. .

[0061] Planar calibration plate P 1， Let the length of line segment L1 be Line L4 is parallel to L1. A perpendicular line is drawn from point O1 to line L4, with the foot of the perpendicular at point O2. The distance between points O2 and O1 is not less than [missing information]. .

[0062] The P1 planar calibration board comes in two color designs; one is a white background with black lines, such as... Figure 1 As shown. Another type is black background with white lines, such as... Figure 2 As shown.

[0063] The P1 is a white-background, black-lined planar calibration board. The background color of the calibration board is light, while lines L1, L2, L3, and L4 are dark, and points 01 and 02 are light. With this calibration board, when the laser line shines from the light-colored background onto the dark line, the attenuation of the laser line brightness can be clearly seen. At point 0102, an easily observable bright spot will appear, which is easy to observe during subsequent calibration operations.

[0064] The P1 is a black-background, white-line planar calibration board. The background of the calibration board is dark, while lines L1, L2, L3, and L4 are light-colored, and points 01 and 02 are dark-colored. For this calibration board, when the laser line shines from the dark background onto the light-colored line, the increase in the brightness of the laser line can be clearly seen. At point 01 and 02, an easily observable dark spot will appear, which is easy to observe during subsequent calibration operations.

[0065] According to the calibration board of the present invention, a calibration method is designed to achieve hand-eye calibration of industrial robots based on line structured light vision, which can be used in conjunction with the calibration board P1. Through the calibration process, the robot can be guided to calibrate its tool coordinate system to a position that coincides with the vision camera coordinate system. The coordinate output measured by the structured light camera is based on both the vision coordinate system and the robot tool coordinate system.

[0066] Structured light vision coordinate systems are typically constructed in two ways, such as... Figure 3 As shown, one system is the oxz system, where the coordinate readings of the points on the laser line are (x, 0, z), and the other system is the oyz system, where the coordinate readings of the points on the laser line are (0, y, z). The following explanation is for the oyz system; for the oxz system, only the coordinate axes are transformed, and the underlying principle remains unchanged.

[0067] Specifically, it includes a total of 5 steps:

[0068] Step 1: As Figure 4 As shown, the robot is controlled to direct the visual laser line perpendicularly onto the calibration board's straight line L1. Simultaneously, the robot moves the laser line up and down along the z-axis, ensuring the laser always illuminates L1, i.e., the Rx angle is close to zero. The robot's pose is adjusted so that the z-coordinates of the laser line readings at all points on line L1 are always equal, i.e., the Ry angle is close to zero. The robot is then translated so that the coordinates of point 01 in the visual system are (0, 0, 0); the robot's pose T1 at this point is recorded.

[0069] Step 2: As Figure 5 As shown, based on pose T1, move the vision laser line, rotate it around the negative z-axis to coincide with L2, and then control the robot to rotate around the y-axis square by no less than 30 degrees, keeping the laser line always coincident with L2. Translate the robot so that the coordinate reading of point 01 in the vision system is (0, 0, 0), reach pose T2, and record the robot pose at this time.

[0070] Step 3: As Figure 6 As shown, the robot is controlled to move the vision laser line based on the pose T1, rotate around the positive z-axis to coincide with L3, and then rotate around the negative y-axis by no less than 30 degrees to keep it coinciding with L3. The robot is then translated so that the coordinate reading of point 01 in the vision system is (0, 0, 0); the robot pose T3 at this time is recorded.

[0071] Step 4: Control the robot to return to pose T1, control the robot to translate along the positive y-direction, and keep the laser line aligned with L1 at all times, with a translation amount not less than [value missing]. Record the robot's pose T4 at this moment, such as Figure 7 .

[0072] Step 5: As Figure 8 As shown, control the robot to return to pose T1, control the robot to translate along the positive x direction and make the laser line coincide with L4, continue to translate the robot, so that the coordinate reading of point 02 in the vision system is (0, 0, 0), and reach pose T5.

[0073] In the above steps, steps 4 and 5 enable the robot to determine the positive directions of the x and y axes, and the direction of the tool coordinate system can be calibrated using the calibration function built into the industrial robot.

[0074] Steps 1-3 enable the robot to align with the same spatial point using three different poses T1, T2, and T3, such as... Figure 9 As shown; steps 4 and 5 enable the robot to determine the positive directions of the x and y axes; the calibration method from steps 1-5 can be combined with the calibration function built into the industrial robot to calibrate the tool coordinate system.

[0075] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0076] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A hand-eye calibration method for industrial robots oriented towards single-line structured light, implemented based on an industrial robot hand-eye calibration board, the calibration board comprising: The calibration plate consists of a calibration board and four straight lines L1, L2, L3, and L4 arranged on the calibration board. The calibration board has a black or white surface and is rectangular. It is positioned directly below a structured light camera, and its dimensions meet the imaging requirements of the camera. Lines L2 and L3 are diagonals on the surface of the calibration board. Lines L1, L2, and L3 intersect at point O1. Line L4 is parallel to line L1 and intersects with L2 and L3. A point O2 is also provided on line L4. The calibration method comprises the following steps: 1) Control the robot to carry a structured light camera and make the laser line shine perpendicularly on the straight line L1 on the calibration plate, and obtain and record the robot's pose T1 when point 01 coincides with the origin of the vision system. 2) Based on pose T1, move the vision laser line so that the laser line illuminates the straight lines L2 and L3 on the calibration plate in sequence, and obtain and record the robot's poses T2 and T3 when point 01 coincides with the origin of the vision system. 3) Control the robot to return to pose T1, control the robot to translate along the positive y-direction, and keep the laser line always aligned with L1, with a translation amount not less than [missing value]. Record the robot's pose T4 at this moment; 4) Control the robot to return to pose T1, control the robot to translate along the positive x direction and make the laser line coincide with L4, continue to translate the robot, make the coordinate reading of point 02 in the vision system (0, 0, 0), and record the robot pose T5 at this time; 5) The robot's poses T1 to T5, combined with the industrial robot's built-in calibration function, complete the hand-eye calibration of the industrial robot.

2. The industrial robot hand-eye calibration method for single-line structured light according to claim 1, characterized in that, The dimensions of the calibration plate are set to meet the imaging requirements of the structured light camera, specifically as follows: Let the effective field of view of the structured light camera, i.e., the imaging width of the laser line in the image, be... Let the width of the calibration plate be... Gao Wei The proportion satisfies the following constraint: , .

3. The method for hand-eye calibration of industrial robots oriented towards single-line structured light according to claim 1, characterized in that, The lines L1, L2, and L3 intersect at point O1, forming three pairs of included angles. Assuming the counter-clockwise angle is positive, then the included angles... , The angles are each no less than 30 degrees. .

4. The industrial robot hand-eye calibration method for single-line structured light according to claim 1, characterized in that, The line L4 and point O2 are set as follows: Let the length of line L1 be Line L4 is parallel to line L1. A perpendicular line is drawn from point O1 to line L4, and the foot of the perpendicular is point O2. The distance between the perpendiculars O2 and O1 is not less than [missing information]. .

5. The method for hand-eye calibration of industrial robots oriented towards single-line structured light according to claim 1, characterized in that, When the surface of the calibration plate is set to black, the straight lines L1, L2, L3, and L4 are set to white, and points 01 and 02 are set to black. When the laser line from the structured light camera shines from a black background onto a white straight line, the observed brightness of the laser line increases, and at point 0102, an easily observable dark spot appears.

6. The industrial robot hand-eye calibration method for single-line structured light according to claim 1, characterized in that, When the surface of the calibration plate is set to white, then lines L1, L2, L3, and L4 are set to black, and points 01 and 02 are set to white; When the laser line from the structured light camera shines from a white background onto a black straight line, the brightness of the observed laser line decreases, and at point 0102, an easily observable bright spot appears.

7. The method for hand-eye calibration of industrial robots oriented towards single-line structured light according to claim 1, characterized in that, Step 1) specifically refers to: 1-1) Control the robot to carry the structured light camera and make the laser line shine perpendicularly on the straight line L1 on the calibration plate; at the same time, control the robot to move the laser line up and down along the z direction to keep the laser always shining on L1, that is, control the robot's Rx angle to be close to zero. 1-2) Control the robot to adjust its pose so that the z-coordinate of the laser line readings at all points on the straight line L1 is always equal, that is, the Ry angle is close to zero; 1-3) Control the robot to translate so that the coordinate reading of point 01 in the vision system is (0, 0, 0); record the robot pose T1 at this time.

8. The method for hand-eye calibration of industrial robots oriented towards single-line structured light according to claim 1, characterized in that, Step 2) specifically refers to: 2-1) Based on pose T1, move the vision laser line, rotate it around the negative z-axis to coincide with L2, and then control the robot to rotate around the positive y-axis by no less than 30 degrees, keeping the laser line always coincident with L2. Translate the robot so that the coordinate reading of point 01 in the vision system is (0, 0, 0), reach pose T2, and record the robot pose at this time. 2-2) Control the robot to move the vision laser line based on the pose T1, rotate around the positive z-axis to coincide with L3, and then rotate around the negative y-axis by no less than 30 degrees to keep it coincident with L3. Translate the robot so that the coordinate reading of point 01 in the vision system is (0, 0, 0); record the robot pose T3 at this time.

9. The method for hand-eye calibration of industrial robots oriented towards single-line structured light according to claim 1, characterized in that, Step 5) specifically involves: 5-1) Based on the robot's poses T1 to T3, perform the operation of aligning with the same spatial point in the visual coordinate system; 5-2) Based on the robot's poses T4 and T5, perform operations to calibrate the positive directions of the x and y axes of the visual coordinate system; 5-3) Use the industrial robot's built-in calibration function to calibrate the orientation of the tool coordinate system; 5-4) Based on steps 5-1) to 5-3), complete the hand-eye calibration of the industrial robot.

Images