Line laser vision sensor and hand-eye calibration method and system of industrial robot

Abstract

The invention discloses a hand-eye calibration method for a line laser vision sensor and an industrial robot. The hand-eye calibration method comprises the steps that a calibration plate is placed in a machining space of the robot; constructing a workpiece coordinate system for the planar substrate of the calibration plate, and obtaining a first sphere center coordinate of a lower hemisphere of the workpiece coordinate system; acquiring image information shot under different poses, extracting second sphere center coordinates of the hemispheroid on-line laser vision sensor coordinate system under each pose in the image information, and converting pose parameters corresponding to the poses into a transformation matrix of an end effector coordinate system relative to a workpiece coordinate system; and constructing a target equation containing the hand-eye calibration matrix, and solving the target equation to obtain a final hand-eye calibration matrix. The invention further provides a hand-eye calibration system. According to the method provided by the invention, the steps of the hand-eye calibration process can be simplified, and the method can be used as a preposed flow for the hand-eye calibration with higher precision to solve the problem that the iterative operation falls into a local minimum value.

Description

Technical Field

[0001] The present invention belongs to the technical field of automated processing and vision calibration, and particularly relates to a hand-eye calibration method and system for a line laser vision sensor and an industrial robot. Background Art

[0002] Due to its good measurement accuracy, high stability, and anti-interference ability, the line laser vision sensor is widely used in various industrial production fields and has become an important device for industrial robots to perceive environmental information. An industrial robot combined with a line laser vision sensor can achieve more autonomous production activities, such as weld seam tracking, defect detection, and positioning. However, to achieve the above production activities, the core lies in converting the coordinate system of the line laser vision sensor to the robot tool coordinate system through hand-eye calibration, and then finally to the robot base coordinate system.

[0003] Currently, common hand-eye calibration methods have some limitations. Some methods have strict requirements for the measurement pose of the robot. For example, it is necessary to adjust the robot pose so that the laser line is parallel to the known scribed line, or the laser line exactly passes through the tip of the calibration object, which increases the operation difficulty of workers. To reduce the pose adjustment requirements, some methods use a standard ball as the calibration object. However, either they need to fix the standard ball on the workbench, increasing additional installation work; or they need to use an additional instrument to measure the coordinates of the ball center in the robot base coordinate system, increasing the required equipment and calibration cost; or they increase the calculation process during the solution process, increasing the complexity of the algorithm.

[0004] Patent document CN118893631B discloses a robot hand-eye calibration method based on a line laser sensor. The specific method is to arrange three laser tracker target balls on the end of the robot equipped with the line laser sensor, adjust the robot pose so that the sensor irradiates on the standard ball, calculate the pose of the end coordinate system formed by the center of the standard ball and the three target balls in the laser tracker coordinate system, collect multiple groups of data to construct an equation set, solve the transformation matrix from the line laser sensor to the end coordinate system, and finally find the optimal solution through an optimization algorithm to complete the hand-eye calibration of the line laser sensor.

[0005] Patent document CN111986268B discloses a 3D line laser scanning camera hand-eye calibration method. This method uses a regular triangular pyramid as the calibration block, takes the vertex of the calibration block as the reference, and the robot collects its coordinates in two different coordinate systems through the 3D line scanning camera: the line laser coordinate system and the base coordinate system, and establishes an equation equivalence relationship based on this same point to obtain the calibration matrix. Summary of the Invention

[0006] The object of the present invention is to provide a method and system for hand-eye calibration of a line laser vision sensor and an industrial robot. This method can simplify the steps of the hand-eye calibration process and can be used as a preprocess for higher-precision hand-eye calibration to solve the problem of iterative calculation falling into a local minimum.

[0007] To achieve the first object of the present invention, the following technical solution is provided: A method for hand-eye calibration of a line laser vision sensor and an industrial robot, comprising the following steps:

[0008] Place a calibration board in the processing space of the robot. The calibration board includes a planar substrate and a hemisphere located on the planar substrate;

[0009] Construct a workpiece coordinate system for the planar substrate of the calibration board and obtain the first spherical center coordinates of the hemisphere in the workpiece coordinate system;

[0010] Take pictures of the calibration board through a line laser vision sensor installed on the end effector of the robot to obtain image information under different poses, and extract the second spherical center coordinates of the hemisphere in the line laser vision sensor coordinate system under each pose in the image information. Convert the pose parameters corresponding to the pose into a transformation matrix of the end effector coordinate system relative to the workpiece coordinate system. The pose parameters are used to adjust the pose of the robot end during shooting;

[0011] Construct a target equation containing the hand-eye calibration matrix, and solve the target equation with the first spherical center coordinates, the transformation matrices corresponding to different poses, and the second spherical center coordinates to obtain the final hand-eye calibration matrix.

[0012] The present invention only needs to measure the position of the spherical center on the calibration board and combine the transformation relationships between the coordinate systems to solve the hand-eye calibration matrix, thereby reducing the calibration process and improving the calibration efficiency.

[0013] Specifically, the spherical center of the hemisphere coincides with the surface center point of the planar substrate, which is convenient for positioning the coordinates of the spherical center of the hemisphere.

[0014] Specifically, the planar substrate adopts a rectangular planar board, and the first spherical center coordinates are (L / 2, W / 2, 0), where L represents the length of the rectangular planar board and W represents the width of the rectangular planar board.

[0015] Specifically, the workpiece coordinate system takes a corner of the planar substrate as the origin, takes the thickness direction of the planar substrate as the Z axis, and the other two adjacent edges forming the origin as the X axis and the Y axis.

[0016] Specifically, the positive direction of the Y axis of the line laser vision sensor coordinate system is determined by the right-hand rule, and the arc profile under the current pose is obtained by arc fitting and the corresponding arc radius and center coordinates are extracted;

[0017] Based on geometric principles, determine the Y-axis coordinate value of the center of the hemisphere in the coordinate system of the on-line laser vision sensor, and combine it with the center coordinate of the circular arc contour to finally obtain the second center coordinate.

[0018] Specifically, the expression of the geometric principle is as follows:

[0019]

[0020] Among them, R represents the radius of the hemisphere, r i represents the radius of the circular arc of the circular arc contour photographed in the i-th pose, y Li represents the Y-axis coordinate value of the second center coordinate in the i-th pose.

[0021] Specifically, the expression of the transformation matrix is as follows:

[0022]

[0023] Among them, A i represents the rotation value of the end effector coordinate system around the Z-axis direction of the workpiece coordinate system in the i-th pose, B i represents the rotation value of the end effector coordinate system around the Y-axis direction of the workpiece coordinate system in the i-th pose, C i represents the rotation value of the end effector coordinate system around the X-axis direction of the workpiece coordinate system in the i-th pose, (x i , y i , z i ) represents the coordinates of the origin of the end effector coordinate system in the workpiece coordinate system in the i-th pose.

[0024] Specifically, the solution process of the target equation is as follows:

[0025] Select the transformation matrix and the second center coordinate in N (N≥3) groups of poses among all poses, and substitute them into the target equation together with the first center coordinate to solve and obtain the hand-eye calibration matrix corresponding to the N groups of poses;

[0026] Repeat the above steps using all poses to solve and obtain multiple hand-eye calibration matrices, and use the mean value of the multiple hand-eye calibration matrices as the final hand-eye calibration matrix for output.

[0027] Specifically, the expression of the target equation is as follows:

[0028]

[0029] Among them, E T Lj represents the j-th hand-eye calibration matrix, P Li represents the second center coordinate in the i-th pose,W T Ei represents the transformation matrix corresponding to the i-th pose, and P w represents the first sphere center coordinate.

[0030] Specifically, the expression of the hand-eye calibration matrix is as follows:

[0031]

[0032] Among them, where r j and t j respectively represent the rotation matrix and the translation matrix in the j-th hand-eye calibration matrix, and q0, q1, q2, q3, t1, t2, and t3 represent the variables in the hand-eye calibration matrix.

[0033] Specifically, the expression for outputting the mean value of multiple hand-eye calibration matrices as the final hand-eye calibration matrix is as follows:

[0034]

[0035] Among them, j represents the hand-eye calibration matrix number, and m represents the total number of hand-eye calibration matrices.

[0036] To achieve the second object of the present invention, the following technical solution is provided: A hand-eye calibration system is implemented by the above-mentioned hand-eye calibration method of the line laser vision sensor and the industrial robot, including a robot, an end effector on the robot, and a calibration board used in cooperation. The end effector is provided with a sensor mounting bracket, and a line laser vision sensor mounted on the sensor mounting bracket.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] There are no special requirements for the shooting pose of the robot. It is not necessary to make the line laser of the line laser vision sensor parallel to some feature structures of the calibration board, and it is not necessary to use an additional measuring device to measure the sphere center position. Compared with the prior art, the calibration process can be reduced and the calibration efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is the flowchart of the hand-eye calibration method provided in this embodiment;

[0040] Figure 2 is the structural schematic diagram of the calibration board provided in this embodiment;

[0041] Figure 3 is the schematic diagram for solving the second sphere center coordinate provided in this embodiment;

[0042] Figure 4 is the schematic diagram of the hand-eye calibration system provided in this embodiment;

[0043] In the figure, 1 is an industrial robot; 2 is an end effector; 3 is a sensor mounting bracket; 4 is a line laser vision sensor; 5 is a calibration plate. Specific embodiments

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, but not all, of the embodiments of the present invention. Components of the embodiments of the present invention usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0045] As Figure 1 shown, a hand-eye calibration method for a line laser vision sensor and an industrial robot provided in this embodiment includes the following steps:

[0046] Place the calibration plate in the machining space of the robot. In this embodiment, the calibration plate mentioned, as Figure 2 shown, uses a rectangular plane plate 502 as the substrate, and a hemisphere 501 is placed at the center of the rectangular plane plate, and the center of the sphere of the hemisphere 501 coincides with the center point of the rectangular plane plate 502.

[0047] Construct a workpiece coordinate system for the planar substrate of the calibration plate and obtain the first center-of-sphere coordinates of the hemisphere in the workpiece coordinate system. More specifically, the size parameters of the rectangular plane plate 502 provided in this embodiment are L×W×H. Then, taking Figure 2 P1 in W as the origin O W -X-Y-Z, the direction from P1 to P2 is the positive X-axis direction, the direction from P1 to P4 is the positive Y-axis direction, to construct the corresponding workpiece coordinate system O W (L / 2, W / 2, 0).

[0048] The calibration board is photographed by a line laser vision sensor installed on the end effector of the robot to obtain image information under different poses, and the second center coordinates of the hemisphere under each pose in the line laser vision sensor coordinate system are extracted from the image information. The pose parameters corresponding to the pose are converted into the transformation matrix of the end effector coordinate system relative to the workpiece coordinate system. In this embodiment, the hemisphere 501 of the calibration board is photographed with different robot poses, and the coordinates of the center of the photographed hemisphere 501 in the line laser vision sensor coordinate system O L -X-Y-Z are recorded as P Li (x Li ,y Li ,z Li ), where i represents the pose number, and i = 1, 2, 3,..., 3n, and n is a positive integer. The specific method is as follows:

[0049] Determine the positive direction of the Y-axis of the line laser vision sensor coordinate system O L -X-Y-Z according to the right-hand rule.

[0050] The circular arc contour photographed in the i-th pose is fitted by a circular arc to obtain its circular arc radius r i and the coordinates (x i ,z Li ) of the center o Li .

[0051] As Figure 3 shown, using the geometric relationship and the position of the center o i relative to the Y-axis in the O L -X-Y-Z coordinate system, the magnitude and sign of y Li are determined respectively, and then P Li (x Li ,y Li ,z Li ) is obtained.

[0052] Through the industrial robot, the pose parameters corresponding to the i-th pose are read. In this embodiment, the pose parameters include position information and attitude information, and its expression is (x i ,y i ,z i ,A i ,B i ,C i ).

[0053] More specifically, (x i ,y i ,z i ) is the position information, and (A i ,B i ,C i) is the pose information, which represents the rotation values of the end - effector coordinate system around the axes of the workpiece coordinate system.

[0054] Convert it to the end - effector coordinate system O E -X - Y - Z with respect to the workpiece coordinate system O W -X - Y - Z transformation matrix W T Ei , and its expression is as follows:

[0055]

[0056] ; where A i represents the rotation value of the end - effector coordinate system around the Z - axis direction of the workpiece coordinate system in the i - th pose, B i represents the rotation value of the end - effector coordinate system around the Y - axis direction of the workpiece coordinate system in the i - th pose, C i represents the rotation value of the end - effector coordinate system around the X - axis direction of the workpiece coordinate system in the i - th pose, (x i , y i , z i ) represents the coordinates of the origin of the end - effector coordinate system in the workpiece coordinate system in the i - th pose.

[0057] Construct a target equation containing the hand - eye calibration matrix, and solve the target equation with the first sphere center coordinates, the transformation matrices corresponding to different poses, and the second sphere center coordinates to obtain the final hand - eye calibration matrix.

[0058] More specifically, use P W , P Li , W T Ei to establish an equation containing the hand - eye calibration matrix E T Lj , and solve the hand - eye calibration matrix E T Lj , where j represents the number of the obtained hand - eye calibration matrix, and j = 1, 2, 3, …, n, n is a positive integer. The specific method is:

[0059] Construct the expression of the hand - eye calibration matrix E T Lj :

[0060]

[0061] ; where r j and t j represent the rotation matrix and the translation matrix in the j - th hand - eye calibration matrix respectively, and q0, q1, q2, q3, t1, t2 and t3 represent the variables in the hand - eye calibration matrix.

[0062] Construct the target equation P W = W Τ Ei E Τ Lj P Li , where E T Lj represents the j-th hand-eye calibration matrix, and P Li represents the second sphere center coordinate in the i-th pose, W T Ei represents the transformation matrix corresponding to the i-th pose, and P w represents the first sphere center coordinate.

[0063] In this embodiment, the transformation matrices and the second sphere center coordinates in three different poses are selected and substituted into the target equation together with the first sphere center coordinate:

[0064]

[0065] Repeat the above steps for all poses to solve for multiple hand-eye calibration matrices, and use the mean value of the multiple hand-eye calibration matrices as the final hand-eye calibration matrix for output.

[0066]

[0067] Among them, j represents the hand-eye calibration matrix number, and m represents the total number of hand-eye calibration matrices.

[0068] This embodiment also provides a hand-eye calibration system, as Figure 4 shown, whose device layout includes an industrial robot 1, an end effector 2, a sensor mounting bracket 3, a line laser vision sensor 4, and a calibration plate 5. The end effector 2 is installed on the industrial robot 1, and the line laser vision sensor 4 is fixedly connected to the end effector 2 through the sensor mounting bracket 3.

[0069] In summary, the hand-eye calibration method for the line laser vision sensor and the industrial robot provided by the present invention has no special requirements for the shooting pose of the robot, does not require the line laser of the line laser vision sensor to be parallel to certain feature structures of the calibration plate, and does not require an additional measuring device to measure the sphere center position, which can reduce the calibration process and improve the calibration efficiency; especially in the case of higher precision requirements, it can be used as a preprocessing process for hand-eye calibration to solve the problem that the iterative operation falls into a local minimum.

[0070] In addition, the terms "upper", "lower", "inner", "outer", "front", and "rear" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0071] Certainly, the above are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes or modifications made according to the structure, features and principles described in the scope of the patent application of the present invention shall be included in the scope of the patent application of the present invention.

[0072] Finally, it should be noted that the above embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A hand-eye calibration method for a line laser vision sensor and an industrial robot, characterized in that Including the following steps: Place the calibration board in the machining space of the robot. The calibration board includes a planar substrate and a hemisphere located on the planar substrate; Construct a workpiece coordinate system for the planar substrate of the calibration board and obtain the first center coordinate of the hemisphere in the workpiece coordinate system; Take pictures of the calibration board through a line laser vision sensor installed on the end effector of the robot to obtain image information under different poses, and extract the second center coordinate of the hemisphere in the line laser vision sensor coordinate system under each pose. Convert the pose parameters corresponding to the poses into the transformation matrix of the end effector coordinate system relative to the workpiece coordinate system. The pose parameters are used to adjust the pose of the robot end during shooting; Construct a target equation including the hand-eye calibration matrix, and solve the target equation with the first center coordinate, the transformation matrices corresponding to different poses, and the second center coordinate to obtain the final hand-eye calibration matrix.

2. The hand-eye calibration method of the line laser vision sensor and the industrial robot according to claim 1, characterized in that, The center of the hemisphere coincides with the surface center point of the planar substrate.

3. The hand-eye calibration method of the line laser vision sensor and the industrial robot according to claim 1, characterized in that, The workpiece coordinate system takes a corner of the planar substrate as the origin, takes the thickness direction of the planar substrate as the Z axis, and the other two adjacent edges forming the origin as the X axis and the Y axis.

4. The hand-eye calibration method of the line laser vision sensor and the industrial robot according to claim 1, wherein The line laser vision sensor coordinate system determines the positive direction of the Y axis according to the right-hand rule, uses the arc fitting method to obtain the arc contour under the current pose and extracts the corresponding arc radius and center coordinate; Based on the geometric principle, determine the Y-axis coordinate value of the hemisphere center in the line laser vision sensor coordinate system, and combine the center coordinate of the arc contour to finally obtain the second center coordinate.

5. The hand-eye calibration method of the line laser vision sensor and the industrial robot according to claim 1, wherein, The expression of the transformation matrix is as follows: Among them, A i represents the rotation value of the end effector coordinate system around the Z-axis direction of the workpiece coordinate system in the i-th pose, B i represents the rotation value of the end effector coordinate system around the Y-axis direction of the workpiece coordinate system in the i-th pose, C i represents the rotation value of the end effector coordinate system around the X-axis direction of the workpiece coordinate system in the i-th pose, (x i , y i , z i ) represents the coordinates of the origin of the end effector coordinate system in the workpiece coordinate system in the i-th pose.

6. The hand-eye calibration method of the line laser vision sensor and the industrial robot according to claim 1, characterized in that, The solution process of the target equation is as follows: Select the transformation matrices and the second center coordinates under N groups of poses among all poses, and substitute them into the target equation together with the first center coordinate to solve for the hand-eye calibration matrices corresponding to the N groups of poses, where N is an integer and N≥3; Repeat the above steps using all poses to solve for multiple hand-eye calibration matrices, and output the mean value of the multiple hand-eye calibration matrices as the final hand-eye calibration matrix.

7. The hand-eye calibration method of the line laser vision sensor and the industrial robot according to claim 1 or 6, characterized in that, The expression of the target equation is as follows: P W = W Τ Ei E Τ Lj P Li ; Among them, E T Lj represents the j-th hand-eye calibration matrix, and P Li represents the second sphere center coordinate in the i-th pose, W T Ei represents the transformation matrix corresponding to the i-th pose, and P w represents the first sphere center coordinate.

8. The hand-eye calibration method of the line laser vision sensor and the industrial robot according to claim 1 or 7, characterized in that, The expression of the hand-eye calibration matrix is as follows: where r j and t j represent the rotation matrix and the translation matrix in the j-th hand-eye calibration matrix respectively, and q0, q1, q2, q3, t1, t2 and t3 represent the variables in the hand-eye calibration matrix.

9. The hand-eye calibration method of the line laser vision sensor and the industrial robot according to claim 7, characterized in that, The expression for outputting the mean value of multiple hand-eye calibration matrices as the final hand-eye calibration matrix is as follows: Where j represents the hand-eye calibration matrix number, and m represents the total number of hand-eye calibration matrices.

10. A hand-eye calibration system, characterized in that, Realized by the hand-eye calibration method of the line laser vision sensor and the industrial robot according to any one of claims 1 to 9, including a robot, an end effector on the robot, and a supporting calibration board. A sensor mounting bracket is provided on the end effector, and a line laser vision sensor is mounted on the sensor mounting bracket.

Images