A method for robot hand-eye calibration on a long material platform using a self-made 3D calibration plate

Abstract

This invention discloses a robot hand-eye calibration method using a self-made 3D calibration plate on a long material platform, belonging to the field of vision-based automated assembly of steel structures. It solves the problems of cumbersome and inaccurate robot hand-eye calibration on long material platforms. The hand-eye calibration method includes: a) designing a calibration plate with four calibration blocks and determining calibration feature points; b) moving the robot to the calibration feature points on the calibration plate and recording the position of the robot's end effector; c) moving a gantry-type support equipped with a line laser sensor on a horizontal guide rail to acquire 3D measurement data of the calibration plate on the scanning platform and extracting the calibration feature points in the line laser sensor coordinate system; d) using the different coordinates of the calibration feature points in the line laser sensor coordinate system and the robot's base coordinate system to solve the hand-eye calibration matrix. This method achieves rapid robot hand-eye calibration, is simple to operate, and fully meets the accuracy requirements of steel structure assembly when using a line laser sensor to scan parts on a long material platform.

Description

Technical Field

[0001] This invention relates to the field of vision-based automated assembly of steel structures, and in particular to a method for robot hand-eye calibration on a long material platform using a self-made three-dimensional calibration plate. Background Art

[0002] In recent years, industrial robots have become a powerful tool for promoting the transformation and upgrading of the manufacturing industry, gradually reducing manual labor in the production of complex steel structural components. During the assembly and welding of steel structural components, industrial robots can overcome the complex conditions of heavy and diverse components, while reducing human intervention and improving assembly efficiency. To ensure the assembly accuracy of robots and to clearly perceive the complex environment of the welding workshop, machine vision must be introduced. Line laser sensors, as a high-precision non-contact measuring device, have been widely used in industrial vision.

[0003] Depending on whether the line laser is fixed to the robot's end flange or external to the robot, it can be divided into two methods: eye-in-hand and eye-to-hand. In the eye-to-hand case, it is generally necessary to find three or more identical points in both the eye and hand coordinate systems, and then calculate the hand-eye matrix based on the coordinates of these points.

[0004] Patent No. 2024115977893 discloses an industrial robot hand-eye calibration plate and calibration method for single-line structured light, including: a calibration plate body and four straight lines L1, L2, L3, and L4 disposed on the calibration plate body; the surface of the calibration plate body is black or white and rectangular; the calibration plate body is disposed directly below the structured light camera, and the size of the calibration plate body is set to meet the imaging requirements of the structured light camera, and the size of the calibration plate body is set to meet the effective field of view width of the structured light camera; straight lines L2 and L3 are diagonals on the surface of the calibration plate body; and straight lines L1, L2, and L3 intersect at point O1; straight line L4 is disposed parallel to straight line L1 and intersects with L2 and L3; a point O2 is also disposed on straight line L4. The calibration method of the above invention is simple in process, easy to operate, and can be used across robot platforms, which is beneficial for application in actual industrial scenarios.

[0005] Patent No. 2023108568464 discloses a method, device, and system for extrinsic parameter calibration of a line laser module. The extrinsic parameter calibration method includes the following steps: Step A: Calculate the yaw angle based on the set of line laser points emitted by the line laser emitter towards a vertical calibration plate; Step B: Calculate the pitch angle when the vertical calibration plate is displaced relative to the line laser module; Step C: Based on the yaw and pitch angles, perform coordinate transformation on the set of line laser points emitted by the line laser emitter towards a horizontal calibration plate to obtain a laser mapping point set; then use this laser mapping point set to calculate the roll angle; Step D: Use the roll angle to perform coordinate transformation on the laser mapping point set to obtain a calibration horizontal plane point set, and then mark the average height of the calibration horizontal plane point set as the calibration extrinsic parameter of the line laser module. This method addresses the calibration needs of line laser modules in scenarios with low precision requirements, and requires less computation.

[0006] However, the above method has certain limitations. For example, when the line laser sensor is mounted on the gantry bracket and scans a large area along the horizontal guide rail, a large number of calibration points need to be determined to calculate the hand-eye matrix, which makes the calibration work cumbersome and computationally intensive. Furthermore, the accuracy of the hand-eye matrix is ​​reduced because it is difficult to ensure the levelness of the long track. Summary of the Invention

[0007] The purpose of this invention is to provide a robot hand-eye calibration method on a long material platform using a self-made 3D calibration plate. This method enables rapid, automatic, and accurate hand-eye calibration of the robot through the special structure of the calibration plate, thereby overcoming the problems of cumbersome and low-accuracy hand-eye calibration of robots on long material platforms.

[0008] This invention utilizes the following technical solution:

[0009] A method for robot hand-eye calibration on a long material platform with a self-made 3D calibration plate, comprising the following steps in sequence;

[0010] Step a: Create a calibration board containing four calibration blocks and determine the calibration feature points;

[0011] Step b: Move the robot to the calibration feature point on the calibration board and record the current position of the robot's end effector;

[0012] Step c: Move the gantry bracket carrying the line laser sensor on the horizontal guide rail to obtain the three-dimensional measurement data of the calibration plate on the scanning platform, thereby extracting the calibration feature points in the coordinate system of the line laser sensor;

[0013] Step d: Using the different coordinates of the calibration feature points in the online laser sensor coordinate system and the robot base coordinate system, solve the hand-eye calibration matrix to complete the hand-eye calibration of the robot on the long material platform.

[0014] Preferably, the calibration feature points are as follows: the calibration plate is a three-dimensional calibration plate, and the calibration plate includes four calibration blocks; the calibration blocks are cuboid calibration blocks, and the cross-sectional area of ​​the calibration blocks is 1 / 4 of the cross-sectional area of ​​the robot end effector; the four calibration blocks are distributed in a circular array along the center of the calibration plate, that is, the four calibration blocks are evenly distributed at equal intervals with the center of the calibration plate as the center, and the inner corner point of each calibration block is used as the calibration feature point.

[0015] Preferably, step b includes the following steps:

[0016] b1. Set the robot's base coordinate system to o b -x b y b z b And set the robot's end effector coordinate system to o t -x t y t z t At the same time, the coordinate system of the line laser sensor is set to o. l -x l y l z l Simultaneously, the coordinate system of the calibration block is set to o. c -x c y c z c ;

[0017] b2. Align the robot end effector with the edge of the calibration block, and align the center point of the actuator with the calibration feature point, ensuring that the robot end effector posture at the feature point position on each calibration block is different, thus obtaining the coordinates of the calibration feature point in the robot end effector coordinate system. That is, the calibration feature points of the robot's end effector coordinate system

[0018] Preferably, step c includes the following steps:

[0019] c1: Determine the scanning start point of the laser sensor, which serves as the origin of the line laser sensor coordinate system. l ;

[0020] c2: Move the gantry bracket equipped with the line laser sensor on the horizontal guide rail to obtain the three-dimensional measurement data of the calibration plate on the scanning platform;

[0021] c3: Remove the lower calibration board data and calibration board handle data from the three-dimensional measurement data of the calibration board, and segment out the calibration block data;

[0022] c4: Extract the boundary data from each calibration block data, and perform line fitting and rectangle fitting on the boundary data in sequence to obtain the fitted rectangle;

[0023] c5: Determine the coordinates of the inner corner points of the four fitted rectangles on each calibration plate in the online laser sensor coordinate system. This leads to the calibration feature points of the line laser sensor coordinate system.

[0024] Preferably, step d includes the following steps:

[0025] d1: Marking matrix via hand-eye calibration The calibration feature points of the line laser sensor coordinate system Calibration feature points transformed to the robot end-effector coordinate system The matrix relation is obtained;

[0026] d2: Expand the matrix relation, set the y-axis coordinate of the calibration feature point in the line laser sensor coordinate system to 0, and convert the matrix relation into the form AX = B to determine the matrix form of A, B and X;

[0027] d3: Solve for X using the least squares method, and use the Cartesian coordinate system to solve for the y-axis coordinates of the calibration feature points in the line laser sensor coordinate system;

[0028] d4: Place calibration plates every X meters on the scanning platform equipped with a line laser sensor for calibration, record the hand-eye calibration matrix for each robot, and then complete the hand-eye calibration of the robot on the long material platform.

[0029] 1) The calibration method of this invention is applicable to hand-eye calibration (eye outside the hand) of all industrial robots and line laser profile scanners that use rectangular end-face tools;

[0030] 2) The self-made three-dimensional calibration plate of the present invention has the characteristics of simple structure and easy processing. Combined with the three-dimensional point cloud algorithm, it can automatically and accurately extract the feature points required for hand-eye calibration.

[0031] 3) This invention utilizes the characteristic that each point cloud data acquired by a line laser profile scanner is a straight line, which simplifies the solution of the hand-eye calibration matrix, is fast to calculate, and uses multiple sets of corresponding points to solve, thereby improving the accuracy of the calibration results. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0033] Figure 1 A flowchart for the robot hand-eye calibration method;

[0034] Figure 2 This is a schematic diagram of the hardware layout;

[0035] Figure 3 A schematic diagram of the self-made 3D calibration plate structure;

[0036] Figure 4 A schematic diagram of the point cloud of the self-made 3D calibration board and the calibration feature points on the calibration block;

[0037] Calibration plate-1; Calibration block-2; Horizontal guide rail-3; Gantry bracket-4; Line laser sensor-5; Scanning platform-6; Calibration feature points on the calibration block Detailed Implementation

[0038] The present invention will now be described in detail with reference to the accompanying drawings and embodiments:

[0039] like Figures 1-4 As shown, the present invention provides a robot hand-eye calibration method on a long material platform using a self-made three-dimensional calibration plate, which includes the following steps in sequence;

[0040] Step a: Create a calibration plate 1 containing four calibration blocks 2 and determine the calibration feature points;

[0041] Step b: Move the robot to the calibration feature point on calibration board 1 and record the current position of the robot's end effector;

[0042] Step c: Move the gantry bracket 4 carrying the line laser sensor 5 on the horizontal guide rail 3 to obtain the three-dimensional measurement data of the calibration plate 1 on the scanning platform 6, thereby extracting the calibration feature points in the coordinate system of the line laser sensor 5.

[0043] Step d: Using the different coordinates of the calibration feature points in the online laser sensor 5 coordinate system and the robot base coordinate system, solve the hand-eye calibration matrix to complete the hand-eye calibration of the robot on the long material platform.

[0044] In this invention, the calibration feature points are as follows: the calibration plate 1 is a three-dimensional calibration plate 1, and the calibration plate 1 includes four calibration blocks 2; the calibration blocks 2 are cuboid calibration blocks 2, and the cross-sectional area of ​​the calibration blocks 2 is 1 / 4 of the cross-sectional area of ​​the robot end effector; the four calibration blocks 2 are distributed in a circular array along the center of the calibration plate 1, that is, the four calibration blocks 2 are evenly distributed at equal intervals with the center of the calibration plate 1 as the center, and the inner corner point of each calibration block 2 is used as a calibration feature point.

[0045] In this invention, step b includes the following steps:

[0046] b1. Set the robot's base coordinate system to o b -x b y b z bAnd set the robot's end effector coordinate system to o t -x t y t z t At the same time, the coordinate system of the line laser sensor 5 is set to o. l -x l y l z l Simultaneously, the coordinate system of calibration block 2 is set to o. c -x c y c z c ;

[0047] b2. Align the robot end effector with the edge of calibration block 2, and align the center point of the actuator with the calibration feature point, ensuring that the robot end effector posture is different for each feature point position on calibration block 2, thus obtaining the coordinates of the calibration feature point in the robot end effector coordinate system. That is, the calibration feature points of the robot's end effector coordinate system

[0048] In this invention, step c includes the following steps:

[0049] c1: Determine the scanning start point of the laser sensor, which serves as the origin of the 5-coordinate system of the line laser sensor. l ;

[0050] c2: Move the gantry bracket 4, which carries the line laser sensor 5, on the horizontal guide rail 3 to obtain the three-dimensional measurement data of the calibration plate 1 on the scanning platform 6;

[0051] c3: Remove the data of the lower calibration plate 1 and the data of the handle of calibration plate 1 from the three-dimensional measurement data of calibration plate 1, and extract the data of calibration block 2;

[0052] c4: Extract the boundary data from each calibration block 2 data, and perform line fitting and rectangle fitting on the boundary data in sequence to obtain the fitted rectangle;

[0053] c5: Determine the coordinates of the inner corner points of the four fitted rectangles on each calibration plate 1 in the coordinate system of the online laser sensor 5. This leads to the calibration feature points of the line laser sensor in coordinate system 5.

[0054] In this invention, step d includes the following steps:

[0055] d1: Marking matrix via hand-eye calibration The calibration feature points of the line laser sensor in the coordinate system of coordinate 5. Calibration feature points transformed to the robot end-effector coordinate system The matrix relation is obtained;

[0056] d2: Expand the matrix relationship, set the y-axis coordinate of the calibration feature point in the coordinate system of the line laser sensor 5 to 0, and convert the matrix relationship into the form AX = B to determine the matrix form of A, B and X;

[0057] d3: Solve for X using the least squares method, and use the Cartesian coordinate system to solve for the y-axis coordinates of the calibration feature points in the coordinate system of the line laser sensor 5.

[0058] d4: Place calibration plates 1 every X meters on the scanning platform 6 equipped with line laser sensor 5 for calibration, record each hand-eye calibration matrix, and thus complete the hand-eye calibration of the robot on the long material platform.

[0059] Example:

[0060] First, by customizing the calibration plate 1, the cross-section of the cuboid calibration blocks 2 on the calibration plate 1 is 1 / 4 the size of the robot end-effector cross-section. The four calibration blocks 2 are arranged in a circular array along the center of the calibration plate 1, that is, the four calibration blocks 2 are evenly distributed at equal intervals with the center of the calibration plate 1 as the center. The inner corner of each calibration block 2 is used as the calibration feature point. A cross-shaped protrusion is designed on the bottom of the calibration plate 1, which can fit into the groove on the scanning platform 6, so that the calibration plate 1 is embedded into the groove of the scanning platform 6, ensuring stable installation without shaking. Each calibration plate 1 can establish four sets of correspondence between the robot end-effector coordinate system and the line laser sensor 5 coordinate system, which are used to solve the hand-eye calibration matrix in the future.

[0061] Then, define the coordinate system and set the robot's base coordinate system as o. b -x b y b z b Set the robot's end effector coordinate system to o. t -x t y t z t The coordinate system of the line laser sensor 5 is set to o. l -x l y l z l Set the coordinate system of calibration block 2 to o. c -x c y c z c ;

[0062] Move the line laser sensor 5 to a safe position to ensure it does not interfere with the robot's movement. Then, teach the robot's end effector to follow the attached... Figure 4The calibration feature points are sequentially aligned with the edges of each calibration block 2, that is, the tool center point is aligned with the calibration feature points, ensuring that the robot end-effector poses are different at the four positions, and ensuring that the robot end-effector coordinate system of each pose remains unchanged relative to the calibration plate 1 coordinate system, thus obtaining the coordinates of the calibration feature points in the robot end-effector coordinate system.

[0063] Subsequently, the scanning starting point of the line laser sensor 5 is determined as the origin of the laser coordinate system. In actual operation, the line laser sensor 5 will start scanning from this point. According to the characteristics of the line laser sensor 5, the scanning direction is the positive Y-axis direction of the laser coordinate system, the positive Z-axis direction is vertically upward, and the X-axis direction is obtained by multiplying the Y-axis direction by the Z-axis direction.

[0064] The gantry bracket 4, which carries the line laser sensor 5, is moved on the horizontal guide rail 3 to acquire three-dimensional measurement data of the calibration plate 1 located on the scanning platform 6.

[0065] The 3D measurement data is preprocessed to remove the point cloud of the lower calibration plate 1 and the point cloud of the upper handle, for example, by using a pass-through filter; the point cloud of calibration block 2 is segmented, for example, by using a Euclidean clustering algorithm.

[0066] Extract the boundaries from the segmented point cloud of the calibration block 2, for example, using the Alpha Shape algorithm for segmentation;

[0067] The extracted boundaries are fitted with straight lines, followed by rectangular fitting, for example using the RANSAC algorithm.

[0068] Determine the inner corner points of the four fitted rectangles on each calibration plate 1 and their corresponding points in step (a), and record their second coordinates in the coordinate system of the online laser sensor 5.

[0069] Hand-eye calibration matrix It can be described as:

[0070]

[0071] Where i is the subscript number. These are the calibration feature points for the robot's end effector coordinate system. These are the calibration feature points for the line laser sensor in the 5-coordinate system;

[0072] Expanding equation (1) yields:

[0073]

[0074] Among them, [r 11 ,r 21 ,r 31 ] TThis represents the direction vector of the X-axis of the line laser sensor 5 coordinate system in the robot's end effector coordinate system, [r 12 ,r 22 ,r 32 ] T This represents the direction vector of the Y-axis of the line laser sensor 5 coordinate system in the robot's end effector coordinate system, [r 13 ,r 23 ,r 33 ] T Let [x, y, z] represent the direction vector of the Z-axis of the line laser sensor's 5-coordinate system in the robot's end effector coordinate system. T This indicates the position of the origin of the line laser sensor's coordinate system 5 relative to the origin of the robot's end effector coordinate system;

[0075] Let y li =0, so equation (2) can be converted to the form AX = B:

[0076]

[0077] in

[0078] Solving for X using the least squares method yields the result shown in equation (4):

[0079] X = (A T A) -1 A T B (4)

[0080] Using the Cartesian coordinate system, vector [r] 12 ,r 22 ,r 32 ] = [r 13 ,r 23 ,r 33 ]×[r 11 ,r 21 ,r 31 ];

[0081] By repeating the above steps, a calibration plate 1 is placed every 1m on the horizontal guide rail 3 equipped with the wired laser sensor 5 for calibration, and a hand-eye matrix is ​​recorded, which can realize the calibration of ultra-long tracks.

Claims

1. A method for robot hand-eye calibration on a long material platform using a self-made 3D calibration plate, characterized in that: The steps are as follows: Step a: Create a calibration board containing four calibration blocks and determine the calibration feature points; Step b: Move the robot to the calibration feature point on the calibration board and record the current position of the robot's end effector; Step c: Move the gantry bracket carrying the line laser sensor on the horizontal guide rail to obtain the three-dimensional measurement data of the calibration plate on the scanning platform, thereby extracting the calibration feature points in the coordinate system of the line laser sensor; Step d: Using the different coordinates of the calibration feature points in the online laser sensor coordinate system and the robot base coordinate system, solve the hand-eye calibration matrix to complete the hand-eye calibration of the robot on the long material platform; Step b includes the following steps: b1. Set the robot's base coordinate system to o b -x b y b z b And set the robot's end effector coordinate system to o t -x t y t z t At the same time, the coordinate system of the line laser sensor is set to o. l -x l y l z l Simultaneously, the coordinate system of the calibration block is set to o. c -x c y c z c ; b2. Align the robot end effector with the edge of the calibration block, and align the center point of the actuator with the calibration feature point. Ensure that the robot end effector posture at the feature point position on each calibration block is different, and obtain the coordinates of the calibration feature point in the robot end coordinate system, i.e., the calibration feature point in the robot end coordinate system. Step c includes the following steps. c1: Determine the scanning start point of the laser sensor, which serves as the origin of the line laser sensor coordinate system. l ; c2: Move the gantry bracket equipped with the line laser sensor on the horizontal guide rail to obtain the three-dimensional measurement data of the calibration plate on the scanning platform; c3: Remove the lower calibration board data and calibration board handle data from the three-dimensional measurement data of the calibration board, and segment out the calibration block data; c4: Extract the boundary data from each calibration block data, and perform line fitting and rectangle fitting on the boundary data in sequence to obtain the fitted rectangle; c5: Determine the coordinates of the inner corner points of the four fitted rectangles on each calibration plate in the linear laser sensor coordinate system, and then obtain the calibration feature points of the linear laser sensor coordinate system. Step d includes the following steps: d1: Transform the calibration feature points of the line laser sensor coordinate system to the calibration feature points of the robot end effector coordinate system using the hand-eye calibration matrix to obtain the matrix relationship; d2: Expand the matrix relation, set the y-axis coordinate of the calibration feature point in the line laser sensor coordinate system to 0, and convert the matrix relation to the form AX=B to determine the matrix form of A, B and X; d3: Solve for X using the least squares method, and use the Cartesian coordinate system to solve for the y-axis coordinates of the calibration feature points in the line laser sensor coordinate system; d4: Place calibration plates every X meters on the scanning platform equipped with a line laser sensor for calibration, record the hand-eye calibration matrix for each robot, and then complete the hand-eye calibration of the robot on the long material platform.

2. The robot hand-eye calibration method on a long material platform using a self-made three-dimensional calibration plate according to claim 1, characterized in that: The calibration feature points are as follows: the calibration plate is a three-dimensional calibration plate, and the calibration plate includes four calibration blocks; the calibration blocks are cuboid calibration blocks; the four calibration blocks are arranged in a circular array along the center of the calibration plate, that is, the four calibration blocks are evenly distributed at equal intervals with the center of the calibration plate as the center, and the inner corner points of each calibration block are used as calibration feature points.

3. The robot hand-eye calibration method on a long material platform using a self-made three-dimensional calibration plate according to claim 1, characterized in that: The cross-sectional area of ​​the calibration block is 1 / n of the cross-sectional area of ​​the robot's end effector, where n is 4.

Images