A line laser 3D camera and mechanical arm hand-eye calibration system and method

Abstract

The application discloses a kind of line laser 3D camera and mechanical arm's eye calibration system and method, belong to industrial manufacturing and automation control technical field, including calibration board, line laser 3D camera and mechanical arm, the line laser 3D camera is connected with the mechanical arm, the calibration board is composed of calibration ball, connecting rod and mounting plate, single frame data of the line laser 3D camera shooting is solved by single frame data solution calibration board pose algorithm.The application uses above-mentioned line laser 3D camera and mechanical arm's eye calibration system and method, calibration board and single frame data solution calibration board pose algorithm can solve the attitude of calibration board in camera coordinate system according to line laser 3D camera single frame data, and then by shooting multiple frame data and combining the posture information of mechanical arm under each frame image, the conversion relationship between camera coordinate system and mechanical arm coordinate system can be solved without camera size and installation information, to solve the inaccurate problem caused by machining and installation error.

Description

Technical Field

[0001] This invention relates to the fields of industrial manufacturing and automation control technology, and in particular to a hand-eye calibration system and method for a line laser 3D camera and a robotic arm. Background Technology

[0002] Establishing a highly precise correlation between the end effector of a robotic arm and its connected actuators is fundamental to achieving high-precision measurement and manipulation. In robotic arm control, advanced robotic technology allows for accurate positioning and movement of the arm to contact and manipulate target objects. However, since the connected actuators are often added later to adapt to specific tasks and functions, achieving high-precision measurement and manipulation places higher demands on the calibration of the correlation between the actuators and the end effector. Laser measurement technology is a high-precision measurement technique. Line laser 3D cameras can scan the surface of a target object, and combined with the motion information of the robotic arm, the three-dimensional shape and geometric information of the measured object can be obtained. However, currently, line laser 3D cameras can generally only calculate the correlation between the camera coordinate system and the robotic arm coordinate system using known installation dimensions and dimensional information provided by the camera manufacturer. This correlation is often inaccurate due to machining, installation errors, and measurement errors, or requires higher installation and machining precision to obtain a more accurate correlation.

[0003] Against this backdrop, this invention proposes a method for hand-eye calibration of a line laser 3D camera and a robotic arm. This method directly uses a line laser 3D camera mounted on the robotic arm to measure a calibration plate. An algorithm calculates the calibration plate's orientation in the camera coordinate system based on single-frame data from the line laser 3D camera. Then, by capturing multiple frames and combining the robotic arm's orientation information from each frame, the correlation between the line laser 3D camera and the robotic arm can be calculated without knowing the camera's size or precise installation. This solves the inaccuracies caused by machining and installation errors, effectively addressing the problems and shortcomings of existing technologies. This invention has broad application prospects in industrial manufacturing, automation control, and other fields. Summary of the Invention

[0004] The purpose of this invention is to provide a hand-eye calibration system and method for a line laser 3D camera and a robotic arm, in order to solve the problem of inaccurate correlation between the line laser 3D camera and the robotic arm due to machining and installation errors in the above-mentioned background art.

[0005] To achieve the above objectives, the present invention provides a hand-eye calibration system for a line laser 3D camera and a robotic arm, comprising a calibration plate, a line laser 3D camera, and a robotic arm. The line laser 3D camera is connected to the robotic arm. The calibration plate consists of a calibration ball, a connecting rod, and a mounting plate. The single-frame data captured by the line laser 3D camera is calculated using a single-frame data calculation calibration plate pose algorithm.

[0006] Preferably, the line laser 3D camera and the robotic arm are connected via a connector, and the line laser 3D camera, the robotic arm, and the connector together form a motion mechanism with the ability to move within a certain range for performing work tasks.

[0007] Preferably, there are three calibration balls, all of which have the same diameter. The three calibration balls are connected to the mounting plate via connecting rods, the length of which is fixed or variable. The mounting plate has three mounting holes, the spacing between adjacent mounting holes is unequal and significantly different, and the connecting rods are connected to the mounting holes.

[0008] Preferably, any one of the three calibration spheres is selected, and its center is used as the origin of the coordinate system. The spatial plane formed by the centers of the three calibration spheres is used as the reference plane to establish the Cartesian coordinate system of the calibration plate. During calibration, the calibration plate is placed within the field of view of the line laser 3D camera.

[0009] Preferably, the single-frame data captured by the line laser 3D camera is used to calculate the calibration plate pose using a single-frame data calculation algorithm. Specifically, when the three calibration spheres on the calibration plate are simultaneously observed to be in any pose of the line laser 3D camera, the pose and position of the calibration plate relative to the line laser 3D camera coordinate system can be calculated using the single-frame data captured by the line laser 3D camera. The specific steps are as follows:

[0010] S1. Determine the dimensions of the calibration plate, including the radius of the calibration ball, the height of each connecting rod, and the spacing between the mounting holes on the mounting plate. Obtain the relative positions of each calibration ball on the calibration plate to form a spatial triangle. Establish a Cartesian coordinate system fixed to the calibration plate using this spatial triangle as a reference plane.

[0011] S2. Adjust the line laser 3D camera so that all three calibration spheres are in its field of view, capture a single frame of data, and obtain the data of three arcs under the line laser 3D camera, that is, the two-dimensional data information of the depth and offset of the points of the three calibration spheres illuminated by the line laser 3D camera relative to the image coordinate system of the line laser 3D camera.

[0012] S3. Remove noise from the data and use the sampling fitting method to fit the two-dimensional circular trajectory equations of the three arcs, so as to obtain the corresponding center positions and radii of the three arcs in the two-dimensional plane sampled by the line laser 3D camera.

[0013] S4. Calculate the position of the center of the calibration sphere in the coordinate system of the line laser 3D camera. The center position and radius of each arc obtained from step S3 are used to calculate the distance between the center of each calibration sphere and the two-dimensional plane sampled by the line laser 3D camera using the Pythagorean theorem. This distance is the position of the center of the calibration sphere in the coordinate system of the line laser 3D camera.

[0014] S5. By comparing and matching the spatial positional relationship of the centers of each calibration ball with the relative positions of each calibration ball calculated in step S1 on the calibration plate, the attitude and positional relationship of the Cartesian coordinate system of the calibration plate relative to the image coordinate system of the line laser 3D camera are obtained.

[0015] A method for a hand-eye calibration system for a line laser 3D camera and a robotic arm is provided. The method utilizes a single-frame data calculation algorithm to obtain a set of positions and orientations of the calibration board relative to the line laser 3D camera, and combines this with the corresponding position and orientation information of the robotic arm to calculate the hand-eye calibration matrix.

[0016] Therefore, this invention employs the aforementioned hand-eye calibration system and method for a line laser 3D camera and robotic arm. Addressing the issue that line laser 3D cameras can only calculate the transformation relationship between the camera coordinate system and the robotic arm coordinate system using known installation dimensions and size information provided by the camera manufacturer, thus failing to resolve the inaccuracy caused by machining and installation errors, this invention designs a calibration plate and a single-frame data calculation algorithm for the calibration plate's pose. Based on a single frame of data from the line laser 3D camera, the calibration plate's pose in the camera coordinate system can be calculated. By capturing multiple frames of data and combining the robotic arm's pose information from each frame, the calculation of the transformation relationship between the camera coordinate system and the robotic arm coordinate system can be completed without requiring camera size and installation information, thus resolving the inaccuracies caused by machining and installation errors. The designed calibration plate consists of three calibration balls, a connecting rod, and a mounting plate. The calculation algorithm works in conjunction with the calibration plate and is applied to the hand-eye calibration of a line laser 3D camera and robotic arm.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the hand-eye calibration system of a line laser 3D camera and robotic arm according to the present invention;

[0019] Figure 2This is a schematic diagram of the calibration plate captured by the laser 3D camera in this invention;

[0020] Figure 3 This is a schematic diagram of the calibration plate composition of the present invention;

[0021] Figure 4 This is a flowchart illustrating the attitude and position in the coordinate system of the online laser 3D camera on the single-frame data calculation calibration board of this invention.

[0022] Figure Labels

[0023] 1. Calibration plate; 101. Calibration ball; 102. Connecting rod; 103. Mounting plate; 2. Line laser 3D camera; 3. Connector; 4. Robotic arm. Detailed Implementation

[0024] Example

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] like Figure 1-3 As shown, a hand-eye calibration system for a line laser 3D camera and a robotic arm includes a calibration plate 1, a line laser 3D camera 2, a robotic arm 4, and a connector 3. The connector 3 connects the line laser 3D camera 2 and the robotic arm 4. The connector 3 can be a fixed connection, a rotating connection, or a connection without a connector 3, meaning the line laser 3D camera 2 and the robotic arm 4 are directly connected. The line laser 3D camera 2, the robotic arm 4, and the connector 3 together form a motion mechanism with the ability to move within a certain range to perform work tasks. Single-frame data captured by the line laser 3D camera 2 is calculated using a pose algorithm on the calibration plate 1 based on the single-frame data calculation.

[0027] The calibration plate 1 consists of three calibration balls 101, connecting rods 102, and a mounting plate 103. Three calibration balls 101 are provided, all with equal diameters. Each of the three calibration balls 101 is connected to the mounting plate 103 via a connecting rod 102, the length of which can be fixed or variable. The mounting plate 103 has three mounting holes, with unequal and significantly different spacing between adjacent holes. The connecting rods 102 connect to the mounting holes. A Cartesian coordinate system for the calibration plate 1 is established by selecting any one of the three calibration balls 101, using its center as the origin, and the spatial plane formed by the centers of the three calibration balls 101 as the reference plane. During calibration, the calibration plate 1 is placed within the field of view of the line laser 3D camera 2.

[0028] like Figure 4 As shown, the single-frame data captured by the line laser 3D camera 2 is used to calculate the pose of the calibration plate 1 using a single-frame data calculation algorithm. Specifically, when the three calibration spheres 101 on the calibration plate 1 are simultaneously observed under any pose of the line laser 3D camera 2, the pose and position of the calibration plate 1 relative to the coordinate system of the line laser 3D camera 2 can be calculated using the single-frame data captured by the line laser 3D camera 2. The specific steps are as follows:

[0029] S1. Determine the dimensions of the calibration plate 1, including the radius of the calibration ball 101, the height of each connecting rod 102, and the spacing between the mounting holes on the mounting plate 103, to obtain the relative position of each calibration ball 101 on the calibration plate 1. 12 , l 23 , l 31 This yields a spatial triangle. A coordinate system is established with the center of one of the calibration spheres 101 as the origin and this spatial triangle as the reference plane. Let l... 12 l 31 The intersection point is the origin O. t Take l 12 The direction from 1 to 2 is the x-axis of the coordinate system. t In the axial direction, take the cross product direction of 1→2 and 1→3 as z. t axial direction, y t axis and x t axis and z t The axes form a right-handed coordinate system O, which is fixed to calibration plate 1. t x t y t z t .

[0030] S2. Adjust the line laser 3D camera 2 so that all three calibration spheres 101 are within its field of view, and capture a single frame of data. This yields data for three arcs from the line laser 3D camera 2, representing the two-dimensional data of the depth and offset of the points on the three calibration spheres 101 illuminated by the line laser 3D camera 2 relative to the image coordinate system of the line laser 3D camera 2. Establish a right-handed coordinate system, taking the depth as the y-axis and the offset as the x-axis, thus establishing the image coordinate system O of the line laser 3D camera 2. c x c y c z c ;

[0031] S3. Remove noise from the data and use the sampling fitting method to fit the two-dimensional circular trajectory equations of the three arc segments. Further obtain the corresponding center positions and radii of the three arc segments in the two-dimensional plane sampled by the online laser 3D camera 2, which are denoted as (x1, y1), (x2, y2), (x3, y3) and r1, r2, r3 respectively.

[0032] S4. Calculate the position of the center of the calibration sphere 101 in the coordinate system of the online laser 3D camera 2. The center positions and radii of each arc obtained from step S3 are then used to apply the Pythagorean theorem...

[0033]

[0034] Simultaneously, by combining the shooting point of the line laser 3D camera 2 with the center position of the calibration sphere 101, the z-value is determined. i The positive and negative values ​​indicate the distance z between the center of each sphere and the two-dimensional plane sampled by the two-dimensional laser 3D camera. i That is, the position of the center of the calibration sphere 101 in the coordinate system of the online laser 3D camera 2 is obtained, namely (x1,y1,z1), (x2,y2,z2), (x3,y3,z3);

[0035] S5. By comparing and matching the spatial relationship between the centers of each calibration ball 101 with the relative positions of each calibration ball 101 on the calibration plate 1 calculated in step S1, the Cartesian coordinate system O defined in step S1 and fixed on the calibration plate 1 can be obtained. t x t y t z t Relative to the line laser 3D camera 2 image coordinate system O c x c y c z c The posture and positional relationship.

[0036] A method for a hand-eye calibration system of a line laser 3D camera 2 and a robotic arm 4 is provided. The method involves continuously changing the spatial pose of the line laser 3D camera 2 to capture images of a calibration board 1, using a single-frame data calculation algorithm to calculate a series of positions and orientations of the calibration board 1 relative to the line laser 3D camera 2, and simultaneously combining the position and orientation information of the corresponding robotic arm 4 to calculate the hand-eye calibration matrix.

[0037] Therefore, the present invention adopts the above-mentioned hand-eye calibration system and method for a line laser 3D camera and a robotic arm. The calibration plate and the single-frame data calculation algorithm for the calibration plate pose can calculate the orientation of the calibration plate in the camera coordinate system based on the single-frame data of the line laser 3D camera. Then, by capturing multiple frames of data and combining the posture information of the robotic arm under each frame image, the conversion relationship between the camera coordinate system and the robotic arm coordinate system can be calculated without the need for camera size and installation information, thus solving the inaccuracy problem caused by machining and installation errors.

[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A hand-eye calibration system for a line laser 3D camera and a robotic arm, characterized in that: It includes a calibration plate, a line laser 3D camera, and a robotic arm. The line laser 3D camera is connected to the robotic arm. The calibration plate consists of a calibration ball, a connecting rod, and a mounting plate. The single-frame data captured by the line laser 3D camera is solved using a single-frame data calculation calibration plate pose algorithm. The calibration ball is provided in three parts, all of which have the same diameter. The three calibration balls are connected to the mounting plate by connecting rods. The mounting plate is provided with three mounting holes, the spacing between adjacent mounting holes is not equal, and the connecting rods are connected to the mounting holes. Choose any one of the three calibration spheres, take its center as the origin, and use the space plane formed by the centers of the three calibration spheres as the reference plane to establish the Cartesian coordinate system of the calibration plate. During calibration, the calibration plate is placed within the field of view of the line laser 3D camera.

2. The hand-eye calibration system for a line laser 3D camera and robotic arm according to claim 1, characterized in that: The line laser 3D camera is connected to the robotic arm via a connector. The line laser 3D camera, the robotic arm, and the connector together form a motion mechanism with motion capabilities for performing work tasks.

3. The hand-eye calibration system for a line laser 3D camera and robotic arm according to claim 1, characterized in that, The single-frame data captured by the line laser 3D camera is used to calculate the calibration board pose using a single-frame data calculation algorithm. Specifically, when the three calibration spheres on the calibration board are simultaneously observed to be in any pose of the line laser 3D camera, the pose and position of the calibration board relative to the line laser 3D camera coordinate system are calculated using the single-frame data captured by the line laser 3D camera. The specific steps are as follows: S1. Determine the dimensions of the calibration plate, including the radius of the calibration ball, the height of each connecting rod, and the spacing between the mounting holes on the mounting plate. Obtain the relative positions of each calibration ball on the calibration plate to form a spatial triangle. Establish a Cartesian coordinate system fixed to the calibration plate using this spatial triangle as a reference plane. S2. Adjust the line laser 3D camera so that all three calibration spheres are in its field of view, capture a single frame of data, and obtain the data of three arc segments under the line laser 3D camera. This data is the two-dimensional data information of the depth and offset of the points of the three calibration spheres illuminated by the line laser 3D camera relative to the image coordinate system of the line laser 3D camera. S3. Remove noise from the data and use the sampling fitting method to fit the two-dimensional circular trajectory equations of the three arcs respectively, so as to obtain the corresponding center positions and radii of the three arcs in the two-dimensional plane sampled by the line laser 3D camera. S4. Calculate the position of the center of the calibration sphere in the coordinate system of the line laser 3D camera. The center position and radius of each arc obtained from step S3 are used to calculate the distance between the center of each calibration sphere and the two-dimensional plane sampled by the line laser 3D camera using the Pythagorean theorem. This distance is the position of the center of the calibration sphere in the coordinate system of the line laser 3D camera. S5. By comparing and matching the spatial positional relationship of the centers of each calibration ball with the relative positions of each calibration ball calculated in step S1 on the calibration plate, the attitude and positional relationship of the Cartesian coordinate system of the calibration plate relative to the image coordinate system of the line laser 3D camera are obtained.

4. A method for applying the hand-eye calibration system of a line laser 3D camera and robotic arm as described in any one of claims 1-3, characterized in that: The position and orientation of the calibration board relative to the line laser 3D camera are calculated using a single-frame data calculation algorithm, and the hand-eye calibration matrix is ​​calculated by combining the corresponding position and orientation information of the robotic arm.

Images