Automatic hand-eye calibration method, hand-eye calibration system and electronic equipment

Abstract

The invention provides an automatic hand-eye calibration method, a hand-eye calibration system and electronic equipment, and the method comprises the steps: obtaining a first machine tool coordinate of the center of a circle of each disc in a machine tool coordinate system based on an arc centering mode; the line laser sensor scans the disc and the machine tool breadth which is not shielded by the disc at least in the second direction of the machine tool coordinate system to obtain a point cloud; camera coordinates of at least four circle centers are obtained in the point cloud; according to the camera coordinates of all the circle centers, all second machine tool coordinates corresponding to the camera coordinates of all the circle centers are obtained; and obtaining a hand-eye matrix according to the first machine tool coordinate, the camera coordinate and the second machine tool coordinate. In the automatic hand-eye calibration method, the first machine tool coordinate is obtained through an arc centering method, namely, based on a program, and manual operation is not needed, so that the automatic hand-eye calibration method can reduce personal errors, and the calibration result consistency, the calibration result precision and the calibration efficiency are further improved.

Description

Technical Field

[0001] This invention relates to the field of integrated laser cutting, and more particularly to an automatic hand-eye calibration method, hand-eye calibration system, and electronic device. Background Technology

[0002] Laser cutting systems typically consist of an actuator and a vision system, but there is a natural coordinate separation between the two. Therefore, in automated laser cutting systems, hand-eye calibration is a core technical link to ensure cutting accuracy and efficiency, which is to establish a precise mapping relationship between the motion coordinates of the actuator (such as machine tool coordinates) and the image coordinates of the vision system (such as camera coordinates).

[0003] However, the manual hand-eye calibration in the existing technology introduces human error, resulting in poor consistency and accuracy of the calibration results. Summary of the Invention

[0004] This invention provides an automatic hand-eye calibration method, a hand-eye calibration system, and an electronic device to improve the accuracy of hand-eye calibration.

[0005] According to a first aspect of the present invention, an automatic hand-eye calibration method is provided, comprising: Place at least two disks on the machine tool web, with each disk arranged along the first direction; Based on the method of finding the center of the circular arc, the first machine tool coordinates of the center of each of the disks in the machine tool coordinate system are obtained respectively; A line laser sensor scans the disk and the machine tool surface not obscured by the disk at least along a second direction of the machine tool coordinate system to obtain a point cloud. The point cloud is used to represent the upper surface of the disk and the machine tool surface not obscured by the disk. The first direction is perpendicular to the second direction, and both the first direction and the second direction are parallel to the machine tool surface. Obtain the camera coordinates of at least four of the circle centers in the point cloud; Based on the camera coordinates of each of the centers, the second machine tool coordinates corresponding to the camera coordinates of each of the centers are obtained. The second machine tool coordinates are used to characterize the position of the nozzle center of the cutting head in the machine tool coordinate system when the laser line emitted by the line laser sensor passes through the center of the disk. A hand-eye matrix is ​​obtained based on the first machine tool coordinates, the camera coordinates, and the second machine tool coordinates. The hand-eye matrix is ​​used to characterize the transformation relationship between the machine tool coordinate system and the camera coordinate system.

[0006] Optionally, the number of disks placed on the machine tool surface is greater than or equal to 2, and the height difference between any two of the disks is greater than 20 mm.

[0007] Optionally, the method for obtaining the hand-eye matrix further includes: based on the first machine tool coordinates The camera coordinates and the second machine tool coordinates and coordinate transformation model Obtain the hand-eye matrix .

[0008] Optionally, the entire disk is located within the scanning area of ​​the line laser sensor.

[0009] Optionally, obtaining the camera coordinates of at least four circle centers in the point cloud includes: Clustering and segmenting the point cloud yields a disk point cloud symbolizing the upper surface of the disk; Based on the point cloud of the disks, extract the contour point set of each disk; Based on the contour point set of each disk, the center of each disk is fitted, and the camera coordinates of each center are obtained.

[0010] Optionally, the number of disks placed on the machine tool surface is 2 or 3, and the line laser sensor scans the disks and the machine tool surface not obscured by the disks at least along a second direction of the machine tool coordinate system to obtain a point cloud, including: The line laser sensor scans the disk and the machine tool surface not obscured by the disk along the second direction to obtain a first point cloud; Move the line laser sensor a first preset distance along the first direction; A line laser sensor scans the disk and the machine tool surface not obscured by the disk along a third direction to obtain a second point cloud, wherein the third direction is opposite to the second direction; The point cloud is obtained, and the point cloud includes the first point cloud and the second point cloud.

[0011] Optionally, the number of disks is greater than or equal to 4, and the line laser sensor scans the disks and the machine tool surface not obscured by the disks at least along the second direction of the machine tool coordinate system to obtain the point cloud.

[0012] Optionally, obtaining the first machine tool coordinates of the center of each of the disks includes: Move the nozzle of the cutting head above the disk; Obtain the capacitance value between the nozzle and the disk; The nozzle is moved in at least three directions respectively, and the third machine tool coordinates of at least three circumferential points of the disk in the machine tool coordinate system are obtained. During the process of moving the nozzle in any of the at least three directions, when the capacitance value changes abruptly, the position of the nozzle at this time is taken as a circumferential point of the disk, and the third machine tool coordinates of the circumferential point in the machine tool coordinate system are obtained. Based on the third machine tool coordinates of at least three circumferential points of the disk in the machine tool coordinate system, the center of the disk is fitted, and the first machine tool coordinates of the center of the disk in the machine tool coordinate system are obtained.

[0013] According to a second aspect of the present invention, a hand-eye calibration system is provided for implementing the above-described automatic hand-eye calibration method, characterized in that the system comprises: The execution processing module is used to obtain the first machine tool coordinates of the center of each disk in the machine tool coordinate system based on the arc centering method; A scanning fitting module is used to control the line laser sensor to move at least along a second direction and to scan each disk through the line laser sensor to obtain the camera coordinates of the centers of at least four of the disks; The hand-eye calibration module is used to obtain a hand-eye matrix based on the first machine tool coordinates, the camera coordinates, and the second machine tool coordinates. The hand-eye matrix is ​​used to characterize the transformation relationship between the machine tool coordinate system and the camera coordinate system. Furthermore, the hand-eye calibration module is also used to calibrate the line laser sensor and the nozzle based on the hand-eye matrix.

[0014] According to a third aspect of the present invention, an electronic device is provided, comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps in the above-described automatic hand-eye calibration method by running the executable instructions.

[0015] Compared with the prior art, the technical solution of the present invention has the following beneficial effects: In the automatic hand-eye calibration method provided by this invention, based on the arc centering method, the first machine tool coordinates of the center of each disk in the machine tool coordinate system are obtained. Then, a line laser sensor scans the disks and the machine tool surface not obscured by the disks along at least a second direction of the machine tool coordinate system to obtain a point cloud. Next, camera coordinates of at least four of the disk centers are obtained from the point cloud. Then, based on the camera coordinates of each disk center, second machine tool coordinates corresponding to the camera coordinates of each disk center are obtained. Finally, the hand-eye matrix is ​​obtained based on the first machine tool coordinates, camera coordinates, and second machine tool coordinates. Since the first machine tool coordinates are obtained through the arc centering method in the automatic hand-eye calibration method, i.e., based on a program and without manual operation, this automatic hand-eye calibration method can reduce human error, thereby further improving the consistency, accuracy, and efficiency of the calibration results. Attached Figure Description

[0016] Figure 1 This is a flowchart of the automatic hand-eye calibration method provided in the embodiments of the present invention; Figure 2 This is a structural block diagram of the hand-eye calibration system provided in an embodiment of the present invention; Figure 3 This is a structural block diagram of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] As described in the background section, manual hand-eye calibration methods introduce human error, leading to poor consistency and accuracy in calibration results. The following provides a detailed explanation of manual hand-eye calibration methods.

[0018] A manual hand-eye calibration method includes the following steps: S1: Place two rectangular plates of different heights on the machine tool surface; S2: Align the nozzles of the cutting head with the intersection points of the rectangular plates and record the machine coordinates of the 8 corner points; S3: A line laser sensor scans the rectangular plate and the exposed machine tool surface to obtain a point cloud, which is used to represent the upper surface of the rectangular plate and the exposed machine tool surface; S4: Obtain the camera coordinates corresponding to the 8 corner points from the point cloud; S5: Substitute the machine tool coordinates and camera coordinates of the 8 corner points into the equation AX=B to obtain the hand-eye matrix X. A is used to represent the camera coordinates of the 8 corner points, and B is used to represent the machine tool coordinates of the 8 corner points.

[0019] In S2, the method of aligning the nozzles of the cutting head to the intersection of the rectangular plates is manual alignment. Therefore, the manual hand-eye calibration method introduces certain human errors, such as inaccurate alignment, resulting in poor consistency and accuracy of the calibration results and a cumbersome process.

[0020] In view of this, the present invention creatively proposes an automatic hand-eye calibration method that can be implemented based on a program without manual operation. This automatic hand-eye calibration method can reduce human error, thereby further improving the consistency, accuracy and efficiency of calibration results.

[0021] To make the above-mentioned objects, features, and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] Figure 1 This is a flowchart of the automatic hand-eye calibration method provided in the embodiments of the present invention.

[0023] Please refer to Figure 1 This invention provides an automatic hand-eye calibration method, comprising: Step S1: Place at least two disks on the machine tool surface, with each disk arranged along the first direction.

[0024] In this embodiment, the number of disks placed on the machine tool surface is greater than or equal to two, and the height difference between any two disks is greater than 20 mm. The laser line projected by the line laser sensor is projected onto the upper surface of the disk, and the undulations of the upper surface of the disk cause the laser line to "bend and deform," and this deformation directly reflects the height difference of the upper surface of the disk. Based on this, setting the height difference between each disk (i.e., the height difference between any two disks is greater than 20 mm) can improve the difference between the point clouds corresponding to each disk obtained subsequently, reduce the calibration error caused by data redundancy, and improve the accuracy of the subsequently obtained hand-eye matrix.

[0025] In this embodiment, all disks are located within the scanning area of ​​the line laser sensor. Furthermore, the disks are spaced apart along the first direction, meaning there is no overlap between any two disks. Therefore, the line laser sensor can scan the upper surface of all disks to obtain a disk point cloud representing the complete upper surface of all disks. This allows for more accurate acquisition of camera coordinates at least four centers from the point cloud, further improving the accuracy of the subsequently obtained hand-eye matrix.

[0026] Step S2: Based on the method of finding the center of the circular arc, obtain the first machine coordinates of the center of each disk in the machine coordinate system.

[0027] Since each machine tool has only one nozzle, the first machine tool coordinates of the center of each disk are obtained sequentially. That is, the center of each disk is found sequentially by performing arc centering on each disk to obtain the first machine tool coordinates of the center of each disk in the machine tool coordinate system.

[0028] In one specific implementation, step S2, based on the method of finding the center of any disk, obtains the first machine tool coordinates in the machine tool coordinate system, specifically including: Step S21: Move the nozzle of the cutting head above the disc; Specifically, the vertical projection of the nozzle is entirely located within the disk.

[0029] Step S22: Obtain the capacitance value between the nozzle and the disk; Among them, through the capacitance model To obtain the capacitance value between the nozzle and the disk. This represents the capacitance between the nozzle and the disk. is the relative permittivity, S is the area between the nozzle and the disk, k is the electrostatic constant, and d is the vertical distance between the nozzle and the disk.

[0030] Step S23: Move the nozzle in at least three directions respectively, and obtain the third machine coordinates of at least three circumferential points of the disk in the machine coordinate system. During the process of moving the nozzle in any of the at least three directions, when the capacitance value changes abruptly, take the position of the nozzle at this time as a circumferential point of the disk, and obtain the third machine coordinates of the circumferential point in the machine coordinate system. Specifically, the circumference is the boundary line of the disk, and the points on the circumference are points on the boundary line of the disk. Therefore, it can be understood that when the area of ​​the nozzle facing the disk and the perpendicular distance between them remain constant—that is, when the vertical projection of the nozzle lies entirely within the disk—the capacitance between the nozzle and the disk is a constant value. If the nozzle maintains a constant perpendicular distance from the disk and moves outward in any direction, as it passes above the circumference, the area S between the nozzle and the disk decreases, causing the capacitance to decrease. As the nozzle continues to move outward beyond the disk, the capacitance abruptly drops to 0. Therefore, by observing the abrupt change in capacitance to 0, the position of the corresponding point on the circumference in any direction of the disk can be determined.

[0031] Step S24: Based on the third machine tool coordinates of at least three circumferential points of the disk in the machine tool coordinate system, fit the center of the disk and obtain the first machine tool coordinates of the center of the disk in the machine tool coordinate system.

[0032] Step S3: The line laser sensor scans the disk and the machine tool surface not obscured by the disk at least along the second direction of the machine tool coordinate system to obtain a point cloud. The point cloud is used to represent the upper surface of the disk and the machine tool surface not obscured by the disk. The first direction is perpendicular to the second direction, and both the first and second directions are parallel to the machine tool surface.

[0033] To accurately calculate the hand-eye matrix, at least four different camera coordinates of the center of the corresponding disk are needed. Therefore, the specific steps for obtaining the point cloud depend on the number of disks. Two specific embodiments are described below.

[0034] In one specific implementation, the number of disks placed on the machine tool surface is 2 or 3. A line laser sensor scans the disks and the unobstructed portion of the machine tool surface along at least a second direction of the machine tool coordinate system to obtain a point cloud. This process includes: the line laser sensor scanning the disks and the unobstructed portion of the machine tool surface along the second direction to obtain a first point cloud; moving the line laser sensor a first preset distance along the first direction; and scanning the disks and the unobstructed portion of the machine tool surface along a third direction (opposite to the second direction) to obtain a second point cloud. The point cloud includes both the first and second point clouds. Since the number of disks is relatively small, to accurately calculate the hand-eye matrix, the linear sensor can be moved to scan the disks at different positions along the second or third direction in the first direction to obtain multiple different camera coordinates of the center. In this specific implementation, the linear sensor only moves once in the first direction. However, to obtain more different camera coordinates of the center, the linear sensor can move multiple times in the first direction.

[0035] In another specific implementation, the number of disks is greater than or equal to four, and the method for obtaining point clouds by having the line laser sensor scan the disks and the unobstructed machine tool surface along at least a second direction of the machine tool coordinate system includes: the line laser sensor scanning the disks and the unobstructed machine tool surface along the second direction to obtain point clouds. Since there are a large number of disks, a single scan by the line laser sensor is sufficient to acquire enough data. Of course, to obtain more different camera coordinates of the center, the line laser sensor can also move in the first direction to scan the disks at different positions along the second or third direction to obtain more data.

[0036] Step S4: Obtain the camera coordinates of at least four circle centers in the point cloud.

[0037] Among them, obtaining the camera coordinates of at least four circle centers in the point cloud includes: Step S41: Cluster and segment the point cloud to obtain the disk point cloud symbolizing the upper surface of the disk. Step S42: Extract the contour point set of each disk based on the disk point cloud; Step S43: Based on the contour point set of each disk, fit the center of each disk and obtain the camera coordinates of each center.

[0038] Step S5: Based on the camera coordinates of each circle center, obtain the second machine tool coordinates corresponding to the camera coordinates of each circle center. The second machine tool coordinates are used to characterize the position of the nozzle center of the cutting head in the machine tool coordinate system when the laser line emitted by the line laser sensor passes through the center of the disk.

[0039] Step S6: Based on the first machine tool coordinates, camera coordinates, and second machine tool coordinates, obtain the hand-eye matrix. The hand-eye matrix is ​​used to characterize the transformation relationship between the machine tool coordinate system and the camera coordinate system.

[0040] The method for obtaining the hand-eye matrix further includes: based on the coordinates of the first machine tool Camera coordinates Second machine tool coordinates and coordinate transformation model Obtain the hand-eye matrix .

[0041] It should be noted that the above-mentioned automatic hand-eye calibration process is all implemented automatically through the designed program.

[0042] In summary, the automatic hand-eye calibration method proposed in this invention can achieve hand-eye calibration based on a program without manual operation, thereby reducing human error and further improving the consistency, accuracy, and efficiency of calibration results. Furthermore, it can reduce the operation process, thus simplifying the hand-eye calibration method.

[0043] Figure 2 This is a structural block diagram of the hand-eye calibration system provided in an embodiment of the present invention.

[0044] Please refer to Figure 2 This invention also provides a hand-eye calibration system for implementing the above-described automatic hand-eye calibration method. The hand-eye calibration system includes: an execution processing module, a scan fitting module, and a hand-eye calibration module.

[0045] The execution processing module is used to obtain the first machine coordinates of the center of each disk in the machine coordinate system based on the arc centering method; The scanning fitting module is used to control the line laser sensor to move at least along the second direction and to scan each disk through the line laser sensor to obtain the camera coordinates of the center of at least four disks; The hand-eye calibration module is used to obtain the hand-eye matrix based on the first machine tool coordinates, camera coordinates, and second machine tool coordinates. The hand-eye matrix is ​​used to characterize the transformation relationship between the machine tool coordinate system and the camera coordinate system. In addition, the hand-eye calibration module is also used to calibrate the line laser sensor and nozzle based on the hand-eye matrix.

[0046] Figure 3 This is a structural block diagram of the electronic device provided in an embodiment of the present invention.

[0047] Please refer to Figure 3 This invention also provides an electronic device, including a processor and a memory for storing processor-executable instructions. The processor executes the executable instructions to implement the steps in the above-described automatic hand-eye calibration method.

[0048] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An automatic hand-eye calibration method, characterized in that, include: Place at least two disks on the machine tool web, with each disk arranged along the first direction; Based on the method of finding the center of the circular arc, the first machine tool coordinates of the center of each of the disks in the machine tool coordinate system are obtained respectively; A line laser sensor scans the disk and the machine tool surface not obscured by the disk at least along a second direction of the machine tool coordinate system to obtain a point cloud. The point cloud is used to represent the upper surface of the disk and the machine tool surface not obscured by the disk. The first direction is perpendicular to the second direction, and both the first direction and the second direction are parallel to the machine tool surface. Obtain the camera coordinates of at least four of the circle centers in the point cloud; Based on the camera coordinates of each of the centers, the second machine tool coordinates corresponding to the camera coordinates of each of the centers are obtained. The second machine tool coordinates are used to characterize the position of the nozzle center of the cutting head in the machine tool coordinate system when the laser line emitted by the line laser sensor passes through the center of the disk. A hand-eye matrix is ​​obtained based on the first machine tool coordinates, the camera coordinates, and the second machine tool coordinates. The hand-eye matrix is ​​used to characterize the transformation relationship between the machine tool coordinate system and the camera coordinate system.

2. The automatic hand-eye calibration method according to claim 1, characterized in that, The number of disks placed on the machine tool surface is greater than or equal to 2, and the height difference between any two of the disks is greater than 20 mm.

3. The automatic hand-eye calibration method according to claim 1, characterized in that, The method for obtaining the hand-eye matrix further includes: based on the first machine tool coordinates The camera coordinates and the second machine tool coordinates and coordinate transformation model Obtain the hand-eye matrix .

4. The automatic hand-eye calibration method according to claim 1, characterized in that, The entire disk lies within the scanning area of ​​the line laser sensor.

5. The automatic hand-eye calibration method according to claim 4, characterized in that, The camera coordinates for obtaining at least four circle centers in the point cloud include: Clustering and segmenting the point cloud yields a disk point cloud symbolizing the upper surface of the disk; Based on the point cloud of the disks, extract the contour point set of each disk; Based on the contour point set of each disk, the center of each disk is fitted, and the camera coordinates of each center are obtained.

6. The automatic hand-eye calibration method according to claim 1 or 2, characterized in that, The number of disks placed on the machine tool surface is 2 or 3, and the line laser sensor scans the disks and the machine tool surface not obscured by the disks at least along a second direction of the machine tool coordinate system to obtain a point cloud, including: The line laser sensor scans the disk and the machine tool surface not obscured by the disk along the second direction to obtain a first point cloud; Move the line laser sensor a first preset distance along the first direction; A line laser sensor scans the disk and the machine tool surface not obscured by the disk along a third direction to obtain a second point cloud, wherein the third direction is opposite to the second direction; The point cloud is obtained, and the point cloud includes the first point cloud and the second point cloud.

7. The automatic hand-eye calibration method according to claim 1 or 2, characterized in that, The number of disks is greater than or equal to 4, and the line laser sensor scans the disks and the machine tool surface not obscured by the disks at least along the second direction of the machine tool coordinate system to obtain the point cloud.

8. The automatic hand-eye calibration method according to claim 1, characterized in that, Obtaining the first machine tool coordinates of the center of each of the disks includes: Move the nozzle of the cutting head above the disk; Obtain the capacitance value between the nozzle and the disk; The nozzle is moved in at least three directions respectively, and the third machine tool coordinates of at least three circumferential points of the disk in the machine tool coordinate system are obtained. During the process of moving the nozzle in any of the at least three directions, when the capacitance value changes abruptly, the position of the nozzle at this time is taken as a circumferential point of the disk, and the third machine tool coordinates of the circumferential point in the machine tool coordinate system are obtained. Based on the third machine tool coordinates of at least three circumferential points of the disk in the machine tool coordinate system, the center of the disk is fitted, and the first machine tool coordinates of the center of the disk in the machine tool coordinate system are obtained.

9. A hand-eye calibration system for implementing the automatic hand-eye calibration method according to any one of claims 1 to 8, characterized in that, The system includes: The execution processing module is used to obtain the first machine tool coordinates of the center of each disk in the machine tool coordinate system based on the arc centering method; A scanning fitting module is used to control the line laser sensor to move at least along a second direction and to scan each disk through the line laser sensor to obtain the camera coordinates of the centers of at least four of the disks; The hand-eye calibration module is used to obtain a hand-eye matrix based on the first machine tool coordinates, the camera coordinates, and the second machine tool coordinates. The hand-eye matrix is ​​used to characterize the transformation relationship between the machine tool coordinate system and the camera coordinate system. Furthermore, the hand-eye calibration module is also used to calibrate the line laser sensor and the nozzle based on the hand-eye matrix.

10. An electronic device, comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the automatic hand-eye calibration method according to any one of claims 1 to 8 by running the executable instructions.

Images