Hand-eye calibration method, hand-eye calibration system, computer device and storage medium

Abstract

The application discloses a hand-eye calibration method, a hand-eye calibration system, a computer device and a computer readable storage medium. The hand-eye calibration method comprises the following steps: acquiring corner point coordinates of a calibration board under a base coordinate system, and generating a reference three-dimensional point cloud A by using an interpolation algorithm based on the corner point coordinates; controlling an end executing mechanism to drive a visual sensor fixed thereon to move along a preset track, collecting a three-dimensional model of the calibration board under a visual sensor coordinate system, and performing contour extraction processing on the three-dimensional model to obtain a contour three-dimensional point cloud B; taking the reference three-dimensional point cloud A as a target point cloud, taking the contour three-dimensional point cloud B as a source point cloud, performing point cloud registration based on a space mapping relationship of corresponding coordinate axes, solving a hand-eye transformation matrix X, and storing the hand-eye transformation matrix X. The technical scheme can realize high-precision, full-automatic and strong anti-interference coordinate mapping, and is widely applicable to industrial scenes such as laser cutting.

Description

Technical Field

[0001] This application relates to the field of visual calibration technology, and in particular to a hand-eye calibration method, a hand-eye calibration system, a computer device, and a computer-readable storage medium. Background Technology

[0002] In industrial automation, the coordinate matching accuracy between vision sensors and actuators directly determines the quality of operations. Hand-eye calibration, as the core step in establishing the mapping relationship between the vision sensor coordinate system and the equipment's base coordinate system, has a critical impact on production operations in terms of accuracy and efficiency. This type of requirement is widespread in various industrial scenarios such as laser cutting, machining, and automated assembly.

[0003] Existing hand-eye calibration methods have several drawbacks: First, traditional calibration relies on manual operation, with manual control of the actuator to move to the calibration point. This is not only inefficient but also prone to insufficient calibration accuracy due to human error, making it difficult to meet the demands of high-precision operations. Second, existing calibration methods often use single feature point acquisition, failing to fully consider the full coverage of the industrial equipment's workspace, resulting in poor adaptability of the calibration results and the potential for error accumulation at different work locations. Third, the accuracy of point cloud processing and registration algorithms is insufficient, making it difficult to effectively eliminate noise interference in the industrial environment, further reducing calibration reliability. For example, in laser cutting scenarios, existing technologies use manual movement of the crossbeam to poke at the marked points for calibration, with a success rate of only about 70% and a high scrap rate, severely impacting production efficiency and cost control.

[0004] Therefore, developing a hand-eye calibration method and system that is highly automated, accurate, efficient, and adaptable to various industrial scenarios has become an urgent technical problem to be solved in the field of visual calibration technology. Summary of the Invention

[0005] This application provides a hand-eye calibration method, a hand-eye calibration system, a computer device, and a computer-readable storage medium, aiming to solve the problems of low accuracy, low efficiency, and reliance on manual operation in traditional calibration processes.

[0006] To achieve the above objectives, this application proposes a hand-eye calibration method. This hand-eye calibration method includes the following steps: Obtain the corner coordinates of the calibration board in the base coordinate system, and generate a reference 3D point cloud A based on the corner coordinates using an interpolation algorithm; control the end effector to drive the vision sensor fixed on it to move along a preset trajectory, collect the 3D model of the calibration board in the vision sensor coordinate system, and perform contour extraction processing on the 3D model to obtain the contour 3D point cloud B; using the reference 3D point cloud A as the target point cloud and the contour 3D point cloud B as the source point cloud, perform point cloud registration based on the spatial mapping relationship of the corresponding coordinate axes, solve for the hand-eye transformation matrix X, and store it; wherein, the hand-eye transformation matrix X satisfies the following relationship: X B = A.

[0007] In some embodiments, obtaining the corner coordinates of the calibration plate in the base coordinate system specifically includes: placing the rectangular calibration plate on the working surface of the base coordinate system; and detecting and recording the coordinates of at least four corner points of the calibration plate in the base coordinate system using a contact or non-contact positioning method via an end effector.

[0008] In some embodiments, the control end effector drives the vision sensor fixed thereon to move along a preset trajectory to collect the three-dimensional model of the calibration plate in the vision sensor coordinate system. Specifically, the preset trajectory includes translational motion along the X-axis, Y-axis, and Z-axis of the base coordinate system, and the range of the translational motion covers at least the area where the calibration plate is located. The vision sensor collects at least one set of three-dimensional data of the calibration plate at different positions.

[0009] In some embodiments, generating a reference 3D point cloud A based on the corner coordinates using an interpolation algorithm specifically includes: generating a reference 3D point cloud A with uniform density using an equal-interval interpolation algorithm within the spatial range defined by the corner coordinates.

[0010] In some embodiments, the contour extraction process of the three-dimensional model to obtain the contour three-dimensional point cloud B specifically includes: using a point cloud filtering algorithm to remove noise points in the three-dimensional model to obtain filtered point cloud data; and using a plane segmentation algorithm to extract the contour edge of the calibration plate from the filtered point cloud data to obtain the contour three-dimensional point cloud B.

[0011] In some embodiments, the point cloud filtering algorithm is a statistical filtering algorithm; the plane segmentation algorithm is a random sampling consensus algorithm.

[0012] In some embodiments, the point cloud registration based on the spatial mapping relationship of corresponding coordinate axes and the solution of the hand-eye transformation matrix X specifically includes: calculating the principal axis directions of the contour 3D point cloud B and the reference 3D point cloud A; rotating the contour 3D point cloud B to be in the same direction as the reference 3D point cloud A according to the corresponding principal axis directions; aligning the centroid of the rotated contour 3D point cloud B with the centroid of the reference 3D point cloud A, and solving for the hand-eye transformation matrix X.

[0013] This application also provides a hand-eye calibration system, which includes: The calibration board position determination module is used to obtain the corner coordinates of the calibration board in the base coordinate system; The scanning module includes a vision sensor and a motion drive unit; the motion drive unit is used to control the end effector to drive the vision sensor to move along a preset trajectory; the vision sensor is used to acquire a three-dimensional model of the calibration plate in the vision sensor coordinate system; The point cloud processing module is connected to the calibration plate position determination module and the scanning module respectively. It is used to generate a reference 3D point cloud A based on the corner coordinates using an interpolation algorithm, and to perform contour extraction processing on the 3D model to obtain a contour 3D point cloud B. The hand-eye calibration module, connected to the point cloud processing module, is used to perform point cloud registration based on the spatial mapping relationship between the corresponding coordinate axes of the reference 3D point cloud A and the contour 3D point cloud B, solve for the hand-eye transformation matrix X, and store the hand-eye transformation matrix X; the hand-eye transformation matrix X satisfies the following relationship: X B = A.

[0014] In some embodiments, the calibration board position determination module includes: The end effector control unit is used to control the movement of the end effector. The corner detection unit is used to detect and obtain the corner coordinates of the calibration plate through contact or non-contact positioning methods.

[0015] In some embodiments, the point cloud processing module has built-in interpolation algorithm, point cloud filtering algorithm and plane segmentation algorithm, which are used to perform the generation of the reference 3D point cloud A, the denoising processing of the 3D model and the extraction of the contour 3D point cloud B, respectively.

[0016] In some embodiments, the hand-eye calibration module has a built-in axis alignment and registration algorithm for performing axial alignment and centroid alignment between the contour 3D point cloud B and the reference 3D point cloud A.

[0017] This application also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the hand-eye calibration method as described above.

[0018] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the hand-eye calibration method as described above.

[0019] This application proposes a hand-eye calibration method, a hand-eye calibration system, a computer device, and a computer-readable storage medium. The hand-eye calibration method includes the following steps: obtaining the corner coordinates of a calibration board in a base coordinate system, and generating a reference 3D point cloud A based on the corner coordinates using an interpolation algorithm; controlling an end effector to move a vision sensor fixed to it along a preset trajectory, acquiring a 3D model of the calibration board in the vision sensor coordinate system, and performing contour extraction processing on the 3D model to obtain a contour 3D point cloud B; using the reference 3D point cloud A as the target point cloud and the contour 3D point cloud B as the source point cloud, performing point cloud registration based on the spatial mapping relationship of corresponding coordinate axes, solving for the hand-eye transformation matrix X, and storing it.

[0020] This application's technical solution constructs a high-precision automated hand-eye calibration scheme through a three-stage process: "reference point cloud generation—contour point cloud extraction—registration solution." First, a reference 3D point cloud A is generated based on corner coordinates using an interpolation algorithm, expanding discrete corner points into a dense point cloud, overcoming the limitations of traditional methods that rely only on a limited number of feature points. Second, a 3D model is acquired and a contour point cloud B is extracted, combined with a preset trajectory for full spatial coverage, ensuring the adaptability of the calibration results across the entire working area. Finally, point cloud registration is performed based on coordinate axis spatial mapping, employing axis alignment and centroid alignment algorithms to quickly solve the transformation matrix without iteration, avoiding local optima and improving efficiency. Based on this collaborative approach, high-precision, fully automated, and highly interference-resistant coordinate mapping is achieved, which can be widely applied in high-precision industrial scenarios such as laser cutting. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a schematic flowchart of a hand-eye alignment method according to an embodiment of this application; Figure 2 This is a modular schematic diagram of a hand-eye calibration system according to an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application.

[0022] Among them, 100-hand-eye calibration system; 110-calibration board position determination module; 120-scanning module; 130-point cloud processing module; 140-hand-eye calibration module; 10-processor; 20-memory; 30-communication interface. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that, unless otherwise stated or limited, all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0025] It should also be noted that, unless otherwise stated or limited, when an element is referred to as "fixed to" or "set on" another element, it may be directly on the other element or there may be an intervening element present. When an element is referred to as "connected to" another element, it may be directly connected to the other element or there may be an intervening element present.

[0026] Furthermore, unless otherwise stated or limited, the descriptions involving "first," "second," etc., in this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0027] This application proposes a hand-eye calibration method. This method is primarily applied in industrial automation scenarios, particularly in equipment requiring vision guidance, such as laser cutting, welding, and grinding. Its core objective is to solve for the transformation matrix X between the vision sensor coordinate system and the equipment's base coordinate system, thereby achieving a precise mapping from visual coordinates to mechanical coordinates.

[0028] like Figure 1 As shown, the method specifically includes steps S10 to S30, and each step is described in detail below: Step S10: Obtain the corner coordinates of the calibration plate in the base coordinate system, and generate the reference 3D point cloud A based on the corner coordinates using an interpolation algorithm.

[0029] This step utilizes the high-precision motion characteristics of the mechanical body to obtain the absolute coordinates of the calibration plate in physical space.

[0030] In this step, a rectangular flat plate is used as the calibration plate, and the material can be selected according to the actual working conditions. For example, in a laser cutting scenario, the cutting scrap can be used directly as the calibration plate, or a dedicated metal calibration plate can be used. The preferred shape of the calibration plate is a rectangle with unequal length and width. The non-square feature helps the subsequent algorithm to identify the direction and avoids registration ambiguities caused by point cloud symmetry (such as misidentifying the major axis as the minor axis).

[0031] The calibration plate is placed randomly or fixedly on the working surface of the equipment. It is not necessary to strictly ensure that it is parallel to the machine tool coordinate axis, which can greatly reduce the preparation threshold for calibration.

[0032] The end effector (such as a laser cutting head or a robotic arm end effector) moves to the vicinity of the calibration plate under the drive of the control system. There are two methods for obtaining corner coordinates: Contact positioning: The control system controls the end effector (such as a laser cutting head or a dedicated probe) to approach the edge of the calibration plate at low speed along a certain axis (such as the X-axis). When the probe tip touches the edge of the calibration plate, a contact sensor triggers a signal (or a current conduction signal or a force feedback signal), and the system instantly locks the current mechanical coordinates of the end effector.

[0033] To determine the spatial position (position and orientation) of the rectangular calibration plate, the coordinates of at least three or four corner points need to be obtained. In this embodiment, a "three-point edge finding" or "five-point edge finding" method can be used. For example, the lower left, lower right, and upper right corners of the calibration plate are touched sequentially, and the base coordinates of these three points are recorded as (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3). Since the calibration plate is planar, the equation of the plane containing the calibration plate can be fitted using these three points, and the coordinates of the fourth corner point can be calculated.

[0034] Non-contact positioning: In certain scenarios involving precision or fragile workpieces, contact methods may cause scratches; in such cases, non-contact methods can be used.

[0035] The end effector is equipped with a laser displacement sensor, infrared range sensor, or capacitive sensor. The control sensor scans at a certain height above the calibration plate. The sensor provides real-time feedback on distance changes; a step change in distance data is identified as an edge position. Through multiple scans in the X and Y directions, a straight line is fitted to the edge; the intersection of two edge lines is the corner coordinate. For example, by utilizing air pressure changes, when the pneumatic nozzle at the end effector approaches the workpiece surface, the back pressure changes; the distance is deduced from the pressure curve, achieving "soft contact" positioning.

[0036] After obtaining the coordinates of the four corner points of the calibration board, the limited number of corner points makes it impossible to directly use them for high-precision point cloud registration. Therefore, in this embodiment, an interpolation algorithm is used to generate a reference 3D point cloud A within the spatial range defined by the corner point coordinates.

[0037] Specifically, an equal-interval interpolation algorithm can be used. Assume the coordinates of the four corner points of the rectangular calibration plate are P1(x1, y1, z1), P2(x2, y2, z2), P3(x3, y3, z3), and P4(x4, y4, z4), respectively, and set the interpolation step size. d (e.g., 1mm or 0.5mm) is used for linear interpolation in the X and Y directions to generate a series of dense spatial points. Since the calibration plate is usually planar, the Z-axis coordinate can be determined based on the mean of the Z-coordinates of the corner points or a plane fitting equation. The resulting reference 3D point cloud A has uniform density, moderate data volume, and can accurately reflect the spatial shape (planar rectangle) of the calibration plate in the base coordinate system.

[0038] Step S20: Control the end effector to drive the vision sensor fixed on it to move along a preset trajectory, collect the three-dimensional model of the calibration plate in the coordinate system of the vision sensor, and perform contour extraction processing on the three-dimensional model to obtain the contour three-dimensional point cloud B.

[0039] This step utilizes a visual sensor to acquire observation data from the calibration board and then uses an algorithm to process and extract effective features.

[0040] The preferred visual sensor is a stereo camera, such as a 3D line scan camera, a binocular camera, or a structured light camera. These cameras can directly acquire depth information of the scene and output 3D point cloud data. The sensor's accuracy and model are determined based on actual on-site requirements (such as field of view, depth of field, and accuracy requirements). The sensor is rigidly fixed to the end effector and moves with it, allowing the camera to flexibly adjust its viewing angle to cover the workspace.

[0041] To ensure the integrity of the scan data, the preset trajectory must cover the area where the calibration board is located. The trajectory planning module automatically plans a scanning path covering the entire area of ​​the calibration board based on the corner coordinates obtained in step S101. The path can be straight, bow-shaped, spiral, or S-shaped. The preset trajectory includes not only translation in the XY plane but also movement along the Z-axis. Notably, Z-axis movement can be used to adjust the camera's focal length. The Z-axis height is determined before scanning the XY plane to ensure the calibration board remains within the camera's optimal depth of field during the plane scan. Preferably, the scanning range can be extended to 0-100% of the working area, meaning that not only the calibration board is scanned but also the surrounding environment. This verifies the camera's imaging quality throughout its entire travel range and makes the calibration results more globally representative, reducing the risk of local optima. More preferably, the scanning range is limited to the area where the calibration board is located; this small area allows the vision sensor to complete the scan at the fastest speed, improving scanning efficiency.

[0042] The camera continuously captures images during its movement, generating a series of local 3D point cloud slices. The system uses the real-time pose information of the end effector (acquired via a grating ruler or encoder) to stitch and merge all the local slices, forming a complete 3D model of the calibration board in the visual sensor coordinate system.

[0043] Due to interference from dust and uneven lighting in industrial environments, raw point cloud data often contains a large amount of noise. This embodiment processes the acquired 3D model as follows: Point cloud denoising: A point cloud filtering algorithm is employed, preferably a statistical filtering algorithm. The statistical filtering algorithm calculates the average distance from each point to its neighbors and assumes that the distance distribution follows a Gaussian distribution, eliminating outliers whose mean distance exceeds several times the standard deviation. This effectively reduces noise interference caused by factors such as dust and reflections in industrial environments.

[0044] Data simplification and plane extraction: After denoising, a plane segmentation algorithm is used to extract the contour edges of the calibration board. Preferably, the RANSAC (Random Sample Consensus) algorithm is used. The RANSAC algorithm fits a plane model by repeatedly selecting a subset of points randomly, finds the interior points that conform to the plane model, and removes the exterior points that do not conform (such as background clutter and scanning error points), thereby extracting the main plane of the calibration board and its contour edges.

[0045] The resulting 3D point cloud B accurately reflects the contour shape of the calibration plate in the visual sensor coordinate system.

[0046] Step S30: Using the reference 3D point cloud A as the target point cloud and the contour 3D point cloud B as the source point cloud, perform point cloud registration based on the spatial mapping relationship of the corresponding coordinate axes, solve for the hand-eye transformation matrix X, and store it; wherein, the hand-eye transformation matrix X satisfies the following relationship: X B = A.

[0047] This step is the core computational component of hand-eye alignment. Traditional ICP (Iterative Closest Point) algorithms are sensitive to initial values ​​and prone to getting trapped in local optima. This embodiment employs a registration algorithm based on the alignment of the point cloud's major and minor axes and centroids. The specific steps are as follows: Calculate the principal axis directions: Calculate the principal axis directions (three axes) of the baseline 3D point cloud A and the contour 3D point cloud B respectively. This can be achieved using the Principal Component Analysis (PCA) algorithm. Calculate the covariance matrix for all points in the point cloud, solve for the eigenvectors of this matrix, and the directions corresponding to the eigenvectors are the principal axis directions of the point cloud. Since the calibration plate is rectangular, its major and minor axes have obvious geometric characteristics, and the calculated principal axis directions are unique (uniquely determined by the centroid position, ignoring 180-degree symmetry).

[0048] Axial alignment (coarse registration): Calculate the rotation matrix Ralign based on the principal axis directions of point clouds A and B. Rotate point cloud B so that its principal axis direction is aligned with that of point cloud A. This step quickly rotates point cloud B to the correct pose, overcoming the dependence of traditional algorithms on initial values.

[0049] Centroid Alignment (Fine Registration): Calculate the centroid C of the rotated point cloud B. B And the centroid C of point cloud A A Calculate the translation vector Talign=C A C B .

[0050] Solving for the transformation matrix X: Combining the rotation and translation described above, we construct the final transformation matrix X. X contains the rotation matrix R and the translation vector T, satisfying X PB=PA, that is, X B=A. Where, P B Let P be a point in point cloud B. A This refers to the corresponding point in point cloud A.

[0051] Compared to the traditional ICP algorithm, this registration algorithm has a faster convergence speed, higher registration accuracy, and is not directly affected by the accuracy of the calibration board corner data acquisition or the accuracy of the scanning calibration board, thus exhibiting strong robustness.

[0052] The obtained transformation matrix X is stored in the device's memory, such as the hard drive of an industrial control computer. The preferred storage format is XML or other common data formats for easy retrieval and parsing in subsequent cutting operations. In subsequent operations, the device can directly access this matrix to quickly convert the workpiece coordinates acquired by the vision sensor into coordinates in the base coordinate system, achieving high-precision operation.

[0053] Through the coordinated operation of steps S10 to S30 described above, this embodiment achieves fully automated, high-precision, and high-efficiency hand-eye calibration. The entire process requires no manual intervention, and the error is controlled within the accuracy of the vision sensor. It can be widely applied to high-precision industrial scenarios such as laser cutting and machining.

[0054] This application also provides a hand-eye calibration system 100, such as Figure 2 As shown, the system has a clear architecture, a high degree of modularity, and is easy to integrate into various industrial equipment. The system mainly includes a calibration board position determination module 110, a scanning module 120, a point cloud processing module 130, and a hand-eye calibration module 140.

[0055] The calibration board position determination module 110 is the system's "tactile" unit, responsible for sensing the reference coordinates of the physical world. This module further includes an end effector unit and a corner detection unit.

[0056] The end effector control unit is the core of motion control. It receives commands from the host computer and controls the rotation or linear motion of the motor via a servo driver. It typically includes a motion control card, servo driver, motor, and reducer. It has trajectory interpolation capabilities, enabling complex movements such as straight lines and circular arcs. During edge finding, the end effector control unit controls the cutting head or probe to approach the edge of the calibration plate at an extremely low speed (e.g., 10 mm / s), ensuring minimal kinetic energy at the moment of contact to avoid damage to the equipment or workpiece. Simultaneously, it reads feedback signals from the grating ruler or encoder in real time to obtain the real-time position of the end effector in the base coordinate system. This unit integrates an edge detection algorithm. When a sensor signal is triggered, the current coordinates are immediately latched with microsecond-level accuracy, ensuring zero-delay position recording.

[0057] The corner detection unit is responsible for sensing "contact" or "proximity" events and converting physical signals into electrical signals. Contact-based implementations can be high-precision mechanical switches, strain gauge force sensors, or simple conductive circuits. When the probe contacts the metal calibration plate, the circuit is activated, generating a high-level signal. This signal is sent to the interrupt port of the end effector control unit, triggering coordinate latching. Non-contact implementations integrate laser displacement sensors or capacitive sensors. The corner detection unit acquires sensor data in real time and processes the signals using a DSP or FPGA. For example, it analyzes abrupt changes in laser reflection intensity or time-of-flight to determine if an edge has been reached. This unit has analog-to-digital (A / D) conversion capabilities, filtering out high-frequency noise and extracting effective edge features. Detected edge points undergo geometric calculations (such as calculating the intersection of two edges), ultimately outputting the coordinate data (x, y, z) of the calibration plate corner points in the base coordinate system.

[0058] The scanning module 120 is the system's "vision" unit, responsible for acquiring 3D information about the scene. This scanning module further includes a vision sensor and a motion drive unit.

[0059] As the core sensing element, a 3D line laser profilometer is preferred for use as the visual sensor. A laser projects a linear laser beam onto the object's surface, and a camera (CMOS / CCD) captures the laser line from a specific angle. Due to variations in the object's surface height, the laser line's position on the camera's imaging plane shifts. Using triangulation principles, the three-dimensional coordinates of the emitted point in the camera's coordinate system can be accurately calculated. The required scanning frequency and Z-axis resolution of the visual sensor can be set based on actual needs, aiming to acquire the point cloud profile of the calibration board and achieve on-site accuracy. Simultaneously, the visual sensor features a large depth-of-field optical lens to adapt to variations in the height of materials in industrial environments.

[0060] The motion drive unit works in conjunction with the vision sensor. While moving the sensor, the motion drive unit must send a synchronous trigger signal to the vision sensor. The camera is triggered to take a picture every fixed distance (e.g., 1 mm). This "equal-spacing triggering" mode ensures uniform point cloud density and avoids uneven data density caused by speed fluctuations.

[0061] The point cloud processing module 130 is the data cleaning center in the "brain" of the system, responsible for processing massive amounts of point cloud data. This module is built on a high-performance CPU or GPU and is capable of parallel processing of large-scale point clouds. This module further includes interpolation algorithm units and filtering and segmentation units.

[0062] Interpolation algorithm unit: Receives corner coordinates, calls the built-in meshing algorithm, and generates a reference point cloud A.

[0063] The filtering and segmentation unit performs neighborhood statistical analysis on the original point cloud B obtained from the scan using a statistical filter. Outliers with density or distance anomalies are marked and removed. Algorithm parameters (such as the number of neighboring points K and the standard deviation factor) are automatically adjusted according to the ambient noise level. The RANSAC segmenter searches for the best-fit plane in the point cloud. Since the calibration board is planar, RANSAC can separate the calibration board point cloud from the worktable and background clutter. The edge extractor projects the segmented planar point cloud, calculates the convex hull, extracts contour points, and obtains the final contour 3D point cloud B.

[0064] The hand-eye calibration module 140 serves as the system's computational decision center, solving for the final transformation matrix. This module incorporates an axis alignment and registration algorithm, and its specific functions include: Registration operation: Based on the spatial mapping relationship between the corresponding axes of point cloud A and point cloud B, perform principal axis calculation, rotation alignment and centroid alignment operations to obtain a high-precision transformation matrix X.

[0065] Data storage: Store the transformation matrix X in the specified memory for subsequent cutting operations.

[0066] To further illustrate the practical effects of this application, the following uses a certain model of high-power fiber laser cutting machine as an example to describe the complete calibration application process.

[0067] 1. Equipment parameters Machine tool type: Gantry fiber laser cutting machine.

[0068] Working area: 3000mm×1500mm.

[0069] Cutting head: Equipped with an autofocus cutting head featuring a capacitive non-contact sensor.

[0070] Vision sensor: A 3D line laser sensor mounted on the side of the cutting head, with a scanning width of 1000mm and a Z-axis resolution of 1mm.

[0071] 2. Calibration process execution Preparation: The operator randomly places a rectangular cold-rolled steel plate (calibration plate) measuring 500mm×300mm in the center of the workbench.

[0072] Step 1 (Location): The system initiates the automatic calibration program. The gantry moves, and the cutting head descends to a height of 5mm above the material.

[0073] Five-point edge detection is performed using a capacitive sensor on the cutting head. The cutting head moves along the positive X direction to detect the left edge of the board; it moves along the positive Y direction to detect the bottom edge of the board; and so on, detecting five points in sequence.

[0074] The system automatically calculates the coordinates of the four corner points of the plate in the machine tool base coordinate system, such as (1500.10, 500.08, 0.00), (1500.10, 500.08, 0.00), etc.

[0075] Based on this, the system generates a baseline point cloud A containing approximately 150,000 points with a point spacing of 1 mm.

[0076] Step 2 (Scanning): The program plans a linear scanning trajectory. The gantry crane drives the cutting head and line laser sensor across the material area.

[0077] The scanning speed is set to 100 mm / s. The camera is triggered at a frequency of 100 Hz to acquire data.

[0078] The raw scan data contains approximately 500,000 points, covering the calibration plate and some surrounding grid areas.

[0079] Step 3 (Processing and Registration): The point cloud processing module first performs statistical filtering on the raw data, removing flypoints (approximately 5%) caused by reflections from the metal surface.

[0080] Next, the RANSAC algorithm is called to segment the calibration plate plane from the background mesh and extract the contour point cloud B.

[0081] The hand-eye calibration module reads point cloud A and point cloud B.

[0082] Calculate the principal axes of point cloud A: X direction (major axis), Y direction (minor axis), and Z direction (normal).

[0083] Calculate the principal axis of point cloud B: Due to the deviation between the camera coordinate system and the machine tool coordinate system, the principal axis of B has an angle with the principal axis of A.

[0084] The algorithm calculates the rotation matrix to rotate B until it is parallel to A. Then, it translates the B and A so that their centroids coincide. Finally, it calculates the hand-eye transformation matrix X.

[0085] Matrix X shows that the camera coordinate system is translated 1200.5mm along the X-axis, 800.3mm along the Y-axis, and rotated 0.5° along the Z-axis relative to the machine tool base coordinate system.

[0086] The matrix is ​​saved as an XML file.

[0087] 3. Effect Verification After calibration, actual verification is performed.

[0088] Test method: Draw a complex irregular outline on the board, and then capture it visually and convert it into a cutting path.

[0089] Results: The laser cutting head moved precisely along the path generated by vision. After cutting, the deviation between the actual cut and the designed contour was measured. The results showed that the deviation across the entire area was less than 1mm, far superior to the deviation level of over 2mm achieved by traditional manual calibration.

[0090] Time taken: The entire process took 20 seconds. Compared to traditional manual teaching and calibration (which requires manual alignment, data recording, and manual calculation, taking about 2-5 minutes), the efficiency is improved by more than 5 times.

[0091] In summary, the hand-eye calibration method provided by this application has the following beneficial effects: (1) High calibration accuracy: The rectangular calibration plate is combined with the axis alignment algorithm, and the registration error is controlled within the accuracy of the vision sensor and is not affected by the accuracy of corner detection.

[0092] (2) High degree of automation: The entire calibration process does not require manual intervention and automatically completes corner detection, data acquisition, point cloud processing, registration solution and storage.

[0093] (3) Strong adaptability: The preset trajectory covers the entire workspace, and the anti-interference point cloud processing algorithm can effectively cope with interference such as dust and light in the industrial environment.

[0094] (4) High practicality: The transformation matrix X obtained by calibration can be directly used for coordinate transformation in subsequent operations, and one calibration can be used for a long time.

[0095] This application also provides a computer device, Figure 3 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 3 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processor can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces).

[0096] In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory units, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 3 Take a processor 10 as an example.

[0097] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0098] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0099] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0100] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0101] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0102] This application also provides a computer-readable storage medium in which the methods described in the embodiments of the present invention can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded via a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the embodiments described above.

[0103] The above are only some or preferred embodiments of this application. Neither the text nor the drawings should limit the scope of protection of this application. All equivalent structural transformations made using the content of this application's specification and drawings under the overall concept of this application, or direct / indirect applications in other related technical fields, are included within the scope of protection of this application.

Claims

1. A hand-eye calibration method, characterized in that, Includes the following steps: Obtain the corner coordinates of the calibration board in the base coordinate system, and generate a reference 3D point cloud A based on the corner coordinates using an interpolation algorithm; control the end effector to drive the vision sensor fixed on it to move along a preset trajectory, collect the 3D model of the calibration board in the vision sensor coordinate system, and perform contour extraction processing on the 3D model to obtain the contour 3D point cloud B; using the reference 3D point cloud A as the target point cloud and the contour 3D point cloud B as the source point cloud, perform point cloud registration based on the spatial mapping relationship of the corresponding coordinate axes, solve for the hand-eye transformation matrix X, and store it; wherein, the hand-eye transformation matrix X satisfies the following relationship: X B = A.

2. The hand-eye calibration method according to claim 1, characterized in that, The acquisition of the corner coordinates of the calibration plate in the base coordinate system specifically includes: placing the rectangular calibration plate on the working surface of the base coordinate system; and using an end effector to detect and record the coordinates of at least four corner points of the calibration plate in the base coordinate system through a contact or non-contact positioning method.

3. The hand-eye calibration method according to claim 1, characterized in that, The control end actuator drives the vision sensor fixed thereon to move along a preset trajectory to collect the three-dimensional model of the calibration plate in the vision sensor coordinate system. Specifically, the preset trajectory includes translational motion along the X-axis, Y-axis, and Z-axis of the base coordinate system, and the range of the translational motion covers at least the area where the calibration plate is located. The vision sensor collects at least one set of three-dimensional data of the calibration plate at different positions.

4. The hand-eye calibration method according to claim 1, characterized in that, The step of generating a reference 3D point cloud A based on the corner coordinates using an interpolation algorithm specifically includes: generating a reference 3D point cloud A with uniform density using an equal-interval interpolation algorithm within the spatial range defined by the corner coordinates.

5. The hand-eye calibration method according to claim 1, characterized in that, The process of extracting the contour of the three-dimensional model to obtain the contour three-dimensional point cloud B specifically includes: using a point cloud filtering algorithm to remove noise points in the three-dimensional model to obtain filtered point cloud data; and using a plane segmentation algorithm to extract the contour edge of the calibration plate from the filtered point cloud data to obtain the contour three-dimensional point cloud B.

6. The hand-eye calibration method according to claim 5, characterized in that, The point cloud filtering algorithm is a statistical filtering algorithm; the plane segmentation algorithm is a random sampling consensus algorithm.

7. The hand-eye calibration method according to claim 1, characterized in that, The point cloud registration based on the spatial mapping relationship of corresponding coordinate axes and the solution of the hand-eye transformation matrix X specifically include: calculating the principal axis directions of the contour 3D point cloud B and the reference 3D point cloud A; rotating the contour 3D point cloud B to be in the same direction as the reference 3D point cloud A according to the corresponding principal axis directions; aligning the centroid of the rotated contour 3D point cloud B with the centroid of the reference 3D point cloud A, and solving for the hand-eye transformation matrix X.

8. A hand-eye calibration system, characterized in that, include: The calibration board position determination module is used to obtain the corner coordinates of the calibration board in the base coordinate system; The scanning module includes a vision sensor and a motion drive unit; The motion drive unit is used to control the end effector to drive the vision sensor to move along a preset trajectory; the vision sensor is used to acquire the three-dimensional model of the calibration plate in the vision sensor coordinate system; The point cloud processing module is connected to the calibration plate position determination module and the scanning module respectively. It is used to generate a reference 3D point cloud A based on the corner coordinates using an interpolation algorithm, and to perform contour extraction processing on the 3D model to obtain a contour 3D point cloud B. The hand-eye calibration module, connected to the point cloud processing module, is used to perform point cloud registration based on the spatial mapping relationship between the corresponding coordinate axes of the reference 3D point cloud A and the contour 3D point cloud B, solve for the hand-eye transformation matrix X, and store the hand-eye transformation matrix X; the hand-eye transformation matrix X satisfies the following relationship: X B = A.

9. The hand-eye calibration system according to claim 8, characterized in that, The calibration board position determination module includes: The end effector control unit is used to control the movement of the end effector. The corner detection unit is used to detect and obtain the corner coordinates of the calibration plate through contact or non-contact positioning methods.

10. The hand-eye calibration system according to claim 8, characterized in that, The point cloud processing module has built-in interpolation algorithm, point cloud filtering algorithm and plane segmentation algorithm, which are used to generate the reference 3D point cloud A, denoise the 3D model and extract the contour 3D point cloud B, respectively.

11. The hand-eye calibration system according to claim 8, characterized in that, The hand-eye calibration module has a built-in axis alignment and registration algorithm, which is used to perform axial alignment and centroid alignment between the contour 3D point cloud B and the reference 3D point cloud A.

12. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the hand-eye calibration method as described in any one of claims 1 to 7.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the hand-eye calibration method as described in any one of claims 1 to 7.

Images