Simple method for robot hand-eye calibration and structured calibration device

Abstract

The invention discloses a simple method for robot hand-eye calibration and a structured calibration device, and aims to solve the problems of complex operation, large calculation amount, low efficiency and poor repeated precision of a calibration method in the prior art. The structured calibration device comprises a calibration plate, a connecting bracket, a disc base and a positioning device for accurate installation, the core of the calibration device is that the spatial transformation relation between the calibration plate and the disc base is a predetermined fixed constant. The method comprises the steps that the device is installed at the tail end of the robot, an image of the calibration plate is collected through the camera, the pose of the camera and the calibration plate is calculated through a pose estimation algorithm, and finally the final hand-eye transformation relation is obtained through calculation in combination with the fixed transformation relation predetermined by the device. Through structure solidification and algorithm simplification, complex motion of the robot is not needed, and the method has the advantages of simple operation, light calculation, high repetition precision, high calibration efficiency and the like.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a robot hand-eye calibration method based on camera vision and a matching structured calibration device. Background Technology

[0002] Hand-eye calibration is a key technology in the field of robot vision. Its purpose is to accurately determine the spatial pose transformation relationship between the camera (eye) mounted on the robot's end effector and the end effector itself. This calibration result is crucial for realizing advanced applications of robots such as vision-guided grasping, precision assembly, and autonomous path planning.

[0003] Currently, mainstream hand-eye calibration methods are typically based on the mathematical model AX=XB, where A represents the pose transformation between two movements of the robot's end effector, B represents the corresponding pose transformation of the camera between these two movements, and X is the hand-eye transformation matrix to be solved. To solve this equation, traditional methods require the operator to control the robot to perform multiple complex spatial movements, acquiring multiple sets of image data from different poses. While these methods are theoretically mature, they suffer from numerous problems in practical applications.

[0004] First, it is computationally complex and resource-intensive. Solving the AX=XB equation typically involves complex matrix operations and nonlinear optimization algorithms such as singular value decomposition (SVD) and quaternion optimization, which places high demands on the performance of the computing platform and makes it difficult to run efficiently in resource-constrained embedded systems or industrial environments requiring rapid response.

[0005] Secondly, the operation process is cumbersome and inefficient. The calibration process requires repeated adjustments to the robot's posture and camera angle, which heavily relies on the operator's professional skills and experience. It is time-consuming and prone to errors due to improper human operation, affecting the final calibration accuracy.

[0006] Secondly, repeatability is poor. Many existing methods rely on temporarily installed calibration boards or randomly selected robot postures, resulting in a lack of mechanical consistency in the physical relationship between the calibration board and the robot end effector. This leads to significant differences in the results of each recalibration, failing to meet the requirements for high stability and high consistency in industrial production.

[0007] Finally, there is a lack of standardized industrial application solutions. Most calibration solutions are primarily geared towards research or development scenarios and are not suitable for large-scale use on automated production lines that require rapid deployment and modular integration. In particular, once robot products are delivered to end customers, if customers need to replace cameras or grippers, there is often a lack of simple, standardized tools for them to perform recalibration independently. This significantly impacts product maintainability and user experience, and increases the cost and difficulty of after-sales service. Summary of the Invention

[0008] This invention aims to solve the technical problems of existing robot hand-eye calibration technology, such as complex operation procedures, large amount of calculation, low repeatability accuracy, and lack of standardized and efficient calibration tools available to end users.

[0009] To address the aforementioned problems, this invention provides a structured calibration device. The device includes: a calibration plate with visual markers; a disc base; a positioning device that mates with the disc base and is used for mounting to a robot end effector; and a connecting bracket that rigidly connects the calibration plate and the disc base. Crucially, through precise design and manufacturing, the spatial transformation relationship between the disc base and the calibration plate is a predetermined constant, and the positioning device ensures a high degree of consistency in pose each time the device is mounted to the robot end effector.

[0010] Another aspect of the present invention provides a simplified hand-eye calibration method using the above-described device. The method includes: mounting the structured calibration device onto a robot end effector; acquiring images of visual markings on a calibration board using a camera; calculating a first transformation relationship between the camera and the calibration board using a pose estimation algorithm; and finally, calculating the final hand-eye transformation relationship by combining the first transformation relationship with a fixed spatial transformation relationship predetermined by the device.

[0011] Compared with the prior art, the present invention has the following significant advantages:

[0012] 1. Easy to operate and low learning cost: The calibration process does not require complex robot posture planning. Users only need to install the device and run the program to complete the calibration in a few minutes, which greatly reduces the threshold for use.

[0013] 2. Lightweight computation and high operating efficiency: It adopts a lightweight single-step pose estimation algorithm, avoiding the complex solution of the AX=XB equation, and can run quickly on low computing power platforms to meet industrial real-time requirements.

[0014] 3. Standardized structure and high repeatability: The precision-machined device ensures the fixity of key transformation relationships and the consistency of installation, fundamentally ensuring high repeatability of calibration results.

[0015] 4. Highly versatile and easy to deploy in industrial applications: The device is compatible with a variety of robot platforms, making it suitable for mass production and rapid deployment in industrial settings. It can also be shipped as a standard accessory with the robot, enhancing product value and after-sales service efficiency. Attached Figure Description

[0016] To more clearly illustrate the embodiments of the present invention, a description will be given below in conjunction with the accompanying drawings, wherein:

[0017] Figure 1 This is a schematic diagram of the overall assembly structure of the structured calibration device for robot hand-eye calibration according to the present invention;

[0018] Figure 2 This is the present invention. Figure 1 Top view of the device shown;

[0019] Figure 3 This is the present invention. Figure 1 A front view of the device shown;

[0020] Figure 4 This is the present invention. Figure 1 Side view of the device shown;

[0021] Figure 5 This is a schematic diagram of the structure of the disc base in the structured calibration device for robot hand-eye calibration of the present invention;

[0022] Figure 6 This is the present invention. Figure 5 Top view of the structure shown;

[0023] Figure 7 This is the present invention. Figure 5 The front view of the structure shown;

[0024] Figure 8 This is the present invention. Figure 5 Side view of the structure shown;

[0025] Figure 9 A schematic diagram of the positioning device in the structured calibration device for robot hand-eye calibration of the present invention.

[0026] Figure 10 This is the present invention. Figure 9 Top view of the structure shown;

[0027] Figure 11 This is the present invention. Figure 9 The front view of the structure shown;

[0028] Figure 12 This is the present invention. Figure 9 Side view of the structure shown;

[0029] Figure 13This is a flowchart of a simplified method for robot hand-eye calibration according to the present invention.

[0030] Explanation of reference numerals in the attached drawings: 1. Calibration plate; 2. Connecting bracket; 3. Disc base; 4. Positioning device Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Example 1: Structured Calibration Device

[0033] Reference Figures 1 to 12 This embodiment provides a structured calibration device for robot hand-eye calibration. The device mainly includes a calibration plate 1, a connecting bracket 2, a disc base 3, and a positioning device 4.

[0034] The calibration plate 1 is made of a metal sheet with good flatness to ensure its structural stability and flatness. Visual markers are provided on the plate for stable recognition by computer vision algorithms. In this preferred embodiment, the visual markers are Aruco codes. Aruco codes are well-suited for pose estimation due to their built-in dictionary, ease of detection, and ID recognition. Of course, in other embodiments, AprilTag, ChArUco checkerboard, high-precision QR codes, or any other pattern with known three-dimensional coordinate feature points can also be used. The pattern of the visual markers can be fabricated on the calibration plate 1 by laser engraving or high-precision lamination to ensure dimensional accuracy.

[0035] The connecting bracket 2 is a high-rigidity connector used to firmly connect the calibration plate 1 and the disc base 3 into a whole. In this embodiment, it is machined from a single piece of aluminum alloy using computer numerical control (CNC) and has locating pins and threaded holes machined in it. This allows for precise alignment between the locating pins and the corresponding positions of the calibration plate 1 and the disc base 3, and then it is tightened with screws to ensure the overall structural strength and dimensional accuracy. In other embodiments, other materials with high rigidity and low coefficient of thermal expansion, such as other metal alloys or carbon fiber composite materials, can also be used.

[0036] The circular base 3 is the mounting base for connecting the bracket 2. (Refer to...) Figures 5 to 8In this embodiment, a planar groove is provided on its surface, which matches the outer contour of the positioning device 4, for circumferential coarse positioning and convenient installation during assembly. Six hexagonally distributed screw holes are also provided on it for rigid connection with the positioning device 4 via screws. The lower middle part of the disc base 3 has four horizontally arranged holes, including two screw holes and two pin holes, for connection with corresponding positions on the connecting bracket 2. A hole is provided at the center of the disc base 3, aligned with the center hole of the positioning device 4.

[0037] Positioning device 4 is crucial for ensuring the high repeatability and accuracy of this device. (Refer to...) Figures 9 to 12 In this embodiment, the outer contour of the positioning device 4 is adapted to the planar slot on the disc base 3 for assembly alignment. At least one wedge-shaped positioning block is provided on the robot-facing side, which provides gripping guidance and coarse positioning for the gripper when the entire calibration device is installed on the robot gripper. The device has two positioning pins for precise alignment with the robot end effector, which is then tightened with screws. This pin-hole fit greatly eliminates installation posture errors caused by gaps between screws and through holes, ensuring that the device's posture relative to the robot end effector is restored to almost exactly the same position after each disassembly and reinstallation. A central hole is provided in the center of the device, aligned with the central hole of the disc base 3, ultimately for connection to the center of the robot end effector flange.

[0038] One of the core aspects of this embodiment is that the internal geometric relationships of this rigid whole, composed of calibration plate 1, connecting bracket 2, and disc base 3, are fixed and unchanging. Specifically, the spatial transformation relationship between the coordinate system defined by the mounting plane of positioning device 4 and the center of disc base 3 (representing the "hand" reference system) and the three-dimensional world coordinate system of the visual mark on calibration plate 1 (representing the "calibration target" reference system) is a predetermined fixed constant. This constant can be directly derived from the computer-aided design (CAD) model of the device. To ensure a high degree of consistency between the CAD model and the actual physical device, the manufacturing tolerances of key components of the device (such as connecting bracket 2, disc base 3, and positioning device 4) are strictly controlled within IT5 level.

[0039] Example 2: A Simple Hand-Eye Marking Method

[0040] This embodiment provides a simple hand-eye calibration method using the device described in Embodiment 1, the process of which is as follows: Figure 13 As shown.

[0041] Step 1: Install the calibration tool. The operator installs the structured calibration device described in Example 1 onto the robot end effector to be calibrated. During installation, ensure that the positioning device 4 (such as a positioning pin) fully mates with the corresponding structure (such as a positioning hole) on the robot end flange, and then tighten the connecting screws.

[0042] Step Two: Connect and Acquire Images. Connect the camera used for calibration (e.g., mounted near the robot base or on an external support) to the computer or robot controller. Adjust the robot's posture so that the camera can clearly and completely capture the calibration plate 1 on the calibration device. Then, start the camera acquisition program. To improve the stability and accuracy of the calibration results, it is preferable to acquire a continuous video stream rather than a single still image. For example, you can set it to continuously acquire 30 to 60 seconds of video data at a resolution of 1920x1080 and a frame rate of 30fps.

[0043] Step 3: Run the calibration program. The calibration program executes the following core algorithms in the background:

[0044] a) Pose Calculation: The program processes the acquired video stream frame by frame. For each frame, it first detects the Aruco code in the image and identifies the two-dimensional pixel coordinates of its corner points. Since the physical dimensions of the Aruco code are known in advance, the three-dimensional coordinates of its corner points are also known. Based on the correspondence between these three-dimensional points and two-dimensional image points, the program calls a lightweight pose estimation algorithm to calculate the pose transformation of the camera coordinate system relative to the calibration board coordinate system in the current frame, i.e., the first transformation relationship (T_cam_board). In this embodiment, this algorithm is preferably implemented by calling the solvePnP function in the open-source computer vision library OpenCV.

[0045] b) Multi-frame averaging (preferred step): To eliminate noise in a single frame or the impact of jitter, the program averages the pose results calculated from multiple consecutive frames. For the translation component (a three-dimensional vector), an arithmetic mean can be directly used. For the rotation component, since rotations cannot be simply linearly added, a method that is fast and accurate enough in engineering practice is to first represent the rotation as a unit quaternion, then perform an arithmetic mean on the four components of the quaternion, and finally normalize the resulting average vector back to a unit quaternion, using this as the averaged rotation result.

[0046] Step 4: Output and save the results. The program performs the final calculation. Let the stable pose obtained after averaging over multiple frames be T_cam_board_avg. The program reads the fixed spatial transformation relationship T_gripper_board (representing the transformation from the robot end effector reference frame defined by the positioning device 4 to the calibration board 1) from the configuration file, which is pre-determined by the calibration device and exported from the CAD model. Finally, the hand-eye transformation relationship T_cam_gripper is calculated through the following matrix operation: T_cam_gripper = T_cam_board_avg·(T_gripper_board)^(-1).

[0047] Where (T_gripper_board)^(-1) represents the inverse matrix of T_gripper_board. The calculated T_cam_gripper matrix is ​​the final calibration result, which the program saves to the robot's configuration file for use in subsequent vision-guided tasks. The entire process is highly automated and requires no manual intervention in the calculations.

[0048] In summary, this invention creatively simplifies the complex AX=XB problem in traditional hand-eye calibration, which requires multiple robot movements to solve, into a direct pose solution problem under a single static observation, through a precision-manufactured calibration device with fixed internal geometric relationships. This greatly improves the efficiency, simplicity, and repeatability of calibration, providing strong technical support for the industrial application and after-sales maintenance of robot vision systems.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A structured calibration device for robot hand-eye calibration, characterized in that, The device comprises: a calibration board provided with visual markers recognizable by a camera; a disc base for supporting the connecting bracket; a positioning device configured to be detachably mounted on a robot end effector and rigidly connected with the disc base; a connecting bracket rigidly connecting the calibration board and the disc base; wherein the spatial transformation relationship between the disc base and the calibration board is a predetermined fixed constant; and the positioning device comprises at least one positioning pin for ensuring the consistency of the mounting pose by cooperating with the positioning hole on the robot end effector when the positioning device is mounted on the robot end effector.

2. A simple method for robot hand-eye calibration, characterized in that, The method comprises the following steps: providing and mounting: providing the structured calibration device as claimed in claim 1 and mounting it on a robot end effector through the positioning device; image acquisition: acquiring the image of the visual markers on the calibration board by a camera; pose solving: based on the two-dimensional information of the visual markers in the image and the known three-dimensional information of the markers, a pose estimation algorithm is used to solve the first transformation relationship of the camera relative to the calibration board; calibration calculation: combining the first transformation relationship and the predetermined fixed spatial transformation relationship between the disc base and the calibration board of the device, the hand-eye transformation relationship of the camera relative to the robot end effector is calculated.

3. The structured calibration device for robot hand-eye calibration of claim 1, wherein, The visual markers are one of Aruco code, AprilTag or ChArUco checkerboard.

4. The structured calibration device of claim 1, wherein, The predetermined fixed spatial transformation relationship is obtained through the computer-aided design (CAD) model of the device.

5. The structured calibration device for robot hand-eye calibration according to claim 1 or 4, characterized in that, The key components of the device, including the connecting bracket, disc base and positioning device, have a manufacturing tolerance of IT5 level.

6. The structured calibration device for robot hand-eye calibration of claim 1, wherein, The positioning device further comprises at least one wedge-shaped positioning block for rough positioning during installation.

7. The structured calibration device for robot hand-eye calibration of claim 1, wherein, The connecting bracket and disc base are made of CNC machined aluminum alloy material.

8. The simple method for robot hand-eye calibration according to claim 2, wherein, The pose estimation algorithm is realized based on the solvePnP function in the OpenCV library.

9. The simple method for robot hand-eye calibration according to claim 2, wherein, The method further comprises a step of multi-frame averaging before the calibration calculation step: In the image acquisition step, a plurality of frames of images are continuously acquired; In the pose solving step, a first transformation relationship is solved for each frame of image; By averaging a plurality of the first transformation relationships, a more robust final first transformation relationship is obtained, which is used for subsequent calibration calculation.

10. The simple method for robot hand-eye calibration according to claim 9, wherein, The step of averaging the first transformation relationship specifically comprises: performing arithmetic averaging on the translation vector representing the translation component; performing component arithmetic averaging on the unit quaternion representing the rotation component, and performing normalization processing after averaging.

11. The simple method for robot hand-eye calibration according to claim 9, wherein, The step of continuously acquiring a plurality of frames of images specifically comprises continuously acquiring a video stream at a frame rate of no less than 30 fps for at least 30 seconds.

Images