A device and method for measuring robot hand-eye calibration accuracy

Abstract

The application provides a device and method for measuring robot hand-eye calibration precision, comprising a camera and a plurality of calibration blocks, wherein each face of each calibration block is printed with a Charuco Tag image, the Charuco Tag image is a checkerboard image formed by a plurality of two-dimensional code patterns, the two-dimensional code patterns carry digital information for camera pose estimation, the Charuco Tag images on each face of the calibration blocks are different, for the eye-on-hand scene, the camera is fixed to the robot end effector, and each calibration block is fixed to the robot end; for the eye-off-hand scene, the camera is fixed to a rigid structure, and each calibration block remains relatively stationary with the robot base. By using the Charuco Tag image as a calibration target, the stability and reliability of calibration can be improved.

Description

Technical Field

[0001] This invention relates to the field of industrial robot technology, and in particular to a device and method for measuring the calibration accuracy of a robot's hand and eye. Background Technology

[0002] Currently, the robotics field lacks a mature and widely accepted standardized standard for evaluating hand-eye calibration accuracy. This leads to different research institutions and manufacturers using different indicators and methods to assess hand-eye calibration accuracy, making cross-platform and cross-system comparisons difficult. Specifically, existing technologies have the following shortcomings:

[0003] (1) There are few existing evaluation devices, and the devices are complex: Currently, there are relatively few dedicated devices available for measuring the accuracy of hand-eye calibration. Most existing devices require customized design, resulting in high costs and complex manufacturing processes. This situation limits the possibility of widespread application, especially for small or medium-sized enterprises, which often find it difficult to afford the high costs of device manufacturing and maintenance.

[0004] (2) Lack of versatility: Existing hand-eye calibration accuracy measurement devices are often designed for specific types of robots or applications and lack versatility. This means that if hand-eye calibration accuracy measurement is required in different robots or applications, it is usually necessary to redesign and manufacture adaptive devices, which increases time and cost.

[0005] (3) Reliance on professional knowledge: Current hand-eye calibration accuracy measurement methods usually require operators with professional knowledge and skills. This limits the popularization and application of the technology, because not every user has enough professional knowledge to perform accurate hand-eye calibration accuracy measurement.

[0006] (4) Lack of real-time capability: Existing hand-eye calibration accuracy measurement methods often take a long time to complete and cannot provide real-time feedback. This may lead to a decrease in production efficiency in application scenarios that require rapid hand-eye calibration, such as robot operation on industrial production lines. Summary of the Invention

[0007] The purpose of this invention is to at least address one of the aforementioned technical deficiencies.

[0008] Therefore, the purpose of this invention is to provide a device and method for measuring the calibration accuracy of a robot's hand and eye.

[0009] To achieve the above objectives, one aspect of the present invention provides an apparatus for measuring the calibration accuracy of a robot's hand and eye, comprising:

[0010] The system includes a camera and multiple calibration blocks, each of which has a Charuco Tag image printed on each face. The Charuco Tag image is a checkerboard pattern composed of multiple QR code graphics, each carrying digital information for camera pose estimation. The Charuco Tag image on each face of the calibration block is unique.

[0011] For the "eye in hand" scenario: the camera is fixed to the robot's end effector, and each calibration block is fixed to the robot's end effector;

[0012] For scenarios where the eye is outside the hand: the camera is fixed to a rigid structure, and each calibration block remains stationary relative to the robot base.

[0013] Furthermore, each of the calibration blocks is a cube.

[0014] Furthermore, the side length of each calibration block is between 10cm and 100cm.

[0015] Another embodiment of the present invention provides a method for measuring the calibration accuracy of a robot's hand and eye, comprising the following steps:

[0016] Step S1: Place the calibration block at a preset position within the robot's workspace to ensure that the calibration block is fully visible within the robot's workspace;

[0017] Step S2: Control the robot system to move the camera on the end effector to a suitable position to take a preliminary positioning image. This position is basically parallel to the first face of the calibration block.

[0018] Step S3: Perform image analysis on the preliminary positioning image, and base the analysis on the image analysis results and hand-eye calibration results. The pose matrix of the calibration block relative to the robot is calculated.

[0019] Step S4: Based on the robot's reachable position and the number of surfaces to be measured, select multiple calibration blocks for hand-eye calibration accuracy measurement, including:

[0020] Step S41: The robot adjusts to the target pose of the corresponding face to ensure that the face is located at the specified position in the camera coordinate system, and captures the Charuco Tag image of the calibration block.

[0021] In step S42, the end effector moves along the x, y, and z axes of the flange coordinate system in preset step sizes. At each step size position, the camera acquires an image, performs image analysis, and calculates the actual pose of the surface relative to the camera.

[0022] Step S43: At the starting position, rotate the flange around the three coordinate axes. At each rotation position, the camera acquires an image, and then the image analysis is used again to calculate... The relative rotation angle between the surface and the camera is obtained, and the difference between this angle and the robot's actual rotation angle is calculated to obtain the angle error.

[0023] Step S44: After each surface to be tested has completed the above test, reset the position and angle of the calibration block and retest.

[0024] Step S45: Take the average value of the above multiple test results to obtain the relationship between the error and displacement angle of the surface. Finally, obtain the relationship between the error, displacement and angle of each surface, which is the hand-eye calibration error chart.

[0025] Furthermore, let the pose of the nth face relative to the calibration block be... Then we get:

[0026]

[0027] in, The relative pose matrix between the camera and the end effector; The pose of the nth face relative to the camera is preset; then, the target poses of all faces are calculated.

[0028] Furthermore, in step S4, the surfaces of 3 to 5 calibration blocks are selected for hand-eye calibration accuracy measurement.

[0029] Furthermore, in step S42, the actual pose of the surface relative to the camera is calculated. Includes the following steps:

[0030] (1) Remove distortion from the image based on the camera distortion coefficient;

[0031] (2) Image preprocessing, including: image filtering and image enhancement;

[0032] (3) Search for control points in the image, obtain their pixel coordinates, establish multiple mapping point relationships, and calculate the homography matrix; based on the actual pose... The relative movement distance between the surface and the camera is obtained and compared with the actual distance moved by the robot, with Euclidean distance as the error.

[0033] Furthermore, in step S43, the flange is rotated about 5° to 5° around the three coordinate axes.

[0034] The apparatus and method for measuring the calibration accuracy of a robot's hand and eye according to embodiments of the present invention have the following beneficial effects:

[0035] 1. Use of Charuco Tag Images and Image Analysis Algorithm: This invention utilizes Charuco Tag images as calibration targets and employs automated image processing methods such as image preprocessing, filtering, and distortion correction to improve image quality and stability. By using Charuco Tag images as calibration targets, the stability and reliability of calibration can be improved.

[0036] 2. Automated Image Acquisition and Processing Method: The apparatus and method provided in this invention aim to reduce the degree of human intervention and achieve automated image acquisition and processing. Automated image acquisition includes camera control and robot movement to ensure sufficient data is acquired on each surface. This invention reduces human intervention and improves calibration efficiency by automating image acquisition and analysis.

[0037] 3. Customizable Cube Design: This invention designs a customizable cube whose side length can be customized according to robot systems of different sizes and applications. This design considers the needs of different application scenarios, providing flexibility and versatility. By adapting to robot systems of different sizes and applications, the customizable cube design provided by this invention makes the device suitable for robot systems of different sizes and applications.

[0038] 4. Standardized evaluation criteria: This invention establishes a documented and quantifiable hand-eye calibration accuracy measurement system.

[0039] 5. High precision and simple operation: The hand-eye calibration method provided by this invention can achieve high-precision pose estimation in robot applications.

[0040] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0041] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0042] Figure 1 This is a flowchart of a method for measuring the calibration accuracy of a robot's hand and eye according to an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of tag36h11 according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of a calibration block according to an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram illustrating the initial positioning during imaging according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram illustrating the movement and rotation direction of the flange according to an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram illustrating the relationship between error and displacement according to an embodiment of the present invention. Detailed Implementation

[0048] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0049] This invention provides an apparatus for measuring the calibration accuracy of a robot's hand and eye, comprising: a camera and multiple calibration blocks. Each face of each calibration block is printed with a Charuco Tag image, which is a checkerboard pattern composed of multiple Apriltags. The Apriltags carry digital information used for camera pose estimation. The Charuco Tag images on each face of the calibration block are all different.

[0050] In embodiments of the present invention, each calibration block is a cube. Preferably, the side length of each calibration block is between 10 cm and 100 cm.

[0051] Specifically, the device for measuring the accuracy of robot hand-eye calibration according to the present invention is an innovative tool for detecting hand-eye calibration accuracy. It uses a customizable cube as its main component, namely the calibration block. The side length of the calibration block can be customized between 10cm and 100cm according to specific needs. This feature allows the device to be adapted to robot systems of various sizes and applications, from small six-axis robots to large industrial robots.

[0052] For the cube-shaped calibration block, each of its six faces is printed with a Charuco Tag image. This image is composed of tiled Apriltag-like graphics. Apriltags carry digital information and have good recognition performance, and are often used in camera calibration and pose estimation in machine vision. Figure 2 A portion of the Apriltag graphic is shown. Among them, Figure 2 The center blank square of each graphic contains a QR code pattern.

[0053] In embodiments of this invention, the Charuco Tag image is a checkerboard image composed of multiple Apriltags, which is commonly used for camera pose estimation. In this invention, it is used for image analysis during the calibration process to measure the accuracy of hand-eye calibration. Figure 3 The physical structure and appearance of the calibration blocks are illustrated. The Charuco Tag images on the six faces of each cube are different, labeled from face 1 to face 6. A coordinate system is established with the vector pointing from the center of the cube to the center of the first face as the x-axis and the vector pointing to the center of the second face as the y-axis. Figure 3 The center blank square of each graphic contains a QR code pattern.

[0054] During the initial preparation phase, the camera is either fixed to the robot's end effector (in a scenario where the eye is on the hand) or attached to a rigid structure (in a scenario where the eye is outside the hand). For the former scenario, the aforementioned calibration block device needs to be fixed to the robot's end effector; for the latter scenario, the calibration block simply needs to remain stationary relative to the robot's base. That is,

[0055] (1) For the eye-on-hand scenario: the camera is fixed to the robot end effector, and each calibration block is fixed to the robot end;

[0056] (2) For scenarios where the eye is outside the hand: the camera is fixed to a rigid structure, and each calibration block remains stationary relative to the robot base.

[0057] Before performing accuracy testing, the following data needs to be input: hand-eye calibration results, including camera pose, intrinsic parameters, distortion coefficients, and calibration block side lengths. The workflow will be described below using a scenario where the eye is on the hand as an example, including camera position and pose calculations, and accuracy testing steps.

[0058] like Figure 1 As shown in the figure, the method for measuring the calibration accuracy of a robot's hand and eye according to an embodiment of the present invention includes the following steps:

[0059] Step S1, Camera position and attitude calculation: Place the calibration block at a preset position in the robot's workspace to ensure that the calibration block is fully visible in the robot's workspace.

[0060] Specifically, when using this invention to test the accuracy of hand-eye calibration, the calibration block is first placed in a suitable fixed position. The choice of this position depends on the robot's workspace, but it must be ensured that the cube can be fully seen within the robot's workspace.

[0061] Step S2: The robot system is manipulated to move the camera on the end effector to a suitable position to take a preliminary positioning image. This position is basically parallel to the first face of the calibration block.

[0062] A manually controlled robot system moves a camera on the end effector to a suitable position to take preliminary positioning images. This position is approximately parallel to the first surface of the calibration block. Figure 4 As shown.

[0063] Step S3: Perform image analysis on the preliminary positioning image, and base the analysis on the image analysis results and the hand-eye calibration results. The pose matrix of the calibration block relative to the robot is calculated.

[0064] Based on image analysis and known hand-eye calibration results (The relative pose matrix between the camera and the end effector) can be used to calculate the pose matrix of the cube relative to the robot. Let the pose of the nth face relative to the calibration block be... The following formula is obtained.

[0065]

[0066] Among them, T c n am The pose of the nth face relative to the camera is preset, usually perpendicular to the camera's optical axis and 100-500mm away from the camera's xoy plane.

[0067] Then, the target pose T of all faces is calculated. b n_ a e s n e d .

[0068] Step S4: Based on the robot's reachable position and the number of surfaces to be measured, select multiple calibration blocks to perform hand-eye calibration accuracy measurement.

[0069] In an embodiment of the invention, 3 to 5 calibration blocks are selected for hand-eye calibration accuracy measurement. The accuracy detection steps for each block are as follows. The following is the programming logic; in actual operation, the robot automatically completes the following actions without human intervention.

[0070] In step S41, the robot adjusts to the target pose of the corresponding face to ensure that the face is located at the specified position in the camera coordinate system. This step is to ensure that the camera can accurately capture the Charuco Tag image of the cube.

[0071] In step S42, the end effector moves along the x, y, and z axes of the flange coordinate system in preset step sizes. At each step size position, the camera acquires an image, performs image analysis, and calculates the actual pose T of the surface relative to the camera. cn am real .

[0072] Specifically, the end effector moves -50 to 50 mm (in 10 mm increments) along the x, y, and z axes of the flange coordinate system. At each position, the camera acquires an image, which is then analyzed. The actual pose T of the surface relative to the camera is calculated. c n am real It includes the following steps:

[0073] (1) Remove distortion from the image based on the camera distortion coefficient;

[0074] (2) Image preprocessing, including: image filtering and image enhancement;

[0075] (3) Search for control points in the image, obtain their pixel coordinates, establish multiple mapping point relationships, and calculate the homography matrix; based on the actual pose T c n am real The relative movement distance between the surface and the camera is obtained and compared with the actual distance moved by the robot, with Euclidean distance as the error.

[0076] Step S43: At the starting position, rotate the flange around the three coordinate axes. At each rotation position, the camera acquires an image, and then T is calculated again through image analysis. c n am real The relative rotation angle between the surface and the camera is obtained, and the difference between this angle and the robot's actual rotation angle is calculated to obtain the angle error.

[0077] Specifically, such as Figure 5 As shown, at the initial position, the flange is rotated from -5° to 5° around the three coordinate axes. At each rotation position, the camera acquires images containing Charuco Tag images of the cube. Then, T is calculated again through image analysis. c n am real The relative rotation angle between the surface and the camera is obtained, and the difference between this angle and the robot's actual rotation angle is calculated to obtain the angle error.

[0078] Step S44: After each surface to be tested has completed the above test, the calibration block is manually moved a certain distance and angle, that is, the position and angle of the calibration block are reset, and the test is repeated. This test can be repeated multiple times.

[0079] Step S45: Based on the average value of the above multiple test results, plot the graph with displacement / angle as the x-axis and error as the y-axis to obtain the relationship between the error and displacement / angle of the surface, as shown below. Figure 6 As shown in the figure. The final result is the relationship between the error of each surface and the displacement / angle, which is the hand-eye calibration error chart.

[0080] The apparatus and method for measuring the calibration accuracy of a robot's hand and eye according to embodiments of the present invention have the following beneficial effects:

[0081] 1. Use of Charuco Tag Images and Image Analysis Algorithm: This invention utilizes Charuco Tag images as calibration targets and employs automated image processing methods such as image preprocessing, filtering, and distortion correction to improve image quality and stability. By using Charuco Tag images as calibration targets, the stability and reliability of calibration can be improved.

[0082] 2. Automated Image Acquisition and Processing Method: The apparatus and method provided in this invention aim to reduce the degree of human intervention and achieve automated image acquisition and processing. Automated image acquisition includes camera control and robot movement to ensure sufficient data is acquired on each surface. This invention reduces human intervention and improves calibration efficiency by automating image acquisition and analysis.

[0083] 3. Customizable Cube Design: This invention designs a customizable cube whose side length can be customized according to robot systems of different sizes and applications. This design considers the needs of different application scenarios, providing flexibility and versatility. By adapting to robot systems of different sizes and applications, the customizable cube design provided by this invention makes the device suitable for robot systems of different sizes and applications.

[0084] 4. Standardized evaluation criteria: This invention establishes a documented and quantifiable hand-eye calibration accuracy measurement system.

[0085] 5. High precision and simple operation: The hand-eye calibration method provided by this invention can achieve high-precision pose estimation in robot applications.

[0086] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0087] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for measuring the accuracy of robot hand-eye calibration, characterized in that, The method is performed using a device for measuring the calibration accuracy of a robot's hand and eye. The device includes a camera and multiple calibration blocks, wherein each face of each calibration block is printed with a Charuco Tag image. The Charuco Tag image is a checkerboard image composed of multiple QR code graphics, wherein the QR code graphics carry digital information for camera pose estimation; the Charuco Tag image on each face of the calibration block is different. The method for measuring the accuracy of robot hand-eye calibration includes the following steps: Step S1: Place the calibration block at a preset position within the robot's workspace to ensure that the calibration block is fully visible within the robot's workspace; Step S2: Control the robot system to move the camera on the end effector to a suitable position to take a preliminary positioning image. This position is basically parallel to the first face of the calibration block. Step S3: Perform image analysis on the preliminary positioning image, and base the analysis on the image analysis results and hand-eye calibration results. The pose matrix of the calibration block relative to the robot is calculated. ; Let the pose of the nth face relative to the calibration block be... Then we get: ； in, The relative pose matrix between the camera and the end effector; The pose of the nth face relative to the camera is preset, perpendicular to the camera's optical axis, and 100~500mm away from the camera's xoy plane; Then, the target pose of all faces is calculated. ; Step S4: Based on the robot's reachable position and the number of surfaces to be measured, select multiple calibration blocks for hand-eye calibration accuracy measurement, including: Step S41: The robot adjusts to the target pose of the corresponding face to ensure that the face is located at the specified position in the camera coordinate system, and captures the Charuco Tag image of the calibration block. In step S42, the end effector moves along the x, y, and z axes of the flange coordinate system in preset step sizes. At each step size position, the camera acquires an image, performs image analysis, and calculates the actual pose of the surface relative to the camera. ; The end effector moves along the x, y, and z axes of the flange coordinate system in steps of 10 mm, from -50 mm to 50 mm respectively; at each position, the camera acquires an image and performs image analysis. Calculate the actual pose of the face and the camera. It includes the following steps: (1) Dedistort the image based on the camera distortion coefficient; (2) Image preprocessing, including: image filtering and image enhancement; (3) Search for control points in the image, obtain their pixel coordinates, establish multiple mapping point relationships, and calculate the homography matrix; based on the actual pose... The relative movement distance between the surface and the camera is obtained and compared with the actual movement distance of the robot, with Euclidean distance as the error. Step S43: At the starting position, rotate the flange around the three coordinate axes. At each rotation position, the camera acquires an image, and then the image analysis is used again to calculate... The relative rotation angle between the face and the camera is obtained, and the difference between this angle and the robot's actual rotation angle is calculated to obtain the angle error. Specifically, at the initial position, the flange is rotated around three coordinate axes from -5° to 5°. At each rotation position, the camera acquires images containing Charuco Tag images of the cube. Then, image analysis is used again to calculate... The relative rotation angle between the surface and the camera is obtained, and the difference between this angle and the robot's actual rotation angle is calculated to obtain the angle error. Step S44: After each surface to be tested has completed the above test, reset the position and angle of the calibration block and retest. Step S45: Take the average value of the above multiple test results to obtain the relationship between the error and displacement angle of the surface. Finally, obtain the relationship between the error, displacement and angle of each surface, which is the hand-eye calibration error chart.

2. The method for measuring the accuracy of robot hand-eye calibration as described in claim 1, characterized in that, In step S4, select 3 to 5 calibration blocks to perform hand-eye calibration accuracy measurement.

Images