A robot hand-eye calibration method and device, robot and storage medium

Abstract

The application discloses a robot hand-eye calibration method and device, a robot and a storage medium, and relates to the technical field of robots. The method comprises the following steps: controlling the robot to grab a calibration object and move to nine preset points, obtaining a nine-point calibration matrix according to the calibration object image of the calibration object at each point, the robot point coordinates, and a preset template matching algorithm; controlling the robot to grab the calibration object to rotate multiple points, obtaining a rotation center according to the template matching algorithm, and correcting the nine-point calibration matrix according to the rotation center. According to the scheme, the robot arm is used to grab the calibration object to perform nine-point calibration, the preset template matching algorithm is used for template matching, so that the nine-point calibration matrix is obtained, and the rotation center obtained by rotation calibration is used to correct the nine-point calibration matrix, thereby solving the problems of complicated operation and low calibration precision of manual calibration operation, and greatly improving the precision of robot hand-eye calibration.

Description

Technical Field

[0001] The present invention belongs to the field of industrial machine vision technology, and specifically relates to a robot hand-eye calibration method, device, robot and storage medium, and more particularly to a robot hand-eye calibration method, device, robot and storage medium based on an automatic nine-point rotation calibration algorithm of sub-pixel image pyramid template matching. Background Art

[0002] In industrial machine vision, a robot typically refers to a multi-joint, multi-degree-of-freedom robotic arm. Robots lack sensors and require an industrial camera to acquire information about the three-dimensional world. The camera acts as the robot's "eyes." The camera converts this information into a two-dimensional plane (a pixel coordinate system). This two-dimensional plane information is then converted into three-dimensional information in the robot's coordinate system through a specific transformation. The overall process involves a 3D-to-2D-to-3D transformation. This "specific transformation" is the relationship between the camera and robot coordinate systems. The process of obtaining this transformation is also known as hand-eye calibration.

[0003] The nine-point calibration method is a classic approach for hand-eye calibration. It collects nine pixel coordinates and nine robot coordinates and calculates the affine transformation matrix between them to complete the hand-eye calibration operation. Traditional nine-point calibration algorithms require manual instruction of the robot coordinates for the nine calibration points. This is a cumbersome operation, and manual instruction of the points is prone to errors, resulting in low overall calibration accuracy.

[0004] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The purpose of the present invention is to provide a robot hand-eye calibration method, device, robot and storage medium to solve the problem that the robot coordinates of nine calibration points are manually taught in the nine-point calibration method, the operation is cumbersome, and the manual teaching points are prone to errors, resulting in low overall calibration accuracy. The method achieves the goal of grasping the calibration object by the robot's mechanical arm for nine-point calibration, performing template matching using a preset template matching algorithm to obtain a nine-point calibration matrix, and correcting the nine-point calibration matrix using the rotation center obtained by rotation calibration, thereby solving the problems of cumbersome operation and low calibration accuracy of manual calibration operations, and greatly improving the accuracy of the robot hand-eye calibration.

[0006] The present invention provides a robot hand-eye calibration method, which is applied to a hand-eye calibration system of an industrial robot. The robot has a robotic arm and a camera. The method comprises: while keeping the robot's camera stationary, controlling the robot's robotic arm to grasp a calibration object, and causing the robot's robotic arm to carry the calibration object and move nine preset positions within the camera's field of view; obtaining a calibration object image recorded by the camera at each of the nine positions of the calibration object, and robot position coordinates recorded by the robot at each position; obtaining an uncorrected nine-point calibration matrix based on the calibration object images at the nine positions, the robot position coordinates at the nine positions, and a preset template matching algorithm; while keeping the robot's camera stationary, controlling the robot's robotic arm to grasp the calibration object and rotate it m positions at the same position, and determining a rotation center based on the preset template matching algorithm, wherein m is a positive integer; and correcting the uncorrected nine-point calibration matrix based on the rotation center to obtain a corrected nine-point calibration matrix.

[0007] In some embodiments, an uncorrected nine-point calibration matrix is ​​obtained based on the calibration object images of the nine points, the robot point coordinates of the nine points, and a preset template matching algorithm, including: determining the calibration center point pixel coordinates of the nine points based on the calibration object images of the nine points and the preset template matching algorithm; and obtaining the uncorrected nine-point calibration matrix based on the robot point coordinates of the nine points and the calibration center point pixel coordinates of the nine points.

[0008] In some embodiments, the preset template matching algorithm includes: processing the matching image and the template image respectively to determine the candidate position of the template image in the image to be matched; based on the candidate position of the template image in the image to be matched and the least squares adjustment theory, adjusting the accuracy of the candidate position to obtain the center point coordinates, rotation angle and scaling factor of the template image in the image to be matched.

[0009] In some embodiments, processing the matching image and the template image separately to determine a candidate position of the template image in the image to be matched includes: smoothing the image to be matched and the template image separately; obtaining an image pyramid of the image to be matched and an image pyramid of the template image based on the smoothed image to be matched and the template image; wherein the image pyramid includes a low-resolution image and a high-resolution image of the image; determining a first position of the template image in the image to be matched based on the low-resolution image in the image pyramid of the image to be matched and the low-resolution image in the image pyramid of the template image; determining a second position of the template image in the image to be matched based on the high-resolution image in the image pyramid of the image to be matched, the high-resolution image in the image pyramid of the template image, and the first position; wherein the second position is a more accurate position than the first position; performing sub-pixel edge detection on the image pyramid of the image to be matched and the image pyramid of the template image separately; and performing similarity calculation based on the results of the sub-pixel edge detection on the image to be matched and the template image to determine a candidate position of the template image in the image to be matched; the formula for calculating the similarity is:

[0010]

[0011] Where d′ i is the edge gradient vector in the template image, q = [x, y] T is the pixel position of the image to be matched, p′ is the pixel of the sub-image in the image to be matched, e q+p′ is the direction vector; n is the pixel value of the template image, t′ i and u′ i is the gradient direction vector value of each point in the template image, and is the gradient direction vector value in the image to be matched.

[0012] In some embodiments, the pixel coordinates of the calibration center points of the nine points are determined based on the calibration object images of the nine points and a preset template matching algorithm, including: identifying the calibration object images of the nine points as images to be matched, and identifying the image containing only the calibration object as a template image; respectively bringing each of the images to be matched and the template image into the preset template matching algorithm to obtain the pixel coordinates of the calibration center point of the template image in each of the images to be matched, that is, determining the pixel coordinates of the calibration center points of the nine points.

[0013] In some embodiments, the robot's robotic arm is controlled to grasp the calibration object and rotate m points at the same position, and the rotation center is determined according to the preset template matching algorithm, including: controlling the robot's robotic arm to grasp the calibration object and rotate m points, and obtaining the calibration object image recorded by the camera at each of the m points, and the robot point coordinates recorded by the robot at each point; identifying the m calibration object images as images to be matched, and identifying the image containing only the calibration object as a template image; respectively bringing each of the images to be matched and the template image into the preset template matching algorithm to obtain the calibration center point pixel coordinates of the template image in each of the images to be matched, that is, determining the calibration center point pixel coordinates of the m points; fitting the rotation center according to the calibration center point pixel coordinates of the m points and the robot point coordinates of the m points.

[0014] Matching the above method, the present invention provides a control device for a voltage detection device on the other hand, which is applied to a hand-eye calibration system of an industrial robot; the robot has a robotic arm and a camera; the device comprises: a control unit, configured to control the robotic arm of the robot to grasp a calibration object while keeping the camera of the robot stationary, and to make the robotic arm of the robot carry the calibration object and move to nine preset points within the field of view of the camera; an acquisition unit, configured to acquire the calibration object image recorded by the camera at each of the nine points, and the calibration object image recorded by the robot at each of the nine points. The robot point coordinates are recorded; the control unit is further configured to obtain an uncorrected nine-point calibration matrix based on the calibration object images at the nine points, the robot point coordinates at the nine points, and a preset template matching algorithm; the control unit is further configured to control the robot's robotic arm to grasp the calibration object and rotate m points at the same position while keeping the robot's camera still, and determine the rotation center according to the preset template matching algorithm; wherein m is a positive integer; the control unit is further configured to correct the uncorrected nine-point calibration matrix according to the rotation center to obtain a corrected nine-point calibration matrix.

[0015] In some embodiments, the control unit obtains an uncorrected nine-point calibration matrix based on the calibration object images of the nine points, the robot point coordinates of the nine points, and a preset template matching algorithm, including: determining the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and the preset template matching algorithm; and obtaining the uncorrected nine-point calibration matrix based on the robot point coordinates of the nine points and the pixel coordinates of the calibration center points of the nine points.

[0016] In some embodiments, the preset template matching algorithm includes: processing the matching image and the template image respectively to determine the candidate position of the template image in the image to be matched; based on the candidate position of the template image in the image to be matched and the least squares adjustment theory, adjusting the accuracy of the candidate position to obtain the center point coordinates, rotation angle and scaling factor of the template image in the image to be matched.

[0017] In some embodiments, processing the matching image and the template image separately to determine a candidate position of the template image in the image to be matched includes: smoothing the image to be matched and the template image separately; obtaining an image pyramid of the image to be matched and an image pyramid of the template image based on the smoothed image to be matched and the template image; wherein the image pyramid includes a low-resolution image and a high-resolution image of the image; determining a first position of the template image in the image to be matched based on the low-resolution image in the image pyramid of the image to be matched and the low-resolution image in the image pyramid of the template image; determining a second position of the template image in the image to be matched based on the high-resolution image in the image pyramid of the image to be matched, the high-resolution image in the image pyramid of the template image, and the first position; wherein the second position is a more accurate position than the first position; performing sub-pixel edge detection on the image pyramid of the image to be matched and the image pyramid of the template image separately; and performing similarity calculation based on the results of the sub-pixel edge detection on the image to be matched and the template image to determine a candidate position of the template image in the image to be matched; the formula for calculating the similarity is:

[0018]

[0019] Where d′ i is the edge gradient vector in the template image, q = [x, y] T is the pixel position of the image to be matched, p′ is the pixel of the sub-image in the image to be matched, e q+p′ is the direction vector; n is the pixel value of the template image, t′ t and u′ t is the gradient direction vector value of each point in the template image, and is the gradient direction vector value in the image to be matched.

[0020] In some embodiments, the control unit determines the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and a preset template matching algorithm, including: identifying the calibration object images of the nine points as images to be matched, and identifying the image containing only the calibration object as a template image; respectively bringing each of the images to be matched and the template image into the preset template matching algorithm to obtain the pixel coordinates of the calibration center point of the template image in each of the images to be matched, that is, determining the pixel coordinates of the calibration center point of the nine points.

[0021] In some embodiments, the control unit controls the robot's robotic arm to grasp the calibration object and rotate m points at the same position, and determines the rotation center according to the preset template matching algorithm, including: controlling the robot's robotic arm to grasp the calibration object and rotate m points, and obtaining the calibration object image recorded by the camera at each of the m points, and the robot point coordinates recorded by the robot at each point; identifying the m calibration object images as images to be matched, and identifying the image containing only the calibration object as a template image; respectively bringing each of the images to be matched and the template image into the preset template matching algorithm to obtain the calibration center point pixel coordinates of the template image in each of the images to be matched, that is, determining the calibration center point pixel coordinates of the m points; fitting the rotation center according to the calibration center point pixel coordinates of the m points and the robot point coordinates of the m points.

[0022] Matching the above-mentioned device, the present invention further provides a robot, comprising: a control device of the voltage detection device described above.

[0023] In accordance with the above method, the present invention further provides a storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above-mentioned robot hand-eye calibration method.

[0024] The solution of the present invention controls the robot's mechanical arm to grasp a calibration object while keeping the robot's camera stationary, and causes the robot's mechanical arm to move the calibration object to nine preset positions within the camera's field of view. The system then obtains an image of the calibration object recorded by the camera at each of the nine positions, as well as the robot's position coordinates recorded by the robot at each position. An uncorrected nine-point calibration matrix is ​​derived based on the nine calibration object images, the nine robot position coordinates, and a preset template matching algorithm. The robot's mechanical arm is controlled to grasp the calibration object and rotate it m positions at the same position, determining the rotation center based on the preset template matching algorithm. The uncorrected nine-point calibration matrix is ​​corrected based on the rotation center to obtain a corrected nine-point calibration matrix. This solves the problems of cumbersome manual operation and low calibration accuracy, significantly improving the accuracy of hand-eye calibration.

[0025] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention.

[0026] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of a flow chart of an embodiment of a robot hand-eye calibration method of the present invention;

[0028] Figure 2 Schematic diagram of a flow chart of an embodiment of determining a nine-point calibration matrix in the method of the present invention;

[0029] Figure 3 1 is a flow chart of an embodiment of a template matching algorithm in the method of the present invention;

[0030] Figure 4 A schematic diagram of a flow chart of an embodiment of determining a candidate position of a template image in an image to be matched in a template matching algorithm in the method of the present invention;

[0031] Figure 5 1. A flow chart of an embodiment of determining the coordinates of the calibration center pixels of nine points according to the template matching algorithm in the method of the present invention;

[0032] Figure 6 Schematic diagram of a flow chart of an embodiment of the method for determining a rotation center in the present invention;

[0033] Figure 7 A schematic structural diagram of an embodiment of a control device for a voltage detection device of the present invention;

[0034] Figure 8Schematic diagram of the structure of template matching in the method of the present invention;

[0035] Figure 9 Schematic diagram of the process of the automatic nine-point rotation calibration method in the hand-eye calibration method of the present invention;

[0036] Figure 10 Schematic diagram of the process of the template matching algorithm based on sub-pixel image pyramid in the hand-eye calibration method of the present invention;

[0037] In conjunction with the accompanying drawings, the reference numerals in the embodiments of the present invention are as follows:

[0038] 102 - acquisition unit; 104 - control unit. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] The nine-point calibration algorithm requires the use of a visual positioning algorithm, such as template matching, to locate the nine calibration points and calculate the corresponding pixel coordinates. Template matching is a computer vision technique that identifies portions of an image that match a predefined template. It involves moving a template across the entire image and calculating the similarity between the template and the covered window on the image. Traditional template matching algorithms perform pixel-level positioning, resulting in low accuracy. This is particularly noticeable at image edges, causing jagged and blurred edges in the matched image. In the case of inaccurate edges, the matching results are also inaccurate. Furthermore, when there are differences in brightness, noise, and other factors between the template and the image to be matched, traditional template matching algorithms cannot achieve consistent accuracy and matching speed. Finally, traditional template matching algorithms lack rotational and scale invariance. Rotational invariance means that the matching score of traditional template matching decreases significantly after rotating the original image. Scale invariance means that scaling the original image significantly affects the template matching score. These factors significantly affect the positioning of the calibration points, resulting in inaccurate calculated pixel coordinates.

[0041] The traditional nine-point calibration algorithm requires manual teaching of the robot coordinates of nine calibration points, and then uses a visual positioning algorithm to calculate the pixel coordinates of the nine calibration points. The nine-time manual teaching operation is tedious and time-consuming, and each teaching is very likely to produce offset errors. The cumulative error of nine times will lead to low calibration accuracy; when the calibration feature point is inconsistent with the tool center point, the rotation center needs to be corrected, but the traditional nine-point calibration algorithm does not take this into account. The calculated nine-point calibration matrix cannot accurately represent the transformation relationship between the camera and the robot.

[0042] Therefore, the present invention provides a robot hand-eye calibration method, which uses a nine-point rotation calibration method by grabbing a calibration object and moving it within the robot's camera field of view. Then, an improved template matching algorithm is used to accurately position the calibration object to obtain a nine-point calibration matrix, and the nine-point calibration matrix is ​​corrected by rotation center calibration. This solves the problems of difficult calibration operation and easy errors in teaching, and improves the accuracy of calibration.

[0043] According to an embodiment of the present invention, a robot hand-eye calibration method is provided, which is applied to a hand-eye calibration system of an industrial robot; the robot has a robotic arm and a camera; Figure 1 The flowchart of an embodiment of the method of the present invention is shown. The robot hand-eye calibration method may include: steps S110 to S150.

[0044] In step S110, while the robot's camera remains stationary, the robot's robotic arm is controlled to grasp the calibration object and move the calibration object within the camera's field of view through nine preset locations. To improve the accuracy of the calibration matrix, the number of preset locations is not limited to nine and can be increased appropriately, such as to sixteen. A greater number of locations results in a more accurate calibration matrix.

[0045] The captured calibration object must also have distinct feature points to facilitate subsequent template matching and improve template matching accuracy. Before controlling the robot's mechanical arm to grasp the calibration object, it is necessary to determine the lighting scheme by adjusting the light source intensity and position, and adjust the camera exposure and focus to ensure a clearer image.

[0046] In step S120 , the calibration object image recorded by the camera at each of the nine points of the calibration object and the robot point coordinates recorded by the robot at each of the nine points are obtained.

[0047] Each time the calibration object moves to a position, the camera records the image of the calibration object at that position, and the robot records the robot point coordinates when the calibration object is at that position, obtaining a total of nine sets of image and coordinate data.

[0048] In step S130 , an uncorrected nine-point calibration matrix is ​​obtained according to the calibration object images at the nine points, the robot point coordinates at the nine points, and a preset template matching algorithm.

[0049] Figure 2 FIG1 is a flow chart of an embodiment of determining a nine-point calibration matrix in the method of the present invention. In some embodiments, in step 130, the specific process of obtaining an uncorrected nine-point calibration matrix is ​​as follows: Figure 2 As shown, it includes: step S210 and step S220.

[0050] Step S210 , determining the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and a preset template matching algorithm.

[0051] Step S220 , obtaining an uncorrected nine-point calibration matrix according to the robot point coordinates of the nine points and the pixel coordinates of the calibration center points of the nine points.

[0052] The nine-point rotation calibration method performs an affine transformation on the robot coordinates at nine positions and the corresponding pixel coordinates in the image to determine the relationship between the robot plane and the camera plane. This relationship is the nine-point calibration matrix.

[0053] Figure 3 FIG. 1 is a flow chart of an embodiment of a template matching algorithm in the method of the present invention. In some implementations, the preset template matching algorithm, such as Figure 3 As shown, it includes: step S310 and step S320.

[0054] Step S310 : Process the matching image and the template image respectively to determine a candidate position of the template image in the image to be matched.

[0055] Figure 4 FIG. 1 is a flow chart of an embodiment of determining a candidate position of a template image in an image to be matched in a template matching algorithm in the method of the present invention. In some embodiments, in step S310, the matching image and the template image are processed separately to determine a specific process of determining a candidate position of the template image in the image to be matched, as shown in FIG. Figure 4 As shown, it includes: step S410 to step S460.

[0056] Step S410 : performing smoothing processing on the image to be matched and the template image respectively, so as to reduce the influence of noise on the image to be matched and the template image.

[0057] Figure 8Schematic diagram of the structure of template matching in the method of the present invention, such as Figure 8 As shown in FIG, on an M×N image to be matched, the matching degree between the template image and the overlapping sub-image is calculated from left to right and from top to bottom. The greater the matching degree, the greater the possibility that the two are the same. The size of the possibility is mainly measured by the similarity metric function.

[0058] Step S420 , obtaining an image pyramid of the image to be matched and an image pyramid of the template image based on the smoothed image to be matched and the template image; wherein the image pyramid includes a low-resolution image and a high-resolution image of the image.

[0059] Step S430 : Determine a first position of the template image in the image to be matched based on the low-resolution image in the image pyramid of the image to be matched and the low-resolution image in the image pyramid of the template image; wherein the first position is an approximate position.

[0060] Step S440: Determine a second position of the template image in the image to be matched based on the high-resolution image in the image pyramid of the image to be matched, the high-resolution image in the image pyramid of the template image, and the first position; wherein the second position is a more accurate position than the first position.

[0061] An image pyramid processes an original image into multiple new images of varying resolutions, arranged from low to high resolution. The number of images is also called the pyramid layers. The matching area is quickly located at low resolution, and precise pixel matching is then performed at high resolution. Because the approximate area is determined at low resolution, there's no need to search the entire image at high resolution, thus saving time.

[0062] Step S450 : performing sub-pixel edge detection on the image pyramid of the image to be matched and the image pyramid of the template image respectively.

[0063] During the camera imaging process, the image data obtained is the discretization of the image. Due to the capacity limitation of the photosensitive element itself, each pixel on the imaging surface only represents the nearby color. For example, there is a 4.5um spacing between the pixels on the two sensory elements, but they are connected together at the macro level, and there are countless tiny things between them at the micro level. These pixels between two actual physical pixels are called sub-pixels.

[0064] By detecting sub-pixel edges, the template image is positioned more accurately in the image to be matched, thereby improving the accuracy of template matching.

[0065] Step S460: Perform similarity calculation based on the results of sub-pixel edge detection on the image to be matched and the template image, thereby determining a candidate position of the template image in the image to be matched; the formula for similarity calculation is:

[0066]

[0067] Where d′ i is the edge gradient vector in the template image, q = [x, y] T is the pixel position of the image to be matched, p′ is the pixel of the sub-image in the image to be matched, e q+p′ is the direction vector; n is the pixel value of the template image, t′ i and u′ i is the gradient direction vector value of each point in the template image, and is the gradient direction vector value in the image to be matched.

[0068] This formula calculates the average dot product of the gradient vectors of corresponding edge points in the template image and the sub-image to be matched. The dot product reaches its maximum value when the gradients of a pair of corresponding edge points are in the same direction. When the gradients are in orthogonal directions, the dot product is zero. When the gradients are in opposite directions, the dot product reaches its negative maximum value. This provides a preliminary measure of template matching accuracy.

[0069] Step S320 , based on the candidate position of the template image in the image to be matched and the least squares adjustment theory, the candidate position is precision-adjusted to obtain the center point coordinates, rotation angle, and scaling factor of the template image in the image to be matched.

[0070] Through the least squares adjustment theory, more accurate matching information and rotation angle information can be calculated.

[0071] Figure 10 FIG. 1 is a flow chart of a template matching algorithm based on a sub-pixel image pyramid in the hand-eye calibration method of the present invention. Figure 10 As shown, the template matching algorithm in the method of the present invention includes:

[0072] In step 11, after obtaining the template image and target image, preprocess them separately, primarily by smoothing the images to reduce the effects of image noise. The target image is the image to be matched. Then, proceed to step 12.

[0073] In step 12, the preprocessed template image and target image are resized, i.e., the images are enlarged or reduced to obtain multiple images of different resolutions. The images are then arranged according to resolution, i.e., from low resolution to high resolution, to obtain image pyramids for the template image and the target image. Step 13 is then executed.

[0074] In step 13, sub-pixel edge detection and gradient direction extraction are performed on each layer in the image pyramid of the template image and the image pyramid of the target image, thereby making the matching of the template image in the target image more accurate. Then, step 14 is executed.

[0075] Step 14: Calculate the similarity metric function for the template image and the target image processed in step 13 to determine the accuracy of the matching of the template image in the target image, and then proceed to step 15. The calculation formula is:

[0076]

[0077] Where d′ i is the edge gradient vector in the template, q = [x, y] T is a pixel in the searched image, p′ is a pixel in the sub-image, e q+p′ Direction vector. ; n is the pixel value of the template image, t′ i and u′ i is the gradient direction vector value of each point in the template image, and is the gradient direction vector value of a certain position in the searched image.

[0078] Step 15, using the least squares adjustment theory, fine-tunes the positioning accuracy of the candidate position of the template image matched in step 14 in the target image, thereby obtaining a matching result with higher accuracy, and obtaining the center point coordinates, rotation angle and scaling factor of the template image in the target image.

[0079] The solution of the present invention improves traditional template matching. During the template matching process, the image is first smoothed to reduce the influence of image noise. Then, an image pyramid is used to quickly find a rough matching position at low resolution and perform accurate matching at high resolution, thereby reducing the matching time of image matching. Image edge detection is performed at a sub-pixel level, thereby making template matching more accurate. Even if the image undergoes changes such as rotation, movement, and scaling, image matching can still be performed accurately, making the template matching rotation-invariant and scale-invariant. Furthermore, through sub-pixel processing, the problems of jagged and blurred image edges are solved. Finally, the accuracy of overall positioning is fine-tuned by the least squares adjustment theory to obtain a final high-precision matching result, thereby making the relationship between the robot plane and the camera plane determined by hand-eye calibration more accurate.

[0080] Figure 5 FIG2 is a flow chart of an embodiment of determining the calibration center pixel coordinates of nine points according to the template matching algorithm in the method of the present invention. In some embodiments, in step S210, the specific process of determining the calibration center pixel coordinates of the nine points according to the calibration object images of the nine points and the preset template matching algorithm is as follows: Figure 5 As shown, it includes: step S510 and step S520.

[0081] Step S510: The image of the calibration object at the nine points is identified as an image to be matched, and the image containing only the calibration object is identified as a template image.

[0082] When performing template matching, it is necessary to determine the template image and the image to be matched, and then match the positions of the image content identical to the template image in the image to be matched. Therefore, the image containing these nine positions needs to be identified as the image to be matched.

[0083] In step S520, each of the images to be matched and the template image is respectively brought into a preset template matching algorithm to obtain the pixel coordinates of the calibration center point of the template image in each of the images to be matched, that is, the pixel coordinates of the calibration center points of the nine points are determined.

[0084] Using the improved template matching algorithm of this invention, the template image is matched with each image to be matched. This determines the coordinates of the template image's center point in the image to be matched. These coordinates are then used to determine the pixel coordinates of the calibration object's center point in the calibration object image. By processing the image using the image pyramid and sub-pixel edge detection used in the template matching algorithm, the resulting pixel coordinates of the calibration object's center point are more precise, ensuring high matching accuracy even after image rotation or scaling.

[0085] In step S140, while keeping the robot's camera still, the robot's robotic arm is controlled to grasp the calibration object and rotate m points at the same position, and the rotation center is determined according to the preset template matching algorithm; wherein m is a positive integer with a minimum value of 3, that is, the robotic arm grasps the calibration object and rotates at least three points at the same position, and the more points, the more accurate the fitting center point.

[0086] Figure 6 FIG1 is a flow chart of an embodiment of determining the rotation center in the method of the present invention. In some embodiments, in step S140, the robot's mechanical arm is controlled to grasp the calibration object and rotate m points at the same position, and the specific process of determining the rotation center is as follows: Figure 6 As shown, it includes: step S610 to step S640.

[0087] Step S610: Control the robot's robotic arm to grasp the calibration object and rotate m points, and obtain the calibration object image recorded by the camera at each of the m points, and the robot point coordinates recorded by the robot at each point.

[0088] Preferably, m = 5, meaning that by grasping the calibration object and rotating it by five points, a circle can be fitted, with the center of this circle being the center of rotation. By fitting a circle from multiple points, the more points are rotated, the more precise the fitted circle is, and thus the more accurate the determination of the center of rotation. The robot gripper grasps the calibration object and rotates it, ensuring that the robot's x, y, and z axis coordinates remain stationary, and the U axis coordinate is simply changed. By changing the U axis, the calibration object can be rotated to five different angles, such as 0°, 10°, 20°, -10°, and -20°.

[0089] Step S620 : identifying the m calibration object images as images to be matched, and identifying the image containing only the calibration object as a template image.

[0090] In step S630, each of the images to be matched and the template image is respectively brought into a preset template matching algorithm to obtain the pixel coordinates of the calibration center point of the template image in each of the images to be matched, that is, the pixel coordinates of the calibration center points of the m points are determined.

[0091] Step S640 , fitting to obtain the rotation center according to the pixel coordinates of the calibration center points of the m points and the robot point coordinates of the m points.

[0092] The solution of the present invention performs two template matching operations during the hand-eye calibration process to obtain a nine-point calibration matrix and a rotation center for correcting the nine-point calibration matrix. The nine-point calibration matrix is ​​corrected using the rotation center to make the relationship between the robot plane and the camera plane more accurate, thereby improving the accuracy of the robot hand-eye calibration in a simpler and more efficient manner.

[0093] In step S150 , the uncorrected nine-point calibration matrix is ​​corrected according to the rotation center to obtain a corrected nine-point calibration matrix.

[0094] In the process of the robot moving the calibration object, the calibration object may be rotated, which will cause errors in the obtained nine-point calibration matrix. Therefore, in order to eliminate the error, the template matching algorithm is used again to calculate the center of rotation, and the nine-point calibration matrix is ​​corrected using the rotation center, thereby eliminating the error caused by the rotation and making the final relationship between the robot plane and the camera plane more accurate.

[0095] Figure 9 FIG. 1 is a flow chart of the automatic nine-point rotation calibration method in the hand-eye calibration method of the present invention. Figure 9 As shown, the hand-eye calibration method of the present invention includes:

[0096] In step 1, adjust the intensity and position of the light source used for hand-eye calibration to determine the lighting scheme and avoid low image matching accuracy due to lighting issues. Also, adjust camera parameters, such as exposure and focal length, to ensure clear image data captured by the camera and avoid overexposure, underexposure, or blurred images. Then proceed to step 2.

[0097] In step 2, the robot is controlled to grasp a calibration object with distinct feature points and move it in a "田" (field) pattern within the camera's field of view. The robot moves to nine locations. At each location, the camera records an image of the calibration object at that location, generating an image of the calibration object. The robot also records the robot's coordinates at that location. Then, proceed to step 3.

[0098] In step 3, each calibration object image is fed into the modified template matching algorithm of this solution to obtain the pixel coordinates of the calibration object center point in each calibration object image, for a total of nine sets of pixel coordinates. Then, proceed to step 4.

[0099] Step 4: Calculate a nine-point calibration matrix based on the nine sets of robot coordinates obtained in step 2 and the nine sets of calibration object center pixel coordinates obtained in step 3.

[0100] In step 5, the robot is controlled to grab the calibration object and rotate it five times. Each time it rotates one point, the camera records the calibration object image, and the robot records the robot coordinates at that point. Then, proceed to step 6.

[0101] In step 6, the calibration object images at the five points in step 5 are respectively fed into the modified template matching algorithm of this solution to obtain the pixel coordinates of the calibration object center point in each calibration object image, for a total of five sets of pixel coordinates. Then, proceed to step 7.

[0102] Step 7: Fit the five sets of calibration object center pixel coordinates in step 6 and the five sets of robot coordinates in step 5 to obtain the true rotation center.

[0103] In step 8, the nine-point calibration matrix obtained in step 4 is corrected using the rotation center obtained in step 7 to make the nine-point calibration matrix more accurate.

[0104] The present invention's solution performs hand-eye calibration by having the robot grasp a calibration object, making the operation more concise and efficient, and avoiding the low precision caused by manual operation. During the hand-eye calibration process, the improved template matching algorithm of the present invention performs two matches: one to obtain a nine-point calibration matrix and another to obtain the rotation center for correcting the nine-point calibration matrix. This solves the problem of low precision of the nine-point calibration matrix after operations such as image movement, rotation, and scaling during the hand-eye calibration process, making the robot's hand-eye calibration more accurate and providing a more reliable visual system for subsequent robot control.

[0105] The technical solution of this embodiment is to control the robot's mechanical arm to grasp the calibration object while keeping the robot's camera stationary, and to move the robot's mechanical arm carrying the calibration object to nine preset positions within the camera's field of view; obtain the calibration object image recorded by the camera at each of the nine positions, and the robot position coordinates recorded by the robot at each position; obtain an uncorrected nine-point calibration matrix based on the calibration object images at the nine positions, the robot position coordinates at the nine positions, and a preset template matching algorithm; control the robot's mechanical arm to grasp the calibration object and rotate it m positions at the same position, and determine the rotation center based on the preset template matching algorithm; and correct the uncorrected nine-point calibration matrix based on the rotation center to obtain a corrected nine-point calibration matrix. This solves the problems of cumbersome manual operation and low calibration accuracy, and greatly improves the accuracy of hand-eye calibration.

[0106] According to an embodiment of the present invention, a control device for a voltage detection device corresponding to a hand-eye calibration method for a robot is also provided. The device is applied to a hand-eye calibration system of an industrial robot; the robot has a robotic arm and a camera. Figure 7 The control device of the voltage detection device may include: an acquisition unit 102 and a control unit 104 .

[0107] Control unit 104 is configured to control the robot's mechanical arm to grasp the calibration object while maintaining the robot's camera stationary, and to cause the robot's mechanical arm to move the calibration object within the camera's field of view to nine preset locations. The specific functions and processing of control unit 104 are described in step S110. To improve the accuracy of the calibration matrix, the number of preset locations is not limited to nine and can be increased as appropriate, such as to sixteen. The greater the number of locations, the higher the accuracy of the calibration matrix.

[0108] The captured calibration object must also have distinct feature points to facilitate subsequent template matching and improve template matching accuracy. Before controlling the robot's mechanical arm to grasp the calibration object, it is necessary to determine the lighting scheme by adjusting the light source intensity and position, and adjust the camera exposure and focus to ensure a clearer image.

[0109] The acquisition unit 102 is configured to acquire the calibration object image recorded by the camera at each of the nine points of the calibration object, as well as the robot point coordinates recorded by the robot at each of the nine points. The specific functions and processing of the acquisition unit 102 are described in step S120.

[0110] Each time the calibration object moves to a position, the camera records the image of the calibration object at that position, and the robot records the robot point coordinates when the calibration object is at that position, obtaining a total of nine sets of image and coordinate data.

[0111] The control unit 104 is further configured to obtain an uncorrected nine-point calibration matrix based on the nine-point calibration object images, the nine-point robot coordinates, and a preset template matching algorithm. The specific functions and processing of the control unit 104 are shown in step S130.

[0112] In some embodiments, the control unit 104 obtains an uncorrected nine-point calibration matrix based on the nine-point calibration object images, the nine-point robot coordinates, and a preset template matching algorithm, including:

[0113] The control unit 104 is further configured to determine the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and a preset template matching algorithm. The specific functions and processing of the control unit 104 are shown in step S210.

[0114] The control unit 104 is further configured to obtain an uncorrected nine-point calibration matrix based on the robot point coordinates of the nine points and the pixel coordinates of the calibration center points of the nine points. The specific functions and processing of the control unit 104 are shown in step S220.

[0115] The nine-point rotation calibration method performs an affine transformation on the robot coordinates at nine positions and the corresponding pixel coordinates in the image to determine the relationship between the robot plane and the camera plane. This relationship is the nine-point calibration matrix.

[0116] In some embodiments, the preset template matching algorithm includes:

[0117] The matching image and the template image are processed respectively to determine a candidate position of the template image in the image to be matched.

[0118] In some embodiments, processing the matching image and the template image separately to determine a candidate position of the template image in the image to be matched includes:

[0119] The image to be matched and the template image are smoothed respectively, thereby reducing the influence of noise on the image to be matched and the template image.

[0120] Figure 8 Schematic diagram of the structure of template matching in the method of the present invention, such as Figure 8 As shown in FIG, on an M×N image to be matched, the matching degree between the template image and the overlapping sub-image is calculated from left to right and from top to bottom. The greater the matching degree, the greater the possibility that the two are the same. The size of the possibility is mainly measured by the similarity metric function.

[0121] An image pyramid of the image to be matched and an image pyramid of the template image are obtained according to the smoothed image to be matched and the template image; wherein the image pyramid includes a low-resolution image and a high-resolution image of the image.

[0122] A first position of the template image in the image to be matched is determined based on a low-resolution image in the image pyramid of the image to be matched and a low-resolution image in the image pyramid of the template image; wherein the first position is an approximate position.

[0123] Determine a second position of the template image in the image to be matched based on a high-resolution image in the image pyramid of the image to be matched, a high-resolution image in the image pyramid of the template image, and the first position; wherein the second position is a more accurate position than the first position.

[0124] An image pyramid processes an original image into multiple new images of varying resolutions, arranged from low to high resolution. The number of images is also called the pyramid layers. The matching area is quickly located at low resolution, and precise pixel matching is then performed at high resolution. Because the approximate area is determined at low resolution, there's no need to search the entire image at high resolution, thus saving time.

[0125] Sub-pixel edge detection is performed on the image pyramid of the image to be matched and the image pyramid of the template image respectively.

[0126] During the camera imaging process, the image data obtained is the discretization of the image. Due to the capacity limitation of the photosensitive element itself, each pixel on the imaging surface only represents the nearby color. For example, there is a 4.5um spacing between the pixels on the two sensory elements, but they are connected together at the macro level, and there are countless tiny things between them at the micro level. These pixels between two actual physical pixels are called sub-pixels.

[0127] By detecting sub-pixel edges, the template image is positioned more accurately in the image to be matched, thereby improving the accuracy of template matching.

[0128] Based on the results of sub-pixel edge detection on the image to be matched and the template image, similarity calculation is performed to determine the candidate position of the template image in the image to be matched; the formula for similarity calculation is:

[0129]

[0130] Where d′ i is the edge gradient vector in the template image, q = [x, y] T is the pixel position of the image to be matched, p′ is the pixel of the sub-image in the image to be matched, e q+p′ is the direction vector; n is the pixel value of the template image, t′ i and u′ i is the gradient direction vector value of each point in the template image, and is the gradient direction vector value in the image to be matched.

[0131] This formula calculates the average dot product of the gradient vectors of corresponding edge points in the template image and the sub-image to be matched. The dot product reaches its maximum value when the gradients of a pair of corresponding edge points are in the same direction. When the gradients are in orthogonal directions, the dot product is zero. When the gradients are in opposite directions, the dot product reaches its negative maximum value. This provides a preliminary measure of template matching accuracy.

[0132] According to the candidate position of the template image in the image to be matched and the least square adjustment theory, the candidate position is adjusted for accuracy to obtain the center point coordinates, rotation angle and scaling factor of the template image in the image to be matched.

[0133] Through the least squares adjustment theory, more accurate matching information and rotation angle information can be calculated.

[0134] Figure 10 FIG. 1 is a flow chart of a template matching algorithm based on a sub-pixel image pyramid in the hand-eye calibration method of the present invention. Figure 10 As shown, the template matching algorithm in the method of the present invention includes:

[0135] In step 11, after obtaining the template image and target image, preprocess them separately, primarily by smoothing the images to reduce the effects of image noise. The target image is the image to be matched. Then, proceed to step 12.

[0136] In step 12, the preprocessed template image and target image are resized, i.e., the images are enlarged or reduced to obtain multiple images of different resolutions. The images are then arranged according to resolution, i.e., from low resolution to high resolution, to obtain image pyramids for the template image and the target image. Step 13 is then executed.

[0137] In step 13, sub-pixel edge detection and gradient direction extraction are performed on each layer in the image pyramid of the template image and the image pyramid of the target image, thereby making the matching of the template image in the target image more accurate. Then, step 14 is executed.

[0138] Step 14: Calculate the similarity metric function for the template image and the target image processed in step 13 to determine the accuracy of the matching of the template image in the target image, and then proceed to step 15. The calculation formula is:

[0139]

[0140] Where d′ i is the edge gradient vector in the template, q = [x, y] T is a pixel in the searched image, p′ is a pixel in the sub-image, e q+p′ Direction vector. ; n is the pixel value of the template image, t′ i and u′ i is the gradient direction vector value of each point in the template image, and is the gradient direction vector value of a certain position in the searched image.

[0141] Step 15, using the least squares adjustment theory, fine-tunes the positioning accuracy of the candidate position of the template image matched in step 14 in the target image, thereby obtaining a matching result with higher accuracy, and obtaining the center point coordinates, rotation angle and scaling factor of the template image in the target image.

[0142] The solution of the present invention improves traditional template matching. During the template matching process, the image is first smoothed to reduce the influence of image noise. Then, an image pyramid is used to quickly find a rough matching position at low resolution and perform accurate matching at high resolution, thereby reducing the matching time of image matching. Image edge detection is performed at a sub-pixel level, thereby making template matching more accurate. Even if the image undergoes changes such as rotation, movement, and scaling, image matching can still be performed accurately, making the template matching rotation-invariant and scale-invariant. Furthermore, through sub-pixel processing, the problems of jagged and blurred image edges are solved. Finally, the accuracy of overall positioning is fine-tuned by the least squares adjustment theory to obtain a final high-precision matching result, thereby making the relationship between the robot plane and the camera plane determined by hand-eye calibration more accurate.

[0143] In some embodiments, the control unit 104 determines the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and a preset template matching algorithm, including:

[0144] The control unit 104 is further configured to identify the calibration object image at the nine points as the image to be matched, and to identify the image containing only the calibration object as the template image. The specific functions and processing of the control unit 104 are shown in step S510.

[0145] When performing template matching, it is necessary to determine the template image and the image to be matched, and then match the positions of the image content identical to the template image in the image to be matched. Therefore, the image containing these nine positions needs to be identified as the image to be matched.

[0146] The control unit 104 is further configured to subject each of the to-be-matched images and the template image to a preset template matching algorithm to obtain the pixel coordinates of the calibration center point of the template image in each of the to-be-matched images, thereby determining the pixel coordinates of the calibration center points of the nine points. The specific functions and processing of the control unit 104 are described in step S520.

[0147] Using the improved template matching algorithm of this invention, the template image is matched with each image to be matched. This determines the coordinates of the template image's center point in the image to be matched. These coordinates are then used to determine the pixel coordinates of the calibration object's center point in the calibration object image. By processing the image using the image pyramid and sub-pixel edge detection used in the template matching algorithm, the resulting pixel coordinates of the calibration object's center point are more precise, ensuring high matching accuracy even after image rotation or scaling.

[0148] The control unit 104 is further configured to control the robot's robotic arm to grasp the calibration object and rotate it m times at the same position, while keeping the robot's camera stationary, and determine the rotation center based on the preset template matching algorithm. m is a positive integer with a minimum value of 3, meaning that the robotic arm grasps the calibration object and rotates it at least three times at the same position. The more times the calibration object is grasped, the more accurate the center point fitting. The specific functions and processing of the control unit 104 are described in step S140.

[0149] In some embodiments, the control unit 104 controls the robot's mechanical arm to grasp the calibration object and rotate m points at the same position, and determines the rotation center according to the preset template matching algorithm, including:

[0150] The control unit 104 is further configured to control the robot's mechanical arm to grasp the calibration object and rotate it through m positions, and to obtain an image of the calibration object recorded by the camera at each of the m positions, as well as the robot position coordinates recorded by the robot at each position. The specific functions and processing of the control unit 104 are described in step S610.

[0151] Preferably, m = 5, meaning that by grasping the calibration object and rotating it by five points, a circle can be fitted, with the center of this circle being the center of rotation. By fitting a circle from multiple points, the more points are rotated, the more precise the fitted circle is, and thus the more accurate the determination of the center of rotation. The robot gripper grasps the calibration object and rotates it, ensuring that the robot's x, y, and z axis coordinates remain stationary, and the U axis coordinate is simply changed. By changing the U axis, the calibration object can be rotated to five different angles, such as 0°, 10°, 20°, -10°, and -20°.

[0152] The control unit 104 is further configured to identify the m calibration object images as images to be matched, and to identify an image containing only the calibration object as a template image. Specific functions and processing of the control unit 104 are described in step S620.

[0153] The control unit 104 is further configured to subject each of the to-be-matched images and the template image to a preset template matching algorithm to obtain the pixel coordinates of the calibration center point of the template image in each of the to-be-matched images, thereby determining the pixel coordinates of the calibration center points of the m points. The specific functions and processing of the control unit 104 are described in step S630.

[0154] The control unit 104 is further configured to obtain the rotation center by fitting the pixel coordinates of the calibration center points of the m points and the robot point coordinates of the m points. The specific functions and processing of the control unit 104 are shown in step S640.

[0155] The solution of the present invention performs two template matching operations during the hand-eye calibration process to obtain a nine-point calibration matrix and a rotation center for correcting the nine-point calibration matrix. The nine-point calibration matrix is ​​corrected using the rotation center to make the relationship between the robot plane and the camera plane more accurate, thereby improving the accuracy of the robot hand-eye calibration in a simpler and more efficient manner.

[0156] The control unit 104 is further configured to correct the uncorrected nine-point calibration matrix according to the rotation center to obtain a corrected nine-point calibration matrix. The specific functions and processing of the control unit 104 are shown in step S150.

[0157] In the process of the robot moving the calibration object, the calibration object may be rotated, which will cause errors in the obtained nine-point calibration matrix. Therefore, in order to eliminate the error, the template matching algorithm is used again to calculate the center of rotation, and the nine-point calibration matrix is ​​corrected using the rotation center, thereby eliminating the error caused by the rotation and making the final relationship between the robot plane and the camera plane more accurate.

[0158] Figure 9 FIG. 1 is a flow chart of the automatic nine-point rotation calibration method in the hand-eye calibration method of the present invention. Figure 9 As shown, the hand-eye calibration method of the present invention includes:

[0159] In step 1, adjust the intensity and position of the light source used for hand-eye calibration to determine the lighting scheme and avoid low image matching accuracy due to lighting issues. Also, adjust camera parameters, such as exposure and focal length, to ensure clear image data captured by the camera and avoid overexposure, underexposure, or blurred images. Then proceed to step 2.

[0160] In step 2, the robot is controlled to grasp a calibration object with distinct feature points and move it in a "田" (field) pattern within the camera's field of view. The robot moves to nine locations. At each location, the camera records an image of the calibration object at that location, generating an image of the calibration object. The robot also records the robot's coordinates at that location. Then, proceed to step 3.

[0161] In step 3, each calibration object image is fed into the modified template matching algorithm of this solution to obtain the pixel coordinates of the calibration object center point in each calibration object image, for a total of nine sets of pixel coordinates. Then, proceed to step 4.

[0162] Step 4: Calculate a nine-point calibration matrix based on the nine sets of robot coordinates obtained in step 2 and the nine sets of calibration object center pixel coordinates obtained in step 3.

[0163] In step 5, the robot is controlled to grab the calibration object and rotate it five times. Each time it rotates one point, the camera records the calibration object image, and the robot records the robot coordinates at that point. Then, proceed to step 6.

[0164] In step 6, the calibration object images at the five points in step 5 are respectively fed into the modified template matching algorithm of this solution to obtain the pixel coordinates of the calibration object center point in each calibration object image, for a total of five sets of pixel coordinates. Then, proceed to step 7.

[0165] Step 7: Fit the five sets of calibration object center pixel coordinates in step 6 and the five sets of robot coordinates in step 5 to obtain the true rotation center.

[0166] In step 8, the nine-point calibration matrix obtained in step 4 is corrected using the rotation center obtained in step 7 to make the nine-point calibration matrix more accurate.

[0167] The present invention's solution performs hand-eye calibration by having the robot grasp a calibration object, making the operation more concise and efficient, and avoiding the low precision caused by manual operation. During the hand-eye calibration process, the improved template matching algorithm of the present invention performs two matches: one to obtain a nine-point calibration matrix and another to obtain the rotation center for correcting the nine-point calibration matrix. This solves the problem of low precision of the nine-point calibration matrix after operations such as image movement, rotation, and scaling during the hand-eye calibration process, making the robot's hand-eye calibration more accurate and providing a more reliable visual system for subsequent robot control.

[0168] Since the processing and functions implemented by the device of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned method, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0169] The technical solution of the present invention controls the robot's mechanical arm to grasp a calibration object while keeping the robot's camera stationary, and causes the robot's mechanical arm to move the calibration object to nine preset positions within the camera's field of view. The camera then obtains an image of the calibration object at each of the nine positions, as well as the robot's position coordinates recorded by the robot at each position. An uncorrected nine-point calibration matrix is ​​obtained based on the nine-position calibration object images, the nine-position robot coordinates, and a preset template matching algorithm. The robot's mechanical arm is controlled to grasp the calibration object and rotate m positions at the same position, and the rotation center is determined based on the preset template matching algorithm. The uncorrected nine-point calibration matrix is ​​corrected based on the rotation center to obtain a corrected nine-point calibration matrix. This solves the problems of cumbersome manual operation and low calibration accuracy, greatly improving the accuracy of hand-eye calibration.

[0170] According to an embodiment of the present invention, a robot corresponding to the control device of the voltage detection device is also provided. The robot may include: the control device of the voltage detection device described above.

[0171] Since the processing and functions implemented by the robot of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned devices, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0172] The technical solution of the present invention controls the robot's mechanical arm to grasp a calibration object while keeping the robot's camera stationary, and causes the robot's mechanical arm to move the calibration object to nine preset positions within the camera's field of view. The camera then obtains an image of the calibration object at each of the nine positions, as well as the robot's position coordinates recorded by the robot at each position. An uncorrected nine-point calibration matrix is ​​obtained based on the nine-position calibration object images, the nine-position robot coordinates, and a preset template matching algorithm. The robot's mechanical arm is controlled to grasp the calibration object and rotate m positions at the same position, and the rotation center is determined based on the preset template matching algorithm. The uncorrected nine-point calibration matrix is ​​corrected based on the rotation center to obtain a corrected nine-point calibration matrix. This solves the problems of cumbersome manual operation and low calibration accuracy, greatly improving the accuracy of hand-eye calibration.

[0173] According to an embodiment of the present invention, a storage medium corresponding to the robot hand-eye calibration method is also provided, wherein the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the robot hand-eye calibration method described above.

[0174] Since the processing and functions implemented by the storage medium of this embodiment basically correspond to the embodiments, principles and examples of the aforementioned method, for any details not fully described in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0175] The technical solution of the present invention controls the robot's mechanical arm to grasp a calibration object while keeping the robot's camera stationary, and causes the robot's mechanical arm to move the calibration object to nine preset positions within the camera's field of view. The camera then obtains an image of the calibration object at each of the nine positions, as well as the robot's position coordinates recorded by the robot at each position. An uncorrected nine-point calibration matrix is ​​obtained based on the nine-position calibration object images, the nine-position robot coordinates, and a preset template matching algorithm. The robot's mechanical arm is controlled to grasp the calibration object and rotate m positions at the same position, and the rotation center is determined based on the preset template matching algorithm. The uncorrected nine-point calibration matrix is ​​corrected based on the rotation center to obtain a corrected nine-point calibration matrix. This solves the problems of cumbersome manual operation and low calibration accuracy, greatly improving the accuracy of hand-eye calibration.

[0176] In summary, it is easy for those skilled in the art to understand that, under the premise of no conflict, the above-mentioned advantageous methods can be freely combined and superimposed.

[0177] The foregoing description is merely an embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of the claims.

Claims

1. A robot hand-eye calibration method, characterized in that: The method is applied to the hand-eye calibration system of industrial robots; The robot has a robotic arm and a camera; the method comprises: While keeping the robot's camera stationary, control the robot's mechanical arm to grab the calibration object, and move the robot's mechanical arm carrying the calibration object to nine preset points within the camera's field of view; Acquire the calibration object image recorded by the camera at each of the nine points of the calibration object, and the robot point coordinates recorded by the robot at each of the nine points; Obtaining an uncorrected nine-point calibration matrix based on the calibration object images at the nine points, the robot point coordinates at the nine points, and a preset template matching algorithm; While keeping the robot's camera stationary, control the robot's mechanical arm to grasp the calibration object and rotate m points at the same position, and determine the rotation center according to the preset template matching algorithm; where m is a positive integer; The uncorrected nine-point calibration matrix is ​​corrected according to the rotation center to obtain a corrected nine-point calibration matrix.

2. The robot hand-eye calibration method according to claim 1, characterized in that: According to the calibration object images of the nine points, the robot point coordinates of the nine points, and the preset template matching algorithm, an uncorrected nine-point calibration matrix is ​​obtained, including: Determine the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and a preset template matching algorithm; An uncorrected nine-point calibration matrix is ​​obtained according to the robot point coordinates of the nine points and the calibration center point pixel coordinates of the nine points.

3. The robot hand-eye calibration method according to claim 1 or 2, characterized in that: The preset template matching algorithm includes: Processing the image to be matched and the template image respectively to determine a candidate position of the template image in the image to be matched; According to the candidate position of the template image in the image to be matched and the least square adjustment theory, the candidate position is adjusted for accuracy to obtain the center point coordinates, rotation angle and scaling factor of the template image in the image to be matched.

4. The robot hand-eye calibration method according to claim 3, characterized in that: Processing the matching image and the template image separately to determine a candidate position of the template image in the image to be matched includes: Perform smoothing on the image to be matched and the template image respectively; Obtaining an image pyramid of the image to be matched and an image pyramid of the template image according to the smoothed image to be matched and the template image; wherein the image pyramid includes a low-resolution image and a high-resolution image of the image; determining a first position of the template image in the image to be matched according to the low-resolution image in the image pyramid of the image to be matched and the low-resolution image in the image pyramid of the template image; determining a second position of the template image in the image to be matched based on the high-resolution image in the image pyramid of the image to be matched, the high-resolution image in the image pyramid of the template image, and the first position; wherein the second position is a more accurate position than the first position; performing sub-pixel edge detection on the image pyramid of the image to be matched and the image pyramid of the template image respectively; Based on the results of sub-pixel edge detection on the image to be matched and the template image, similarity calculation is performed to determine the candidate position of the template image in the image to be matched; the formula for similarity calculation is: Among them, d i ′ is the edge gradient vector in the template image, q = [x, y] T is the pixel position of the image to be matched, p ′ is the pixel of the sub-image in the image to be matched, e q+p′ is the direction vector; n is the pixel value of the template image, t i ′ and u i ′ is the gradient direction vector value of each point in the template image, and is the gradient direction vector value in the image to be matched.

5. The robot hand-eye calibration method according to claim 2, characterized in that: Determining the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and a preset template matching algorithm includes: Identifying the calibration object images at the nine points as images to be matched, and identifying the image containing only the calibration object as a template image; Each of the images to be matched and the template image is respectively brought into the preset template matching algorithm to obtain the pixel coordinates of the calibration center point of the template image in each of the images to be matched, that is, the pixel coordinates of the calibration center point of the nine points are determined.

6. The robot hand-eye calibration method according to claim 1, characterized in that: Controlling the robot's mechanical arm to grasp the calibration object and rotate m points at the same position, and determining the rotation center according to the preset template matching algorithm, including: Controlling the robot's mechanical arm to grasp the calibration object and rotate m points, and obtaining the calibration object image recorded by the camera at each of the m points, as well as the robot point coordinates recorded by the robot at each point; Identifying the m calibration object images as images to be matched, and identifying an image containing only the calibration objects as a template image; Submitting each of the images to be matched and the template image into a preset template matching algorithm respectively to obtain the pixel coordinates of the calibration center point of the template image in each of the images to be matched, that is, determining the pixel coordinates of the calibration center points of the m points; The rotation center is obtained by fitting according to the pixel coordinates of the calibration center points of the m points and the robot point coordinates of the m points.

7. A robot hand-eye calibration device, characterized in that: The device is applied to the hand-eye calibration system of industrial robots; The robot has a mechanical arm and a camera; the device includes: a control unit configured to control the robot's mechanical arm to grasp the calibration object while keeping the robot's camera stationary, and to move the robot's mechanical arm carrying the calibration object to nine preset points within the field of view of the camera; an acquisition unit configured to acquire an image of the calibration object recorded by the camera at each of the nine positions of the calibration object, and a robot position coordinate recorded by the robot at each of the nine positions; The control unit is further configured to obtain an uncorrected nine-point calibration matrix based on the calibration object images at the nine points, the robot point coordinates at the nine points, and a preset template matching algorithm; The control unit is further configured to control the robot's mechanical arm to grasp the calibration object and rotate m points at the same position while keeping the robot's camera stationary, and determine the rotation center according to the preset template matching algorithm; wherein m is a positive integer; The control unit is further configured to correct the uncorrected nine-point calibration matrix according to the rotation center to obtain a corrected nine-point calibration matrix.

8. The robot hand-eye calibration device according to claim 7, characterized in that: The control unit obtains an uncorrected nine-point calibration matrix based on the calibration object images at the nine points, the robot point coordinates at the nine points, and a preset template matching algorithm, including: Determine the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and a preset template matching algorithm; An uncorrected nine-point calibration matrix is ​​obtained according to the robot point coordinates of the nine points and the calibration center point pixel coordinates of the nine points.

9. The robot hand-eye calibration device according to claim 7 or 8, characterized in that: The preset template matching algorithm includes: Processing the image to be matched and the template image respectively to determine a candidate position of the template image in the image to be matched; According to the candidate position of the template image in the image to be matched and the least square adjustment theory, the candidate position is adjusted for accuracy to obtain the center point coordinates, rotation angle and scaling factor of the template image in the image to be matched.

10. The robot hand-eye calibration device according to claim 9, characterized in that: Processing the matching image and the template image separately to determine a candidate position of the template image in the image to be matched includes: Perform smoothing on the image to be matched and the template image respectively; Obtaining an image pyramid of the image to be matched and an image pyramid of the template image according to the smoothed image to be matched and the template image; wherein the image pyramid includes a low-resolution image and a high-resolution image of the image; determining a first position of the template image in the image to be matched according to the low-resolution image in the image pyramid of the image to be matched and the low-resolution image in the image pyramid of the template image; determining a second position of the template image in the image to be matched based on the high-resolution image in the image pyramid of the image to be matched, the high-resolution image in the image pyramid of the template image, and the first position; wherein the second position is a more accurate position than the first position; performing sub-pixel edge detection on the image pyramid of the image to be matched and the image pyramid of the template image respectively; Based on the results of sub-pixel edge detection on the image to be matched and the template image, similarity calculation is performed to determine the candidate position of the template image in the image to be matched; the formula for similarity calculation is: Among them, d i ′ is the edge gradient vector in the template image, q = [x, y] T is the pixel position of the image to be matched, p ′ is the pixel of the sub-image in the image to be matched, e q+p′ is the direction vector; n is the pixel value of the template image, t i ′ and u i ′ is the gradient direction vector value of each point in the template image, and is the gradient direction vector value in the image to be matched.

11. The robot hand-eye calibration device according to claim 8, characterized in that: The control unit determines the pixel coordinates of the calibration center points of the nine points based on the calibration object images of the nine points and a preset template matching algorithm, including: Identifying the calibration object images at the nine points as images to be matched, and identifying the image containing only the calibration object as a template image; Each of the images to be matched and the template image is respectively brought into the preset template matching algorithm to obtain the pixel coordinates of the calibration center point of the template image in each of the images to be matched, that is, the pixel coordinates of the calibration center point of the nine points are determined.

12. The robot hand-eye calibration device according to claim 7, characterized in that: The control unit controls the robot's mechanical arm to grasp the calibration object and rotate m points at the same position, and determines the rotation center according to the preset template matching algorithm, including: Controlling the robot's mechanical arm to grasp the calibration object and rotate m points, and obtaining the calibration object image recorded by the camera at each of the m points, as well as the robot point coordinates recorded by the robot at each point; Identifying the m calibration object images as images to be matched, and identifying an image containing only the calibration objects as a template image; Submitting each of the images to be matched and the template image into a preset template matching algorithm respectively to obtain the pixel coordinates of the calibration center point of the template image in each of the images to be matched, that is, determining the pixel coordinates of the calibration center points of the m points; The rotation center is obtained by fitting according to the pixel coordinates of the calibration center points of the m points and the robot point coordinates of the m points.

13. A robot, characterized in that: include: The hand-eye calibration device for a robot according to any one of claims 7 to 12.

14. A storage medium, characterized in that The storage medium includes a stored program, wherein when the program is run, the device where the storage medium is located is controlled to execute the robot hand-eye calibration method according to any one of claims 1 to 6.

Images