Robot hand-eye calibration method and device, computer device and storage medium

Abstract

The application relates to a robot hand-eye calibration method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring first calibration data and second calibration data of calibration points on the straight sides of a calibration object; the first calibration data is collected by translating a robot flange, and the second calibration data is collected by translating and rotating the robot flange; based on the first calibration data of a first calibration point in the calibration points, a rotation matrix between a camera coordinate system and a flange end coordinate system is determined; according to the rotation matrix and the second calibration data of the first calibration point, a translation matrix between the camera coordinate system and the flange end coordinate system is determined; based on the calibration data of a plurality of groups of second calibration points in the calibration points, the rotation matrix and the translation matrix are optimized, and according to the target rotation matrix and the target translation matrix after optimization stops, a pose transformation matrix is constructed. The method can efficiently and accurately realize robot vision calibration (hand-eye calibration).

Description

Technical Field

[0001] This application relates to the field of visual calibration technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for robot hand-eye calibration. Background Technology

[0002] Visual servo-based robotic arms have been widely used in industrial automation production. They can accurately plan the movement path of the robotic arm through visual guidance, improve the intelligence level of the robotic arm, and effectively improve production efficiency.

[0003] Traditional technologies typically use calibration blocks for robot visual calibration (such as hand-eye calibration) to ensure the accurate operation of vision-servo-based robotic arms. For example, a sphere is used as a calibration block, and visual calibration is performed through the center of the sphere or calibration points on its surface. However, this calibration process requires time-consuming calculations of the point cloud on the calibration block's surface. Furthermore, when the robot changes pose and scans the point cloud data on the calibration block's surface, the surface features obtained from point cloud processing are not obvious, leading to inaccurate calibration results. Therefore, traditional technologies cannot achieve efficient and accurate robot visual calibration. Summary of the Invention

[0004] Therefore, it is necessary to provide a robot hand-eye calibration method, device, computer equipment, computer-readable storage medium, and computer program product that can efficiently and accurately achieve robot vision calibration, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a method for robot hand-eye calibration, including:

[0006] Acquire the first and second calibration data for each calibration point on the right-angled side of the calibration object; the first calibration data is acquired by translating the robot flange, and the second calibration data is acquired by translating and rotating the robot flange, with a camera installed at the end of the robot flange;

[0007] Select a set of first calibration points from each calibration point, and determine the rotation matrix between the camera coordinate system and the flange end coordinate system based on the first calibration data of each first calibration point.

[0008] Based on the rotation matrix and the second calibration data for each first calibration point, determine the translation matrix between the camera coordinate system and the flange end coordinate system;

[0009] Multiple sets of second calibration points are selected from each calibration point. Based on the first and second calibration data of the multiple sets of second calibration points, the rotation matrix and translation matrix are iteratively optimized. Based on the target rotation matrix and target translation matrix obtained after optimization stops, a pose transformation matrix for robot hand-eye calibration is constructed.

[0010] In one embodiment, the process of acquiring the first calibration data includes:

[0011] Determine the data acquisition location corresponding to each calibration point;

[0012] For each data acquisition location, when the camera is moved to the data acquisition location, the first calibration data of the corresponding calibration point in the camera coordinate system is obtained.

[0013] In one embodiment, the process of acquiring the second calibration data includes:

[0014] Determine the data acquisition location corresponding to each calibration point;

[0015] For each data acquisition location, when the camera is moved to the data acquisition location and the camera's data acquisition angle is different from the initial acquisition angle, the second calibration data of the corresponding calibration point in the camera coordinate system is obtained; the initial acquisition angle is the angle at which the camera acquires the first calibration data.

[0016] In one embodiment, a first set of calibration points is selected from the calibration points, including:

[0017] Determine the initial calibration point from among the calibration points, and use the remaining calibration points as non-initial calibration points;

[0018] Two non-initial calibration points are selected as a set of first calibration points.

[0019] In one embodiment, each calibration point includes an initial calibration point and a set of first calibration points; based on the first calibration data of each first calibration point, a rotation matrix between the camera coordinate system and the flange end coordinate system is determined, including:

[0020] For each first calibration point, based on the first calibration data of the first calibration point and the initial calibration point respectively, the first distance vector from the initial calibration point to the first calibration point is determined in the robot base coordinate system according to the coordinate transformation rules between the camera coordinate system and the robot base coordinate system.

[0021] Based on the parallelism between the first distance vectors corresponding to different first calibration points, the least squares method is used to determine the rotation matrix between the camera coordinate system and the flange end coordinate system.

[0022] In one embodiment, the translation matrix between the camera coordinate system and the flange end coordinate system is determined based on the rotation matrix and the second calibration data for each first calibration point, including:

[0023] Based on the second calibration data of each first calibration point and the coordinate transformation rules between the camera coordinate system and the robot base coordinate system, the second distance vector between different first calibration points in the robot base coordinate system is determined.

[0024] Based on the distance from the second distance vector to the right-angled side of the calibration object, and using the least squares method and the determined rotation matrix, the translation matrix between the camera coordinate system and the flange end coordinate system is determined.

[0025] Secondly, this application also provides a robot hand-eye calibration device, comprising:

[0026] The calibration data acquisition module is used to acquire the first calibration data and the second calibration data of each calibration point on the right-angled side of the calibration object; the first calibration data is acquired by translating the robot flange, and the second calibration data is acquired by translating and rotating the robot flange, with a camera installed at the end of the robot flange;

[0027] The rotation matrix determination module is used to select a set of first calibration points from each calibration point, and determine the rotation matrix between the camera coordinate system and the flange end coordinate system based on the first calibration data of each first calibration point.

[0028] The translation matrix determination module is used to determine the translation matrix between the camera coordinate system and the flange end coordinate system based on the rotation matrix and the second calibration data for each first calibration point.

[0029] The pose transformation matrix determination module is used to select multiple sets of second calibration points from each calibration point, and based on the first calibration data and second calibration data of the multiple sets of second calibration points, iteratively optimize the rotation matrix and translation matrix, and construct the pose transformation matrix for robot hand-eye calibration according to the target rotation matrix and target translation matrix obtained after optimization stops.

[0030] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described robot hand-eye calibration method.

[0031] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described robot hand-eye calibration method.

[0032] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described robot hand-eye calibration method.

[0033] The aforementioned robot hand-eye calibration method, apparatus, computer equipment, storage medium, and computer program product first acquire first calibration data and second calibration data for each calibration point on the right-angled side of the calibration object. The first calibration data is acquired through a translational robot flange, and the second calibration data is acquired through a translational and rotational robot flange. Then, a set of first calibration points is selected from each calibration point. Based on the first calibration data for each first calibration point, a rotation matrix between the camera coordinate system and the flange end coordinate system is determined. Then, based on the rotation matrix and the second calibration data for each first calibration point, a translation matrix between the camera coordinate system and the flange end coordinate system is determined. Finally, multiple sets of second calibration points are selected from each calibration point. Based on the first and second calibration data for multiple sets of second calibration points, the rotation matrix and translation matrix are iteratively optimized. Finally, based on the target rotation matrix and target translation matrix obtained after optimization, a pose transformation matrix for robot hand-eye calibration is constructed. Throughout the process, there is no need to spend a lot of time on the point cloud of the calibration object surface for calculation. Moreover, by using calibration objects with more obvious surface features and right-angled sides for calibration, the calibration accuracy can be improved. Furthermore, by iteratively optimizing the preliminary calibration results, the calibration accuracy can be further improved. Therefore, robot vision calibration, such as robot hand-eye calibration, can be achieved efficiently and accurately. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is an application environment diagram of the robot hand-eye calibration method in one embodiment;

[0036] Figure 2 This is a flowchart illustrating a robot hand-eye calibration method in one embodiment;

[0037] Figure 3 This is a schematic diagram of a camera acquiring calibration data in one embodiment;

[0038] Figure 4 This is a flowchart illustrating the robot hand-eye calibration method in another embodiment;

[0039] Figure 5 This is a structural block diagram of a robot hand-eye calibration device in one embodiment;

[0040] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] The robot hand-eye calibration method provided in this application can be applied to, for example... Figure 1 In the application environment shown, server 102 is connected to camera 104. Server 102 can acquire the first calibration data and the second calibration data collected by camera 104. The connection methods between server 102 and camera 104 include, but are not limited to, network communication and electrical connection. Server 102 can be implemented using a standalone server or a server cluster composed of multiple servers, and has functions such as image processing and data calculation. Camera 104 is a 2D (two-dimensional) vision camera, specifically a 2D line laser scanning camera, which can be used to measure key parameters such as the contour shape, size, and coordinate position of an object. Typically, 2D line laser scanning cameras are considered the "eyes" of robots. They can be mounted on the end of the robot flange or fixed in a position relative to the robot base, closely integrated with the production line. In this embodiment, camera 104 can be mounted on the end of the robot flange, serving as the "eye" mounted on the robot's "hand". The robot flange is installed at the end joint of the robot and can be used as a part to connect shafts to each other or to connect two devices. Different tools can be installed at the end of the robot flange to meet actual production needs, such as installing a camera as the robot's "eye". The camera is connected to the flange, and the flange is connected to the robot arm.

[0043] Specifically, server 102 can acquire first and second calibration data for each calibration point on the right-angled side of the calibration object through the connected camera 104. Then, it selects a set of first calibration points from each calibration point. Based on the first calibration data of each first calibration point, it determines the rotation matrix between the camera coordinate system and the flange end coordinate system. Then, based on the rotation matrix and the second calibration data of each first calibration point, it determines the translation matrix between the camera coordinate system and the flange end coordinate system. Finally, server 102 can select multiple sets of second calibration points from each calibration point. Based on the first and second calibration data of multiple sets of second calibration points, it iteratively optimizes the rotation matrix and translation matrix. Based on the target rotation matrix and target translation matrix obtained after optimization stops, it constructs a pose transformation matrix for robot hand-eye calibration.

[0044] In one exemplary embodiment, such as Figure 2 As shown, a robot hand-eye calibration method is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 202 to 208. Wherein:

[0045] Step 202: Obtain the first calibration data and the second calibration data for each calibration point on the right-angled side of the calibration object; the first calibration data is collected by translating the robot flange, and the second calibration data is collected by translating and rotating the robot flange, with a camera installed at the end of the robot flange.

[0046] The calibration object can be any object with right-angled sides, and the calibration points are located on the cross-section of the right-angled sides of the calibration object. The calibration object is located within the robot's range of motion so that the robot flange can be moved by driving the robot, and calibration data for each calibration point on the right-angled side of the calibration object can be collected by the camera. The first and second calibration data include: the coordinates of each calibration point in the camera coordinate system, collected by the camera in different postures.

[0047] Optionally, the server can connect to the camera and read the first and second calibration data of each calibration point on the right-angled side of the calibration object from the camera.

[0048] Step 204: Select a set of first calibration points from each calibration point, and determine the rotation matrix between the camera coordinate system and the flange end coordinate system based on the first calibration data of each first calibration point.

[0049] Optionally, the server can first determine the coordinate transformation rules from the camera coordinate system to the robot base coordinate system. These rules include the translation matrix (a parameter to be solved during robot hand-eye calibration) and rotation matrix (also a parameter to be solved during robot hand-eye calibration) from the camera coordinate system to the flange end coordinate system, as well as the translation and rotation matrices from the flange end coordinate system to the robot base coordinate system. The robot base coordinate system is constructed based on the robot base and can be determined according to the robot manufacturer's settings. The server can directly read and determine the robot base coordinate system from the robot controller, which stores the robot manufacturer's settings.

[0050] Furthermore, the server can randomly select two calibration points from each calibration point as a set of first calibration points. Then, it can substitute the first calibration data of each first calibration point into the coordinate transformation rules from the camera coordinate system to the robot base coordinate system, eliminate the unknown parameter of the translation matrix from the camera coordinate system to the flange end coordinate system, and calculate the rotation matrix from the camera coordinate system to the flange end coordinate system based on the correspondence between the coordinates of each first calibration point in the camera coordinate system and the robot base coordinate system.

[0051] Step 206: Determine the translation matrix between the camera coordinate system and the flange end coordinate system based on the rotation matrix and the second calibration data for each first calibration point.

[0052] Optionally, the server can substitute the second calibration data of each first calibration point and the calculated rotation matrix into the coordinate transformation rules from the camera coordinate system to the robot base coordinate system, and calculate the translation matrix from the camera coordinate system to the flange end coordinate system based on the correspondence between the coordinates of each first calibration point in the camera coordinate system and the robot base coordinate system.

[0053] Step 208: Select multiple sets of second calibration points from each calibration point. Based on the first and second calibration data of the multiple sets of second calibration points, iteratively optimize the rotation matrix and translation matrix. Based on the target rotation matrix and target translation matrix obtained after optimization stops, construct the pose transformation matrix for robot hand-eye calibration.

[0054] Both the target rotation matrix and the target translation matrix are 3x3 matrices.

[0055] Optionally, the server can randomly select any two second calibration points from each calibration point, forming multiple sets of second calibration points. Based on the first and second calibration data of these multiple sets of second calibration points, the server iteratively optimizes the rotation and translation matrices until the errors of the rotation and translation matrices after the latest few iterations are less than a set error threshold, at which point the optimization stops. Finally, the server can construct a 4*4 pose transformation matrix for robot hand-eye calibration based on the target rotation and target translation matrices obtained after the optimization stops, with the fourth row of the pose transformation matrix set to [0 0 0 1].

[0056] For example, the server can first group all calibration points pairwise to obtain multiple sets of calibration points, and then select one set as the first calibration point, and use the other sets as the second calibration points. It can then iteratively optimize the translation and rotation matrices obtained based on the first calibration points using each set of second calibration points. Alternatively, the server can first select one set of first calibration points, calculate the translation and rotation matrices based on the calibration data of that set of first calibration points, then select multiple sets of second calibration points, and iteratively optimize the calculated translation and rotation matrices using the calibration data of those sets.

[0057] The aforementioned robot hand-eye calibration method first acquires first and second calibration data for each calibration point on the right-angled side of the calibration object. The first calibration data is acquired by translating the robot flange, and the second calibration data is acquired by translating and rotating the robot flange. Then, a set of first calibration points is selected from each calibration point. Based on the first calibration data of each first calibration point, the rotation matrix between the camera coordinate system and the flange end coordinate system is determined. Then, based on the rotation matrix and the second calibration data of each first calibration point, the translation matrix between the camera coordinate system and the flange end coordinate system is determined. Finally, multiple sets of second calibration points are selected from each calibration point. Based on the first and second calibration data of multiple sets of second calibration points, the rotation matrix and translation matrix are iteratively optimized. Based on the target rotation matrix and target translation matrix obtained after optimization, a pose transformation matrix for robot hand-eye calibration is constructed. Throughout the process, there is no need to spend a lot of time on the point cloud of the calibration object surface for calculation. Moreover, by using calibration objects with more obvious surface features and right-angled sides for calibration, the calibration accuracy can be improved. Furthermore, by iteratively optimizing the preliminary calibration results, the calibration accuracy can be further improved. Therefore, robot vision calibration, such as robot hand-eye calibration, can be achieved efficiently and accurately.

[0058] In one exemplary embodiment, the process of acquiring the first calibration data includes:

[0059] Determine the data acquisition location corresponding to each calibration point;

[0060] For each data acquisition location, when the camera is moved to the data acquisition location, the first calibration data of the corresponding calibration point in the camera coordinate system is obtained.

[0061] The first calibration data includes an initial calibration point, a set of first calibration points, and the first calibration data for each of multiple sets of second calibration points.

[0062] Optionally, the server can determine the data acquisition position corresponding to each calibration point. For each data acquisition position, if it is determined that the robot has moved the camera to the data acquisition position through the translation flange, the server can obtain the first calibration data of the calibration point corresponding to the data acquisition position in the camera coordinate system from the calibration surface data acquired by the camera.

[0063] For example, such as Figure 3As shown, a schematic diagram illustrates a camera acquiring calibration data. The operating object can send a drive command to the robot controller, driving the robot to translate the flange, causing the camera to translate above the calibration object, thereby causing the laser line emitted by the camera to translate across the surface of the calibration object. When the laser line reaches the location of the calibration point, the camera reaches the data acquisition position corresponding to that calibration point. The data acquired by the camera at this data acquisition position on the surface of the calibration object is the first calibration data for that calibration point. In the figure... This indicates the coordinates of the calibration point in the camera coordinate system. p s =[ x s ,0, z s ] The coordinates of each calibration point in the camera coordinate system are as follows: p s[1] , p s[2] , , p s[n] ; This indicates the coordinates of the calibration point in the robot's base coordinate system. p b =[ x b , y b , z b ] The coordinates of each calibration point in the robot's base coordinate system are as follows: p b[1] , p b[2] , , p b[n] Since the camera is a 2D vision camera, therefore, Theoretically, the superpoint is the two-dimensional point cloud coordinate of the camera coordinate system. To better represent the coordinate correspondence between the camera coordinate system and the robot base coordinate system, the y-axis coordinate of the superpoint in the camera coordinate system can be represented by 0.

[0064] In this embodiment, the first calibration data of each calibration point on the right-angle side of the calibration object can be collected by translating only the robot flange. Then, by substituting the first calibration data into the coordinate transformation rules from the camera coordinate system to the robot base coordinate system, the translation matrix from the camera coordinate system to the flange end coordinate system can be eliminated. Then, based on the least squares method, the rotation matrix from the camera coordinate system to the flange end coordinate system can be calculated, that is, the rotation matrix required for robot hand-eye calibration can be determined.

[0065] In one exemplary embodiment, the process of acquiring the second calibration data includes:

[0066] Determine the data acquisition location corresponding to each calibration point;

[0067] For each data acquisition location, when the camera is moved to the data acquisition location and the camera's data acquisition angle is different from the initial acquisition angle, the second calibration data of the corresponding calibration point in the camera coordinate system is obtained; the initial acquisition angle is the angle at which the camera acquires the first calibration data.

[0068] The second calibration data includes an initial calibration point, a set of first calibration points, and multiple sets of second calibration points, each occupying a portion of the second calibration data.

[0069] Optionally, the server can first determine the data acquisition position corresponding to each calibration point. For each data acquisition position, if it is determined that the robot has moved the camera to the data acquisition position and the data acquisition angle of the camera is different from the initial acquisition angle (because the robot flange rotates, the data acquisition angle of the camera will change), the server can obtain the second calibration data of the calibration point corresponding to the data acquisition position in the camera coordinate system from the calibration object surface data acquired by the camera.

[0070] For example, the rotational posture of the robot flange can be multiple. The rotational posture of the robot flange can also be different at different data acquisition positions. That is, the robot can translate while continuously adjusting the rotation angle of the robot flange to collect second calibration data under multiple flange rotation directions, so as to make the second calibration data richer.

[0071] In this embodiment, second calibration data for each calibration point under different robot postures can be obtained, so that the translation matrix determined based on the first calibration point can be accurately iteratively optimized based on the second calibration data under different robot postures, thereby improving the accuracy of the final pose change matrix.

[0072] In an exemplary embodiment, selecting a first set of calibration points from the calibration points includes:

[0073] Determine the initial calibration point from among the calibration points, and use the remaining calibration points as non-initial calibration points;

[0074] Two non-initial calibration points are selected as a set of first calibration points.

[0075] Among them, when the robot drives the camera to scan the surface of the calibration object, the first calibration point scanned by the camera's laser line is the initial calibration point.

[0076] Optionally, the server can take the first calibration point scanned by the camera's laser line as the initial calibration point, and take the remaining calibration points other than the initial calibration point as non-initial calibration points, and randomly select two non-initial calibration points as a group of first calibration points.

[0077] It should be noted that when selecting multiple sets of second calibration points from each calibration point, the server also selects the second calibration points from each non-initial calibration point.

[0078] In this embodiment, the server can arbitrarily select two calibration points from each non-initial calibration point as a group, so that the translation matrix and rotation matrix from the camera coordinate system to the flange end coordinate system can be derived based on the first calibration data and the second calibration data of each selected calibration point, combined with the coordinate transformation rules from the camera coordinate system to the robot base coordinate system, thereby realizing robot hand-eye calibration.

[0079] In an exemplary embodiment, each calibration point includes an initial calibration point and a set of first calibration points; based on the first calibration data of each first calibration point, the rotation matrix between the camera coordinate system and the flange end coordinate system is determined, including:

[0080] For each first calibration point, based on the first calibration data of the first calibration point and the initial calibration point respectively, the first distance vector from the initial calibration point to the first calibration point is determined in the robot base coordinate system according to the coordinate transformation rules between the camera coordinate system and the robot base coordinate system.

[0081] Based on the parallelism between the first distance vectors corresponding to different first calibration points, the least squares method is used to determine the rotation matrix between the camera coordinate system and the flange end coordinate system.

[0082] The first calibration data for both the first calibration point and the initial calibration point can be read from the surface data of the calibration object acquired by the camera. The least squares method can find the best function match for the data by minimizing the sum of squares of the errors. In this embodiment, the least squares method can be used to easily obtain the unknown data and minimize the sum of squares of the errors between the obtained data and the actual data.

[0083] Optionally, for each first calibration point, the server can substitute the first calibration data of both the first calibration point and the initial calibration point into the coordinate transformation rules between the camera coordinate system and the robot base coordinate system to determine the first distance vector from the initial calibration point to the first calibration point in the robot base coordinate system. In the equation representing the first distance vector, the translation matrix from the camera coordinate system to the flange end coordinate system has been eliminated. Furthermore, based on the parallelism relationship between the first distance vectors corresponding to different first calibration points (i.e., the parallelism relationship between two first distance vectors), the server can use the least squares method to calculate the rotation matrix between the camera coordinate system and the flange end coordinate system.

[0084] For example, the coordinates of the calibration points on the right-angled side of the calibration object in the camera coordinate system are expressed as follows: p s[1] , p s[2] ... p s[n] For example, the coordinates of the initial calibration point in the camera coordinate system are: p s[1] If a set of first calibration points are the i-th and j-th calibration points respectively, then the coordinates of the first calibration points in the camera coordinate system are as follows: p s [i] , p s[j] The coordinate transformation rule between the camera coordinate system and the robot base coordinate system can be specifically shown in formula (1):

[0085] (1)

[0086] in, Represents the coordinates in the robot's base coordinate system. Represents the coordinates in the camera coordinate system. This represents the rotation matrix from the camera coordinate system to the flange end coordinate system. This represents the translation matrix from the camera coordinate system to the flange end coordinate system. and To achieve the required parameters for robot hand-eye calibration, This represents the rotation matrix from the flange end coordinate system to the robot base coordinate system. This represents the translation matrix from the flange end coordinate system to the robot base coordinate system.

[0087] Furthermore, in the robot's base coordinate system, the first distance vector from the initial calibration point to each first calibration point is shown in formulas (2) and (3), respectively:

[0088] (2)

[0089] (3)

[0090] in, This represents the first distance vector from the initial calibration point to the i-th calibration point in the robot's base coordinate system. This represents the first distance vector from the initial calibration point to the j-th calibration point in the robot's base coordinate system. Since all calibration points lie on the cross-section of the right-angled side of the calibration object, therefore... or It can represent the direction vector of the edge where the right-angled side of the calibration object is located in the camera coordinate system. or It can represent the motion vector generated by the translation of the robot flange in the robot's base coordinate system.

[0091] The initial calibration data for each calibration point was collected only when the robot flange was being translated. Therefore, during the data acquisition process, Nothing has changed; everything is the same as the robot's default state, but... There are changes, and This can be read from the manufacturer's settings parameters when the robot is in its default state. In representing the first distance vector... and In the middle, the translation matrix to be solved from the camera coordinate system to the flange end coordinate system is... It has been eliminated.

[0092] Furthermore, the server can use formulas (4) to (8), based on the parallelism relationship between the first distance vectors corresponding to different first calibration points, and employ the least squares method to determine the rotation matrix between the camera coordinate system and the flange end coordinate system:

[0093] (4)

[0094] Wherein, formula (4) represents the two first distance vectors and The parallelism relationship between them.

[0095] Furthermore, the rotation matrix between the camera coordinate system and the flange end coordinate system will be... Transform into As shown in formula (5):

[0096] (5)

[0097] in, It can be used to convert a matrix into a column vector.

[0098] Furthermore, combining with formula (4), formula (5) is transformed into the form shown in formula (6):

[0099] (6)

[0100] In formula (6), , , , .

[0101] Equation (6) can be expressed in linear form as shown in equation (7):

[0102] (7)

[0103] Based on formula (7), the least squares method is used to obtain... Solution As shown in formula (8):

[0104] (8)

[0105] Finally, the server can Substituting into formula (5), the solution for the rotation matrix is ​​obtained. .

[0106] In this embodiment, the rotation matrix can be used. Convert to column vector The solution is derived by considering the parallelism between the initial calibration point and the first distance vectors from different first calibration points. The least squares method is employed in the solution process, resulting in low computational complexity and high accuracy, enabling fast and accurate calculation of the distance vectors from the initial calibration point to different first calibration points. The solution is obtained, thereby enabling rapid and accurate determination of the... Solving this problem, i.e. determining the rotation matrix required for robot hand-eye calibration, is beneficial for achieving efficient and accurate robot hand-eye calibration.

[0107] In an exemplary embodiment, determining the translation matrix between the camera coordinate system and the flange end coordinate system based on the rotation matrix and the second calibration data for each first calibration point includes:

[0108] Based on the second calibration data of each first calibration point and the coordinate transformation rules between the camera coordinate system and the robot base coordinate system, the second distance vector between different first calibration points in the robot base coordinate system is determined.

[0109] Based on the distance from the second distance vector to the right-angled side of the calibration object, and using the least squares method and the determined rotation matrix, the translation matrix between the camera coordinate system and the flange end coordinate system is determined.

[0110] Optionally, the server can substitute the second calibration data of each first calibration point into the coordinate transformation rules between the camera coordinate system and the robot base coordinate system to determine the second distance vector between different first calibration points in the robot base coordinate system. Then, based on the distance of the second distance vector to the right-angle side of the calibration object, an equation is constructed. The determined rotation matrix is ​​substituted into the equation, and the least squares method is used to solve it, thereby determining the translation matrix between the camera coordinate system and the flange end coordinate system.

[0111] For example, let a set of first calibration points be the i-th and j-th calibration points respectively, and the coordinates of the first calibration points in the camera coordinate system are as follows: p s [i] , p s[j] Taking this as an example, the server can first determine the second distance vector between different first calibration points in the robot's base coordinate system. .in, , The second calibration data for each calibration point was collected while the robot flange was being translated and rotated. Therefore, during the data acquisition process, All of these are subject to change.

[0112] Furthermore, the server can use formulas (9) to (12), based on the distance from the second distance vector to the right-angled side of the calibration object, and using the least squares method and the determined rotation matrix, to determine the translation matrix between the camera coordinate system and the flange end coordinate system. Specifically, the solution of the rotation matrix from the camera coordinate system to the flange end coordinate system will be... Substitution As shown in formula (9):

[0113] (9)

[0114] in, It can be: Substitution , back, and Any one of them. Since the calibration points are all located on the cross-section of the right-angled side of the calibration object, therefore, It can characterize the cross section where the right-angled side of the calibration object is located.

[0115] Furthermore, the distance from the second distance vector to the right-angled side of the calibration object is shown in formula (10):

[0116] (10)

[0117] Simplify formula (10) to the form shown in formula (11):

[0118] (11)

[0119] In formula (11), , .

[0120] Finally, using the least squares method and based on formula (11), the solution of the translation matrix from the camera coordinate system to the flange end coordinate system is obtained. As shown in formula (12):

[0121] (12)

[0122] In this embodiment, the least squares method is used, which results in low computational complexity and high accuracy, enabling rapid and accurate calculation of... Solving this problem allows us to determine the rotation matrix required for robot hand-eye calibration, which is beneficial for achieving efficient and accurate robot hand-eye calibration.

[0123] In one embodiment, such as Figure 4 As shown, a flowchart of another robot hand-eye calibration method is provided, which mainly includes the following steps:

[0124] Step 402: Establish a hand-eye calibration system; the hand-eye calibration system includes a calibration object with right-angled sides, a camera coordinate system, a flange end coordinate system, and a robot base coordinate system;

[0125] Step 404: Initialize hand-eye calibration parameters; such as determining calibration points, data acquisition locations of calibration points, determining robot manufacturer settings, transformation equations from camera coordinate system to robot base coordinate system, etc.

[0126] Step 406: Perform attitude sampling on the camera; subsequently, the camera can be controlled to collect data from the surface of the target object according to different attitudes.

[0127] Step 408: Based on the first calibration data and transformation equations acquired by the camera, the least squares method is used to calculate the rotation matrix from the camera coordinate system to the flange end coordinate system.

[0128] Step 410: Based on the second calibration data acquired by the camera, the transformation equation, and the calculated rotation matrix, the least squares method is used to calculate the translation matrix from the camera coordinate system to the flange end coordinate system.

[0129] Step 412: Based on the first and second calibration data collected by the camera in different poses, the rotation matrix and translation matrix are iteratively optimized. Based on the iteratively optimized target rotation matrix and target translation matrix, a pose transformation matrix from the camera coordinate system to the flange end coordinate system is constructed.

[0130] In this embodiment, using any calibration object with right-angled sides, equations can be established based on the correspondence between the coordinates of each calibration point on the right-angled side of the calibration block in the camera coordinate system and the robot base coordinate system, thereby calculating the pose transformation matrix and achieving hand-eye calibration. Compared with the traditional method of using spheres for calibration, the calculation process based on the least squares method in this embodiment is simpler, easier to understand, has lower computational load, higher execution efficiency, and higher accuracy. It can solve calibration problems, which is beneficial for the robotic arm to accurately perform tasks and can improve the automation level of the entire production process.

[0131] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0132] Based on the same inventive concept, this application also provides a robot hand-eye calibration device for implementing the robot hand-eye calibration method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more robot hand-eye calibration device embodiments provided below can be found in the limitations of the robot hand-eye calibration method described above, and will not be repeated here.

[0133] In one exemplary embodiment, such as Figure 5 As shown, a robot hand-eye calibration device is provided, including: a calibration data acquisition module 502, a rotation matrix determination module 504, a translation matrix determination module 506, and a pose transformation matrix determination module 508, wherein:

[0134] The calibration data acquisition module is used to acquire the first calibration data and the second calibration data of each calibration point on the right-angled side of the calibration object; the first calibration data is acquired by translating the robot flange, and the second calibration data is acquired by translating and rotating the robot flange, with a camera installed at the end of the robot flange;

[0135] The rotation matrix determination module is used to select a set of first calibration points from each calibration point, and determine the rotation matrix between the camera coordinate system and the flange end coordinate system based on the first calibration data of each first calibration point.

[0136] The translation matrix determination module is used to determine the translation matrix between the camera coordinate system and the flange end coordinate system based on the rotation matrix and the second calibration data for each first calibration point.

[0137] The pose transformation matrix determination module is used to select multiple sets of second calibration points from each calibration point, and based on the first calibration data and second calibration data of the multiple sets of second calibration points, iteratively optimize the rotation matrix and translation matrix, and construct the pose transformation matrix for robot hand-eye calibration according to the target rotation matrix and target translation matrix obtained after optimization stops.

[0138] In the aforementioned robot hand-eye calibration device, firstly, first calibration data and second calibration data are acquired for each calibration point on the right-angled side of the calibration object. The first calibration data is acquired by translating the robot flange, and the second calibration data is acquired by translating and rotating the robot flange. Then, a set of first calibration points is selected from each calibration point. Based on the first calibration data of each first calibration point, the rotation matrix between the camera coordinate system and the flange end coordinate system is determined. Then, based on the rotation matrix and the second calibration data of each first calibration point, the translation matrix between the camera coordinate system and the flange end coordinate system is determined. Finally, multiple sets of second calibration points are selected from each calibration point. Based on the first and second calibration data of multiple sets of second calibration points, the rotation matrix and translation matrix are iteratively optimized. Based on the target rotation matrix and target translation matrix obtained after optimization stops, a pose transformation matrix for robot hand-eye calibration is constructed. Throughout the process, there is no need to spend a lot of time on the point cloud of the calibration object surface for calculation. Moreover, by using calibration objects with more obvious surface features and right-angled sides for calibration, the calibration accuracy can be improved. Furthermore, by iteratively optimizing the preliminary calibration results, the calibration accuracy can be further improved. Therefore, robot vision calibration, such as robot hand-eye calibration, can be achieved efficiently and accurately.

[0139] In one embodiment, the calibration data acquisition module is further configured to determine the data acquisition position corresponding to each calibration point; and for each data acquisition position, when the camera is moved to the data acquisition position, to acquire the first calibration data of the calibration point corresponding to the data acquisition position in the camera coordinate system.

[0140] In one embodiment, the calibration data acquisition module is further configured to determine the data acquisition position corresponding to each calibration point; for each data acquisition position, when the camera is moved to the data acquisition position and the camera's data acquisition angle is different from the initial acquisition angle, the second calibration data of the calibration point corresponding to the data acquisition position in the camera coordinate system is acquired; the initial acquisition angle is the angle at which the camera acquires the first calibration data.

[0141] In one embodiment, the rotation matrix determination module is further configured to determine an initial calibration point from each calibration point, use the remaining calibration points as non-initial calibration points, and select two non-initial calibration points as a first set of calibration points.

[0142] In one embodiment, each calibration point includes an initial calibration point and a set of first calibration points; the rotation matrix determination module is further configured to, for each first calibration point, determine, based on the first calibration data of the first calibration point and the initial calibration point respectively, and in accordance with the coordinate transformation rules between the camera coordinate system and the robot base coordinate system, the first distance vector from the initial calibration point to the first calibration point in the robot base coordinate system; and, based on the parallelism relationship between the first distance vectors corresponding to different first calibration points, determine the rotation matrix between the camera coordinate system and the flange end coordinate system using the least squares method.

[0143] In one embodiment, the translation matrix determination module is further configured to determine a second distance vector between different first calibration points in the robot base coordinate system based on the second calibration data for each first calibration point and the coordinate transformation rules between the camera coordinate system and the robot base coordinate system; and to determine the translation matrix between the camera coordinate system and the flange end coordinate system based on the distance from the second distance vector to the right-angled side of the calibration object, using the least squares method and the determined rotation matrix.

[0144] Each module in the aforementioned robot hand-eye calibration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0145] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores robot hand-eye calibration data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a robot hand-eye calibration method.

[0146] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0147] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the robot hand-eye calibration method described above.

[0148] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the robot hand-eye calibration method described above.

[0149] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the robot hand-eye calibration method described above.

[0150] It should be noted that the information (including but not limited to robot equipment information) and data (including but not limited to data used for analysis, data stored, data displayed) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for robot hand-eye calibration, characterized in that, The method includes: Acquire first and second calibration data for each calibration point on the right-angled side of the calibration object; the first calibration data is acquired by translating the robot flange, and the second calibration data is acquired by translating and rotating the robot flange, with a camera mounted at the end of the robot flange; Select a set of first calibration points from each of the calibration points. For each first calibration point, based on the first calibration data of the first calibration point and the initial calibration point, determine the first distance vector from the initial calibration point to the first calibration point in the robot base coordinate system according to the coordinate transformation rules between the camera coordinate system and the robot base coordinate system. Based on the parallelism between the first distance vectors corresponding to different first calibration points, the least squares method is used to determine the rotation matrix between the camera coordinate system and the flange end coordinate system; Based on the second calibration data for each of the first calibration points and the coordinate transformation rules between the camera coordinate system and the robot base coordinate system, a second distance vector between different first calibration points in the robot base coordinate system is determined. Based on the distance from the second distance vector to the right-angled side of the calibration object, and using the least squares method and the determined rotation matrix, the translation matrix between the camera coordinate system and the flange end coordinate system is determined. Multiple sets of second calibration points are selected from the calibration points. Based on the first calibration data and the second calibration data of the multiple sets of second calibration points, the rotation matrix and the translation matrix are iteratively optimized. Based on the target rotation matrix and the target translation matrix obtained after the optimization stops, a pose transformation matrix for robot hand-eye calibration is constructed.

2. The method according to claim 1, characterized in that, The process of acquiring the first calibration data includes: Determine the data acquisition location corresponding to each of the aforementioned calibration points; For each data acquisition location, when the camera is moved to the data acquisition location, the first calibration data of the calibration point corresponding to the data acquisition location in the camera coordinate system is obtained.

3. The method according to claim 1, characterized in that, The process of acquiring the second calibration data includes: Determine the data acquisition location corresponding to each of the aforementioned calibration points; For each data acquisition location, when the camera is moved to the data acquisition location and the data acquisition angle of the camera is different from the initial acquisition angle, the second calibration data of the calibration point corresponding to the data acquisition location in the camera coordinate system is obtained; the initial acquisition angle is the angle at which the camera acquires the first calibration data.

4. The method according to claim 1, characterized in that, The step of selecting a first set of calibration points from the calibration points includes: Determine the initial calibration point from each of the calibration points, and use the remaining calibration points as non-initial calibration points; Two of the non-initial calibration points are selected as a set of first calibration points.

5. A robot hand-eye calibration device, characterized in that, The device includes: The calibration data acquisition module is used to acquire first calibration data and second calibration data for each calibration point on the right-angled side of the calibration object; the first calibration data is acquired by translating the robot flange, and the second calibration data is acquired by translating and rotating the robot flange, and a camera is installed at the end of the robot flange; The rotation matrix determination module is used to select a set of first calibration points from the calibration points, and for each first calibration point, based on the first calibration data of the first calibration point and the initial calibration point, determine the first distance vector from the initial calibration point to the first calibration point in the robot base coordinate system according to the coordinate transformation rules between the camera coordinate system and the robot base coordinate system; and based on the parallelism relationship between the first distance vectors corresponding to different first calibration points, determine the rotation matrix between the camera coordinate system and the flange end coordinate system using the least squares method. The translation matrix determination module is used to determine the second distance vector between different first calibration points in the robot base coordinate system based on the second calibration data of each first calibration point and the coordinate transformation rules between the camera coordinate system and the robot base coordinate system; and to determine the translation matrix between the camera coordinate system and the flange end coordinate system based on the distance of the second distance vector to the right-angle side of the calibration object, using the least squares method and the determined rotation matrix. The pose transformation matrix determination module is used to select multiple sets of second calibration points from each of the calibration points, and based on the first calibration data and the second calibration data of the multiple sets of second calibration points, iteratively optimize the rotation matrix and the translation matrix, and construct the pose transformation matrix for robot hand-eye calibration according to the target rotation matrix and the target translation matrix obtained after the optimization stops.

6. The apparatus according to claim 5, characterized in that, The calibration data acquisition module is also used for: Determine the data acquisition location corresponding to each of the aforementioned calibration points; For each data acquisition location, when the camera is moved to the data acquisition location, the first calibration data of the calibration point corresponding to the data acquisition location in the camera coordinate system is obtained.

7. The apparatus according to claim 5, characterized in that, The calibration data acquisition module is also used for: Determine the data acquisition location corresponding to each of the aforementioned calibration points; For each data acquisition location, when the camera is moved to the data acquisition location and the data acquisition angle of the camera is different from the initial acquisition angle, the second calibration data of the calibration point corresponding to the data acquisition location in the camera coordinate system is obtained; the initial acquisition angle is the angle at which the camera acquires the first calibration data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images