A device and method for calibrating a TCP of a robot arm based on a binocular camera

Abstract

The application provides a device and method for calibrating a TCP of a mechanical arm based on a binocular camera, the device comprising an infrared binocular camera, a mechanical arm, a probe, an end tool, and an end array, the method using an initially inaccurate mechanical arm TCP to move to a target position reached by the inaccurate TCP, then using a visual servoing method to control the mechanical arm to move to an accurate target position in vision, calculating the difference between the two positions, compensating the difference into the initial inaccurate TCP data, and obtaining accurate TCP data, if the TCP data is replaced into the TCP data before the first movement, the target position in vision can be reached by one movement. In addition, the application calibrates the TCP of the mechanical arm through a series of matrix conversions based on the high-precision characteristics of the binocular camera, and a high calibration precision can be obtained, so that the execution precision of the whole mechanical arm system is improved, and the precision of the subsequent use of the mechanical arm is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of robotic arms, and more specifically to a device and method for calibrating the TCP of a robotic arm based on a binocular camera. Background Technology

[0002] Tool center point (TCP) calibration of a robotic arm is a crucial step in robotic arm applications, used to determine the relationship between the end effector and the robotic arm's coordinate system. There are generally two existing TCP calibration methods: one is a machining-based method, which requires machining a set of feature points with known spatial positions on a worktable, then measuring these points with the robotic arm to determine its posture and position; the other is an inverse kinematics-based method, which determines the robotic arm's position and posture through inverse kinematics. First, the transformation matrix between the end effector coordinate system and the robotic arm coordinate system is solved using the robotic arm's kinematic model. Then, by calibrating the known feature points on the workpiece, the computer can inversely solve for the robotic arm's position and posture.

[0003] However, calibration methods based on machining require special processing of the workpiece, which is time-consuming and costly. Calibration methods based on inverse kinematics are relatively simple, but their accuracy may not be high enough due to the nonlinear characteristics of the robotic arm. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art described in the background section and to provide a device and method for calibrating the TCP of a robotic arm based on a binocular camera. By using a binocular camera to calibrate the TCP of the robotic arm, the calibration accuracy of the TCP is improved by using visual algorithm technology.

[0005] The present invention is achieved through the following technical solution: providing a device for calibrating the TCP of a robotic arm based on a binocular camera, including an infrared binocular camera, a robotic arm, a probe, an end effector, and an end effector array.

[0006] The infrared binocular camera includes two lenses, left and right, which, as a visual navigator, can obtain spatial information of objects in the field of view through the principle of parallax, and is used to obtain the coordinate positions of the end array and the reflective ball on the probe.

[0007] Furthermore, the end flange of the robotic arm is connected to an end tool and an end array is mounted thereon.

[0008] Furthermore, one end of the end tool is connected to the end flange of the robotic arm, and the other end is connected to the specific actuator at the tool's center point.

[0009] Furthermore, the end effector array is connected to the end flange of the robotic arm and is equipped with reflective balls, which can be used as passive markers to identify the position of the infrared binocular camera as a first visual marker.

[0010] Furthermore, one end of the probe is equipped with a reflective ball, which serves as a passive marker point that can be identified by an infrared binocular camera and acts as a second visual marker. The other end, the probe tip, is spherical and used for end calibration.

[0011] This invention also provides a method for calibrating the TCP of a robotic arm based on a binocular camera, comprising the following steps: S101, Equipment preparation: Install the end-effector tool and end-effector array onto the end flange of the robotic arm; S102, End-effector calibration, obtain the relative positional relationship between the end-effector array and the center point of the end-effector tool Matrix_tool2flange_ndi, and record this relationship as the initial robot arm TCP data, represented by a matrix Matrix_tool2flange_original; S103, obtain the current pose of the robotic arm, set the initial robotic arm TCP data into the robotic arm controller, the robotic arm controller will send the current pose of the robotic arm (x,y,z,rx,ry,rz) in real time in the format of rotation vector, and convert the rotation vector into a rotation matrix by calculation, denoted as Matrix_currenttool2base; S104. Using an infrared binocular camera, a target position in a camera coordinate system is given, denoted as Matrix_targetflange2ndi. Combined with the transformation matrix Matrix_tool2flange_ndi of the end-effector array relative to the center point of the end-effector tool, the positional relationship of the center point of the target end-effector tool in the camera coordinate system is calculated and represented by a matrix Matrix_targettool2ndi. S105, by calculating the transformation relationship between each coordinate system, the transformation matrix Matrix_targettool2base of the center point of the target end tool in the robot arm base coordinate system is obtained, and the matrix is ​​converted into the rotation vector parameters required by the robot arm motion command; S106, control the movement of the robotic arm, move the center point of the end tool to the target position, and after the first movement is completed, record the position of the current end tool center point in the robotic arm base coordinate system, which is represented by a matrix Matrix_tool2base_original; S107 uses visual servoing technology to continuously calculate the pose of the end-effector center point in the robot arm base coordinate system based on the current pose of the end-effector center point in the visual and robot arm base coordinate system, and finally moves to the correct pose. The position of the end-effector center point in the robot arm base coordinate system at this time is recorded and represented by a matrix as Matrix_tool2base_final. S108: Calculate the relative positional relationship between the end-tool center points (Matrix_toolfinal2original) and (Matrix_tool2base_original) using Matrix_tool2base_final. Combine this with the transformation relationship matrix (Matrix_tool2flange_original) from S102 to calculate the final transformation relationship between the end-tool center point and the robot arm flange coordinate system, i.e., the robot arm TCP (tool center point), represented by the matrix Matrix_tool2flange, where Matrix_tool2flange = Matrix_tool2flange_original * Matrix_toolfinal2original.

[0012] Further, obtaining the relative positional relationship between the end effector array and the center point of the end effector tool in S102 includes: Use a probe to calibrate the tool center point of the end-effector; The infrared binocular camera acquires the position information of the probe and the position information of the end array in real time. The relative positional relationship between the end array and the center point of the end tool is calculated.

[0013] Furthermore, as described in S105, the transformation matrix of the target end-effector center point in the robot arm base coordinate system is obtained by calculating the transformation relationship between the coordinate systems, including: The position matrix of the current robotic arm end effector array in the camera coordinate system is denoted as Matrix_currentflange2ndi; The position of the end tool center point in the camera coordinate system is calculated from Matrix_tool2flange_ndi and Matrix_currentflange2ndi in S102, and is represented by the matrix Matrix_currenttool2ndi. Using S104 and Matrix_currenttool2ndi, the transformation relationship between the current end array position and the target end array position is calculated and represented by a matrix as Matrix_current2target; The pose transformation matrix Matrix_currenttool2base of the end-effector center point in the robot arm base coordinate system obtained from S103, combined with Matrix_current2target, can be calculated to obtain Matrix_targettool2base; The matrix is ​​converted into rotation vector parameters (x, y, z, rx, ry, rz) required for the robotic arm's motion commands. R is extracted from the matrix, and the rotation matrix R is converted into an axis-angle expression using the axis-angle expression method in the Eigen matrix library, denoted as rotation_vector(kx, ky, kz, theta), where rx = kx * theta, ry = ky * theta, rz = kz * theta, and x, y, z are T0, T1, and T2 corresponding to T in RT, respectively.

[0014] This invention provides a device and method for calibrating the TCP of a robotic arm based on a binocular camera. Compared with the prior art, this invention has the following technical advantages: As a high-precision product, the infrared binocular camera can achieve sub-millimeter level accuracy. Using a binocular camera to calibrate the TCP of the robotic arm can achieve a very high calibration accuracy, thereby improving the execution accuracy of the entire robotic arm system and providing an accuracy guarantee for the subsequent use of the robotic arm. The operation method is simple. The precise robotic arm TCP can be calculated after the robotic arm moves. It adopts an automated program, which is efficient and fast. The calibration method is simple, and calibration can be quickly performed anywhere after the tool is disassembled and reinstalled. Therefore, there are no high requirements for the machining accuracy of the end tool, which also reduces production costs. Attached Figure Description

[0015] The features, advantages and technical effects of exemplary embodiments of the present invention will now be described with reference to the accompanying drawings.

[0016] Figure 1 This is a schematic diagram of an infrared binocular camera provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the robotic arm provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the end-effector tool and end-effector array provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the end array provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the probe provided in an embodiment of the present invention; Figure 6 This is a schematic flowchart of a method for calibrating the TCP of a robotic arm based on a binocular camera, provided in an embodiment of the present invention. Detailed Implementation

[0017] The features and exemplary embodiments of various aspects of this disclosure will now be described in detail. To make the objectives, technical solutions, and advantages of this disclosure clearer, the disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are intended to explain this disclosure only and not to limit it. For those skilled in the art, this disclosure can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this disclosure by illustrating examples.

[0018] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0019] To better understand the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0020] This invention provides a device for calibrating the TCP of a robotic arm based on a binocular camera, including an infrared binocular camera, a robotic arm, a probe, an end effector, and an end effector array.

[0021] like Figure 1 As shown, the infrared binocular camera includes two lenses, left and right. As a visual navigator, it can obtain spatial information of objects in the field of view through the principle of parallax, and is used to obtain the coordinate positions of the end array and the reflective ball on the probe.

[0022] As an optional implementation, the infrared binocular camera can be one of an Atracsys camera or an NDIPolaris camera, or other cameras capable of recognizing spatial information of objects.

[0023] As an optional implementation, the robotic arm is as follows: Figure 2 As shown, the end effector flange of the robotic arm connects to the end effector tool and mounts the end effector array. A schematic diagram of the installation of the end effector tool and end effector array is shown below. Figure 3 As shown.

[0024] As an optional implementation, the robotic arm may be one of the following: UR robotic arm, Kuka robotic arm, Aubo robotic arm, etc.

[0025] As an optional implementation, one end of the end tool is connected to the end flange of the robotic arm, and the other end is connected to the specific actuator at the tool's center point.

[0026] As an optional implementation, the end array is as follows: Figure 4 As shown, it is used to connect to the flange at the end of the robotic arm and is equipped with a reflective ball. As a passive marker, its position can be identified by an infrared binocular camera and used as a primary visual marker.

[0027] As an alternative implementation, the end array can also be replaced with an array of three passive markers, or an array of other shapes.

[0028] like Figure 5 As shown, one end of the probe is equipped with a reflective ball, which serves as a passive marker point that can be identified by an infrared binocular camera and acts as a second visual marker. The other end, the probe tip, is spherical and used for end calibration.

[0029] like Figure 6 As shown, the present invention also provides a method for calibrating the TCP of a robotic arm based on a binocular camera, comprising the following steps: S101, Equipment preparation: Install the end-effector tool and end-effector array onto the end flange of the robotic arm; S102, End-effector calibration, obtain the relative positional relationship between the end-effector array and the center point of the end-effector tool Matrix_tool2flange_ndi, and record this relationship as the initial robot arm TCP data, represented by a matrix Matrix_tool2flange_original; S103, obtain the current pose of the robotic arm, set the initial robotic arm TCP data into the robotic arm controller, the robotic arm controller will send the current pose of the robotic arm (x,y,z,rx,ry,rz) in real time in the format of rotation vector, and convert the rotation vector into a rotation matrix by calculation, denoted as Matrix_currenttool2base; S104. Using an infrared binocular camera, a target position in a camera coordinate system is given, denoted as Matrix_targetflange2ndi. Combined with the transformation matrix Matrix_tool2flange_ndi of the end-effector array relative to the center point of the end-effector tool, the positional relationship of the center point of the target end-effector tool in the camera coordinate system is calculated and represented by a matrix Matrix_targettool2ndi. S105, by calculating the transformation relationship between each coordinate system, the transformation matrix Matrix_targettool2base of the center point of the target end tool in the robot arm base coordinate system is obtained, and the matrix is ​​converted into the rotation vector parameters required by the robot arm motion command; S106, control the movement of the robotic arm, move the center point of the end tool to the target position, and after the first movement is completed, record the position of the current end tool center point in the robotic arm base coordinate system, which is represented by a matrix Matrix_tool2base_original; S107 uses visual servoing technology to continuously calculate the pose of the end-effector center point in the robot arm base coordinate system based on the current pose of the end-effector center point in the visual and robot arm base coordinate system, and finally moves to the correct pose. The position of the end-effector center point in the robot arm base coordinate system at this time is recorded and represented by a matrix as Matrix_tool2base_final. S108: Calculate the relative positional relationship between the end-tool center points (Matrix_toolfinal2original) and (Matrix_tool2base_original) using Matrix_tool2base_final. Combine this with the transformation relationship matrix (Matrix_tool2flange_original) from S102 to calculate the final transformation relationship between the end-tool center point and the robot arm flange coordinate system, i.e., the robot arm TCP (tool center point), represented by the matrix Matrix_tool2flange, where Matrix_tool2flange = Matrix_tool2flange_original * Matrix_toolfinal2original.

[0030] As an optional implementation, obtaining the relative positional relationship between the end effector array and the center point of the end effector tool in S102 includes: Use a probe to calibrate the tool center point of the end-effector; The infrared binocular camera acquires the position information of the probe and the position information of the end array in real time. The relative positional relationship between the end array and the center point of the end tool is calculated.

[0031] As an optional implementation, the calculation of converting the rotation vector into a rotation matrix in S103 includes: Obtain the current pose (x, y, z, rx, ry, rz) of the robotic arm's end effector. Based on rx, ry, and rz in the pose, we can calculate (kx, ky, kz, theta) in the axis-angle expression of the pose representation method, where theta = sqrt(rx*rx + rry*ry + rz*rz), kx = rx / theta, ky = ry / theta, and kz = rz / theta. According to Rodriguez's formula, R = I + (sinθ)K + (1 - cosθ)K 2 Where K represents a matrix: ; Substituting the values ​​of kx, ky, kz, and θ into the formula yields the complete R; Construct a fourth-order homogeneous matrix based on x, y, z, and R, and obtain Matrix_currenttool2base.

[0032] As an optional implementation, step S105, which involves calculating the transformation relationship between various coordinate systems to obtain the transformation matrix of the target end-effector center point in the robot arm's base coordinate system, includes: The position matrix of the current robotic arm end effector array in the camera coordinate system is denoted as Matrix_currentflange2ndi; The position of the end tool center point in the camera coordinate system is calculated from Matrix_tool2flange_ndi and Matrix_currentflange2ndi in S102, and is represented by the matrix Matrix_currenttool2ndi. Using S104 and Matrix_currenttool2ndi, the transformation relationship between the current end array position and the target end array position is calculated and represented by a matrix as Matrix_current2target; The pose transformation matrix Matrix_currenttool2base of the end-effector center point in the robot arm base coordinate system obtained from S103, combined with Matrix_current2target, can be calculated to obtain Matrix_targettool2base; The matrix is ​​converted into rotation vector parameters (x, y, z, rx, ry, rz) required for the robotic arm's motion commands. R is extracted from the matrix, and the rotation matrix R is converted into an axis-angle expression using the axis-angle expression method in the Eigen matrix library, denoted as rotation_vector(kx, ky, kz, theta), where rx = kx * theta, ry = ky * theta, rz = kz * theta, and x, y, z are T0, T1, and T2 corresponding to T in RT, respectively.

[0033] This invention provides a device and method for calibrating the TCP of a robotic arm based on a binocular camera. Compared with the prior art, this invention has the following technical advantages: As a high-precision product, the infrared binocular camera can achieve sub-millimeter level accuracy. Using a binocular camera to calibrate the TCP of the robotic arm can achieve a very high calibration accuracy, thereby improving the execution accuracy of the entire robotic arm system and providing an accuracy guarantee for the subsequent use of the robotic arm. The operation method is simple. The precise robotic arm TCP can be calculated after the robotic arm moves. It adopts an automated program, which is efficient and fast. The calibration method is simple, and calibration can be quickly performed anywhere after the tool is disassembled and reinstalled. Therefore, the machining accuracy requirements for the end tool are not very high, which also reduces production costs. The above description is merely a specific embodiment of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be clarified that this disclosure is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this disclosure is not limited to the specific steps described and shown. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and these modifications or substitutions should all be covered within the protection scope of this disclosure.

Claims

1. A method for calibrating the TCP of a robotic arm based on a binocular camera, characterized in that, The method for calibrating the TCP of a robotic arm based on a binocular camera uses a device for calibrating the TCP of a robotic arm based on a binocular camera. The device for calibrating the TCP of a robotic arm based on a binocular camera includes: an infrared binocular camera, a robotic arm, a probe, an end effector, and an end effector array. The infrared binocular camera includes two lenses, left and right, which, as a visual navigator, can obtain spatial information of objects in the field of view through the principle of parallax, and is used to obtain the coordinate positions of the end array and the reflective ball on the probe. The end flange of the robotic arm is connected to an end tool and an end array is mounted thereon; One end of the end tool is connected to the end flange of the robotic arm, and the other end is connected to the specific actuator at the center point of the tool. The end effector array is connected to the end flange of the robotic arm and is equipped with reflective balls. As a passive marker, its position can be identified by an infrared binocular camera and used as a first visual marker. One end of the probe is equipped with a reflective ball, which serves as a passive marker point that can be identified by an infrared binocular camera and acts as a second visual marker. The other end, the probe tip, is spherical and used for end calibration. The method for calibrating the TCP of a robotic arm based on a binocular camera includes the following steps: S101, Equipment preparation: Install the end-effector tool and end-effector array onto the end flange of the robotic arm; S102, End-effector calibration, obtain the relative positional relationship between the end-effector array and the center point of the end-effector tool Matrix_tool2flange_ndi, and record this relationship as the initial robot arm TCP data, represented by a matrix Matrix_tool2flange_original; S103, obtain the current pose of the robotic arm, set the initial robotic arm TCP data into the robotic arm controller, the robotic arm controller will send the current pose of the robotic arm (x, y, z, rx, ry, rz) in real time in the format of rotation vector, and convert the rotation vector into a rotation matrix by calculation, denoted as Matrix_currenttool2base; S104. Using an infrared binocular camera, a target position in a camera coordinate system is given, denoted as Matrix_targetflange2ndi. Combined with the transformation matrix Matrix_tool2flange_ndi of the end-effector array relative to the center point of the end-effector tool, the positional relationship of the center point of the target end-effector tool in the camera coordinate system is calculated and represented by a matrix Matrix_targettool2ndi. S105, by calculating the transformation relationship between each coordinate system, the transformation matrix Matrix_targettool2base of the center point of the target end tool in the robot arm base coordinate system is obtained, and the matrix is ​​converted into the rotation vector parameters required by the robot arm motion command; S106, control the movement of the robotic arm, move the center point of the end tool to the target position, and after the first movement is completed, record the position of the current end tool center point in the robotic arm base coordinate system, which is represented by a matrix Matrix_tool2base_original; S107 uses visual servoing technology to continuously calculate the pose of the end-effector center point in the robot arm base coordinate system based on the current pose of the end-effector center point in the visual and robot arm base coordinate system, and finally moves to the correct pose. The position of the end-effector center point in the robot arm base coordinate system at this time is recorded and represented by a matrix as Matrix_tool2base_final. S108: Calculate the relative positional relationship between the end-effector center points (Matrix_tool2base_original and Matrix_tool2base_final) using Matrix_tool2base_original and Matrix_tool2base_final. Combine this with the transformation relationship matrix (Matrix_tool2flange_original) from S102 to calculate the final transformation relationship between the end-effector center point and the robotic arm flange coordinate system, i.e., the robotic arm TCP, represented by the matrix Matrix_tool2flange, where Matrix_tool2flange = Matrix_tool2flange_original * Matrix_toolfinal2original.

2. The method for calibrating the TCP of a robotic arm based on a binocular camera according to claim 1, characterized in that, The acquisition of the relative positional relationship between the end effector array and the center point of the end effector tool as described in S102 includes: Use a probe to calibrate the tool center point of the end-effector; The infrared binocular camera acquires the position information of the probe and the position information of the end array in real time. The relative positional relationship between the end array and the center point of the end tool is calculated.

3. The method for calibrating the TCP of a robotic arm based on a binocular camera according to claim 1, characterized in that, The transformation matrix of the target end-effector center point in the robot arm base coordinate system, obtained by calculating the transformation relationship between various coordinate systems as described in S105, includes: The position matrix of the current robotic arm end effector array in the camera coordinate system is denoted as Matrix_currentflange2ndi; The position of the end tool center point in the camera coordinate system is calculated from Matrix_tool2flange_ndi and Matrix_currentflange2ndi in S102, and is represented by the matrix Matrix_currenttool2ndi. Using S104 and Matrix_currenttool2ndi, the transformation relationship between the current end array position and the target end array position is calculated and represented by a matrix as Matrix_current2target; The pose transformation matrix Matrix_currenttool2base of the end-effector center point in the robot arm base coordinate system obtained from S103, combined with Matrix_current2target, can be calculated to obtain Matrix_targettool2base; The matrix is ​​converted into rotation vector parameters (x, y, z, rx, ry, rz) required for the robotic arm's motion commands. R is extracted from the matrix, and the rotation matrix R is converted into an axis-angle expression using the axis-angle expression method in the Eigen matrix library, denoted as rotation_vector(kx, ky, kz, theta), where rx = kx * theta, ry = ky * theta, rz = kz * theta, and x, y, z are T0, T1, and T2 corresponding to T in RT, respectively.

Images