A robot hand-eye calibration method, device, equipment and medium

Abstract

This application discloses a robot hand-eye calibration method, apparatus, device, and medium, comprising the following steps: acquiring point cloud data of a calibration tool scanned by a line laser camera; wherein the calibration tool has a polygonal cross-section, and a polygonal line is drawn on the surface of the calibration tool, and the line laser camera is fixed to the end effector of the robot; based on the point cloud data, acquiring the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system; wherein the intersection point is the inflection point of the polygonal line; controlling the end effector of the robot tool to move to the corresponding intersection point position, and acquiring the coordinates of the end effector of the robot tool at the corresponding intersection point position based on the current tool coordinate system to obtain the coordinates of the second intersection point; and acquiring the transformation matrix T from the current camera coordinate system to the current tool coordinate system based on the first intersection point coordinates and the second intersection point coordinates. This application has the advantages of a simpler calibration method and improved calibration efficiency.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a method, apparatus, device, and medium for robot hand-eye calibration. Background Technology

[0002] In a wide range of robotic applications, vision devices are responsible for environmental perception, while the robot, as the actuator, is responsible for outputting actions. Hand-eye calibration can transform environmental perception information from the camera coordinate system to the actuator coordinate system, thereby enabling the robot to effectively perceive the environment and control outputs.

[0003] Currently, there are many calibration methods for the combination of line laser cameras and robotic arm end-effectors with eyes attached to the hand. However, existing calibration methods are very complex in pursuit of high precision. They not only have demanding requirements for calibration boards and are difficult to manufacture, but the calibration methods themselves are also extremely complex to implement, with very cumbersome operation procedures and high data processing pressure, resulting in low calibration efficiency. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, and medium for robot hand-eye calibration, aiming to solve the technical problem of low efficiency in robot hand-eye calibration methods.

[0005] To achieve the above objectives, this application provides a robot hand-eye calibration method, comprising the following steps:

[0006] Acquire point cloud data of a calibration tool based on line laser camera scanning; wherein the calibration tool has a polygonal cross-section, the surface of the calibration tool is drawn with polygonal lines, and the line laser camera is fixed to the end effector of the robot;

[0007] Based on point cloud data, obtain the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system; where the intersection point is the inflection point of the polyline, and the current camera coordinate system is a coordinate system constructed based on the line laser camera at the current position;

[0008] The robot tool's end effector is moved to the corresponding intersection point. The coordinates of the robot tool's end effector at the corresponding intersection point are obtained based on the current tool coordinate system to obtain the coordinates of the second intersection point. The current tool coordinate system is a coordinate system constructed based on the robot tool's current position.

[0009] Based on the coordinates of the first and second intersection points, obtain the transformation matrix T that transforms the current camera coordinate system to the current tool coordinate system.

[0010] Optionally, the polygonal line can be W-shaped with three intersection points;

[0011] Based on the point cloud data, obtain the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system, including:

[0012] Linear fitting is performed on the point cloud data to obtain four line segments corresponding to the calibration tool;

[0013] Extract the three intersection points P0, P1, and P2 corresponding to the four line segments;

[0014] Obtain the coordinates of the first intersection point of the three intersection points P0, P1, and P2 in the current camera coordinate system.

[0015] Optionally, the coordinates of the first intersection point are transformed from the current camera coordinate system to the current tool coordinate system to obtain the coordinates of the second intersection point, including:

[0016] Obtain the transformation matrix of the current tool coordinate system when the end effector of the robot tool is translated to the three intersection points P0, P1, and P2 respectively. Where, transformation matrix This indicates a transformation from the current tool coordinate system to the initial tool coordinate system, which is a coordinate system built based on the robot tool's initial position.

[0017] Obtain the coordinates of the third intersection point in the initial tool coordinate system for each of the three intersection points P0, P1, and P2.

[0018] The coordinates of the third intersection point are determined according to the transformation matrix. Transform to the current tool coordinate system when the line laser camera takes the picture to obtain the coordinates of the second intersection point.

[0019] Optionally, the transformation matrix The expression is:

[0020]

[0021] Where R is a 3x3 rotation matrix, t calib It is a 3x1 translation vector.

[0022] Optionally, the coordinates of the first intersection point are denoted as:

[0023] p0_cam(x0_cam,y0_cam,0,1);

[0024] p1_cam(x1_cam,y1_cam,0,1);

[0025] p2_cam(x2_cam,y2_cam,0,1);

[0026] The coordinates of the second intersection point are denoted as:

[0027] p0_tool(x0_tool,y0_tool,z0_tool,1);

[0028] P1_tool(x1_tool,y1_tool,z1_tool,1);

[0029] P2_tool(x2_tool,y2_tool,z2_tool,1);

[0030] The transformation formula from the current camera coordinate system to the current tool coordinate system is:

[0031] [p0_tool,p1_tool,p2_tool] 4x3 =T*[p0_cam,p1_cam,p2_cam] 4x3 .

[0032] Optionally, based on the coordinates of the first intersection point and the second intersection point, the transformation matrix T for transforming the current camera coordinate system to the current tool coordinate system is obtained, including:

[0033] Based on the current camera coordinate system and the current tool coordinate system, obtain the vectors respectively. sum vector

[0034] Obtain vectors respectively sum vector The cross products are denoted as cross product v_cam and cross product v_tool, respectively.

[0035] By adding the cross product v_cam and the cross product v_tool to the transformation relation, we obtain the new transformation relation as follows:

[0036] [p0_tool,p1_tool,p2_tool,v_tool] 4x4 =T*[p0_cam,p1_cam,p2_cam,v_cam] 4x4 ;

[0037] Based on the new transformation relation, the transformation matrix T is obtained.

[0038] Optionally, before acquiring the point cloud data of the calibration tool based on line laser camera scanning, the following steps are also included:

[0039] Place the calibration tool on the workbench;

[0040] The end effector of the mobile robot ensures that the triangular field of view captured by the line laser camera is parallel to the cross-sectional direction of the calibration tool; wherein, the projection direction of the line laser camera is kept vertically downward, so that the light rays projected by the line laser camera coincide with the broken line drawn on the calibration tool.

[0041] To achieve the above objectives, this application also provides a robot hand-eye calibration device, comprising:

[0042] The point cloud acquisition module is used to acquire point cloud data of the calibration tool based on line laser camera scanning; wherein, the calibration tool has a polygonal cross-section, the surface of the calibration tool is drawn with polygonal lines, and the line laser camera is fixed to the end of the robot;

[0043] The first coordinate acquisition module is used to obtain the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system based on the point cloud data; wherein, the intersection point is the inflection point of the polyline, and the current camera coordinate system is a coordinate system constructed based on the line laser camera at the current position;

[0044] The second coordinate acquisition module is used to control the end effector of the robot tool to move to the corresponding intersection point position, and to obtain the coordinates of the end effector of the robot tool at the corresponding intersection point position based on the current tool coordinate system, so as to obtain the coordinates of the second intersection point; wherein, the current tool coordinate system is a coordinate system constructed based on the current position of the robot tool;

[0045] The matrix calculation module is used to obtain the transformation matrix T from the current camera coordinate system to the current tool coordinate system based on the coordinates of the first and second intersection points.

[0046] To achieve the above objectives, this application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0047] To achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program, on which a processor executes the computer program to implement the above-described method.

[0048] The beneficial effects that this application can achieve are as follows:

[0049] The hand-eye calibration in this application derives a transformation matrix T through calibration. This matrix T transforms points from the current camera coordinate system to the current tool coordinate system, and then back to the initial tool coordinate system, guiding the robot to perform specific actions at designated points. Therefore, this application uses a polygonal calibration tool to adjust the orientation of the linear laser camera during calibration. This tool consists of multiple V-grooves. If the projection direction of the linear laser camera is not parallel to the cross-section of the polygonal calibration tool, the difference between the projected linear laser and the polygonal line marked on the tool will be very obvious at the bottom of the V-grooves. Therefore, the orientation of the linear laser camera can be adjusted by visually observing the difference between the stripes projected by the linear laser and the polygonal calibration tool, saving the steps of controlling camera orientation through complex algorithms and improving efficiency. Furthermore, after obtaining the point cloud data from the scanning calibration tool, the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system can be obtained. Moreover, multiple points in the current camera coordinate system coincide with multiple inflection points of the polygonal line. Therefore, in... When the end effector of the mobile robot records the positions of multiple points in the robot's current tool coordinate system, the corresponding target inflection points are always clearly defined. Therefore, the end effector can be moved simply by translation, without rotational changes, only changes in translation amount. No additional annotation is required, making the process very simple and reliable. Then, the end effector is controlled to move to the corresponding intersection point position. Based on the current tool coordinate system, the coordinates of the end effector at the corresponding intersection point position are obtained, i.e., the coordinates of the second intersection point are obtained. Based on the coordinates of the first and second intersection points, the transformation matrix T for transforming the current camera coordinate system to the current tool coordinate system can be deduced in reverse, ensuring calibration accuracy. This transformation matrix T can be used to transform any point captured by the line laser camera at any position of the robot end effector to the current tool coordinate system. In summary, the calibration method of this application, while ensuring calibration accuracy, has a simpler calibration process, improves calibration efficiency, and has higher practicality. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0051] Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of this application;

[0052] Figure 2 This is a flowchart illustrating a robot hand-eye calibration method according to an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of the calibration tool with polylines drawn in an embodiment of this application;

[0054] Figure 4 This is a schematic diagram illustrating the principle of acquiring point cloud data of a calibration tool based on scanning with a line laser camera in an embodiment of this application.

[0055] Figure 5 This is a schematic diagram illustrating the extraction of three intersection points P0, P1, and P2 from the calibration tool in an embodiment of this application.

[0056] Figure 6 This is a schematic diagram illustrating the principle of a robot hand-eye calibration method according to an embodiment of this application.

[0057] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0060] In this application, unless otherwise expressly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0061] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0062] Example 1

[0063] Reference Figure 1 , Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in this embodiment, such as... Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0064] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0065] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and electronic programs.

[0066] exist Figure 1 In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the computer device of this embodiment can be set in the computer device. The computer device calls the robot hand-eye calibration device stored in the memory 1005 through the processor 1001 and executes the robot hand-eye calibration method provided in this embodiment.

[0067] Reference Figures 2-6 Based on the aforementioned hardware environment, this embodiment provides a robot hand-eye calibration method, including the following steps:

[0068] Step S100: Acquire point cloud data of the calibration tool based on line laser camera scanning; wherein, the cross-section of the calibration tool is polygonal, the surface of the calibration tool is drawn with polygonal lines, and the line laser camera is fixed to the end of the robot;

[0069] Step S200: Based on the point cloud data, obtain the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system; where the intersection point is the inflection point of the polyline, and the current camera coordinate system is a coordinate system constructed based on the line laser camera at the current position;

[0070] Step S300: Control the end effector of the robot tool to move to the corresponding intersection point position respectively, and obtain the coordinates of the end effector of the robot tool at the corresponding intersection point position based on the current tool coordinate system to obtain the second intersection point coordinates; wherein, the current tool coordinate system is a coordinate system constructed based on the robot tool at the current position;

[0071] Step S400: Based on the coordinates of the first intersection point and the second intersection point, obtain the transformation matrix T that transforms the current camera coordinate system to the current tool coordinate system.

[0072] The scenario where a line laser camera is attached to a robot's end effector and moves along with it is also known as an "eye-on-hand" structure. In this structure, the camera acts as the robot's eye, attached to the end effector of the robot arm and moving with it. Typically, the robot tool is calibrated, and its initial coordinate system is called the initial tool coordinate system. As the robot controls the arm to move and perform actions at specific positions, both the camera and the tool have their own coordinate systems. Hand-eye calibration involves calculating a 4x4 transformation matrix T. During subsequent operation, points in the current camera coordinate system are transformed to the current tool coordinate system, and then back to the initial tool coordinate system, guiding the robot to perform specific actions at designated points.

[0073] Therefore, in this embodiment, a polygonal calibration tool is used to adjust the attitude of the linear laser camera during calibration. The polygonal calibration tool is composed of multiple V-grooves. If the projection direction of the linear laser camera and the cross-section of the polygonal calibration tool are not parallel, the difference between the projected linear laser and the polygonal line marked on the calibration tool will be very obvious at the bottom of the V-grooves. Therefore, the attitude of the linear laser camera during calibration can be adjusted by visually observing the difference between the stripes projected by the linear laser and the polygonal calibration tool, thus saving the step of controlling the camera attitude through complex algorithms and improving efficiency. Based on this, after obtaining the point cloud data of the scanning calibration tool, the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system can be obtained. Moreover, multiple points in the current camera coordinate system coincide with multiple inflection points of the polygonal line. Therefore, when moving the machine... When the end effector records the positions of multiple points in the robot's current tool coordinate system, the corresponding target inflection points are always clearly defined. Therefore, the end effector can be moved simply by translation, without rotational changes, only changes in translation amount. No additional annotation is required, making the process very simple and reliable. Then, the end effector is controlled to move to the corresponding intersection point. Based on the current tool coordinate system, the coordinates of the end effector at the corresponding intersection point are obtained (coordinates can be read through a teach pendant), i.e., the coordinates of the second intersection point are obtained. Based on the coordinates of the first and second intersection points, the transformation matrix T for transforming the current camera coordinate system to the current tool coordinate system can be deduced in reverse, ensuring calibration accuracy. This transformation matrix T can be used to transform any point captured by the line laser camera at any position of the robot end effector to the current tool coordinate system. In summary, the calibration method of this application, while ensuring calibration accuracy, has a simpler calibration process, improves calibration efficiency, and has higher practicality.

[0074] As an alternative implementation, the polygonal line is W-shaped with three intersection points;

[0075] In step S200: Based on the point cloud data, obtain the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system, including:

[0076] Step S210: Perform line fitting on the point cloud data to obtain four line segments corresponding to the calibration tool;

[0077] Step S220: Extract the three intersection points P0, P1, and P2 corresponding to the four line segments;

[0078] Step S230: Obtain the coordinates of the first intersection point of the three intersection points P0, P1, and P2 in the current camera coordinate system.

[0079] In this embodiment, the calibration tool adopts a W-shape, which is a combination of two V-shapes, thus forming three intersection points. Its size and material are not particularly important. The angles between each side of the W-shaped calibration tool (i.e., the V-angles) are also not particularly important. A broken line is drawn on the surface of the calibration tool. Projecting this broken line downwards from directly above should result in a straight line. This is because the line projected by the line laser appears as a straight line when viewed from directly above, and the broken line on the W-shaped calibration tool also appears as a straight line when projected from directly above. Therefore, aligning the line projected by the line laser with the broken line on the W-shaped calibration tool is relatively easy compared to other line laser hand-eye calibration methods. The calibration board, with its W-shaped design, is a very simple calibration tool with almost no special requirements, thus reducing the complexity of calibration. Furthermore, the W-shaped design ensures that the three intersection points extracted by the calibration tool are coplanar but not collinear. If a planar calibration board were used, achieving this coplanar but non-collinear intersection point would require adjusting the position of the TCP point at the end of the tool using a teach pendant. The data generated by the W-shaped calibration board inherently possesses this coplanar but non-collinear characteristic. Using three or more V-shaped combinations results in more intersection points, increasing the difficulty of data processing. Therefore, using a W-shaped calibration tool with two V-shaped combinations satisfies the requirements for accurate calibration while simplifying the process as much as possible, thereby improving efficiency.

[0080] After obtaining the point cloud data corresponding to the W-shaped calibration tool, the specific format is a series of point cloud data (p1, p2, ..., p...). n ),p i =(x i ,y i ,z i Let i = 1, 2, 3...n, meaning there are n points in total. Each point consists of three coordinate values: x, y, and z. For the point cloud of line laser scanning, the RANSAC algorithm can be used to extract the fitted straight line to obtain the four sides of the W-shaped calibration tool. The W-shaped scanning data generated by the calibration tool is itself a natural four-segment structure, so it is very easy to extract the four sides. Arranged from left to right, they can be denoted as line0, line1, line2, and line3. By calculating the intersection points between the edges, three intersection points can be obtained. Calculate the intersection point of edge line0 and edge line1 to obtain p1, calculate the intersection point of edge line1 and edge line2 to obtain p0, and calculate the intersection point of edge line2 and edge line3 to obtain p2. These three intersection points are denoted as p0, p1, and p2, respectively. Then, the coordinates of these three intersection points are simplified by coplanarization to obtain the first intersection point coordinates of the three intersection points P0, P1, and P2 in the current camera coordinate system.

[0081] As an optional implementation, in step S300: controlling the end effector of the robot tool to move to the corresponding intersection point position, and obtaining the coordinates of the end effector of the robot tool at the corresponding intersection point position based on the current tool coordinate system to obtain the coordinates of the second intersection point, including:

[0082] Step S310: Obtain the transformation matrix of the current tool coordinate system when the end effector of the robot tool is translated to the three intersection points P0, P1, and P2 respectively. Where, transformation matrix This indicates a transformation from the current tool coordinate system to the initial tool coordinate system, which is a coordinate system built based on the robot tool's initial position.

[0083] Step S320: Obtain the coordinates of the third intersection point in the initial tool coordinate system for the three intersection points P0, P1, and P2 respectively;

[0084] Step S330: Calculate the coordinates of the third intersection point using the transformation matrix. Transform to the current tool coordinate system when the line laser camera takes the picture to obtain the coordinates of the second intersection point.

[0085] Where, transformation matrix The expression is:

[0086]

[0087] Where R is a 3x3 rotation matrix, t calib It is a 3x1 translation vector.

[0088] In this embodiment, the transformation matrix is ​​obtained first. Then, obtain the coordinates of the third intersection point in the initial tool coordinate system from the three intersection points P0, P1, and P2, and then apply the transformation matrix to the coordinates of the third intersection point. The coordinates of the three intersection points P0, P1, and P2 are transformed to the current tool coordinate system when the line laser camera takes pictures, so as to obtain the coordinates of the second intersection point after the transformation from the current camera coordinate system to the current tool coordinate system.

[0089] It should be noted that the information of the robot's end effector's current tool coordinate system during laser imaging can be recorded in homogeneous coordinates. This information represents the transformation matrix from the current tool coordinate system to the initial tool coordinate system. For example, the origin of the current tool coordinate system is denoted as p. calib =(0,0,0,1) T ,pass Transform to the initial tool coordinate system Then, when the robot tool end effector is moved to point p0 by translation and the current tool coordinate system information is recorded... The coordinates of point p0 in the initial tool coordinate system can then be calculated. Transforming this further to the current tool coordinate system used when the line laser camera took the picture, we get: in That is, the above transformation matrix The corresponding inverse transformation, repeated in the same way, yields the coordinates of point p1 in the current tool coordinate system. And the coordinates of point p2 in the current tool coordinate system are

[0090] Because the coordinates of points p0, p1, and p2 in the current tool coordinate system are calculated by translating the robot tool's end effector, there is no rotational change, only a change in translation amount. Therefore, the information is very similar to the image capture position, only the translation amount differs. Without the pre-built polyline on the W-shaped calibration tool, the line laser is bound to the robot tool's end effector when moving it, so the position of the line laser projection will also move with the robot tool's end effector, making it impossible to know the intersection point position before the movement. However, with the pre-built polyline on the W-shaped calibration tool, it can be aligned with the laser projection line, and the calculated coordinates of the three first intersection points in the camera coordinate system coincide with the three inflection points of the W-shaped polyline. Therefore, when recording the positions of these three points in the robot tool coordinate system by moving the robot tool's end effector, the three target inflection points are always clear, requiring no additional annotation, making the process simple and reliable.

[0091] As an optional implementation, the coordinates of the first intersection point are denoted as:

[0092] p0_cam(x0_cam,y0_cam,0,1);

[0093] p1_cam(x1_cam,y1_cam,0,1);

[0094] p2_cam(x2_cam,y2_cam,0,1);

[0095] The coordinates of the second intersection point are denoted as:

[0096] p0_tool(x0_tool,y0_tool,z0_tool,1);

[0097] P1_tool(x1_tool,y1_tool,z1_tool,1);

[0098] P2_tool(x2_tool,y2_tool,z2_tool,1);

[0099] The transformation formula from the current camera coordinate system to the current tool coordinate system is:

[0100] [p0_tool,p1_tool,p2_tool] 4x3 =T*[p0_cam,p1_cam,p2_cam] 4x3 ...

[0101] In this embodiment, when obtaining the coordinates of the three intersection points in the current camera coordinate system, they are denoted as: p0_cam(x0_cam,y0_cam,z0_cam,1), p1_cam(x1_cam,y1_cam,z1_cam,1), and p2_cam(x2_cam,y2_cam,z2_cam,1). Since the three intersection points are all within the current imaging plane of the line laser camera, the z-components are all 0, i.e., p0_cam(x0_cam,y0_cam,0,1), p1_cam(x1_cam,y1_cam,0,1), and p2_cam(x2_cam,y2_cam,0,1). After calculating the specific coordinates of p0_tool, p1_tool, p2_tool, and p0_cam, p1_cam, p2_cam, the transformation matrix T can be calculated by reverse calculation based on the above transformation relationship.

[0102] As an optional implementation, step S400: obtaining the transformation matrix T from the current camera coordinate system to the current tool coordinate system based on the coordinates of the first intersection point and the coordinates of the second intersection point, including:

[0103] Step S410: Obtain vectors based on the current camera coordinate system and the current tool coordinate system respectively. sum vector

[0104] Step S420: Obtain vectors respectively sum vector The cross products are denoted as cross product v_cam and cross product v_tool, respectively.

[0105] Step S430: Add the cross product v_cam and cross product v_tool to the transformation relation to obtain the new transformation relation as follows:

[0106] [p0_tool,p1_tool,p2_tool,v_tool] 4x4 =T*[p0_cam,p1_cam,p2_cam,v_cam] 4x4 ;

[0107] Based on the new transformation relation, the transformation matrix T is obtained.

[0108] In this embodiment, since the above transformation relation is an underdetermined equation, the transformation matrix T still cannot be solved. To make T easier to solve, an additional constraint relation can be added to the existing points to directly solve T. Here, consider vectors. sum vector The cross product v, which can be obtained in both the current camera coordinate system and the current tool coordinate system, namely v_cam and v_tool, respectively, yields the new transformation relation mentioned above, namely B. tool =T*A cam Given a matrix T in the form of invertible matrices, T can be solved. 4x4 =B tool *A cam -1 This allows us to calculate the calibration matrix (i.e., the transformation matrix T), which can be used to transform a point captured by a line laser camera at any position of the robot end effector to the current tool coordinate system.

[0109] Example of using the obtained transformation matrix T:

[0110] The robot's end effector reaches any reachable position in space; let the coordinate system information of the tool at this point be denoted as . A line laser camera attached to the end of the tool scans and images the work scene, and an algorithm is used to analyze and calculate a location point. First, transform this point from the current camera coordinate system to the current tool coordinate system using the transformation matrix T. Next, rotate this point to the robot's initial tool coordinate system. Sending this point to the robot allows the robot to control the end effector to move to that point and perform the specified operation.

[0111] As an optional implementation, step S100, before acquiring the point cloud data of the calibration tool based on line laser camera scanning, further includes:

[0112] Place the calibration tool on the workbench;

[0113] The end effector of the mobile robot ensures that the triangular field of view captured by the line laser camera is parallel to the cross-sectional direction of the calibration tool; wherein, the projection direction of the line laser camera is kept vertically downward, so that the light rays projected by the line laser camera coincide with the broken line drawn on the calibration tool.

[0114] Example 2

[0115] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a robot hand-eye calibration device, comprising:

[0116] The point cloud acquisition module is used to acquire point cloud data of the calibration tool based on line laser camera scanning; wherein, the calibration tool has a polygonal cross-section, the surface of the calibration tool is drawn with polygonal lines, and the line laser camera is fixed to the end of the robot;

[0117] The first coordinate acquisition module is used to obtain the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system based on the point cloud data; wherein, the intersection point is the inflection point of the polyline, and the current camera coordinate system is a coordinate system constructed based on the line laser camera at the current position;

[0118] The second coordinate acquisition module is used to control the end effector of the robot tool to move to the corresponding intersection point position, and to obtain the coordinates of the end effector of the robot tool at the corresponding intersection point position based on the current tool coordinate system, so as to obtain the coordinates of the second intersection point; wherein, the current tool coordinate system is a coordinate system constructed based on the current position of the robot tool;

[0119] The matrix calculation module is used to obtain the transformation matrix T from the current camera coordinate system to the current tool coordinate system based on the coordinates of the first and second intersection points.

[0120] The explanations and examples of each module in the device of this embodiment can be referred to the methods of the foregoing embodiments, and will not be repeated here.

[0121] Example 3

[0122] Based on the same inventive concept as the foregoing embodiments, this embodiment provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0123] Example 4

[0124] Based on the same inventive concept as the foregoing embodiments, this embodiment provides a computer-readable storage medium storing a computer program, and a processor executes the computer program to implement the above-described method.

[0125] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for robot hand-eye calibration, characterized in that, Includes the following steps: Point cloud data of a calibration tool based on line laser camera scanning is acquired; wherein the cross-section of the calibration tool is polygonal, the polygonal shape is W-shaped, the surface of the calibration tool is drawn with polygonal lines, and the line laser camera is fixed to the end effector of the robot; Based on the point cloud data, the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system are obtained; wherein, the intersection point is the inflection point of the polyline, there are three intersection points, and the current camera coordinate system is a coordinate system constructed based on the line laser camera at the current position; The end effector of the robot tool is moved to the corresponding intersection point position, and the coordinates of the end effector of the robot tool at the corresponding intersection point position are obtained based on the current tool coordinate system to obtain the second intersection point coordinates; wherein, the current tool coordinate system is a coordinate system constructed based on the robot tool at the current position; Based on the coordinates of the first intersection point and the coordinates of the second intersection point, obtain the transformation matrix T that transforms the current camera coordinate system to the current tool coordinate system.

2. The robot hand-eye calibration method as described in claim 1, characterized in that, The step of obtaining the coordinates of the first intersection point in the current camera coordinate system of multiple intersection points on the calibration tool based on the point cloud data includes: The point cloud data is fitted with straight lines to obtain four straight line segments corresponding to the calibration tool; Extract the three intersection points p0, p1, and p2 corresponding to the four line segments; Obtain the coordinates of the first intersection point of the three intersection points p0, p1, and p2 in the current camera coordinate system.

3. The robot hand-eye calibration method as described in claim 2, characterized in that, The control robot tool's end effector moves to the corresponding intersection point position, and the coordinates of the robot tool's end effector at the corresponding intersection point position are obtained based on the current tool coordinate system to obtain the second intersection point coordinates, including: Obtain the transformation matrix of the current tool coordinate system when the end effector of the robot tool is translated to the three intersection points p0, p1, and p2 respectively. Wherein, the transformation matrix This indicates a transformation from the current tool coordinate system to the initial tool coordinate system, which is a coordinate system constructed based on the robot tool's initial position. Obtain the coordinates of the third intersection point (p0, p1, p2) in the initial tool coordinate system; The coordinates of the third intersection point are determined according to the transformation matrix. Transform to the current tool coordinate system when the line laser camera takes the picture to obtain the coordinates of the second intersection point.

4. The robot hand-eye calibration method as described in claim 3, characterized in that, The transformation matrix The expression is: in, R It is a 3x3 rotation matrix. It is a 3x1 translation vector.

5. A robot hand-eye calibration method as described in claim 3 or 4, characterized in that, The coordinates of the first intersection point are denoted as: p0_cam(x0_cam,y0_cam,0,1); p1_cam(x1_cam,y1_cam,0,1); p2_cam(x2_cam,y2_cam,0,1); The coordinates of the second intersection point are denoted as: p0_tool(x0_tool,y0_tool,z0_tool,1); p1_tool(x1_tool,y1_tool,z1_tool,1); p2_tool(x2_tool,y2_tool,z2_tool,1); The transformation formula from the current camera coordinate system to the current tool coordinate system is as follows: 。 6. The robot hand-eye calibration method as described in claim 5, characterized in that, The step of obtaining the transformation matrix T from the current camera coordinate system to the current tool coordinate system based on the coordinates of the first intersection point and the coordinates of the second intersection point includes: Based on the current camera coordinate system and the current tool coordinate system, obtain the vectors respectively. sum vector ; Obtain the vectors respectively and the vector The cross products are denoted as cross product v_cam and cross product v_tool, respectively. Adding the cross product v_cam and the cross product v_tool to the transformation relation, we obtain the new transformation relation as follows: ； Based on the new transformation relation, the transformation matrix T is obtained.

7. The robot hand-eye calibration method as described in claim 1, characterized in that, Before acquiring the point cloud data of the calibration tool based on line laser camera scanning, the method further includes: Place the calibration tool on the workbench; The mobile robot end effector ensures that the triangular field of view captured by the line laser camera is parallel to the cross-sectional direction of the calibration tool; wherein, the projection direction of the line laser camera is kept vertically downward, so that the light rays projected by the line laser camera coincide with the broken line drawn on the calibration tool.

8. A robot hand-eye calibration device, characterized in that, include: A point cloud acquisition module is used to acquire point cloud data of a calibration tool based on line laser camera scanning; wherein, the calibration tool has a polygonal cross-section, the polygonal shape is W-shaped, the surface of the calibration tool is drawn with polygonal lines, and the line laser camera is fixed to the end effector of the robot; The first coordinate acquisition module is used to acquire the coordinates of the first intersection point of multiple intersection points on the calibration tool in the current camera coordinate system based on the point cloud data; wherein, the intersection point is the inflection point of the polyline, there are three intersection points, and the current camera coordinate system is a coordinate system constructed based on the line laser camera at the current position; The second coordinate acquisition module is used to control the end effector of the robot tool to move to the corresponding intersection position, and to acquire the coordinates of the end effector of the robot tool at the corresponding intersection position based on the current tool coordinate system, so as to obtain the second intersection coordinates; wherein, the current tool coordinate system is a coordinate system constructed based on the robot tool at the current position; The matrix calculation module is used to obtain the transformation matrix T from the current camera coordinate system to the current tool coordinate system based on the coordinates of the first intersection point and the coordinates of the second intersection point.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-6.

Images