Robot hand-eye coordinate conversion method and device, computer device and storage medium

Abstract

The application relates to a robot hand-eye coordinate conversion method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring an image of an end effector on which a calibration board is installed by a scanner of a robot, to obtain calibration board images of the end effector in multiple poses; when the difference value between the calibration board images in multiple poses does not fall into a difference value interval, controlling the end effector to run in a direction corresponding to the difference value, and acquiring an image of a target calibration board by the scanner during the running process; determining a tool origin position of the robot according to the positions of the mark points in the target calibration board image; and calibrating a coordinate conversion relationship between the end effector and the scanner based on the tool origin position. The method can accurately obtain the tool origin position in the robot coordinate system, and then accurately calibrate the coordinate conversion relationship between the end effector and the scanner.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for robot hand-eye coordinate transformation. Background Technology

[0002] With the development of artificial intelligence technology, robots have been widely used in various industries. In industrial applications, robots possess visual perception systems. Using the three-dimensional information acquired by these systems, robots can control end effectors to perform machining and installation tasks. Simply put, the three-dimensional perception system is like the human eye, and the end effector is like the human hand; through the coordination of hand and eye, pre-set tasks are completed.

[0003] To ensure that the robot accurately moves objects in space to their target positions, it is necessary to determine the transformation relationship between the vision system coordinate system and the robot arm coordinate system. Traditional methods for determining the transformation relationship have poor accuracy, and the results obtained will deviate significantly from the actual true values. Summary of the Invention

[0004] Therefore, it is necessary to provide a robot hand-eye coordinate transformation method, device, computer equipment, computer-readable storage medium, and computer program product that can improve accuracy in addressing the above-mentioned technical problems.

[0005] Firstly, this application provides a method for robot hand-eye coordinate transformation. The method includes:

[0006] The end effector with a calibration plate mounted on the robot is imaged using the robot's scanner, resulting in calibration plate images of the end effector in multiple poses.

[0007] When the difference value between the multiple pose calibration board images does not fall within the difference value range, the end effector is controlled to run in the direction corresponding to the difference value, and during the operation, the end effector is image acquired by the scanner to obtain the target calibration board image;

[0008] The tool origin position of the robot is determined based on the scan values ​​corresponding to the marker points in the target calibration plate image;

[0009] Based on the origin position of the tool, the coordinate transformation relationship between the end effector and the scanner is determined.

[0010] In one embodiment, controlling the end effector to move in the direction corresponding to the difference value, and during the operation, acquiring an image of the end effector via the scanner to obtain a target calibration plate image, includes:

[0011] Based on the respective marker point positions of the calibration board images of the multiple poses, scan values ​​of the calibration board images of the multiple poses are generated respectively.

[0012] Generate a difference vector based on the scan values ​​of the calibration plate image at each pose;

[0013] The end effector is controlled to operate in the opposite direction of the difference vector;

[0014] During operation, the scanner acquires images of the end effector to obtain an image of the target calibration board.

[0015] In one embodiment, generating scan values ​​for the calibration board images of the plurality of poses based on the respective marker point positions of the calibration board images of the plurality of poses includes:

[0016] Obtain the position of the marker point corresponding to the marker point in the calibration board image of the multiple poses;

[0017] The positions of the marker points in each pose are averaged to obtain the scan values ​​of the calibration plate images in each pose.

[0018] In one embodiment, generating a difference vector based on the scan values ​​of the calibration board image at each pose includes:

[0019] The initial coordinate transformation relationship between the end effector and the scanner is obtained by calculating the homogeneous transformation matrices of each pose in the tool coordinate system and the homogeneous transformation matrices of each pose in the scanner coordinate system.

[0020] According to the initial coordinate transformation relationship, the scan values ​​of the calibration plate images of adjacent poses are combined and transformed into the robot coordinate system, and the difference vector of each group is generated based on the scan values ​​of adjacent poses of each combination after transformation.

[0021] In one embodiment, determining the tool origin position of the robot based on the scan value corresponding to the marker point in the target calibration plate image includes:

[0022] The scan value corresponding to the marker point in the target calibration board image is used as the initial working origin position;

[0023] Transform the initial working origin position to the robot coordinate system to obtain the target tool origin position in the robot coordinate system;

[0024] The step of determining the coordinate transformation relationship between the end effector and the scanner based on the origin position of the tool includes:

[0025] Based on the origin position of the target tool, the coordinate transformation relationship between the end effector and the scanner is determined.

[0026] In one embodiment, determining the coordinate transformation relationship between the end effector and the scanner based on the tool origin position includes:

[0027] The calibration plate image for pose matching is obtained based on the origin position of the tool under different poses;

[0028] Based on the set of marker points corresponding to the calibration plate images in different poses, the coordinate rotation transformation relationship and translation transformation relationship between the end effector and the scanner are calculated.

[0029] In one embodiment, the step of acquiring images of the end effector with a calibration plate mounted on the robot using the robot's scanner includes:

[0030] The position of the calibration plate in the robot coordinate system is kept within a preset range, and the movement of the end effector is controlled so that the calibration plate undergoes multiple pose changes.

[0031] For each pose change of the end effector, images are acquired using the robot's scanner.

[0032] Secondly, this application also provides a robot hand-eye coordinate transformation device. The device includes:

[0033] The image acquisition module is used to acquire images of the end effector on the robot with a calibration plate mounted on it using the robot's scanner, and obtain calibration plate images of the end effector in multiple poses.

[0034] The adjustment module is used to control the end effector to run in the direction corresponding to the difference value when the difference value between the multiple pose calibration board images does not fall within the difference value range, and to acquire images of the end effector through the scanner during the operation to obtain the target calibration board image.

[0035] The origin calibration module is used to determine the tool origin position of the robot based on the scan value corresponding to the marker point in the target calibration board image;

[0036] The transformation relationship determination module is used to determine the coordinate transformation relationship between the end effector and the scanner based on the origin position of the tool.

[0037] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the robot hand-eye coordinate transformation steps in any of the above embodiments.

[0038] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the robot hand-eye coordinate transformation steps in any of the above embodiments.

[0039] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the robot hand-eye coordinate transformation steps in any of the above embodiments.

[0040] The aforementioned robot hand-eye coordinate transformation method, device, computer equipment, storage medium, and computer program product, when the difference value between the calibration board images of the multiple poses does not fall within the difference value range, controls the end effector to run in the direction corresponding to the difference value, and during the operation, the scanner acquires images of the end effector to obtain a target calibration board image; based on the scan value corresponding to the marker point in the target calibration board image, the tool origin position of the robot is determined, replacing the traditional teaching pendant calibration process using a needle tip. Setting the robot origin coordinates can avoid manual alignment errors and the limitation of only being able to set the tool coordinates to a feature position that can be recognized by the human eye, so as to obtain the tool origin position in the robot coordinate system more accurately, and thus more accurately calibrate the coordinate transformation relationship between the end effector and the scanner. Attached Figure Description

[0041] Figure 1 This is an application environment diagram of the robot hand-eye coordinate transformation method in one embodiment;

[0042] Figure 2 This is a flowchart illustrating a robot hand-eye coordinate transformation method in one embodiment;

[0043] Figure 3 This is a schematic diagram showing the distribution of marker points on a marker board in one embodiment.

[0044] Figure 4 This is a flowchart illustrating the robot hand-eye coordinate transformation method in another embodiment;

[0045] Figure 5 This is a schematic diagram of the process for generating a target calibration board image in another embodiment;

[0046] Figure 6 This is a structural block diagram of a robot hand-eye coordinate transformation device in one embodiment;

[0047] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] The robot hand-eye coordinate transformation method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 uses the robot's scanner to acquire images of the end effector with a calibration plate mounted on it, obtaining calibration plate images of the end effector in multiple poses. When the difference values ​​between the multiple pose calibration plate images do not fall within the difference value range, the terminal controls the end effector to move in the direction corresponding to the difference value, and during the movement, it acquires images of the end effector through the scanner to obtain a target calibration plate image. Based on the scan values ​​corresponding to the marker points in the target calibration plate image, the tool origin position of the robot is determined. Based on the tool origin position, the coordinate transformation relationship between the end effector and the scanner is calibrated.

[0050] The terminal 102 can be, but is not limited to, various robots, personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0051] In one embodiment, such as Figure 2 As shown, a robot hand-eye coordinate transformation method is provided, which can be applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0052] Step 202: Using the robot's scanner, images of the end effector on which the calibration plate is mounted are acquired, resulting in calibration plate images of the end effector in multiple poses.

[0053] The robot comprises a robot body, a scanner, and an end effector. The robot body is equipped with a scanner, and the transformation relationship between the robot's coordinate system and the scanner's coordinate system is fixed. The scanner includes a depth map and a grayscale camera. Correspondingly, an end effector is installed on one axis of the robot body. The end effector has a tool coordinate system. To calibrate the position of the tool origin in the tool coordinate system within the robot's coordinate system, the robot, controlled by the end effector, installs a marker plate. The marker plate has marker points distributed as follows: Figure 3 As shown.

[0054] In one embodiment, an end effector with a calibration plate mounted on a robot is image-acquired using a robot scanner. This includes: the robot controlling the end effector to move multiple times within its working range; during each movement, the calibration plate mounted on the end effector is image-acquired by the scanner to obtain calibration plate images in multiple poses; wherein the range of motion of the end effector within its working range is positively correlated with the accuracy of the tool origin.

[0055] In one embodiment, the robot controls the end effector to move multiple times within its working range, including: the robot keeping the position of the calibration plate in the robot coordinate system within a preset range, and controlling the end effector to move so that the calibration plate undergoes multiple pose changes; correspondingly, the calibration plate mounted on the end effector during each movement is image-captured by a scanner, including: image-captured by the robot's scanner for the end effector during each pose change.

[0056] Step 204: When the difference value between multiple pose calibration board images does not fall within the difference value range, control the end effector to run in the direction corresponding to the difference value, and during the operation, use a scanner to acquire images of the robot's end effector to obtain the target calibration board image.

[0057] The difference value between calibration board images of multiple poses is obtained by calculating the difference based on the robot coordinate system in the calibration board images of multiple poses. The process of calculating the difference can be based on the position of a preset point in the robot coordinate system or on the difference between the scan values ​​of the calibration board images of multiple poses.

[0058] In one embodiment, when the difference value does not fall within the difference value range, the position of the preset point in the calibration plate image of each pose is calculated in the robot coordinate system based on the position of the marker point in the calibration plate image of each pose. Based on the calculated result, the end effector is controlled to run multiple times, and the end effector of the robot is imaged by a scanner during the operation. This process is repeated until the difference value falls within the difference value range, and the target calibration plate image is obtained.

[0059] The scan values ​​of the target calibration board image are generated based on the positions of the marker points in the calibration board image at each pose. When the marker point position consists of multiple 3D marker point coordinates, the set of 3D marker point coordinates for a certain pose is subjected to stability control by the robot, and the average value of the 3D marker point coordinates after stability control is obtained.

[0060] When a calibration board has a marker point, and each pose of the calibration board image contains a marker point location, the end effector with the same pose is scanned using scanners of each pose. The obtained marker point location is the scan value of the corresponding pose's calibration board image. The marker point location is the coordinate position of the marker point in the calibration board images of multiple poses in the scanner coordinate system, and the marker point location is used to generate the scan value for each pose.

[0061] The scan value can be either an input value for the tool origin calibration process of the robot, used to calculate the difference between calibration plate images; or an input value for calibrating the relationship between the 3D scanner and the robot body, used to calibrate the coordinate transformation relationship between the tool coordinate system and the scanner coordinate system of the end effector.

[0062] In one embodiment, controlling the end effector to run in the direction corresponding to the difference value, and acquiring images of the end effector through a scanner during operation to obtain a target calibration plate image, includes: generating scan values ​​for multiple poses of the calibration plate image based on the position of each marker point of the calibration plate image; generating a difference vector based on the scan values ​​of each pose of the calibration plate image; controlling the end effector to run in the opposite direction of the difference vector; and acquiring images of the end effector through a scanner during operation to obtain a target calibration plate image.

[0063] To further improve the accuracy of the tool origin position, the calibration board has multiple marker points distributed in a ring, and each pose of the calibration board image contains multiple marker points. Correspondingly, based on the marker point positions of the calibration board images for each of the multiple poses, scan values ​​for each calibration board image in each pose are generated, including: obtaining the marker point positions corresponding to the marker points in the calibration board images for each of the multiple poses; and averaging the marker point positions for each pose to obtain the scan values ​​for each calibration board image in each pose.

[0064] In one embodiment, the marker position is averaged for each pose, including: scanning an end effector with the same pose using a scanner with multiple poses, averaging the marker positions with the same pose obtained from the scan, and obtaining the averaged marker position results for each pose. The averaged marker position results for each pose are the scan values ​​of the calibration plate image for the corresponding pose.

[0065] Step 206: Determine the tool origin position of the robot based on the position of the marker point corresponding to the marker point in the target calibration board image.

[0066] In one embodiment, determining the robot's tool origin position based on the marker positions corresponding to marker points in the target calibration plate image includes: using the scan values ​​corresponding to the marker points in the target calibration plate image as the initial working origin position; and transforming the initial working origin position to the robot coordinate system to obtain the target tool origin position in the robot coordinate system. Correspondingly, calibrating the coordinate transformation relationship between the end effector and the scanner based on the tool origin position includes: calibrating the coordinate transformation relationship between the end effector and the scanner based on the target tool origin position.

[0067] The scan value corresponding to the marker position is the scan value of the marker position corresponding to each target calibration image. It can be understood that when the difference value of a certain group falls into the difference value range of the calibration plate image, then there exists a target calibration image for that group. The target calibration images of different groups obtain the origin position of the target tool in the robot coordinate system based on the robot's teach pendant function, so as to calibrate the coordinate transformation relationship between the end effector and the scanner.

[0068] The process of obtaining the origin position of the target tool in the robot coordinate system specifically includes: based on the robot teach pendant function, using the difference value moving end effector, after the scan value (the averaged result of the marker points) determined by different groups of target calibration images are coincident, the position of the scan value can be set as the origin of the tool coordinate system, and its coordinate position in the robot coordinate system can be obtained.

[0069] Step 208: Based on the tool origin position, calibrate the coordinate transformation relationship between the end effector and the scanner.

[0070] In one embodiment, calibrating the coordinate transformation relationship between the end effector and the scanner based on the tool origin position includes: obtaining calibration plate images with pose matching based on the tool origin position under different poses; and calculating the coordinate rotation transformation relationship and translation transformation relationship between the end effector and the scanner based on the set of marker points corresponding to the calibration plate images under different poses.

[0071] Specifically, when calibrating the coordinates of the robot and the 3D scanner, the angle between the tool coordinate system and the robot body coordinate system is kept constant during calibration. After each robot movement, the robot's XYZ information is recorded, and the 3D average value of the 3D scanner marker points is also recorded. At least 8 sets of data are collected in total, and the positions are as close to their working range as possible to reduce calibration errors. Finally, the rotation and translation values ​​between the robot and the 3D scanner can be obtained using PNP (Perspective-n-Point). The tool origin position is the position in the robot coordinate system, thus obtaining the corresponding tool origin position.

[0072] Among them, PnP is a method for solving the motion of 3D to 2D point pairs. It is a method for estimating the pose of the camera based on 3D spatial points and their projected positions. This method does not require solving the AX=XB equation, so it does not have very strict requirements for data accuracy.

[0073] In the above-mentioned robot hand-eye coordinate transformation method, when the difference value between multiple pose calibration board images does not fall within the difference value range, the end effector is controlled to move in the direction corresponding to the difference value. During the operation, the end effector is image-captured by a scanner to obtain the target calibration board image. Based on the position of the marker point corresponding to the marker point in the target calibration board image, the tool origin position of the robot is determined, replacing the traditional process of calibration with a needle tip using a teach pendant. Setting the robot origin coordinates can avoid manual alignment errors and the limitation of only being able to set the tool coordinates to the feature position that can be recognized by the human eye. This allows for a more accurate acquisition of the tool origin position in the robot coordinate system, thereby more accurately calibrating the coordinate transformation relationship between the end effector and the scanner.

[0074] Furthermore, since the robot's tool origin position is determined based on the location of the marker points corresponding to the marker points in the target calibration board image, the solution process does not implicitly include the RT relationship between the tool coordinate origin and the target coordinate system origin. Therefore, the coordinate transformation relationship between the end effector and the scanner can be directly calibrated without using the tsai-lenz algorithm. Thus, this embodiment omits the calculation of the AX=XB matrix compared to the classic tsai-lenz-based method. The input for this matrix calculation is the change between two postures, which is not very intuitive. Actual simulation tests show that adding 0.1mm noise to the input results in an error of about 1mm. This error can be introduced by the 3D scanner or the robot arm, causing this situation. This solution can directly adjust the calibration working range to distribute the quantitative error over a larger range.

[0075] In one embodiment, to more clearly illustrate the process of generating the origin position of the scanning target tool, a more specific example is provided, the method comprising:

[0076] Step 402: Using the robot's scanner, images of the end effector with the calibration plate mounted on the robot are acquired, resulting in calibration plate images of the end effector in multiple poses. These multiple pose images are then combined into a first group according to the acquisition order of each pose. It is then determined whether the difference value of the scanned values ​​in the first group falls within the difference value range. For example, two poses acquired in adjacent sequences are combined into the first group.

[0077] Step 404: When the difference value of the first group of scan values ​​does not fall into the difference value range, control the end effector to run in the opposite direction corresponding to the difference value, and during the operation, use a scanner to acquire images of the robot's end effector. This process is repeated until the difference value of the first group of scan values ​​falls into the difference value range, thus obtaining the target calibration plate image of the first group.

[0078] Step 406: Based on the position of the marker point corresponding to the marker point in the target calibration board image of the first group, determine the tool origin position of the first group. Then, follow steps 402 to 406 to obtain the tool origin position of the second group, and so on, until the tool origin positions of four groups, six groups, etc. are obtained. The tool origin positions of each group are then used to calibrate the tool origin position of the robot through the robot's teach pendant function.

[0079] Step 408: Based on the tool origin position, calibrate the coordinate transformation relationship between the end effector and the scanner.

[0080] Thus, steps 402-408 more clearly demonstrate the technical solution for determining the tool origin position through multiple groups in the case of pose grouping.

[0081] In one embodiment, such as Figure 5 As shown, based on any of the above embodiments, a further limitation is made to achieve faster hand-eye calibration. Correspondingly, the end effector is controlled to move in the opposite direction to the difference value, and during operation, an image of the robot's end effector is acquired by a scanner, including:

[0082] Step 502: When the difference value between the calibration board images of multiple poses does not fall within the difference value range, generate the scan value of each calibration board image of multiple poses based on the position of the respective marker point of each calibration board image of multiple poses.

[0083] Step 504: Calculate the initial coordinate transformation relationship between the end effector and the scanner based on the homogeneous transformation matrices of each pose in the tool coordinate system and the homogeneous transformation matrices of each pose in the scanner coordinate system.

[0084] Among them, the homogeneous transformation matrix of each pose in the tool coordinate system is used to characterize the transformation relationship between different poses in the tool coordinate system when the positions of the calibration plate and the robot body remain unchanged; the homogeneous transformation matrix of each pose in the scanner coordinate system is used to characterize the transformation relationship between different poses in the scanner coordinate system when the positions of the calibration plate and the robot body remain unchanged.

[0085] It is understandable that even if the initial coordinate transformation relationship has a large error, it can still provide a certain reference, reducing the number of iterations required to generate the difference vector. This, in turn, reduces the number of iterations required to transform the scan values ​​of the calibration board images for each adjacent pose, making it easier for the difference values ​​to fall within the aforementioned difference value range, thereby improving the generation speed of the difference vector. Furthermore, since the initial coordinate transformation relationship is part of the coordinate transformation between the tool coordinate system and the robot coordinate system, and this step involves multiple poses, the various corresponding difference vectors for these multiple poses can help avoid accuracy problems caused by the initial coordinate transformation relationship.

[0086] In one embodiment, the initial coordinate transformation relationship is calculated using the Tsai-Lenz algorithm. Correspondingly, with the position between the calibration board and the robot body remaining unchanged, the calculation formula for the initial coordinate transformation relationship is as follows:

[0087] H gi jH gc =H gc H ci j;

[0088] Among them, H gc It is the initial coordinate transformation relationship between the tool coordinate system and the scanner coordinate system; H gi j is the homogeneous transformation matrix used to characterize the transformation relationship between the tool coordinate systems of pose i and pose j; H gi j is a homogeneous transformation matrix used to characterize the transformation relationship between the scanner coordinate systems of pose i and pose j.

[0089] Step 506: According to the initial coordinate transformation relationship, combine and transform the scan values ​​of the calibration plate images of adjacent poses into the robot coordinate system, and generate the difference vector of each group based on the adjacent pose scan values ​​of each combination after transformation.

[0090] A combination of adjacent pose scan values ​​can be the scan values ​​of at least two adjacent poses in a group. By combining multiple values, the tool origin position can be more accurately calibrated based on the robot's teach pendant function.

[0091] In one embodiment, step 506 includes: according to the initial coordinate transformation relationship, sequentially transforming the first scan value and the second scan value of the calibration plate image of adjacent poses of each combination into the robot coordinate system, and calculating the difference vector of the first scan value and the second scan value of each group in the robot coordinate system.

[0092] Step 508: Control the end effector to run repeatedly in the opposite direction of the difference vector of each group so that the difference vector of each group falls into the above difference vector interval.

[0093] It is understandable that after obtaining the first scan value of a certain group, if the difference between the second scan value of the group and the first scan value is too large, the end effector is controlled to move once based on the opposite direction of the current difference vector to obtain the third scan value to replace the second scan value. If the difference between the third scan value and the first scan value is still too large, the movement is reversed again based on the difference between the first scan value and the third scan value. This process is repeated multiple times to reduce the difference vector of the group to the above-mentioned difference vector range.

[0094] Step 510: During operation, the end effector is image acquired by a scanner to obtain an image of the target calibration board.

[0095] In this embodiment, the initial coordinate transformation relationship between the end effector and the scanner is calculated based on the homogeneous transformation matrices of each pose in the tool coordinate system and the homogeneous transformation matrices of each pose in the scanner coordinate system. Although the initial coordinate transformation relationship cannot accurately calibrate the coordinate transformation relationship between the end effector and the scanner, it can provide a general direction for modification, thereby reducing the number of loop operations to calculate the difference vector and improving the calibration speed while ensuring accuracy.

[0096] Furthermore, although the initial coordinate transformation relationship is based on the Tsai-Lenz algorithm, this process is used to assist in calibrating the origin of the tool. By repeatedly running the end effector in the opposite direction of the difference vectors of each group, so that the difference vectors of each group fall into the above difference vector interval, the error brought about by the Tsai-Lenz algorithm is eliminated.

[0097] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0098] Based on the same inventive concept, this application also provides a robot hand-eye coordinate transformation device for implementing the robot hand-eye coordinate transformation method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more robot hand-eye coordinate transformation device embodiments provided below can be found in the limitations of the robot hand-eye coordinate transformation method described above, and will not be repeated here.

[0099] In one embodiment, such as Figure 6 As shown, a robot hand-eye coordinate transformation device is provided, including: an image acquisition module 602, an adjustment module 604, an origin calibration module 606, and a transformation relationship determination module 608, wherein:

[0100] The image acquisition module 602 is used to acquire images of the end effector on the robot with a calibration plate mounted on it using the robot's scanner, and obtain calibration plate images of the end effector in multiple poses.

[0101] The adjustment module 604 is used to control the end effector to run in the direction corresponding to the difference value when the difference value between the multiple pose calibration board images does not fall into the difference value range, and to acquire images of the end effector through the scanner during the operation to obtain the target calibration board image.

[0102] The origin calibration module 606 is used to determine the tool origin position of the robot based on the scan value corresponding to the marker point in the target calibration plate image;

[0103] The transformation relationship determination module 608 is used to determine the coordinate transformation relationship between the end effector and the scanner based on the origin position of the tool.

[0104] In one embodiment, the image acquisition module 602 includes:

[0105] The pose change unit is used to keep the position change of the calibration plate in the robot coordinate system within a preset range and control the movement of the end effector so that the calibration plate can perform multiple pose changes.

[0106] The image acquisition unit is used to acquire images of the end effector during each pose change using the robot's scanner.

[0107] In one embodiment, the adjustment module 604 includes:

[0108] The scan value generation unit is used to generate scan values ​​for the calibration board images of the multiple poses based on the respective marker point positions of the calibration board images of the multiple poses.

[0109] The difference vector calculation unit is used to generate difference vectors based on the scan values ​​of the calibration plate image at each pose.

[0110] An adjustment unit is used to control the end effector to operate in the opposite direction of the difference vector;

[0111] The target calibration board image determination unit is used to acquire images of the end effector through the scanner during operation to obtain the target calibration board image.

[0112] In one embodiment, the scan value generation unit includes:

[0113] The marker position acquisition subunit is used to acquire the marker position corresponding to the marker in the calibration board image of the multiple poses;

[0114] The scan value calculation subunit is used to average the position of the marker point under each pose to obtain the scan value of the calibration plate image under each pose.

[0115] In one embodiment, the difference vector calculation unit includes:

[0116] The initial calibration subunit is used to calculate the initial coordinate transformation relationship between the end effector and the scanner based on the homogeneous transformation matrix of each pose in the tool coordinate system and the homogeneous transformation matrix of each pose in the scanner coordinate system.

[0117] The loop calculation subunit is used to combine and transform the scan values ​​of the calibration plate images of adjacent poses into the robot coordinate system according to the initial coordinate transformation relationship, and generate the difference vector of each group based on the adjacent pose scan values ​​of each combination after transformation.

[0118] In one embodiment, the origin calibration module 606 includes:

[0119] The initial working origin determination unit is used to take the scan value corresponding to the marker point in the target calibration board image as the initial working origin position.

[0120] The tool origin position calculation unit is used to transform the initial working origin position to the robot coordinate system to obtain the target tool origin position in the robot coordinate system.

[0121] Correspondingly, the transformation relationship determination module 608 is used to determine the coordinate transformation relationship between the end effector and the scanner based on the origin position of the target tool.

[0122] In one embodiment, the transformation relationship determination module 608 includes:

[0123] The calibration board image matching unit is used to obtain the calibration board image with pose matching based on the tool origin position under different poses;

[0124] The transformation relationship calibration unit is used to calculate the coordinate rotation transformation relationship and translation transformation relationship between the end effector and the scanner based on the set of marker points corresponding to the calibration plate images in different poses.

[0125] Each module in the aforementioned robot hand-eye coordinate transformation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0126] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a robot hand-eye coordinate transformation method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0127] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0128] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0129] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0130] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0132] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0133] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for transforming robot hand-eye coordinates, characterized in that, The method includes: The end effector with a calibration plate mounted on the robot is imaged using the robot's scanner, resulting in calibration plate images of the end effector in multiple poses. When the difference value between the multiple pose calibration board images does not fall within the difference value range, the end effector is controlled to run in the direction corresponding to the difference value, and during the operation, the end effector is image acquired by the scanner to obtain the target calibration board image; Based on the robot teach pendant function, the end effector is moved using the difference value. After the scan values ​​corresponding to the marker points in the target calibration board images of different groups are made to coincide, the scan value is set as the tool origin position of the robot. A calibration plate image matching the pose is obtained based on the same origin position of the tool under different poses; based on the set of marker points corresponding to the re-acquired pose-matched and different calibration plate images, the coordinate rotation transformation relationship and translation transformation relationship between the end effector and the scanner are calculated.

2. The method according to claim 1, characterized in that, The process of controlling the end effector to move in the direction corresponding to the difference value, and acquiring an image of the end effector via the scanner during operation to obtain an image of the target calibration plate, includes: Based on the respective marker point positions of the calibration board images of the multiple poses, scan values ​​of the calibration board images of the multiple poses are generated respectively. Generate a difference vector based on the scan values ​​of the calibration plate image at each pose; The end effector is controlled to operate in the opposite direction of the difference vector; During operation, the scanner acquires images of the end effector to obtain an image of the target calibration board.

3. The method according to claim 2, characterized in that, The step of generating scan values ​​for each of the multiple poses of the calibration board images based on the respective marker point positions of the multiple poses includes: Obtain the position of the marker point corresponding to the marker point in the calibration board image of the multiple poses; The positions of the marker points in each pose are averaged to obtain the scan values ​​of the calibration plate images in each pose.

4. The method according to claim 2, characterized in that, The generation of difference vectors from the scan values ​​of the calibration board image at each pose includes: The initial coordinate transformation relationship between the end effector and the scanner is obtained by calculating the homogeneous transformation matrices of each pose in the tool coordinate system and the homogeneous transformation matrices of each pose in the scanner coordinate system. According to the initial coordinate transformation relationship, the scan values ​​of the calibration plate images of adjacent poses are combined and transformed into the robot coordinate system, and the difference vector of each group is generated based on the scan values ​​of adjacent poses of each combination after transformation.

5. The method according to claim 1, characterized in that, Determining the tool origin position of the robot based on the scan values ​​corresponding to the marker points in the target calibration board image includes: The scan value corresponding to the marker point in the target calibration board image is used as the initial working origin position; Transform the initial working origin position to the robot coordinate system to obtain the target tool origin position in the robot coordinate system; The step of obtaining the pose-matching calibration board images based on the tool origin position under different poses includes: The calibration board image is obtained based on the origin position of the target tool under different poses to achieve pose matching.

6. The method according to claim 1, characterized in that, The step of acquiring images of the end effector with a calibration plate mounted on the robot using the robot's scanner includes: The position of the calibration plate in the robot coordinate system is kept within a preset range, and the movement of the end effector is controlled so that the pose of the calibration plate changes multiple times. For each pose change of the end effector, images are acquired using the robot's scanner.

7. The method according to claim 1, characterized in that, The coordinate rotation and translation transformation relationships between the end effector and the scanner are calculated based on the set of marker points corresponding to the re-acquired pose-matched and different calibration plate images, including: Based on the set of marker points corresponding to the re-acquired pose-matched and different calibration plate images, and the PNP algorithm, the coordinate rotation transformation relationship and translation transformation relationship between the end effector and the scanner are calculated.

8. A robot hand-eye coordinate transformation device, characterized in that, The device includes: The image acquisition module is used to acquire images of the end effector on the robot with a calibration plate mounted on it using the robot's scanner, and obtain calibration plate images of the end effector in multiple poses. The adjustment module is used to control the end effector to run in the direction corresponding to the difference value when the difference value between the multiple pose calibration board images does not fall within the difference value range, and to acquire images of the end effector through the scanner during the operation to obtain the target calibration board image. The origin calibration module is used to move the end effector based on the difference value using the robot teach pendant function, and after the scan values ​​corresponding to the marker points in the target calibration board images of different groups are made to coincide, the scan value is set as the tool origin position of the robot. The transformation relationship determination module is used to acquire calibration plate images with pose matching based on the same origin position of the tool under different poses; and to calculate the coordinate rotation transformation relationship and translation transformation relationship between the end effector and the scanner based on the set of marker points corresponding to the reacquired pose matching and different calibration plate images.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images