Hand-eye calibration method, system, terminal and medium for robot vision system

Abstract

The present invention provides a hand-eye calibration method and system for a robot vision system, wherein the method comprises: installing a calibration sphere at the end of the robot, moving the calibration sphere into the workspace of a three-dimensional point cloud camera, controlling the position of the calibration sphere in the workspace, and calculating the coordinates of the center of the calibration sphere in the robot coordinate system; obtaining three-dimensional point cloud information of the calibration sphere in the workspace, and performing point cloud screening to obtain calibration sphere point cloud information; extracting the calibration sphere based on the calibration sphere point cloud information to obtain the coordinates of the center of the calibration sphere in the three-dimensional camera coordinate system; and calculating the hand-eye calibration relationship between the camera and the robot based on the coordinates of the center of the calibration sphere in each calibration in the camera coordinate system and the robot coordinate system, thereby completing the hand-eye calibration of the robot vision system. The present invention is easy to deploy, low-cost, and widely applicable, and is of great significance for promoting the development of the manufacturing industry, reducing enterprise production costs, and improving production efficiency.

Description

Technical Field

[0001] The present invention relates to a camera-robot system calibration technology in the field of recognition and grasping technology in industrial production, and specifically to a hand-eye calibration method, system, terminal and medium for a high-precision robot vision system. Background Art

[0002] With the advancement of machine vision and robotics technologies, depth cameras and industrial robots are increasingly collaborating in industrial production environments. Robots are required to perceive scene information in real time through their cameras, which requires high-precision pre-calibration of 3D camera-robot systems. Conventional camera-robot systems often require recalibration due to changes in the camera or robot position due to changes in the task. Therefore, a fast and highly accurate calibration system is needed.

[0003] Currently, the commonly used calibration system is based on a flat calibration plate. The camera identifies the calibration plate's position and, combined with the robot's coordinates, determines the camera-robot coordinate relationship. This method requires the two-dimensional calibration plate to be within the camera's field of view. However, this approach is significantly limited by the large size of the calibration plate and the robot's limited motion space. Another common method uses auxiliary devices, such as laser trackers or three-axis trackers, to achieve hand-eye calibration. This method greatly increases the cost and operational difficulty of the calibration system. Therefore, a fast, universal, and low-cost three-dimensional camera-robot calibration system is of great significance.

[0004] After searching the prior art, we found that:

[0005] The Chinese invention patent with authorization announcement number CN110842901B discloses a "robot hand-eye calibration method and device based on a new three-dimensional calibration block". By adjusting the robot posture and the placement of the three-dimensional calibration block, the three-dimensional vision device can obtain a point cloud containing three key points on the three-dimensional calibration block. Based on the coordinates of the key points in the camera coordinate system and the robot coordinate system, the camera-robot hand-eye relationship can be solved.

[0006] The Chinese invention patent with authorization announcement number CN112091971B discloses a "robot hand-eye calibration method, device, electronic device and system", which includes the following steps: 1. Obtaining three-dimensional point cloud image information obtained by a three-dimensional camera, the three-dimensional point cloud image information includes three-dimensional point cloud information of at least three marking balls that are not all in the same straight line; 2. Based on the three-dimensional point cloud image information, calculating the first position data of the center of mass of the marking ball in the camera coordinate system; 3. Obtaining the second position data of the center of mass of the marking ball in the robot base coordinate system; 4. Calculating the transformation matrix between the camera coordinate system and the robot base coordinate system based on the first position data and the second position data.

[0007] The above two invention patents have the characteristics of low cost and high speed, but there are still problems such as the calibration parts are too complicated, the information of three key points needs to be extracted simultaneously, there are restrictions on the position of the calibration parts, and the applicability is low. Summary of the Invention

[0008] In view of the above-mentioned deficiencies in the prior art, the present invention provides a hand-eye calibration method, system, terminal and medium for a robot vision system.

[0009] According to one aspect of the present invention, a hand-eye calibration method for a robot vision system is provided, comprising:

[0010] Set the working space of the 3D point cloud camera;

[0011] A calibration ball is installed at the end of the robot, and the calibration ball is moved into the working space of the three-dimensional point cloud camera, the position of the calibration ball in the working space is controlled, the coordinates of the center of the calibration ball in the robot coordinate system are calculated, and the coordinates are sent to the calibration module;

[0012] Acquire three-dimensional point cloud information of the calibration sphere in the workspace, and perform point cloud screening to obtain calibration sphere point cloud information;

[0013] Extracting the calibration sphere according to the calibration sphere point cloud information to obtain the coordinates of the center of the calibration sphere in the three-dimensional camera coordinate system;

[0014] Repeat the above steps to obtain the coordinates of the center of the calibration ball in the robot coordinate system and the three-dimensional camera coordinate system multiple times;

[0015] According to the coordinates of the center of the calibration ball in the camera coordinate system and the robot coordinate system each time, the hand-eye calibration relationship between the camera and the robot is calculated to complete the hand-eye calibration of the robot vision system.

[0016] Optionally, setting the working space of the three-dimensional point cloud camera includes:

[0017] The 3D point cloud camera is fixed at a specific working position, and the working parameters of the 3D point cloud camera are adjusted according to environmental factors to obtain the working space of the 3D point cloud camera.

[0018] Optionally, the step of installing a calibration ball on the end of the robot, moving the calibration ball into the working space of the three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, and calculating the coordinates of the center of the calibration ball in the robot coordinate system includes:

[0019] A customized calibration ball, comprising a calibration member and a positioning member; wherein the calibration member comprises a metal ball, a connecting rod, and a base flange, and the positioning member is provided with a cylindrical groove adapted to the radius of the metal ball;

[0020] The metal ball is fixed to the end of the robot through the connecting rod and the base flange in sequence; the positioning piece is fixed to the working platform; the metal ball contacts the bottom and side surfaces of the cylindrical groove to obtain a determined matching state between the calibration piece and the positioning piece, thereby fixing the position of the calibration ball in the working space;

[0021] The robot's posture is changed multiple times, and the coordination between the calibration component and the positioning component is utilized to ensure that the position of the center of the calibration ball remains unchanged, and the coordinates of the center of the calibration ball in the robot coordinate system are calculated.

[0022] Optionally, the method further includes: changing the position of the calibration ball in the workspace multiple times to obtain the coordinates of the center of the calibration ball at the corresponding position in the robot coordinate system.

[0023] Optionally, the point cloud screening includes:

[0024] According to the distance between the 3D point cloud camera and the workbench and the size of the workspace, the point cloud outside the workspace is removed and only the point cloud within the workspace is retained;

[0025] The voxel filtering method or the straight-through filtering method is used to filter out irrelevant point cloud information and obtain the calibration sphere point cloud information.

[0026] Optionally, extracting the calibration sphere includes:

[0027] Use the outlier removal algorithm to remove point clouds with uneven distribution;

[0028] Using clustering algorithm, the point cloud is divided into different parts, and the point cloud where the calibration sphere is located is selected according to the volume of the point cloud bounding box;

[0029] The calibration sphere is preliminarily fitted using the random sampling consensus algorithm;

[0030] According to the fitting results, the point cloud is further screened;

[0031] The random sampling consistency algorithm is performed again to extract the fine coordinates of the calibration sphere and obtain the coordinates of the center of the calibration sphere in the three-dimensional camera coordinate system.

[0032] Optionally, calculating the hand-eye calibration relationship between the camera and the robot based on the coordinates of the center of the calibration ball in the camera coordinate system and the robot coordinate system each time includes:

[0033] Calculate the mean coordinates of the sphere center in the camera coordinate system and the robot coordinate system respectively, and use the difference between the mean coordinates of the sphere center in the two coordinate systems as the translation between the two coordinate systems;

[0034] Obtain a set of sphere center coordinates in the camera coordinate system and a set of sphere center coordinates in the robot coordinate system respectively, and subtract the mean of the sphere center coordinates respectively to complete the centralization process and obtain two sets of sphere center coordinates;

[0035] The covariance matrix H of the two sets of sphere center coordinates is calculated, and the covariance matrix H is subjected to singular value decomposition to obtain the pose matrix of the robot coordinate system relative to the camera coordinate system, thereby obtaining the hand-eye calibration relationship between the camera and the robot.

[0036] According to another aspect of the present invention, a hand-eye calibration system for a robot vision system is provided, comprising:

[0037] A robot module includes a calibration sphere pre-installed at the end of the robot. The calibration sphere is moved into the working space of the 3D point cloud camera to obtain the position of the calibration sphere in the working space and calculate the coordinates of the center of the calibration sphere in the robot coordinate system.

[0038] A point cloud acquisition module, which is based on a three-dimensional point cloud camera, obtains the three-dimensional point cloud information of the calibration sphere in the three-dimensional point cloud camera workspace, performs point cloud screening, obtains the calibration sphere point cloud information, and sends it to the calibration module;

[0039] The calibration module is used to control the position of the robot module in the working space of the three-dimensional point cloud camera. According to the point cloud information of the calibration ball, the calibration ball is extracted to obtain the coordinates of the center of the calibration ball in the three-dimensional camera coordinate system; according to the coordinates of the center of the calibration ball in the camera coordinate system and the robot coordinate system each time, the hand-eye calibration relationship between the camera and the robot is calculated to complete the hand-eye calibration of the robot vision system.

[0040] According to a third aspect of the present invention, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor may be configured to execute any one of the above methods when executing the program.

[0041] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it can be used to perform any of the methods described above.

[0042] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0043] The hand-eye calibration method, system, terminal, and medium for the robot vision system provided by the present invention are easy to deploy, low-cost, and widely applicable. They can be quickly deployed in a system consisting of any model of industrial robot and three-dimensional camera, and are of great significance for promoting the development of the manufacturing industry, reducing enterprise production costs, and improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0045] Figure 1 This is a workflow diagram of a hand-eye calibration method for a robot vision system according to one embodiment of the present invention;

[0046] Figure 2 Schematic diagram of the components of a hand-eye calibration system for a robot vision system according to one embodiment of the present invention;

[0047] Figure 3 A schematic structural diagram of a calibration sphere in a preferred embodiment of the present invention;

[0048] Figure 4 Schematic diagram of the positioning of the calibration ball in the robot coordinate system in a preferred embodiment of the present invention.

[0049] In the figure, 1 is a calibration part, 2 is a positioning part, 11 is a metal ball, 12 is a connecting rod, 13 is a base flange, and 21 is a cylindrical groove. DETAILED DESCRIPTION

[0050] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. It should be noted that those skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention.

[0051] An embodiment of the present invention provides a hand-eye calibration method for a robot vision system.

[0052] like Figure 1 As shown, the hand-eye calibration method for the robot vision system provided in this embodiment may include the following steps:

[0053] S100, setting the working space of the 3D point cloud camera;

[0054] S200: Install a calibration sphere at the end of the robot, move the calibration sphere into the workspace of the 3D point cloud camera, control the position of the calibration sphere in the workspace, and calculate the coordinates of the center of the calibration sphere in the robot coordinate system.

[0055] S300, obtaining three-dimensional point cloud information of the calibration sphere in the workspace, and performing point cloud screening to obtain calibration sphere point cloud information;

[0056] S400, extracting the calibration sphere according to the calibration sphere point cloud information, and obtaining the coordinates of the center of the calibration sphere in the three-dimensional camera coordinate system;

[0057] Repeat the above steps to obtain the coordinates of the center of the calibration ball in the robot coordinate system and the 3D camera coordinate system multiple times;

[0058] S500 , based on the coordinates of the center of each calibration ball in the camera coordinate system and the robot coordinate system, calculate the hand-eye calibration relationship between the camera and the robot, and complete the hand-eye calibration of the robot vision system.

[0059] In a preferred embodiment of S100, setting the working space of the 3D point cloud camera includes:

[0060] The 3D point cloud camera is fixed at a specific working position, and the working parameters of the 3D point cloud camera are adjusted according to environmental factors to obtain the working space of the 3D point cloud camera.

[0061] In a preferred embodiment of S200, a calibration sphere is mounted on the end of the robot, and the calibration sphere is moved into the workspace of the three-dimensional point cloud camera. The position of the calibration sphere in the workspace is controlled, and the coordinates of the center of the calibration sphere in the robot coordinate system are calculated, including:

[0062] Customized calibration sphere, the calibration sphere includes a calibration piece and a positioning piece; wherein the calibration piece includes a metal ball, a connecting rod and a base flange, and the positioning piece is provided with a cylindrical groove adapted to the radius of the metal ball;

[0063] The metal ball is fixed to the end of the robot through the connecting rod and the base flange in sequence; the positioning piece is fixed to the working platform; the metal ball contacts the bottom and side surfaces of the cylindrical groove to obtain a determined matching state between the calibration piece and the positioning piece, thereby fixing the position of the calibration ball in the working space;

[0064] The robot's posture is changed multiple times, and the coordination between the calibration part and the positioning part is used to ensure that the position of the center of the calibration ball remains unchanged, and the coordinates of the center of the calibration ball in the robot coordinate system are calculated.

[0065] In a preferred embodiment of S200, the position of the calibration ball in the workspace is changed multiple times to obtain the coordinates of the center of the calibration ball at the corresponding position in the robot coordinate system.

[0066] In a preferred embodiment of S300, performing point cloud screening includes:

[0067] According to the distance between the 3D point cloud camera and the workbench and the size of the workspace, the point cloud outside the workspace is removed and only the point cloud within the workspace is retained;

[0068] The voxel filtering method or the straight-through filtering method is used to filter out irrelevant point cloud information and obtain the calibration sphere point cloud information.

[0069] In a preferred embodiment of S400, extracting the calibration sphere includes:

[0070] Use the outlier removal algorithm to remove point clouds with uneven distribution;

[0071] Using a clustering algorithm, the point cloud is segmented into different parts, and the point cloud containing the calibration sphere is selected based on the volume of the point cloud bounding box. In this step, the point cloud bounding box is the smallest cube that encloses a specific point cloud. The point cloud containing the calibration sphere is spherical, so the length, width, and height of its bounding box are similar to the radius of the sphere. The bounding boxes of other point clouds, such as those of the robot end, debris, and noise, do not have these characteristics, so the calibration sphere point cloud can be identified based on this.

[0072] The calibration sphere is preliminarily fitted using the random sampling consensus algorithm;

[0073] According to the fitting results, the point cloud is further screened;

[0074] The random sampling consensus algorithm is performed again to extract the fine coordinates of the calibration sphere and obtain the coordinates of the center of the calibration sphere in the 3D camera coordinate system. In this step, during the initial RANSAC (random sampling consensus), since the point cloud of the calibration sphere contains part of the point cloud of the connecting rod, in order to ensure that the ball is recognized, the RANSAC used this time adopts a lower standard, resulting in less accurate calibration sphere information. Before the second RANSAC, the point cloud outside the initially identified sphere is first filtered out. At this time, the point cloud basically only retains the point cloud of the sphere. Then, the high-precision RANSAC is used to calculate the information of the sphere, and the coordinates of the center of the sphere are more accurate. The fine coordinates and the coordinates of the center of the sphere both refer to the coordinates of the center of the sphere in the camera coordinate system. The fine coordinates obtained after the second RANSAC indicate that the result is more accurate.

[0075] In a preferred embodiment of S500, the hand-eye calibration relationship between the camera and the robot is calculated based on the coordinates of the center of the calibration ball in the camera coordinate system and the robot coordinate system, including:

[0076] Calculate the mean coordinates of the sphere center in the camera coordinate system and the robot coordinate system respectively, and use the difference between the mean coordinates of the sphere center in the two coordinate systems as the translation between the two coordinate systems;

[0077] Obtain a set of sphere center coordinates in the camera coordinate system and a set of sphere center coordinates in the robot coordinate system respectively, and subtract the mean of the sphere center coordinates respectively to complete the centralization process and obtain two sets of sphere center coordinates;

[0078] Calculate the covariance matrix H of the two sets of sphere center coordinates and perform singular value decomposition on the covariance matrix H to obtain the pose matrix of the robot coordinate system relative to the camera coordinate system. This allows us to derive the hand-eye calibration relationship between the camera and the robot. In one specific application example, the pose matrix is ​​a 4x4 matrix.

[0079] The hand-eye calibration method for a robot vision system provided in the above-mentioned embodiment of the present invention obtains point cloud information of the calibration part, and performs noise reduction and segmentation on the point cloud information based on the shooting results; a special spherical calibration ball is fixed at the end of the robot, and the special calibration ball and flange ensure that the installation error of the calibration part is less than 0.1 mm; by controlling the robot, the spatial coordinates of the calibration ball are transformed, the position of the calibration ball is identified, and the hand-eye transformation matrix of the three-dimensional camera-robot is solved by combining multiple recognition results.

[0080] An embodiment of the present invention provides a hand-eye calibration system for a robot vision system.

[0081] like Figure 2 As shown, the hand-eye calibration system of the robot vision system provided in this embodiment may include the following modules:

[0082] The robot module includes a calibration sphere pre-installed on the end of the robot. By moving the calibration sphere into the workspace of the 3D point cloud camera, the position of the calibration sphere in the workspace is obtained, and the coordinates of the center of the calibration sphere in the robot coordinate system are calculated and sent to the calibration module;

[0083] Point cloud acquisition module, based on the 3D point cloud camera, obtains the 3D point cloud information of the calibration sphere in the 3D point cloud camera workspace, performs point cloud screening, obtains the calibration sphere point cloud information, and sends it to the calibration module;

[0084] The calibration module is used to control the position of the robot module in the working space of the 3D point cloud camera. According to the calibration ball point cloud information, the calibration ball is extracted to obtain the coordinates of the center of the calibration ball in the 3D camera coordinate system. According to the coordinates of the center of the calibration ball in the camera coordinate system and the robot coordinate system each time, the hand-eye calibration relationship between the camera and the robot is calculated to complete the hand-eye calibration of the robot vision system.

[0085] It should be noted that the steps in the method provided by the present invention can be implemented using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to implement the composition of the system, that is, the embodiments in the method can be understood as preferred examples of constructing the system.

[0086] Furthermore, the hand-eye calibration system of the robot vision system provided by the present invention includes the following modules:

[0087] A robot module, which includes a calibration ball that is compatible with a robot (industrial robot or collaborative robot). The calibration ball is fixed to the end of the robot, and the coordinates of the center of the calibration ball in the robot coordinate system are determined through a special calibration ball calibration tool (i.e., a positioning part). During the calibration stage, the module will change the position of the calibration ball multiple times, and output the coordinates of the center of the calibration ball in the robot coordinate system to the calibration module. In a preferred embodiment, the calibration ball can be customized according to the accuracy and optimal working distance of the three-dimensional camera, including a metal ball of a specific radius and a connecting rod of a specific height, which is fixed to the end of the robot through a base flange; the calibration ball has been surface-treated to avoid the problem of the three-dimensional camera being unable to shoot due to the reflection of the metal ball. The cylindrical groove of the calibration ball calibration tool can be matched with the metal ball of the calibration ball with high precision, and is used to calibrate the calibration ball in the robot coordinate system.

[0088] The point cloud acquisition module is based on a 3D camera and acquires 3D point cloud information of the calibration sphere in the workspace. It adjusts the working parameters of the 3D camera according to environmental factors such as ambient lighting and shooting noise to reduce the interference of external factors on the shooting effect. It then performs point cloud filtering on the 3D point cloud information to filter out irrelevant point cloud information such as the workbench, material frame, robot, flange, and calibration sphere connecting rod. After each point cloud acquisition, the acquired point cloud is filtered to remove irrelevant point cloud information and only retain the point cloud information of the calibration sphere, greatly reducing the size of the point cloud file. After completing the above operations, the obtained calibration sphere point cloud information is transmitted to the calibration module for subsequent calibration part detection and hand-eye calibration.

[0089] The calibration module controls the robot's motion and, at different locations, calls the point cloud acquisition module to acquire point cloud information from a calibration sphere. Next, the calibration sphere is extracted and fitted based on its spherical point cloud to obtain the coordinates of its center in the 3D camera coordinate system. After completing at least three sphere center identifications, the hand-eye calibration relationship between the camera and robot is calculated based on the coordinates of the sphere center in both the camera and robot coordinate systems, and the calibration error is output.

[0090] Furthermore, the point cloud acquisition module includes:

[0091] The point cloud acquisition module uses a 3D point cloud camera to acquire 3D point cloud information from a calibration sphere within the workspace. The camera can capture 3D point cloud information for objects within a range of 2 meters. This information includes the 3D coordinates of each point in the camera's coordinate system, as well as its grayscale and RGB color information. The 3D camera uses structured light, eliminating the need for additional light sources.

[0092] A camera bracket fixes the 3D point cloud camera at a specific working position.

[0093] Point cloud screening algorithm module, which uses a point cloud screening algorithm to delete unnecessary point cloud data (i.e., irrelevant point cloud information) based on the pre-input working distance and the position and size parameters of the material frame to obtain the calibration sphere cloud information.

[0094] Furthermore, the robot module includes:

[0095] Based on industrial robots or collaborative robots, a special calibration ball is fixed on the end of the robot. After the calibration ball is installed, the calibration program provided by the robot is used in combination with the calibration tooling that is compatible with the calibration ball to complete the tool calibration of the calibration ball. The structure of the calibration ball is as follows: Figure 3 The calibration ball is positioned in the robot coordinate system, as shown in Figure 4 shown.

[0096] Furthermore, the calibration module includes:

[0097] Calibration sphere extraction algorithm module, which uses the calibration sphere extraction algorithm to extract the precise position of the calibration sphere from the complex point cloud and return the coordinates of its center;

[0098] The hand-eye calibration algorithm module calculates the hand-eye relationship between the camera and the robot based on the coordinates of at least three calibration spheres in the robot coordinate system and the camera coordinate system, and returns the calibration matrix and error results.

[0099] Furthermore, the point cloud screening algorithm includes the following steps:

[0100] Step 1: Based on the distance between the 3D point cloud camera and the workbench and the size of the workspace, remove the point cloud outside the workspace and only keep the point cloud within the workspace;

[0101] Step 2: Use voxel filtering or straight-through filtering to filter out irrelevant point cloud information, reduce the density of the point cloud, and maintain the distribution characteristics of the point cloud to obtain the calibration sphere point cloud information.

[0102] Furthermore, the calibration sphere extraction algorithm includes the following steps:

[0103] In the first step, the outlier removal algorithm is used to remove the point clouds with scattered distribution, so as to retain the calibration sphere point cloud and separate the calibration sphere from other point clouds;

[0104] In the second step, the point cloud is divided into different parts using a clustering algorithm, and the point cloud where the calibration sphere is located is selected according to the volume of the point cloud bounding box;

[0105] In the third step, the calibration sphere is preliminarily fitted using the RANSAC (Random Sample Consensus) algorithm;

[0106] The fourth step is to further screen the point cloud based on the fitting results;

[0107] In the fifth step, the RANSAC (Random Sampling Consensus) algorithm is performed again to extract the fine coordinates of the calibration sphere and return the coordinates of the sphere center obtained by calibration as the result.

[0108] In the hand-eye calibration system of the robot vision system provided by the above embodiment of the present invention, the robot module pre-installs the calibration sphere and calculates the tool coordinate system of the calibration sphere, moves the calibration sphere into the working space of the camera, and transmits the center coordinates of the calibration sphere in the camera coordinate system at this time to the calibration module; the point cloud acquisition module is used to obtain three-dimensional point cloud information in the working space, and downsamples the three-dimensional point cloud information (point cloud screening processing), and the processed point cloud is transmitted to the calibration module; the calibration module starts working after obtaining the center coordinates and point cloud data of at least three groups of calibration spheres in different positions, extracts the calibration sphere, obtains the center coordinates of the calibration sphere in the camera coordinate system, and solves the camera-robot calibration matrix according to the calibration algorithm.

[0109] An embodiment of the present invention provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute any one of the methods in the above embodiments of the present invention.

[0110] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above-mentioned methods), computer instructions, etc., and the above-mentioned computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. In addition, the above-mentioned computer programs, computer instructions, data, etc. can be called by the processor.

[0111] The aforementioned computer programs, computer instructions, etc. may be partitioned and stored in one or more memories, and the aforementioned computer programs, computer instructions, data, etc. may be called by a processor.

[0112] The processor is configured to execute the computer program stored in the memory to implement the various steps of the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0113] The processor and memory can be independent structures or integrated structures. When the processor and memory are independent structures, the memory and processor can be coupled via a bus.

[0114] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, it can be used to execute any one of the methods in the above embodiments of the present invention.

[0115] The hand-eye calibration method, system, terminal, and medium for the robot vision system provided by the above-mentioned embodiments of the present invention are easy to deploy, low-cost, and widely applicable. They can be quickly deployed in a system consisting of any type of industrial robot and a three-dimensional camera, and are of great significance for promoting the development of the manufacturing industry, reducing enterprise production costs, and improving production efficiency.

[0116] Matters not mentioned in the above embodiments of the present invention are well known in the art.

[0117] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A hand-eye calibration method for a robot vision system, characterized in that: include: Set the working space of the 3D point cloud camera; Installing a calibration ball at the end of the robot, moving the calibration ball into the working space of the three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, and calculating the coordinates of the center of the calibration ball in the robot coordinate system; Acquire three-dimensional point cloud information of the calibration sphere in the workspace, and perform point cloud screening to obtain calibration sphere point cloud information; Extracting the calibration sphere according to the calibration sphere point cloud information to obtain the coordinates of the center of the calibration sphere in the three-dimensional camera coordinate system; Repeat the above steps to obtain the coordinates of the center of the calibration ball in the robot coordinate system and the three-dimensional camera coordinate system multiple times; Calculate the hand-eye calibration relationship between the camera and the robot based on the coordinates of the center of the calibration ball in the camera coordinate system and the robot coordinate system each time, and complete the hand-eye calibration of the robot vision system; The method includes installing a calibration ball on the end of the robot, moving the calibration ball into the working space of the three-dimensional point cloud camera, controlling the position of the calibration ball in the working space, and calculating the coordinates of the center of the calibration ball in the robot coordinate system, including: A customized calibration ball, comprising a calibration member and a positioning member; wherein the calibration member comprises a metal ball, a connecting rod, and a base flange, and the positioning member is provided with a cylindrical groove adapted to the radius of the metal ball; The metal ball is fixed to the end of the robot through the connecting rod and the base flange in sequence; the positioning piece is fixed to the working platform; the metal ball contacts the bottom and side surfaces of the cylindrical groove to obtain a determined matching state between the calibration piece and the positioning piece, thereby fixing the position of the calibration ball in the working space; The robot's posture is changed multiple times, and the coordination between the calibration component and the positioning component is utilized to ensure that the position of the center of the calibration ball remains unchanged, and the coordinates of the center of the calibration ball in the robot coordinate system are calculated.

2. The hand-eye calibration method of the robot vision system according to claim 1, characterized in that: Setting the working space of the 3D point cloud camera includes: The 3D point cloud camera is fixed at a specific working position, and the working parameters of the 3D point cloud camera are adjusted according to environmental factors to obtain the working space of the 3D point cloud camera.

3. The hand-eye calibration method for a robot vision system according to claim 1, characterized in that: Also includes: The position of the calibration ball in the workspace is changed multiple times to obtain the coordinates of the center of the calibration ball at the corresponding position in the robot coordinate system.

4. The hand-eye calibration method for a robot vision system according to claim 1, characterized in that: The point cloud screening includes: According to the distance between the 3D point cloud camera and the workbench and the size of the workspace, the point cloud outside the workspace is removed and only the point cloud within the workspace is retained; The voxel filtering method or the straight-through filtering method is used to filter out irrelevant point cloud information and obtain the calibration sphere point cloud information.

5. The hand-eye calibration method for a robot vision system according to claim 1, characterized in that: The extracting of the calibration sphere comprises: Use the outlier removal algorithm to remove point clouds with uneven distribution; Using the clustering algorithm, the point cloud is divided into different parts, and then the point cloud where the calibration sphere is located is selected according to the volume of the point cloud bounding box; The calibration sphere is preliminarily fitted using the random sampling consensus algorithm; According to the fitting results, the point cloud is further screened; The random sampling consistency algorithm is performed again to extract the fine coordinates of the calibration sphere and obtain the coordinates of the center of the calibration sphere in the three-dimensional camera coordinate system.

6. The hand-eye calibration method for a robot vision system according to claim 1, characterized in that: The hand-eye calibration relationship between the camera and the robot is calculated based on the coordinates of the center of the calibration ball in the camera coordinate system and the robot coordinate system each time, including: Calculate the mean coordinates of the sphere center in the camera coordinate system and the robot coordinate system respectively, and use the difference between the mean coordinates of the sphere center in the two coordinate systems as the translation between the two coordinate systems; Obtain a set of sphere center coordinates in the camera coordinate system and a set of sphere center coordinates in the robot coordinate system respectively, and subtract the mean of the sphere center coordinates respectively to complete the centralization process and obtain two sets of sphere center coordinates; The covariance matrix H of the two sets of sphere center coordinates is calculated, and the covariance matrix H is subjected to singular value decomposition to obtain the pose matrix of the robot coordinate system relative to the camera coordinate system, thereby obtaining the hand-eye calibration relationship between the camera and the robot.

7. A hand-eye calibration system for a robot vision system, characterized in that: include: The robot module includes a calibration sphere pre-installed on the end of the robot. By moving the calibration sphere into the working space of the 3D point cloud camera, the position of the calibration sphere in the working space is obtained, and the coordinates of the center of the calibration sphere in the robot coordinate system are calculated and sent to the calibration module; A point cloud acquisition module, which is based on a three-dimensional point cloud camera, obtains the three-dimensional point cloud information of the calibration sphere in the three-dimensional point cloud camera workspace, performs point cloud screening, obtains the calibration sphere point cloud information, and sends it to the calibration module; A calibration module is used to control the position of the robot module within the workspace of the 3D point cloud camera. The module extracts the calibration sphere based on the calibration sphere point cloud information and obtains the coordinates of the center of the calibration sphere in the 3D camera coordinate system. The module calculates the hand-eye calibration relationship between the camera and the robot based on the coordinates of the center of the calibration sphere in the camera coordinate system and the robot coordinate system each time, thereby completing the hand-eye calibration of the robot vision system. The robot module further comprises: A customized calibration ball, comprising a calibration member and a positioning member; wherein the calibration member comprises a metal ball, a connecting rod, and a base flange, and the positioning member is provided with a cylindrical groove adapted to the radius of the metal ball; The metal ball is fixed to the end of the robot through the connecting rod and the base flange in sequence; the positioning piece is fixed to the working platform; the metal ball contacts the bottom and side surfaces of the cylindrical groove to obtain a determined matching state between the calibration piece and the positioning piece, thereby fixing the position of the calibration ball in the working space; The robot's posture is changed multiple times, and the coordination between the calibration component and the positioning component is utilized to ensure that the position of the center of the calibration ball remains unchanged, and the coordinates of the center of the calibration ball in the robot coordinate system are calculated.

8. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it can be used to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to perform the method according to any one of claims 1 to 6.

Images