Hand-eye calibration method and system based on clamping adsorption component

Abstract

The embodiment of the application provides a hand-eye calibration method and system based on clamping and adsorbing components, which comprises the following steps: placing a material at a first position in the field of view of a camera to obtain N sets of robot end position and pixel coordinates; solving the uncorrected camera calibration parameters from the N sets of robot end position and pixel coordinates, and obtaining a conversion function according to the uncorrected camera calibration parameters; controlling the robot end to move to a calibration plane at the center of the field of view of the camera, and controlling the clamping and adsorbing components to place the material at a third position in the field of view of the camera to obtain M sets of robot end position and pixel coordinates; solving the tool coordinates; calculating the coordinates of the material in the robot base coordinate system to obtain the camera calibration parameters, so that the hand-eye calibration can be realized by obtaining the pixel coordinates of the feature points of the material through vision, the extension rod is not needed to be installed, and the height difference problem between the calibration plane and the working plane caused by the thickness of the extension rod is fundamentally avoided, and the calibration precision is improved.

Description

Technical Field

[0001] This invention relates to the field of hand-eye calibration technology, and in particular to a hand-eye calibration method and a hand-eye calibration system based on a clamping and adsorption component, a computer device, and a storage medium. Background Technology

[0002] Some high-precision processes in industrial settings (such as vision-based screw fastening and labeling) require a vision system as the robot's "eyes" to identify and calculate the position of marker points in the robot's base coordinate system, thereby enabling precise robot positioning.

[0003] When a camera is fixedly mounted above a robot, the robot's obstruction prevents the camera from directly photographing the marker point at the robot's end effector. In existing technologies, a common hand-eye calibration scheme involves fixing an extension rod to the robot's end effector, attaching a marker point to the extension rod, controlling the robot's movement to move the marker point to several positions within the camera's field of view, and then using the camera to capture the pixel coordinates of the marker point while simultaneously acquiring the robot's end effector pose for camera calibration.

[0004] However, the existing solutions mentioned above have the following problems:

[0005] (1) The extension rod has a certain thickness, so there is a height difference between the calibration plane and the process operation plane. This height difference will affect the calibration accuracy and thus the process accuracy.

[0006] (2) When there are obstacles such as protrusions near the calibration area, the calibration scheme with extension rod is difficult to implement, and the extension rod is prone to interference with the protrusion.

[0007] Therefore, there is an urgent need for a hand-eye calibration method that does not rely on an extension rod to overcome the problems existing in the above-mentioned prior art. Summary of the Invention

[0008] In view of the above problems, embodiments of the present invention are proposed to provide a hand-eye calibration method based on a clamping and adsorption component, a hand-eye calibration system based on a clamping and adsorption component, a computer device, and a storage medium to overcome or at least partially solve the above problems.

[0009] To address the aforementioned problems, this invention discloses a hand-eye calibration method based on a clamping and adsorption component. An industrial camera is fixedly mounted above a robot, and a clamping and adsorption component is fixedly mounted at the robot's end effector. The clamping and adsorption component is used to adsorb the calibration object, including:

[0010] Control the gripping and adsorption components to place the material at a first position within the camera's field of view, and control the gripping and adsorption components to place the material at a second position within the camera's field of view. Control the camera to take pictures of the material, and obtain N sets of robot end-effector poses and pixel coordinates.

[0011] The uncorrected camera calibration parameters are obtained by solving N sets of robot end-effector poses and pixel coordinates, and the transformation function is obtained based on the uncorrected camera calibration parameters.

[0012] Control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption components to place the material in the third position within the camera's field of view, and control the gripping and adsorption components to place the material in the fourth position within the camera's field of view. Control the camera to take pictures of the material, and obtain M sets of robot end effector poses and pixel coordinates.

[0013] The tool coordinates are obtained by using the robot end-effector pose and pixel coordinates corresponding to the M groups through a transformation function;

[0014] Calculate the coordinates of the material in the robot's base coordinate system, and use N sets of robot end-effector poses and pixel coordinates to solve for the camera calibration parameters.

[0015] Preferably, the control gripping and adsorption component places the material at a first position within the camera's field of view, and controls the gripping and adsorption component to place the material at a second position within the camera's field of view, controlling the camera to photograph the material, thereby obtaining N sets of robot end-effector poses and pixel coordinates, including:

[0016] The clamping and adsorption components are controlled to place the material in the first position within the camera's field of view. The robot's end-effector pose is obtained. The robot's end-effector is then controlled to move to the standby position. The camera is then controlled to take a picture of the material, and the pixel coordinates of the material's outline center are obtained.

[0017] The clamping and adsorption components are controlled to place the material in the second position within the camera's field of view. The robot's end-effector pose is obtained. The robot is then controlled to move to the standby position. The camera is then controlled to take a picture of the material, and the pixel coordinates of the material's outline center are obtained. The above steps are repeated to obtain N sets of robot end-effector poses and pixel coordinates.

[0018] Preferably, the robot end effector moves to the calibration plane at the center of the camera's field of view, the gripping and adsorption component is controlled to place the material at a third position within the camera's field of view, and the gripping and adsorption component is controlled to place the material at a fourth position within the camera's field of view. The camera is then controlled to photograph the material, obtaining M sets of robot end effector poses and pixel coordinates, including:

[0019] Control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption components to place the material in the third position within the camera's field of view, and obtain the robot end effector pose; control the robot end effector to move to the standby position, control the camera to take a picture of the material, and obtain the pixel coordinates of the material's outline center;

[0020] Control the robot end effector to return to the position where the material was placed last time, control the gripping and adsorption components to pick up the material and lift it to a certain height, after the robot end effector rotation center rotates a certain angle, it descends to the calibration plane, control the gripping and adsorption components to release the material, and place the material in the fourth position within the camera's field of view to obtain the robot end effector pose;

[0021] Control the robot end effector to move to the standby position, control the camera to take a picture of the material, and obtain the pixel coordinates of the center of the material outline. Repeat the above steps to obtain M sets of robot end effector poses and pixel coordinates.

[0022] Preferably, the method further includes:

[0023] By subtracting the equations obtained from the i-th and j-th rotations of the robot's end effector rotation center and eliminating the fixed values, a linear equation is obtained.

[0024] Preferably, the step of obtaining the tool coordinates by using the robot end-effector poses and pixel coordinates corresponding to M groups through a transformation function includes:

[0025] By using the transformation function and the robot end-effector pose and pixel coordinates corresponding to the M groups, the linear equation is expanded into multiple linear equations, and the tool coordinates are obtained by solving the least squares method.

[0026] Preferably, the calculation of the material's coordinates in the robot's base coordinate system involves using N sets of robot end-effector poses and pixel coordinates to obtain camera calibration parameters, including:

[0027] The coordinates of the material in the robot's base coordinate system are obtained by converting the tool coordinates. Using N sets of robot end poses and pixel coordinates, the camera calibration linear model is expanded into 2N linear equations. The camera calibration parameters can be obtained by solving the least squares method.

[0028] This invention discloses a hand-eye calibration system based on a clamping and adsorption component. An industrial camera is fixedly mounted above a robot, and a clamping and adsorption component is fixedly mounted on the robot's end effector. The clamping and adsorption component is used to adsorb calibration materials and includes:

[0029] The first data acquisition module is used to control the clamping and adsorption component to place the material at a first position within the camera's field of view, and to control the clamping and adsorption component to place the material at a second position within the camera's field of view, and to control the camera to take pictures of the material, thereby obtaining N sets of robot end-effector poses and pixel coordinates.

[0030] The first solution module is used to obtain uncorrected camera calibration parameters by solving N sets of robot end poses and pixel coordinates, and to obtain a transformation function based on the uncorrected camera calibration parameters;

[0031] The second data acquisition module is used to control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption components to place the material in the third position within the camera's field of view, and control the gripping and adsorption components to place the material in the fourth position within the camera's field of view, and control the camera to take pictures of the material, thereby obtaining M sets of robot end effector poses and pixel coordinates.

[0032] The second solution module is used to solve for the tool coordinates by using the robot end pose and pixel coordinates corresponding to M groups through a transformation function.

[0033] The third solution module is used to calculate the coordinates of the material in the robot's base coordinate system. It uses N sets of robot end-effector poses and pixel coordinates to solve for the camera calibration parameters.

[0034] Preferably, the first data acquisition module includes:

[0035] The first placement submodule is used to control the clamping and adsorption component to place the material in the first position within the camera's field of view, obtain the robot end pose, control the robot end to move to the standby position, control the camera to take a picture of the material, and obtain the pixel coordinates of the material's outline center.

[0036] The second placement submodule is used to control the clamping and adsorption components to place the material in the second position within the camera's field of view, obtain the robot's end-effector pose, control the robot to move to the standby position, control the camera to take a picture of the material, obtain the pixel coordinates of the material's outline center, and repeat the above steps to obtain N sets of robot end-effector poses and pixel coordinates.

[0037] This invention also discloses a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described hand-eye calibration steps based on the clamping and adsorption component.

[0038] This invention also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described hand-eye calibration steps based on the clamping and adsorption component.

[0039] The embodiments of the present invention have the following advantages:

[0040] The hand-eye calibration method based on a gripping and adsorption component in this embodiment of the invention includes: controlling the gripping and adsorption component to place the material at a first position within the camera's field of view, and controlling the gripping and adsorption component to place the material at a second position within the camera's field of view; controlling the camera to photograph the material to obtain N sets of robot end-effector poses and pixel coordinates; obtaining uncorrected camera calibration parameters by solving for the N sets of robot end-effector poses and pixel coordinates; obtaining a transformation function based on the uncorrected camera calibration parameters; controlling the robot end-effector to move to the calibration plane at the center of the camera's field of view; controlling the gripping and adsorption component to place the material at the first position within the camera's field of view; and controlling the gripping and adsorption component to place the material at the second position within the camera's field of view. In the second position within the robot, the camera is controlled to photograph the material, obtaining M sets of robot end-effector poses and pixel coordinates. Using a transformation function with the corresponding M sets of robot end-effector poses and pixel coordinates, the tool coordinates are calculated. The coordinates of the material in the robot's base coordinate system are calculated, and using N sets of robot end-effector poses and pixel coordinates, the camera calibration parameters are obtained. This invention utilizes a gripping and adsorption component installed at the robot end-effector to adsorb the material, placing it at several positions within the camera's field of view. Hand-eye calibration is achieved by visually acquiring the pixel coordinates of the material's feature points, eliminating the need for an extension rod. This fundamentally avoids the height difference between the calibration plane and the working plane caused by the thickness of the extension rod, improving calibration accuracy. The absence of an extension rod avoids interference problems between the extension rod and obstacles such as protrusions near the calibration area, broadening its applicability, especially suitable for industrial sites with limited calibration space. This invention employs a two-stage calibration process: first, obtaining uncorrected camera calibration parameters, then solving for the tool coordinates, ultimately obtaining accurate camera calibration parameters with high calibration accuracy. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the steps of a hand-eye calibration method based on a clamping and adsorption component according to an embodiment of the present invention.

[0043] Figure 2 This is a schematic diagram of the structure of a calibration system according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the field of view of a camera according to an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the field of view of a camera according to an embodiment of the present invention;

[0046] Figure 5 This is a structural block diagram of an embodiment of a hand-eye calibration system based on a clamping and adsorption component according to the present invention;

[0047] Figure 6 This is an internal structural diagram of a computer device according to one embodiment. Detailed Implementation

[0048] To make the technical problems, technical solutions, and beneficial effects solved by the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0049] Reference Figure 1 This diagram illustrates a step-by-step flowchart of a hand-eye calibration method based on a clamping and adsorption component according to an embodiment of the present invention. An industrial camera is fixedly mounted above a robot, and a clamping and adsorption component is fixedly mounted on the robot's end effector. The clamping and adsorption component is used to adsorb the calibration object, and the method may specifically include the following steps:

[0050] Step 101: Control the clamping and adsorption component to place the material at a first position within the camera's field of view, and control the clamping and adsorption component to place the material at a second position within the camera's field of view. Control the camera to take pictures of the material and obtain N sets of robot end-effector poses and pixel coordinates.

[0051] like Figure 2 As shown, the clamping and adsorption component may include a suction nozzle, an electromagnet, a pneumatic gripper, or other clamping or adsorption components. This embodiment of the invention does not impose excessive limitations on this. Correspondingly, the material may include materials of different materials such as cardboard or metal sheets. This embodiment of the invention does not impose excessive limitations on this. The calibration system of this application may include a frame, an industrial camera, a robot, a clamping and adsorption component (taking a suction nozzle as an example), and materials. The calibration area is the field of view of the industrial camera. Figure 3 As shown.

[0052] In this embodiment of the invention, the control gripping and adsorption component places the material at a first position within the camera's field of view, and the control gripping and adsorption component places the material at a second position within the camera's field of view, controlling the camera to photograph the material, thereby obtaining N sets of robot end-effector poses and pixel coordinates, including:

[0053] The clamping and adsorption components are controlled to place the material in the first position within the camera's field of view. The robot's end-effector pose is obtained. The robot's end-effector is then controlled to move to the standby position. The camera is then controlled to take a picture of the material, and the pixel coordinates of the material's outline center are obtained.

[0054] The clamping and adsorption components are controlled to place the material at a second position within the camera's field of view, the robot's end-effector pose is acquired, and the robot is then moved to a standby position. The camera takes a picture of the material, obtaining the pixel coordinates of the material's outline center. The above steps are repeated to obtain N sets of robot end-effector poses and pixel coordinates.

[0055] A suction nozzle is fixedly mounted on the end effector of a robot, which picks up a material. The robot's end effector is moved to a point within the camera's field of view, its pose is acquired, the nozzle breaks the vacuum, and the material is placed at position 1 (the first position) within the camera's field of view. The robot's end effector is then moved to a standby position (ensuring it doesn't obstruct the camera from photographing the material). The camera photographs the material, obtaining the pixel coordinates of its outline center. The robot's end effector returns to the previous position, the nozzle picks up the material, maintaining the same pose, and the robot's end effector is moved to another point within the camera's field of view, its pose is acquired, the nozzle breaks the vacuum, and the material is placed at position 2 (the second position) within the camera's field of view. The robot is then moved to a standby position. The camera photographs the material, obtaining the pixel coordinates of its outline center. This process is repeated multiple times to obtain N sets of one-to-one robot end effector poses (maintaining the same pose) and pixel coordinates.

[0056] Step 102: Obtain uncorrected camera calibration parameters by solving N sets of robot end-effector poses and pixel coordinates, and obtain a transformation function based on the uncorrected camera calibration parameters;

[0057] Based on the camera calibration linear model:

[0058] ;

[0059] Where (u,v) are pixel coordinates and (x,y) are Cartesian coordinates. To calibrate the camera parameters.

[0060] Using N sets of one-to-one robot end-effector poses (with the end-effector pose remaining unchanged) and pixel coordinates, equation (1) is expanded into 2*N linear equations, and the uncorrected camera calibration parameters are obtained by solving them using the least squares method. (Because the Cartesian coordinates used here are for the robot's end position, not for the material's coordinates in the robot's base coordinate system).

[0061] After obtaining the uncorrected camera calibration parameters, the conversion function between Cartesian coordinates and pixel coordinates can be obtained:

[0062] ;

[0063] in, , It is a fixed value.

[0064] Step 103: Control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption components to place the material in the third position within the camera's field of view, and control the gripping and adsorption components to place the material in the fourth position within the camera's field of view. Control the camera to take pictures of the material and obtain M sets of robot end effector poses and pixel coordinates.

[0065] The robot's end effector is controlled to move to the calibration plane at the center of the camera's field of view. The clamping and adsorption components are then controlled to place the material at a third position within the camera's field of view, and at a fourth position within the camera's field of view. The camera is then controlled to photograph the material, resulting in M ​​sets of robot end effector poses and pixel coordinates, including:

[0066] Control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption components to place the material in the third position within the camera's field of view, and obtain the robot end effector pose;

[0067] Control the robot's end effector to move to the standby position. The camera takes a picture of the material, obtaining the pixel coordinates of the material's outline center.

[0068] Control the robot end effector to return to the position where the material was placed last time, control the gripping and adsorption components to pick up the material and lift it to a certain height, after the robot end effector rotation center rotates a certain angle, it descends to the calibration plane, control the gripping and adsorption components to release the material, and place the material in the fourth position within the camera's field of view to obtain the robot end effector pose;

[0069] Control the robot's end effector to move to the standby position. Take a picture of the material with the camera to obtain the pixel coordinates of the material's outline center, and obtain the pose and pixel coordinates of the M robot end effectors.

[0070] Specifically, the robot's end effector can be controlled to move to a point P (located above the calibration plane) near the center of the camera's field of view, such as... Figure 4 As shown, after the robot's end effector rotates a certain angle, it descends to the calibration plane. The suction nozzle breaks the vacuum, placing the material at position A (the third position) within the camera's field of view. The robot's end effector pose is acquired, and the robot's end effector moves to the standby position. The camera takes a picture of the material, obtaining the pixel coordinates of the material's outline center. The robot's end effector returns to the position where the material was previously placed, the suction nozzle picks up the material and raises it to a certain height. After the robot's end effector rotates a certain angle, it descends to the calibration plane. The suction nozzle breaks the vacuum, placing the material at position B (the fourth position) within the camera's field of view. The robot's end effector pose is acquired, and the robot's end effector moves to the standby position. The camera takes a picture of the material, obtaining the pixel coordinates of the material's outline center. This process is repeated multiple times to obtain M sets of one-to-one robot end effector poses (with the end effector position remaining unchanged) and pixel coordinates.

[0071] Step 104: Using the robot end-effector pose and pixel coordinates corresponding to group M, the tool coordinates are obtained by solving the transformation function;

[0072] In this embodiment of the invention, the method further includes: subtracting the equations obtained from the i-th rotation and the j-th rotation of the robot's end effector rotation center, eliminating fixed values, and obtaining a linear equation.

[0073] Specifically, the step of obtaining the tool coordinates by using the robot end-effector poses and pixel coordinates corresponding to M groups through a transformation function includes:

[0074] By using the transformation function and the robot end-effector pose and pixel coordinates corresponding to the M groups, the linear equation is expanded into multiple linear equations, and the tool coordinates are obtained by solving the least squares method.

[0075] In this embodiment of the invention, the coordinates of the material in the robot's base coordinate system are:

[0076]

[0077] in, The coordinates of the robot's end effector (since the robot's end effector only performs rotational motion, therefore...) , (for fixed values) For tool coordinates, , , , The rotation matrix elements corresponding to the end pose.

[0078] From equations (2) and (3), we obtain:

[0079]

[0080] According to equation (4), the difference between the equations obtained from the i-th and j-th rotations of the robot's end effector rotation center is used to eliminate the fixed values. , , , ,get:

[0081]

[0082] Using the M sets of one-to-one robot end-effector poses (with the end-effector position remaining unchanged) and pixel coordinates, equation (5) can be expanded into multiple linear equations, and the tool coordinates can be obtained by solving them using the least squares method. .

[0083] Step 105: Calculate the coordinates of the material in the robot's base coordinate system. Using N sets of robot end-effector poses and pixel coordinates, solve for the camera calibration parameters.

[0084] Further applied to embodiments of the present invention, the calculation of the material's coordinates in the robot's base coordinate system, using N sets of robot end-effector poses and pixel coordinates, to obtain camera calibration parameters, includes:

[0085] The coordinates of the material in the robot's base coordinate system are obtained by converting the tool coordinates. Using N sets of robot end poses and pixel coordinates, the camera calibration linear model is expanded into 2N linear equations. The camera calibration parameters can be obtained by solving the least squares method.

[0086] After obtaining the tool coordinates, the material coordinates in the robot base coordinate system are converted according to equation (3). Using N sets of one-to-one material coordinates and pixel coordinates in the robot base coordinate system, equation (1) is expanded into 2*N linear equations. The camera calibration parameters can be obtained by solving the least squares method. This enables high-precision hand-eye calibration.

[0087] The hand-eye calibration method based on a gripping and adsorption component in this embodiment of the invention includes: controlling the gripping and adsorption component to place the material at a first position within the camera's field of view, and controlling the gripping and adsorption component to place the material at a second position within the camera's field of view; controlling the camera to photograph the material to obtain N sets of robot end-effector poses and pixel coordinates; obtaining uncorrected camera calibration parameters by solving for the N sets of robot end-effector poses and pixel coordinates; obtaining a transformation function based on the uncorrected camera calibration parameters; controlling the robot end-effector to move to the calibration plane at the center of the camera's field of view; controlling the gripping and adsorption component to place the material at the first position within the camera's field of view; and controlling the gripping and adsorption component to place the material at the second position within the camera's field of view. In the second position within the robot, the camera is controlled to photograph the material, obtaining M sets of robot end-effector poses and pixel coordinates. Using a transformation function with the corresponding M sets of robot end-effector poses and pixel coordinates, the tool coordinates are calculated. The coordinates of the material in the robot's base coordinate system are calculated, and using N sets of robot end-effector poses and pixel coordinates, the camera calibration parameters are obtained. This invention utilizes a gripping and adsorption component installed at the robot end-effector to adsorb the material, placing it at several positions within the camera's field of view. Hand-eye calibration is achieved by visually acquiring the pixel coordinates of the material's feature points, eliminating the need for an extension rod. This fundamentally avoids the height difference between the calibration plane and the working plane caused by the thickness of the extension rod, improving calibration accuracy. The absence of an extension rod avoids interference problems between the extension rod and obstacles such as protrusions near the calibration area, broadening its applicability, especially suitable for industrial sites with limited calibration space. This invention achieves high calibration accuracy through two calibration processes—first obtaining uncorrected camera calibration parameters, then solving for the tool coordinates.

[0088] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0089] Reference Figure 2 The diagram illustrates a structural block diagram of a hand-eye calibration system based on a clamping and adsorption component according to an embodiment of the present invention, which may specifically include the following modules:

[0090] The first data acquisition module 301 is used to control the clamping and adsorption component to place the material at a first position within the camera's field of view, and to control the clamping and adsorption component to place the material at a second position within the camera's field of view, and to control the camera to take pictures of the material, thereby obtaining N sets of robot end poses and pixel coordinates.

[0091] The first solving module 302 is used to obtain uncorrected camera calibration parameters by solving N sets of robot end poses and pixel coordinates, and to obtain a transformation function based on the uncorrected camera calibration parameters.

[0092] The second data acquisition module 303 is used to control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption component to place the material in the third position within the camera's field of view, and control the gripping and adsorption component to place the material in the fourth position within the camera's field of view, control the camera to take pictures of the material, and obtain M sets of robot end effector poses and pixel coordinates.

[0093] The second solving module 304 is used to solve for the tool coordinates by using the robot end pose and pixel coordinates corresponding to the M groups through a transformation function.

[0094] The third solution module 305 is used to calculate the coordinates of the material in the robot's base coordinate system. It uses N sets of robot end poses and pixel coordinates to solve for the camera calibration parameters.

[0095] Preferably, the first data acquisition module includes:

[0096] The first placement submodule is used to control the clamping and adsorption component to place the material in the first position within the camera's field of view, obtain the robot end pose, control the robot end to move to the standby position, control the camera to take a picture of the material, and obtain the pixel coordinates of the material's outline center.

[0097] The second placement submodule controls the clamping and adsorption components to place the material at a second position within the camera's field of view, thereby acquiring the robot's end-effector pose. It controls the robot to move to a standby position, controls the camera to photograph the material, and obtains the pixel coordinates of the material's outline center. The above steps are repeated to obtain N sets of robot end-effector poses and pixel coordinates.

[0098] Preferably, the second data acquisition module includes:

[0099] The third placement submodule is used to control the robot end effector to move to the calibration plane at the center of the camera's field of view, and to control the gripping and adsorption components to place the material in the third position within the camera's field of view, thereby obtaining the robot end effector's pose.

[0100] The first imaging submodule is used to control the robot end effector to move to the standby position, control the camera to take pictures of the material, and obtain the pixel coordinates of the center of the material outline.

[0101] The fourth placement submodule is used to control the robot end effector to return to the position where the material was placed last time, control the gripping and adsorption components to pick up the material and lift it to a certain height, and after the robot end effector rotation center rotates a certain angle, it descends to the calibration plane, controls the gripping and adsorption components to release the material, and places the material in the fourth position within the camera's field of view to obtain the robot end effector pose.

[0102] The second imaging submodule is used to control the robot end effector to move to the standby position, control the camera to take pictures of the material, obtain the pixel coordinates of the material outline center, and repeat the above steps to obtain M sets of robot end effector poses and pixel coordinates.

[0103] Preferably, the system further includes:

[0104] The equation construction module is used to subtract the equations obtained from the i-th and j-th rotations of the robot's end effector rotation center, eliminate fixed values, and obtain linear equations.

[0105] Preferably, the second solution module includes:

[0106] The extended submodule is used to extend the linear equation into multiple linear equations by using the robot end pose and pixel coordinates corresponding to the M groups through the transformation function, and then solves the tool coordinates using the least squares method.

[0107] Preferably, the third solution module includes:

[0108] The calibration submodule is used to calculate the coordinates of the material in the robot's base coordinate system based on the tool coordinates. It uses N sets of robot end poses and pixel coordinates to expand the camera calibration linear model into 2N linear equations, which can be solved using the least squares method to obtain the camera calibration parameters.

[0109] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0110] Specific limitations regarding the hand-eye calibration system based on clamping and adsorption components can be found in the limitations of the hand-eye calibration method based on clamping and adsorption components described above, and will not be repeated here. Each module in the aforementioned hand-eye calibration system based on clamping and adsorption components can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0111] The hand-eye calibration system based on the clamping and adsorption component provided above can be used to execute the hand-eye calibration method based on the clamping and adsorption component provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0112] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a hand-eye calibration method based on a gripping and adsorption component. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0113] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0114] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the above steps.

[0115] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the above steps.

[0116] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of apparatus, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction methods implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0123] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or terminal device that includes said element.

[0124] The foregoing has provided a detailed description of a hand-eye calibration method and system based on a clamping and adsorption component, a computer device, and a storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A hand-eye calibration method based on a clamping and adsorption component, characterized in that, An industrial camera is fixedly mounted above the robot, and a clamping and adsorption component is fixedly mounted at the robot's end effector. This clamping and adsorption component is used to adsorb a calibration sample, and includes: Control the gripping and adsorption components to place the material at a first position within the camera's field of view, and control the gripping and adsorption components to place the material at a second position within the camera's field of view. Control the camera to take pictures of the material, and obtain N sets of robot end-effector poses and pixel coordinates. The uncorrected camera calibration parameters are obtained by solving N sets of robot end-effector poses and pixel coordinates, and the transformation function is obtained based on the uncorrected camera calibration parameters. Control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption components to place the material in the third position within the camera's field of view, and control the gripping and adsorption components to place the material in the fourth position within the camera's field of view. Control the camera to take pictures of the material, and obtain M sets of robot end effector poses and pixel coordinates. The tool coordinates are obtained by using the robot end-effector pose and pixel coordinates corresponding to the M groups through a transformation function; Calculate the coordinates of the material in the robot's base coordinate system, and use N sets of robot end-effector poses and pixel coordinates to solve for the camera calibration parameters.

2. The method according to claim 1, characterized in that, The control gripping and adsorption component places the material at a first position within the camera's field of view, and the control gripping and adsorption component places the material at a second position within the camera's field of view, controlling the camera to photograph the material, obtaining N sets of robot end-effector poses and pixel coordinates, including: The clamping and adsorption components are controlled to place the material in the first position within the camera's field of view. The robot's end-effector pose is obtained. The robot's end-effector is then controlled to move to the standby position. The camera is then controlled to take a picture of the material, and the pixel coordinates of the material's outline center are obtained. The clamping and adsorption components are controlled to place the material in the second position within the camera's field of view. The robot's end-effector pose is obtained. The robot is then controlled to move to the standby position. The camera is then controlled to take a picture of the material, and the pixel coordinates of the material's outline center are obtained. The above steps are repeated to obtain N sets of robot end-effector poses and pixel coordinates.

3. The method according to claim 1, characterized in that, The robot's end effector is controlled to move to the calibration plane at the center of the camera's field of view. The clamping and adsorption components are then controlled to place the material at a third position within the camera's field of view, and at a fourth position within the camera's field of view. The camera is then controlled to photograph the material, resulting in M ​​sets of robot end effector poses and pixel coordinates, including: Control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption components to place the material in the third position within the camera's field of view, and obtain the robot end effector pose; control the robot end effector to move to the standby position, control the camera to take a picture of the material, and obtain the pixel coordinates of the material's outline center; Control the robot end effector to return to the position where the material was placed last time, control the gripping and adsorption components to pick up the material and lift it to a certain height, after the robot end effector rotation center rotates a certain angle, it descends to the calibration plane, control the gripping and adsorption components to release the material, and place the material in the fourth position within the camera's field of view to obtain the robot end effector pose; Control the robot end effector to move to the standby position, control the camera to take a picture of the material, and obtain the pixel coordinates of the center of the material outline. Repeat the above steps to obtain M sets of robot end effector poses and pixel coordinates.

4. The method according to claim 1, characterized in that, The method further includes: By subtracting the equations obtained from the i-th and j-th rotations of the robot's end effector rotation center and eliminating the fixed values, a linear equation is obtained.

5. The method according to claim 1, characterized in that, The step of obtaining tool coordinates by using the robot end-effector poses and pixel coordinates corresponding to M groups through a transformation function includes: By using the transformation function and the robot end-effector pose and pixel coordinates corresponding to the M groups, the linear equation is expanded into multiple linear equations, and the tool coordinates are obtained by solving the least squares method.

6. The method according to claim 5, characterized in that, The coordinates of the material in the robot's base coordinate system are calculated using N sets of robot end-effector poses and pixel coordinates to obtain camera calibration parameters, including: The coordinates of the material in the robot's base coordinate system are obtained by converting the tool coordinates. Using N sets of robot end poses and pixel coordinates, the camera calibration linear model is expanded into 2N linear equations. The camera calibration parameters can be obtained by solving the least squares method.

7. A hand-eye calibration system based on a clamping and adsorption component, characterized in that, An industrial camera is fixedly mounted above the robot, and a clamping and adsorption component is fixedly mounted at the robot's end effector. This clamping and adsorption component is used to adsorb a calibration sample, and includes: The first data acquisition module is used to control the clamping and adsorption component to place the material at a first position within the camera's field of view, and to control the clamping and adsorption component to place the material at a second position within the camera's field of view, and to control the camera to take pictures of the material, thereby obtaining N sets of robot end-effector poses and pixel coordinates. The first solution module is used to obtain uncorrected camera calibration parameters by solving N sets of robot end poses and pixel coordinates, and to obtain a transformation function based on the uncorrected camera calibration parameters; The second data acquisition module is used to control the robot end effector to move to the calibration plane at the center of the camera's field of view, control the gripping and adsorption components to place the material in the third position within the camera's field of view, and control the gripping and adsorption components to place the material in the fourth position within the camera's field of view, and control the camera to take pictures of the material, thereby obtaining M sets of robot end effector poses and pixel coordinates. The second solution module is used to solve for the tool coordinates by using the robot end pose and pixel coordinates corresponding to M groups through a transformation function. The third solution module is used to calculate the coordinates of the material in the robot's base coordinate system. It uses N sets of robot end-effector poses and pixel coordinates to solve for the camera calibration parameters.

8. The system according to claim 7, characterized in that, The first data acquisition module includes: The first placement submodule is used to control the clamping and adsorption component to place the material in the first position within the camera's field of view, obtain the robot end pose, control the robot end to move to the standby position, control the camera to take a picture of the material, and obtain the pixel coordinates of the material's outline center. The second placement submodule is used to control the clamping and adsorption components to place the material in the second position within the camera's field of view, obtain the robot's end-effector pose, control the robot to move to the standby position, control the camera to take a picture of the material, obtain the pixel coordinates of the material's outline center, and repeat the above steps to obtain N sets of robot end-effector poses and pixel coordinates.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the hand-eye calibration method based on the clamping and adsorption component as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the hand-eye calibration method based on the clamping and adsorption component as described in any one of claims 1 to 6.

Images