Collaborative robot automatic calibration method and device, computer device and storage medium

Abstract

The application relates to a collaborative robot automatic calibration method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: generating a first shooting track according to a first distance between a camera arranged on the end of a collaborative robot and a charuco calibration board placed on a work plane of the collaborative robot; performing camera calibration and hand-eye calibration on the collaborative robot according to the first shooting track, to obtain first camera calibration parameters and first hand-eye calibration results; performing camera calibration and hand-eye calibration on the collaborative robot according to a second shooting track generated according to the first camera calibration parameters and the first hand-eye calibration results, to obtain second camera calibration parameters and second hand-eye calibration results; and performing work plane calibration on the collaborative robot according to the second camera calibration parameters and the second hand-eye calibration results. The method can improve the accuracy of work plane calibration of the collaborative robot.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for automatic calibration of collaborative robots. Background Technology

[0002] With the widespread application of collaborative robots, they have played an irreplaceable role in fields such as industrial 3C, automotive electronics, daily necessities, and biosafety, greatly improving product quality and production efficiency while effectively protecting human safety. To better enable collaborative robots to serve these applications, machine vision technology has emerged. This technology can significantly enhance collaborative robots' perception of their surroundings, allowing them to more acutely and intelligently perceive human activities in scenarios involving human interaction, thereby preventing dangerous actions.

[0003] To better achieve the linkage between robots and vision, it is inevitable to complete the relevant calibration first. The commonly used calibration method is to first determine the camera parameters and hand-eye relationship through camera calibration and hand-eye calibration, and then calibrate the working plane of the collaborative robot according to the camera parameters and hand-eye relationship.

[0004] However, the above calibration method requires the distance parameter to be determined manually to generate the shooting trajectory. This distance parameter is usually determined manually and will have a certain error. Therefore, the accuracy of camera calibration and hand-eye calibration based on this shooting trajectory is low, which leads to low accuracy of working plane calibration for collaborative robots and cannot meet the requirements of high-precision application scenarios. Summary of the Invention

[0005] Therefore, it is necessary to provide an automatic calibration method, apparatus, computer equipment, computer-readable storage medium, and computer program product for collaborative robots that can improve the calibration accuracy of the working plane of collaborative robots, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides an automatic calibration method for collaborative robots. The method includes:

[0007] A first shooting trajectory is generated based on the first distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot;

[0008] Based on the first shooting trajectory, the collaborative robot is calibrated with both camera and hand-eye calibration to obtain the first camera calibration parameters and the first hand-eye calibration result.

[0009] Based on the second shooting trajectory generated by the first camera calibration parameters and the first hand-eye calibration result, the collaborative robot is calibrated for both camera and hand-eye calibration to obtain the second camera calibration parameters and the second hand-eye calibration result.

[0010] The working plane of the collaborative robot is calibrated based on the second camera calibration parameters and the second hand-eye calibration results.

[0011] In one embodiment, the step of performing camera calibration and hand-eye calibration on the collaborative robot based on the second shooting trajectory generated according to the first camera calibration parameters and the first hand-eye calibration result, to obtain second camera calibration parameters and second hand-eye calibration results, includes:

[0012] Based on the first camera calibration parameters and the first hand-eye calibration result, a second shooting trajectory is generated; based on the second shooting trajectory, the camera set on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a first captured image; based on the first captured image, the collaborative robot is calibrated using a camera to obtain second camera calibration parameters; based on the first captured image and the second camera calibration parameters, the collaborative robot is calibrated using hand-eye calibration to obtain a second hand-eye calibration result.

[0013] In one embodiment, generating the second shooting trajectory based on the first camera calibration parameters and the first hand-eye calibration result includes:

[0014] Based on the first camera calibration parameters, a second distance between the camera and the Charuco calibration board is determined; based on the first hand-eye calibration result and the second distance, spherical coordinates are generated, and the spherical coordinates within a preset angle range are used as the second shooting trajectory.

[0015] In one embodiment, the step of controlling a camera mounted on the end effector of the collaborative robot to capture an image of the Charuco calibration board based on the second shooting trajectory to obtain a first captured image includes:

[0016] Obtain the ideal affine transformation image of the Charuco calibration board; according to the second shooting trajectory, control the camera set on the end of the collaborative robot to take an image of the Charuco calibration board to obtain a second shooting image; if the error between the second shooting image and the ideal affine transformation image is less than a preset affine transformation error, then output the second shooting image as the first shooting image.

[0017] In one embodiment, calibrating the working plane of the collaborative robot based on the second camera calibration parameters and the second hand-eye calibration result includes:

[0018] Based on the second camera calibration parameters and the second hand-eye calibration results, the camera set on the end of the collaborative robot is controlled to capture an image of the Charuco calibration board to obtain a third captured image.

[0019] The working plane of the collaborative robot is calibrated based on the third captured image, the second camera calibration parameters, and the second hand-eye calibration results.

[0020] In one embodiment, controlling the camera mounted on the end effector of the collaborative robot to capture an image of the Charuco calibration board based on the second camera calibration parameters and the second hand-eye calibration result, to obtain a third captured image, includes:

[0021] Based on the second camera calibration parameters and the second hand-eye calibration results, a planar shooting point is generated; based on the planar shooting point, the camera set on the end of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a fourth captured image; based on the second distortion coefficient in the second camera calibration parameters, the fourth captured image is subjected to distortion correction processing to obtain a third captured image.

[0022] Secondly, this application also provides an automatic calibration device for collaborative robots. The device includes:

[0023] The shooting trajectory generation module is used to generate a first shooting trajectory based on a first distance between the camera set on the end of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot;

[0024] The coarse calibration module is used to perform camera calibration and hand-eye calibration on the collaborative robot based on the first shooting trajectory, and obtain the first camera calibration parameters and the first hand-eye calibration result;

[0025] The fine calibration module is used to perform camera calibration and hand-eye calibration on the collaborative robot based on the second shooting trajectory generated by the first camera calibration parameters and the first hand-eye calibration result, so as to obtain the second camera calibration parameters and the second hand-eye calibration result;

[0026] The working plane calibration module is used to calibrate the working plane of the collaborative robot based on the second camera calibration parameters and the second hand-eye calibration results.

[0027] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0028] A first shooting trajectory is generated based on a first distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot. Based on the first shooting trajectory, camera calibration and hand-eye calibration are performed on the collaborative robot to obtain first camera calibration parameters and first hand-eye calibration results. A second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration results, and camera calibration and hand-eye calibration are performed on the collaborative robot to obtain second camera calibration parameters and second hand-eye calibration results. Based on the second camera calibration parameters and the second hand-eye calibration results, the working plane of the collaborative robot is calibrated.

[0029] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0030] A first shooting trajectory is generated based on a first distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot. Based on the first shooting trajectory, camera calibration and hand-eye calibration are performed on the collaborative robot to obtain first camera calibration parameters and first hand-eye calibration results. A second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration results, and camera calibration and hand-eye calibration are performed on the collaborative robot to obtain second camera calibration parameters and second hand-eye calibration results. Based on the second camera calibration parameters and the second hand-eye calibration results, the working plane of the collaborative robot is calibrated.

[0031] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0032] A first shooting trajectory is generated based on a first distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot. Based on the first shooting trajectory, camera calibration and hand-eye calibration are performed on the collaborative robot to obtain first camera calibration parameters and first hand-eye calibration results. A second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration results, and camera calibration and hand-eye calibration are performed on the collaborative robot to obtain second camera calibration parameters and second hand-eye calibration results. Based on the second camera calibration parameters and the second hand-eye calibration results, the working plane of the collaborative robot is calibrated.

[0033] The aforementioned collaborative robot automatic calibration method, apparatus, computer equipment, storage medium, and computer program product perform camera calibration and hand-eye calibration on the collaborative robot by generating a second shooting trajectory based on the first camera calibration parameters and the first hand-eye calibration result. This yields second camera calibration parameters and second hand-eye calibration results. Since the second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration result, the trajectory coordinates of the second shooting trajectory are more accurate. Therefore, the second camera calibration parameters and second hand-eye calibration results obtained using this shooting trajectory have higher accuracy. Consequently, calibrating the collaborative robot's working plane based on the second camera calibration parameters and the second hand-eye calibration result results in a more accurate coordinate relationship between the collaborative robot's base and its working plane, thereby improving the calibration accuracy of the collaborative robot's working plane calibration. Attached Figure Description

[0034] Figure 1 This is an application environment diagram of the automatic calibration method for collaborative robots in one embodiment;

[0035] Figure 2 This is a diagram illustrating the working environment of an automatic calibration method for collaborative robots in one embodiment.

[0036] Figure 3 This is a flowchart illustrating an automatic calibration method for collaborative robots in one embodiment;

[0037] Figure 4 This is a flowchart illustrating the fine calibration steps in one embodiment;

[0038] Figure 5 This is a structural block diagram of an automatic calibration device for a collaborative robot in one embodiment;

[0039] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] The collaborative robot automatic calibration method of this application obtains more accurate camera calibration parameters and hand-eye calibration results through two calibrations: coarse calibration and fine calibration. Based on the above camera calibration parameters and hand-eye calibration results, the working plane of the collaborative robot is calibrated, which can improve the calibration accuracy of the working plane calibration.

[0042] The automatic calibration method for collaborative robots provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with collaborative robot 104 via a network. Terminal 102 generates a first shooting trajectory based on a first distance between a camera mounted on the end effector of collaborative robot 104 and a Charuco calibration plate placed on the working plane of collaborative robot 104. Terminal 102 controls the camera mounted on the end effector of collaborative robot 104 to take pictures of the Charuco calibration plate at various coordinate points along the first shooting trajectory. Terminal 102 performs camera calibration and hand-eye calibration on collaborative robot 104 based on the captured images, obtaining first camera calibration parameters and first hand-eye calibration results. Based on the second shooting trajectory generated from the first camera calibration parameters and first hand-eye calibration results, terminal 102 performs camera calibration and hand-eye calibration on collaborative robot 104, obtaining second camera calibration parameters and second hand-eye calibration results. Based on the second camera calibration parameters and second hand-eye calibration results, terminal 102 performs working plane calibration on collaborative robot 104. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc.

[0043] The automatic calibration method for collaborative robots provided in this application can be applied to, for example... Figure 2 The work environment shown. Among them, the industrial control computer corresponds to... Figure 1 Terminal 102, corresponding to collaborative robots Figure 1 The collaborative robot 104 communicates with the industrial control computer via network. A camera is installed on the end effector of the collaborative robot, and a Charuco calibration plate is placed on the working plane of the collaborative robot.

[0044] In one embodiment, such as Figure 3 As shown, an automatic calibration method for collaborative robots is provided. This embodiment applies this method to... Figure 1 The method is illustrated using terminal 102 as an example. In this embodiment, the method includes the following steps:

[0045] Step 202: Generate a first shooting trajectory based on the first distance between the camera set on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot.

[0046] The camera installed on the end effector of the collaborative robot can be set by the robot manufacturer before the robot leaves the factory, or it can be added by the user when expanding the functionality of the collaborative robot; the working plane of the collaborative robot can be the working plane of the collaborative robot, and the work items of the collaborative robot are stored on this plane; the Charuco calibration board is a special calibration board. The Charuco calibration board can use the Aruco encoding function, so it is not necessary to take a picture of the entire calibration board, and the calibration task can be completed even if there is obstruction.

[0047] As an example, step 202 includes: obtaining the distance between the camera set on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot, and generating a spherical trajectory with a preset angle range as the first shooting trajectory based on this distance.

[0048] As an example, the preset angle range can be 0~60°.

[0049] As an example, the Charuco calibration board has a grid size of 7×5 and the Aruco encoding format is DICT_6*6.

[0050] Step 204: Based on the first shooting trajectory, perform camera calibration and hand-eye calibration on the collaborative robot to obtain the first camera calibration parameters and the first hand-eye calibration results.

[0051] The first shooting trajectory includes multiple shooting trajectory coordinate points. Camera calibration is used to obtain camera calibration parameters, which may include camera intrinsic parameters, camera extrinsic parameters, and distortion coefficients, etc. Hand-eye calibration is used to obtain hand-eye calibration results, which may be the coordinate transformation relationship between the collaborative robot end effector and the camera.

[0052] As an example, step 204 includes: according to the coordinate points of the shooting trajectory of the first shooting trajectory, controlling the camera set on the end of the collaborative robot to move to each shooting trajectory coordinate point to take pictures of the calibration board respectively; based on the images taken by the camera, performing camera calibration and hand-eye calibration on the collaborative robot to obtain the first camera calibration parameters and the first hand-eye calibration results.

[0053] Step 206: Based on the second shooting trajectory generated by the first camera calibration parameters and the first hand-eye calibration results, perform camera calibration and hand-eye calibration on the collaborative robot to obtain the second camera calibration parameters and the second hand-eye calibration results.

[0054] The second shooting trajectory includes multiple shooting trajectory coordinate points. The intrinsic and extrinsic parameters in the first camera calibration parameters, as well as the first hand-eye calibration results, help to calculate the distance between the camera set on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot, thus generating more accurate shooting trajectory coordinates.

[0055] As an example, step 206 includes: generating a second shooting trajectory based on the first camera calibration parameters and the first hand-eye calibration result; controlling the camera set on the end of the collaborative robot to move to each shooting trajectory coordinate point to take pictures of the calibration board respectively based on the shooting trajectory coordinate points of the second shooting trajectory; and performing camera calibration and hand-eye calibration on the collaborative robot based on the captured images to obtain the second camera calibration parameters and the second hand-eye calibration result.

[0056] Step 208: Based on the calibration parameters of the second camera and the calibration results of the second hand-eye alignment, perform work plane calibration on the collaborative robot.

[0057] Among them, the working plane calibration is used to generate the coordinate relationship between the working plane and the collaborative robot base. The more accurate this relationship is, the higher the accuracy of the working plane calibration.

[0058] As an example, step 208 includes: generating a preset number of shooting coordinate points based on the second calibration parameters and the second hand-eye calibration results; controlling the camera set on the end of the collaborative robot to take pictures of the calibration board; and calibrating the working plane of the collaborative robot based on the captured pictures, the second camera calibration parameters, and the second hand-eye calibration results.

[0059] The aforementioned automatic calibration method for collaborative robots generates a second shooting trajectory based on the calibration parameters of the first camera and the first hand-eye calibration results. Since the calibration parameters of the first camera include the camera's intrinsic and extrinsic parameters, and the first hand-eye calibration results include the coordinate transformation relationship between the collaborative robot's end effector and the camera, the coordinates of the second shooting trajectory can be made more accurate. Therefore, the second camera calibration parameters and the second hand-eye calibration results obtained by calibrating based on the calibration board image captured at the coordinates of this shooting trajectory are more accurate. Thus, by calibrating the collaborative robot's working plane based on the more accurate second camera calibration parameters and the more accurate second hand-eye calibration results, the coordinate relationship between the collaborative robot's base and the collaborative robot's working plane is obtained more accurately.

[0060] In one embodiment, such as Figure 4 As shown, based on the second shooting trajectory generated by the first camera calibration parameters and the first hand-eye calibration results, camera calibration and hand-eye calibration are performed on the collaborative robot to obtain the second camera calibration parameters and the second hand-eye calibration results, including:

[0061] Step 302: Generate a second shooting trajectory based on the first camera calibration parameters and the first hand-eye calibration results.

[0062] As an example, step 302 includes: determining the distance between the camera and the Charuco calibration board based on the first camera calibration parameters and the first hand-eye calibration results, and generating trajectory coordinate points at equal distances between the camera and the Charuco calibration board as the second shooting trajectory.

[0063] Step 304: Based on the second shooting trajectory, control the camera set on the end effector of the collaborative robot to take an image of the Charuco calibration board and obtain the first image.

[0064] The second shooting trajectory includes the trajectory coordinates of multiple collaborative robot end-effectors, and the first shooting image includes multiple images taken by the calibration board.

[0065] As an example, step 304 includes: controlling the end effector of the collaborative robot to move to each coordinate point according to the trajectory coordinate points of each end effector in the second shooting trajectory, controlling the camera set on the end effector of the collaborative robot to take pictures of the Charuco calibration board, and taking the first picture of each captured picture.

[0066] As an example, the first image can be a photograph taken of each calibration plate that meets the affine transformation error.

[0067] Step 306: Based on the first captured image, perform camera calibration on the collaborative robot to obtain the second camera calibration parameters.

[0068] The first captured image includes pictures taken at various shooting trajectory points. The second camera calibration parameters may include camera intrinsic parameters, camera extrinsic parameters, and distortion coefficients. The camera intrinsic parameters are only related to the camera itself and depend on the camera's internal parameters. The camera extrinsic parameters change with the relative position of the world coordinate system and the camera coordinate system. That is, the corresponding camera parameters will change depending on the different pictures taken by the camera. The distortion coefficients include radial distortion coefficients and tangential distortion coefficients. Radial distortion is distortion distributed along the lens radius direction. Tangential distortion is caused by the mirror itself not being parallel to the camera sensor plane or image plane.

[0069] As an example, step 306 includes: detecting the corner points of the calibration plate in the captured images based on the captured images of each captured trajectory point in the first captured image, obtaining the pixel coordinate values ​​of each corner point, and then calculating the physical coordinate values ​​of the corner points through the correspondence between the actual size of the grid and the world coordinate system; and calculating the second camera calibration parameters based on the physical coordinate values ​​of the corner points and the pixel coordinate values.

[0070] Step 308: Based on the first captured image and the second camera calibration parameters, perform hand-eye calibration on the collaborative robot to obtain the second hand-eye calibration result.

[0071] Among them, the factory parameters of the collaborative robot include the coordinate transformation relationship between the collaborative robot base and the collaborative robot end effector.

[0072] As an example, step 308 includes: obtaining the first coordinate transformation relationship between the collaborative robot base and the collaborative robot end effector; obtaining the second coordinate relationship between the camera and the calibration plate based on the second camera calibration parameters; calculating the hand-eye relationship based on the relationship between the first coordinate relationship, the second coordinate relationship and the hand-eye relationship; and using the hand-eye relationship as the hand-eye calibration result.

[0073] In this embodiment, a second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration results. Since the first camera calibration parameters include the camera's intrinsic and extrinsic parameters, and the first hand-eye calibration results include the coordinate transformation relationship between the collaborative robot's end effector and the camera, the shooting trajectory coordinates of the second shooting trajectory can be more accurate. Therefore, the second camera calibration parameters and the second hand-eye calibration results obtained by calibrating based on the calibration board image captured at the shooting trajectory coordinates have higher accuracy. More accurate second camera calibration parameters and more accurate second hand-eye calibration results help improve the calibration accuracy of the collaborative robot's working plane.

[0074] In one embodiment, generating a second shooting trajectory based on the first camera calibration parameters and the first hand-eye calibration result includes:

[0075] Based on the first camera calibration parameters, determine the second distance between the camera and the Charuco calibration board; based on the first hand-eye calibration results and the second distance, generate spherical coordinates, and use the spherical coordinates within a preset angle range as the second shooting trajectory.

[0076] Among them, the factory parameters of the collaborative robot include the coordinate transformation relationship between the collaborative robot base and the collaborative robot end effector; the first camera calibration parameters include the first camera extrinsic parameters and the first camera intrinsic parameters; the first hand-eye calibration result can be the coordinate transformation relationship between the collaborative robot end effector and the camera; and the second shooting trajectory can be the trajectory coordinate points of the collaborative robot end effector.

[0077] Specifically, the distance between the camera and the Charuco calibration board is calculated based on the first camera extrinsic and intrinsic parameters in the first camera calibration parameters; the distance between the collaborative robot end effector and the calibration board is calculated based on the coordinate transformation relationship between the collaborative robot end effector and the camera and the distance between the camera and the calibration board; spherical coordinates are generated with the distance between the collaborative robot end effector and the calibration board as the radius; and the spherical coordinates within a preset angle range are used as the second shooting trajectory.

[0078] In this embodiment, the second shooting trajectory is spherical coordinates. The shooting method of controlling the collaborative robot to randomly rotate on the sphere can obtain a more uniform distribution of corner points and collaborative robot posture, which helps to improve the quality of the images captured by the calibration board, thereby helping to obtain more accurate camera calibration parameters and hand-eye calibration results. The distance between the camera and the calibration board is calculated based on the world coordinates and camera coordinates in the first camera calibration parameters. Compared with manual measurement, it can be more accurate, which helps to generate more accurate shooting trajectory coordinates. When controlling the camera set on the end of the collaborative robot to shoot, the shooting trajectory coordinates can be more accurate, thereby helping to obtain more accurate camera calibration parameters and hand-eye calibration results.

[0079] In one embodiment, according to a second shooting trajectory, a camera mounted on the end effector of the collaborative robot is controlled to capture images of the Charuco calibration board, resulting in a first captured image, including:

[0080] Obtain the ideal affine transformation image of the Charuco calibration board; according to the second shooting trajectory, control the camera set on the end effector of the collaborative robot to take an image of the Charuco calibration board, and obtain the second shooting image; if the error between the second shooting image and the ideal affine transformation image is less than the preset affine transformation error, then output the second shooting image as the first shooting image.

[0081] The error between the second captured image and the ideal affine transformation image can be the total pixel error between the corner positions of the calibration board in the second captured image and the corner positions of the calibration board in the ideal affine transformation image. The preset affine transformation error can be set according to the required accuracy of the actual working environment of the collaborative robot. The lower the preset affine transformation error is set, the higher the accuracy of the captured image. A high-precision captured image helps to improve the accuracy of the working plane calibration.

[0082] Specifically, the ideal affine transformation image of the Charuco calibration board is obtained; according to the second shooting trajectory, the camera set on the end effector of the collaborative robot is controlled to take an image of the Charuco calibration board to obtain the second shooting image; the second shooting image is compared with the ideal affine transformation image to obtain the total pixel error; if the pixel error is less than the preset affine transformation error, the second shooting image is output as the first shooting image.

[0083] In this embodiment, the captured image is compared with the ideal affine transformation image. The captured image that meets the preset affine transformation error is retained, while the captured image with large affine transformation error is discarded. Therefore, the accuracy of the captured image can be guaranteed, which helps to improve the accuracy of the second camera calibration parameters and the second hand-eye calibration results, thereby helping to improve the accuracy of the working plane calibration.

[0084] In one embodiment, the collaborative robot's work plane is calibrated based on the second camera calibration parameters and the second hand-eye calibration results, including:

[0085] Based on the calibration parameters of the second camera and the calibration results of the second hand-eye calibration, the camera set on the end effector of the collaborative robot is controlled to capture images of the Charuco calibration board, resulting in a third captured image. Based on the third captured image, the calibration parameters of the second camera, and the calibration results of the second hand-eye calibration, the working plane of the collaborative robot is calibrated.

[0086] The third captured image can be a captured image that meets the preset affine error. The factory parameters of the collaborative robot include the coordinate transformation relationship between the collaborative robot base and the collaborative robot end effector. Since the collaborative robot base and the calibration plate are both fixed, the coordinate transformation relationship between the collaborative robot base and the calibration plate is fixed. The second hand-eye calibration result includes the hand-eye relationship, which is the coordinate change relationship between the camera and the collaborative robot end effector.

[0087] Specifically, based on the second camera calibration parameters and the second hand-eye calibration results, planar shooting points are generated; based on the planar shooting points, the camera set on the end effector of the collaborative robot is controlled to capture images of the Charuco calibration board at each planar shooting point, resulting in a third captured image; based on the third captured image and the second camera calibration parameters, the coordinate transformation relationship between the camera and the calibration board is calculated; based on the hand-eye relationship in the second hand-eye calibration results and the coordinate transformation relationship between the camera and the calibration board, a transformation is performed to obtain the coordinate transformation relationship between the end effector of the collaborative robot and the calibration board; based on the coordinate transformation relationship between the base of the collaborative robot and the end effector in the collaborative robot's factory parameters, the coordinate transformation relationship between the end effector of the collaborative robot and the calibration board is transformed to obtain the coordinate transformation relationship between the base of the collaborative robot and the calibration board; the coordinate transformation relationship between the base of the collaborative robot and the calibration board is then converted into the coordinate transformation relationship between the base of the collaborative robot and the working plane on which the calibration board is placed.

[0088] In this embodiment, the collaborative robot's working plane is calibrated based on the third captured image, the second camera calibration parameters, and the second hand-eye calibration results. Since the second camera calibration parameters and second hand-eye calibration results obtained from the calibration board image captured based on the second captured trajectory coordinates are more accurate, more accurate second camera calibration parameters and second hand-eye calibration results can improve the accuracy of the collaborative robot's working plane calibration.

[0089] In one embodiment, based on the second camera calibration parameters and the second hand-eye calibration results, the camera mounted on the end effector of the collaborative robot is controlled to capture images of the Charuco calibration board, resulting in a third captured image, including:

[0090] Based on the calibration parameters of the second camera and the calibration results of the second hand-eye alignment, planar shooting points are generated; based on the planar shooting points, the camera set on the end effector of the collaborative robot is controlled to capture images of the Charuco calibration board, resulting in a fourth image; based on the second distortion coefficient in the calibration parameters of the second camera, the fourth image is processed to remove distortion, resulting in a third image.

[0091] Among them, the planar shooting points include multiple shooting points on the same plane, and the number of shooting points can be set in advance according to the actual environment. The fourth shooting image includes multiple images taken by calibration plates, and the second camera calibration parameters include the second distortion coefficient.

[0092] Specifically, based on the second camera calibration parameters and the second hand-eye calibration results, the vertical distance between the collaborative robot and the calibration board is determined, and a preset number of planar shooting points are randomly generated on the plane of this vertical distance. Based on the planar shooting points, the camera set on the end of the collaborative robot is controlled to take images of the Charuco calibration board at each shooting point to obtain a fourth image. Based on the second distortion coefficient in the second camera calibration parameters, the fourth image is subjected to distortion correction processing to obtain a third image.

[0093] In this embodiment, the third image is obtained by performing distortion correction processing on the fourth image. The third image after distortion correction has higher imaging accuracy, and the calibration of the working plane of the collaborative robot based on the third image helps to improve the calibration accuracy.

[0094] In one embodiment, the collaborative robot automatic calibration method includes:

[0095] A first shooting trajectory is generated based on the distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration board placed on the working plane of the collaborative robot. Based on the first shooting trajectory, camera calibration and hand-eye calibration are performed on the collaborative robot to obtain first camera calibration parameters and first hand-eye calibration results. The distance between the camera and the Charuco calibration board is determined based on the first camera calibration parameters. Spherical coordinates are generated based on the first hand-eye calibration results and the distance. The spherical coordinates within a preset angle range are used as the second shooting trajectory. The second shooting trajectory, using spherical coordinates, controls the collaborative robot's random rotation on the spherical surface for shooting. This method can obtain a more uniform distribution of corner points and the collaborative robot's posture, which helps improve the quality of the images captured by the calibration board, thus contributing to more accurate camera calibration parameters and hand-eye calibration results. The distance between the camera and the calibration board is calculated based on the world coordinates and camera coordinates in the first camera calibration parameters, which is more accurate than manual measurement. Therefore, the generated shooting trajectory coordinates are more precise, helping to improve the accuracy of the second camera calibration parameters and the second hand-eye calibration results.

[0096] After generating the second shooting trajectory, an ideal affine transformation image of the Charuco calibration board is acquired. Based on the second shooting trajectory, the camera on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a second captured image. If the error between the second captured image and the ideal affine transformation image is less than a preset affine transformation error, the second captured image is output as the first captured image. Based on the first captured image, the collaborative robot undergoes camera calibration to obtain second camera calibration parameters. Based on the first captured image and the second camera calibration parameters, the collaborative robot undergoes hand-eye calibration to obtain a second hand-eye calibration result. The captured images are compared with the ideal affine transformation image, retaining images that meet the preset affine transformation error and discarding images with large affine transformation errors. This ensures the accuracy of the retained images. Therefore, the second camera calibration parameters and the second hand-eye calibration result obtained from the first captured image based on the second shooting trajectory have higher accuracy, thus helping to improve the accuracy of the working plane calibration.

[0097] Based on the second camera calibration parameters and the second hand-eye calibration results, planar shooting points are generated. Using these points, the camera on the end effector of the collaborative robot is controlled to capture images of the Charuco calibration board, resulting in a fourth image. The fourth image is then processed to remove distortion using the second distortion coefficient in the second camera calibration parameters, yielding a third image. Based on the third image, the second camera calibration parameters, and the second hand-eye calibration results, the collaborative robot's working plane is calibrated. This high-precision second camera calibration parameters and second hand-eye calibration results result in a more accurate coordinate relationship between the collaborative robot's base and its working plane, thus improving the calibration accuracy of the collaborative robot's working plane.

[0098] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0099] Based on the same inventive concept, this application also provides an automatic calibration device for collaborative robots to implement the aforementioned automatic calibration method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the automatic calibration device for collaborative robots provided below can be found in the limitations of the automatic calibration method for collaborative robots described above, and will not be repeated here.

[0100] In one embodiment, such as Figure 5 As shown, an automatic calibration device for collaborative robots is provided, comprising: a shooting trajectory generation module 402, a coarse calibration module 404, a fine calibration module 406, and a working plane calibration module 408, wherein:

[0101] The shooting trajectory generation module 402 is used to generate a first shooting trajectory based on the first distance between the camera set on the end of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot;

[0102] The coarse calibration module 404 is used to perform camera calibration and hand-eye calibration on the collaborative robot according to the first shooting trajectory, and obtain the first camera calibration parameters and the first hand-eye calibration result;

[0103] The fine calibration module 406 is used to perform camera calibration and hand-eye calibration on the collaborative robot based on the second shooting trajectory generated by the first camera calibration parameters and the first hand-eye calibration result, and obtain the second camera calibration parameters and the second hand-eye calibration result;

[0104] The working plane calibration module 408 is used to calibrate the working plane of the collaborative robot according to the second camera calibration parameters and the second hand-eye calibration results.

[0105] In one embodiment, the calibration module 406 further includes:

[0106] Based on the first camera calibration parameters and the first hand-eye calibration result, a second shooting trajectory is generated; based on the second shooting trajectory, the camera set on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a first captured image; based on the first captured image, the collaborative robot is calibrated using a camera to obtain second camera calibration parameters; based on the first captured image and the second camera calibration parameters, the collaborative robot is calibrated using hand-eye calibration to obtain a second hand-eye calibration result.

[0107] In one embodiment, the calibration module 406 further includes:

[0108] Based on the first camera calibration parameters, a second distance between the camera and the Charuco calibration board is determined; based on the first hand-eye calibration result and the second distance, spherical coordinates are generated, and the spherical coordinates within a preset angle range are used as the second shooting trajectory.

[0109] In one embodiment, the calibration module 406 further includes:

[0110] Obtain the ideal affine transformation image of the Charuco calibration board; according to the second shooting trajectory, control the camera set on the end of the collaborative robot to take an image of the Charuco calibration board to obtain a second shooting image; if the error between the second shooting image and the ideal affine transformation image is less than a preset affine transformation error, then output the second shooting image as the first shooting image.

[0111] In one embodiment, the working plane calibration module 408 further includes:

[0112] Based on the second camera calibration parameters and the second hand-eye calibration results, the camera on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a third captured image. Based on the third captured image, the second camera calibration parameters, and the second hand-eye calibration results, the collaborative robot is calibrated on its working plane.

[0113] In one embodiment, the working plane calibration module 408 further includes:

[0114] Based on the second camera calibration parameters and the second hand-eye calibration results, a planar shooting point is generated; based on the planar shooting point, the camera set on the end of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a fourth captured image; based on the second distortion coefficient in the second camera calibration parameters, the fourth captured image is subjected to distortion correction processing to obtain a third captured image.

[0115] Each module in the aforementioned collaborative robot automatic calibration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0116] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores relevant data for the automatic calibration of collaborative robots. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an automatic calibration method for collaborative robots.

[0117] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0118] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0119] A first shooting trajectory is generated based on a first distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot. Based on the first shooting trajectory, camera calibration and hand-eye calibration are performed on the collaborative robot to obtain first camera calibration parameters and first hand-eye calibration results. A second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration results, and camera calibration and hand-eye calibration are performed on the collaborative robot to obtain second camera calibration parameters and second hand-eye calibration results. Based on the second camera calibration parameters and the second hand-eye calibration results, the working plane of the collaborative robot is calibrated.

[0120] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0121] Based on the first camera calibration parameters and the first hand-eye calibration result, a second shooting trajectory is generated; based on the second shooting trajectory, the camera set on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a first captured image; based on the first captured image, the collaborative robot is calibrated using a camera to obtain second camera calibration parameters; based on the first captured image and the second camera calibration parameters, the collaborative robot is calibrated using hand-eye calibration to obtain a second hand-eye calibration result.

[0122] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0123] Based on the first camera calibration parameters, a second distance between the camera and the Charuco calibration board is determined; based on the first hand-eye calibration result and the second distance, spherical coordinates are generated, and the spherical coordinates within a preset angle range are used as the second shooting trajectory.

[0124] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0125] Obtain the ideal affine transformation image of the Charuco calibration board; according to the second shooting trajectory, control the camera set on the end of the collaborative robot to take an image of the Charuco calibration board to obtain a second shooting image; if the error between the second shooting image and the ideal affine transformation image is less than a preset affine transformation error, then output the second shooting image as the first shooting image.

[0126] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0127] Based on the second camera calibration parameters and the second hand-eye calibration results, the camera on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a third captured image. Based on the third captured image, the second camera calibration parameters, and the second hand-eye calibration results, the collaborative robot is calibrated on its working plane.

[0128] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0129] Based on the second camera calibration parameters and the second hand-eye calibration results, a planar shooting point is generated; based on the planar shooting point, the camera set on the end of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a fourth captured image; based on the second distortion coefficient in the second camera calibration parameters, the fourth captured image is subjected to distortion correction processing to obtain a third captured image.

[0130] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0131] A first shooting trajectory is generated based on a first distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot. Based on the first shooting trajectory, camera calibration and hand-eye calibration are performed on the collaborative robot to obtain first camera calibration parameters and first hand-eye calibration results. A second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration results, and camera calibration and hand-eye calibration are performed on the collaborative robot to obtain second camera calibration parameters and second hand-eye calibration results. Based on the second camera calibration parameters and the second hand-eye calibration results, the working plane of the collaborative robot is calibrated.

[0132] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0133] Based on the first camera calibration parameters and the first hand-eye calibration result, a second shooting trajectory is generated; based on the second shooting trajectory, the camera set on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a first captured image; based on the first captured image, the collaborative robot is calibrated using a camera to obtain second camera calibration parameters; based on the first captured image and the second camera calibration parameters, the collaborative robot is calibrated using hand-eye calibration to obtain a second hand-eye calibration result.

[0134] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0135] Based on the first camera calibration parameters, a second distance between the camera and the Charuco calibration board is determined; based on the first hand-eye calibration result and the second distance, spherical coordinates are generated, and the spherical coordinates within a preset angle range are used as the second shooting trajectory.

[0136] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0137] Obtain the ideal affine transformation image of the Charuco calibration board; according to the second shooting trajectory, control the camera set on the end of the collaborative robot to take an image of the Charuco calibration board to obtain a second shooting image; if the error between the second shooting image and the ideal affine transformation image is less than a preset affine transformation error, then output the second shooting image as the first shooting image.

[0138] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0139] Based on the second camera calibration parameters and the second hand-eye calibration results, the camera on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a third captured image. Based on the third captured image, the second camera calibration parameters, and the second hand-eye calibration results, the collaborative robot is calibrated on its working plane.

[0140] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0141] Based on the second camera calibration parameters and the second hand-eye calibration results, a planar shooting point is generated; based on the planar shooting point, the camera set on the end of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a fourth captured image; based on the second distortion coefficient in the second camera calibration parameters, the fourth captured image is subjected to distortion correction processing to obtain a third captured image.

[0142] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0143] A first shooting trajectory is generated based on a first distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot. Based on the first shooting trajectory, camera calibration and hand-eye calibration are performed on the collaborative robot to obtain first camera calibration parameters and first hand-eye calibration results. A second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration results, and camera calibration and hand-eye calibration are performed on the collaborative robot to obtain second camera calibration parameters and second hand-eye calibration results. Based on the second camera calibration parameters and the second hand-eye calibration results, the working plane of the collaborative robot is calibrated.

[0144] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0145] Based on the first camera calibration parameters and the first hand-eye calibration result, a second shooting trajectory is generated; based on the second shooting trajectory, the camera set on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a first captured image; based on the first captured image, the collaborative robot is calibrated using a camera to obtain second camera calibration parameters; based on the first captured image and the second camera calibration parameters, the collaborative robot is calibrated using hand-eye calibration to obtain a second hand-eye calibration result.

[0146] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0147] Based on the first camera calibration parameters, a second distance between the camera and the Charuco calibration board is determined; based on the first hand-eye calibration result and the second distance, spherical coordinates are generated, and the spherical coordinates within a preset angle range are used as the second shooting trajectory.

[0148] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0149] Obtain the ideal affine transformation image of the Charuco calibration board; according to the second shooting trajectory, control the camera set on the end of the collaborative robot to take an image of the Charuco calibration board to obtain a second shooting image; if the error between the second shooting image and the ideal affine transformation image is less than a preset affine transformation error, then output the second shooting image as the first shooting image.

[0150] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0151] Based on the second camera calibration parameters and the second hand-eye calibration results, the camera on the end effector of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a third captured image. Based on the third captured image, the second camera calibration parameters, and the second hand-eye calibration results, the collaborative robot is calibrated on its working plane.

[0152] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0153] Based on the second camera calibration parameters and the second hand-eye calibration results, a planar shooting point is generated; based on the planar shooting point, the camera set on the end of the collaborative robot is controlled to capture an image of the Charuco calibration board, resulting in a fourth captured image; based on the second distortion coefficient in the second camera calibration parameters, the fourth captured image is subjected to distortion correction processing to obtain a third captured image.

[0154] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0155] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of collaborative robot auto-calibration, the method comprising: The method includes: A first shooting trajectory is generated based on the first distance between the camera mounted on the end effector of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot; the first shooting trajectory is a spherical trajectory within a preset angle range. Based on the first shooting trajectory, the collaborative robot is calibrated with both camera and hand-eye calibration to obtain the first camera calibration parameters and the first hand-eye calibration result. Based on the second shooting trajectory generated by the first camera calibration parameters and the first hand-eye calibration result, the collaborative robot is calibrated for both camera and hand-eye calibration to obtain the second camera calibration parameters and the second hand-eye calibration result. Based on the second camera calibration parameters and the second hand-eye calibration results, generate planar shooting points; Based on the planar shooting points, the camera installed on the end effector of the collaborative robot is controlled to capture images of the Charuco calibration board, thereby obtaining a fourth captured image. Based on the second distortion coefficient in the second camera calibration parameters, the fourth captured image is subjected to distortion correction processing to obtain the third captured image; The working plane of the collaborative robot is calibrated based on the third captured image, the second camera calibration parameters, and the second hand-eye calibration results.

2. The method of claim 1, wherein, The step of performing camera calibration and hand-eye calibration on the collaborative robot based on the second shooting trajectory generated according to the first camera calibration parameters and the first hand-eye calibration result, to obtain the second camera calibration parameters and the second hand-eye calibration result, includes: A second shooting trajectory is generated based on the first camera calibration parameters and the first hand-eye calibration results; Based on the second shooting trajectory, the camera set on the end of the collaborative robot is controlled to take an image of the Charuco calibration board to obtain a first captured image. Based on the first captured image, the camera of the collaborative robot is calibrated to obtain the second camera calibration parameters; Based on the first captured image and the second camera calibration parameters, the collaborative robot is calibrated using hand-eye calibration to obtain a second hand-eye calibration result.

3. The method according to claim 2, characterized in that, The step of generating a second shooting trajectory based on the first camera calibration parameters and the first hand-eye calibration result includes: Based on the first camera calibration parameters, determine the second distance between the camera and the Charuco calibration plate; Based on the first hand-eye calibration result and the second distance, spherical coordinates are generated, and the spherical coordinates within a preset angle range are used as the second shooting trajectory.

4. The method according to claim 2, characterized in that, The step of controlling the camera mounted on the end effector of the collaborative robot to capture an image of the Charuco calibration board according to the second shooting trajectory, thereby obtaining a first captured image, includes: Obtain the ideal affine transformation image of the Charuco calibration plate; Based on the second shooting trajectory, the camera set on the end of the collaborative robot is controlled to take an image of the Charuco calibration board to obtain the second captured image. If the error between the second captured image and the ideal affine transformation image is less than the preset affine transformation error, then the second captured image is output as the first captured image.

5. An automatic calibration device for collaborative robots, characterized in that, The device includes: The shooting trajectory generation module is used to generate a first shooting trajectory based on a first distance between the camera set on the end of the collaborative robot and the Charuco calibration plate placed on the working plane of the collaborative robot; the first shooting trajectory is a spherical trajectory within a preset angle range; The coarse calibration module is used to perform camera calibration and hand-eye calibration on the collaborative robot based on the first shooting trajectory, and obtain the first camera calibration parameters and the first hand-eye calibration result; The fine calibration module is used to perform camera calibration and hand-eye calibration on the collaborative robot based on the second shooting trajectory generated by the first camera calibration parameters and the first hand-eye calibration result, so as to obtain the second camera calibration parameters and the second hand-eye calibration result; The working plane calibration module is used to calibrate the working plane of the collaborative robot according to the second camera calibration parameters and the second hand-eye calibration results; The working plane calibration module is further configured to generate planar shooting points based on the second camera calibration parameters and the second hand-eye calibration results; control the camera set on the end effector of the collaborative robot to capture images of the Charuco calibration board based on the planar shooting points, thereby obtaining a fourth captured image; perform distortion correction processing on the fourth captured image based on the second distortion coefficient in the second camera calibration parameters, thereby obtaining a third captured image; and perform working plane calibration on the collaborative robot based on the third captured image, the second camera calibration parameters, and the second hand-eye calibration results.

6. The apparatus according to claim 5, characterized in that, The precision calibration module is further configured to: generate a second shooting trajectory based on the first camera calibration parameters and the first hand-eye calibration result; control the camera mounted on the end effector of the collaborative robot to capture an image of the Charuco calibration board based on the second shooting trajectory, thereby obtaining a first captured image; perform camera calibration on the collaborative robot based on the first captured image, thereby obtaining a second camera calibration parameter; and perform hand-eye calibration on the collaborative robot based on the first captured image and the second camera calibration parameter, thereby obtaining a second hand-eye calibration result.

7. The apparatus according to claim 5, characterized in that, The precision calibration module is further configured to: determine a second distance between the camera and the Charuco calibration board based on the first camera calibration parameters; generate spherical coordinates based on the first hand-eye calibration result and the second distance; and use the spherical coordinates within a preset angle range as the second shooting trajectory.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images