Robot automatic hand-eye calibration method and system based on deep reinforcement learning

Abstract

The present invention discloses a robot automatic hand-eye calibration method and system based on deep reinforcement learning, the method comprising the following steps: obtaining an initial hand-eye matrix; screening the generated random postures, clustering analysis to obtain multiple initial calibration postures; obtaining the calibration plate image corresponding to the initial calibration posture, inputting the view-distance convolutional neural network VDCNN after preprocessing, the view-distance convolutional neural network VDCNN fitting the view-distance action value function based on deep reinforcement learning, predicting the optimal view-distance adjustment action, outputting the view value map and the distance value vector; obtaining the view adjustment pixel points based on the view value map, converting the view adjustment pixel points into a rotation matrix, rotating the robot according to the rotation matrix, and realizing the adjustment of the view; obtaining the distance adjustment amplitude based on the distance value vector, adjusting the distance of the robot, and obtaining the final hand-eye calibration posture. The present invention significantly improves the calibration accuracy and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of robot hand-eye calibration, and in particular to a robot automated hand-eye calibration method and system based on deep reinforcement learning. Background Art

[0002] In industrial automation and smart manufacturing, accurate hand-eye calibration is essential to improve production efficiency, ensure quality, and automate complex tasks. Hand-eye calibration involves determining the spatial position and orientation of the camera mounted on the robot's end effector, which is necessary for vision-guided operation. As robots are increasingly used in tasks such as assembly, welding, and material handling, the accuracy of hand-eye calibration directly affects the overall performance of the robot.

[0003] Traditional calibration methods mainly rely on external calibration objects and complex mathematical calculations. These methods require precise control of the robot's movement and rely on external equipment for calibration. They are not adaptable enough in dynamic environments and often require manual intervention in actual operations, resulting in low efficiency and complex operations.

[0004] With the development of artificial intelligence technology, especially deep learning, machine learning and optimization methods, the automation and accuracy of hand-eye calibration have been significantly improved. Modern hand-eye calibration methods not only rely on traditional optimization techniques, but also combine innovative methods of data-driven and autonomous learning. For example, deep learning and reinforcement learning can optimize the posture selection during the calibration process through autonomous learning, thereby improving the calibration accuracy and system robustness. At the same time, traditional machine learning methods, such as K-means clustering, are also used to optimize the posture selection and reduce manual calibration errors. In order to further optimize the calibration process, the constrained random increment method is used. In order to improve the calibration accuracy, image processing and optimization techniques are also combined. Specifically, the least squares method based on scene features can be used to optimize camera parameters and robot posture; the board-based online calibration algorithm provides more accurate and flexible calibration by detecting feature points. In addition, the active learning method selects the best viewing angle by estimating the mass field online to enhance the robustness of the system. Combining technologies such as RANSAC, K-means clustering and dual quaternion theory effectively solves the accuracy problem in traditional methods.

[0005] However, these methods still have some shortcomings, which are manifested in the following aspects:

[0006] (1) Inability to adapt to environmental changes: Although these methods perform well in static environments, in complex and dynamic environments, the calibration system may have difficulty adapting to environmental changes. For example, lighting, background complexity, or other external interference may affect the accuracy of image processing and optimization techniques.

[0007] (2) High dependence on manual hand-eye calibration: Many existing hand-eye calibration methods still rely on manual calibration, especially in the initial calibration stage. Manual intervention not only increases the operational complexity of the system, but may also introduce human errors, reducing the calibration accuracy and the automation level of the system.

[0008] (3) Accuracy limitation: Although the calibration accuracy can be improved through optimization methods such as K-means clustering and RANSAC, the accuracy of existing methods is still difficult to meet the requirements in some applications with high precision requirements. For example, in some small objects or high-precision processing, the accuracy of existing hand-eye calibration may not meet the expected standards.

[0009] (4) Dependence on data sets and computing resources: Many calibration methods based on deep learning and image processing require a large amount of high-quality calibration data sets to ensure model training and optimization. However, collecting enough high-quality calibration data is not only time-consuming and costly, but also the algorithm performance may be greatly limited when the data volume is insufficient or computing resources are limited.

[0010] (5) Insufficient robustness: Although active learning and other optimization methods improve the robustness of the system, existing methods may perform poorly in some extreme or special environments. For example, the system may not be able to effectively handle strong noise, occlusion, or non-ideal viewing angles, resulting in increased calibration errors.

[0011] Therefore, although existing methods have made some progress in improving calibration accuracy and robustness, there are still many challenges when facing complex environments, increasing the degree of automation and accuracy requirements, especially in complex, dynamic and resource-limited environments. Solving these challenges and enhancing the robustness and adaptability of the calibration system are the key to future development. Summary of the invention

[0012] In order to overcome the defects and shortcomings of the prior art, the present invention provides a robot automated hand-eye calibration method and system based on deep reinforcement learning. The present invention frames the viewpoint-distance selection problem as a Markov decision process, and uses a viewpoint-distance convolutional neural network (VDCNN) to approximate the viewpoint-distance action value function, and evaluates the optimal relationship between viewpoint and distance during the calibration process, so that the robot can autonomously learn and make optimal decisions during the hand-eye calibration process, thereby improving efficiency and accuracy. Through a reward mechanism and an experience replay strategy, the robot is helped to select the optimal viewpoint and distance, which significantly improves the calibration accuracy and robustness.

[0013] In order to achieve the above object, the present invention adopts the following technical solutions:

[0014] The present invention provides a robot automatic hand-eye calibration method based on deep reinforcement learning, comprising the following steps:

[0015] Get the initial hand-eye matrix;

[0016] Generate and filter the postures of the end of the random robot arm, perform cluster analysis on the filtered posture data, and obtain multiple initial calibration postures;

[0017] The calibration plate images under each initial calibration posture are obtained and image preprocessing is performed, and the images are input into the view-distance convolutional neural network VDCNN, which fits the view-distance action value function based on deep reinforcement learning and predicts the optimal view-distance adjustment action. The view-distance convolutional neural network VDCNN outputs a view value map and a distance value vector based on an encoder-decoder structure;

[0018] Calculate the position with the maximum predicted value in the perspective value map to obtain the perspective adjustment pixel point;

[0019] Based on the initial hand-eye matrix, the view adjustment pixel points are converted into robot coordinates expressed in the robot coordinate system, and the corresponding rotation matrix is ​​obtained. The robot is rotated according to the rotation matrix to achieve the view adjustment;

[0020] Calculate the position with the largest predicted value in the distance value vector, obtain the distance adjustment amplitude, adjust the distance of the robot, and obtain the final hand-eye calibration posture.

[0021] As a preferred technical solution, when generating the posture of the random robot end, its rotation vector is constrained to point to the desktop projection of the initial position, specifically including:

[0022] Get a random point in the set space , the coordinates of the desktop projection of the initial position are , the vector from the random point to the projection point on the desktop is , the corresponding rotation matrix is ​​obtained through the Rodriguez formula, expressed as:

[0023] ;

[0024] in, is a vector and vector The angle of represents an antisymmetric matrix, For vector With vector The rotation matrix between the robot Rotate it to achieve random posture pointing to the desktop projection. and The rotation matrix is ​​the matrix corresponding to the known unit vector and vector.

[0025] As a preferred technical solution, cluster analysis is performed on the screened posture data to obtain multiple initial calibration postures, specifically including:

[0026] The rotation matrix Convert to rotation vector Based on the K-means clustering method, select the cluster center in each class The closest vector As a representative of this class, a set number of robot end-arm postures are retained, specifically expressed as:

[0027] ;

[0028] in, Indicates the distance from the cluster center The nearest vector, and Represents the rotation vector The and indivual, represents the Euclidean distance.

[0029] As a preferred technical solution, the view-distance convolutional neural network VDCNN includes multiple convolutional layers, multiple deconvolutional layers and fully connected layers. The calibration plate image after image preprocessing passes through multiple convolutional layers and multiple deconvolutional layers in sequence to obtain a view value map. The calibration plate image after image preprocessing passes through multiple convolutional layers, multiple deconvolutional layers and fully connected layers in sequence to obtain a distance value vector.

[0030] As the preferred technical solution, the view-distance convolutional neural network (VDCNN) is based on deep reinforcement learning to fit the view-distance action value function. ε -greedy strategy to select the next best view-distance adjustment action , specifically expressed as:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] in, , Perspective Value Map Resolution, To explore the probability, Indicated in The uniform distribution of the interval, represents the view-distance action-value function, Indicates the status, Indicates action, Indicates strategy, represents the reward function, Indicates reward, represents the discount factor, Indicates duration, Indicates the detection center coordinates of the calibration plate, represents the image center coordinates, Represents the preset Manhattan distance threshold, Indicates the preset depth threshold, Indicates the depth of the calibration plate detection center. express function.

[0039] As a preferred technical solution, the view-distance convolutional neural network VDCNN fits the view-distance action value function based on deep reinforcement learning, and the loss function is:

[0040] ;

[0041] in, represents the loss function, For the The estimated network parameters based on VDCNN in the iteration, is the target network parameter based on VDCNN, represents the view-distance action-value function corresponding to the estimated network parameters, Respectively represent The status of the iteration, Indicates The action of the iteration, Represents the view-distance action-value function corresponding to the target network parameters.

[0042] As a preferred technical solution, the position with the largest predicted value in the perspective value map is calculated to obtain the perspective adjustment pixel point, which is specifically expressed as:

[0043] ;

[0044] in, Indicates the viewing angle adjustment pixel point, Represents a perspective value graph;

[0045] Based on the initial hand-eye matrix, the perspective adjustment pixel points are converted into robot coordinates expressed in the robot coordinate system, which is specifically expressed as:

[0046] ;

[0047] in, represents the initial hand-eye matrix, Indicates that the pixel position in the preprocessing image Restore to the pixel position in the actual color image, Representation of the optimal view-distance adjustment action through the camera pinhole model The position of the best view-distance adjustment action obtained by conversion in the camera coordinate system;

[0048] Get the robot coordinates corresponding to the current robot end , we get the vector , the corresponding rotation matrix is ​​obtained through the Rodriguez formula, expressed as:

[0049] ;

[0050] in, To calculate the rotation matrix using the Rodriguez formula, and is the matrix corresponding to the unit vector and vector known in the rotation matrix, is a vector and vector Angle.

[0051] As a preferred technical solution, the position with the largest predicted value in the distance value vector is calculated to obtain the distance adjustment amplitude, which is specifically expressed as:

[0052] ;

[0053] in, Indicates the distance adjustment amplitude. Represents the distance value vector.

[0054] As a preferred technical solution, the robot is adjusted in distance to obtain the final hand-eye calibration posture, which specifically includes:

[0055] The calibration plate image after adjusting the viewing angle is detected by the calibration plate to obtain the center pixel coordinates , the robot coordinates corresponding to the calibration plate It is expressed as:

[0056] ;

[0057] in, Indicates the depth of the calibration plate detection center. represents the camera pinhole model, Indicates that the calibration plate will be tested at the center position Restore to the pixel position in the actual color image;

[0058] Based on the initial hand-eye matrix and the robot coordinates corresponding to the current robot end, the robot coordinates corresponding to the current camera are obtained , specifically expressed as:

[0059] ;

[0060] in, Indicates the robot coordinates corresponding to the current robot end;

[0061] Get the camera's depth map at the center of the current camera pixel Depth value , get the current camera pixel center position in the base coordinate system , specifically expressed as:

[0062] ;

[0063] Based on robot coordinates The coordinates of the robot end after the distance adjustment are:

[0064] ;

[0065] in, Indicates the coordinates of the robot end after distance adjustment.

[0066] The present invention provides a robot automatic hand-eye calibration system based on deep reinforcement learning, which is used to implement the above-mentioned robot automatic hand-eye calibration method based on deep reinforcement learning. The system includes: an initial hand-eye matrix acquisition module, an initial calibration posture acquisition module, an image generation module, a network prediction module, a perspective adjustment pixel point acquisition module, a perspective adjustment module, and a distance adjustment module;

[0067] The initial hand-eye matrix acquisition module is used to acquire the initial hand-eye matrix;

[0068] The initial calibration posture acquisition module is used to generate and filter the postures of the end of the random robot arm, and perform cluster analysis on the filtered posture data to obtain multiple initial calibration postures;

[0069] The image generation module is used to obtain the calibration plate image in each initial calibration posture and perform image preprocessing;

[0070] The network prediction module is used to input the image after image preprocessing into the perspective-distance convolutional neural network VDCNN, the perspective-distance convolutional neural network VDCNN fits the perspective-distance action value function based on deep reinforcement learning, and predicts the optimal perspective-distance adjustment action. The perspective-distance convolutional neural network VDCNN outputs a perspective value map and a distance value vector based on an encoder-decoder structure;

[0071] The viewing angle adjustment pixel point acquisition module is used to calculate the position with the maximum predicted value in the viewing angle value map to obtain the viewing angle adjustment pixel point;

[0072] The viewing angle adjustment module is used to convert the viewing angle adjustment pixel points into robot coordinates represented by the robot coordinate system based on the initial hand-eye matrix, obtain the corresponding rotation matrix, and rotate the robot according to the rotation matrix to achieve viewing angle adjustment;

[0073] The distance adjustment module is used to calculate the position with the largest predicted value in the distance value vector, obtain the distance adjustment amplitude, perform distance adjustment on the robot, and obtain the final hand-eye calibration posture.

[0074] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0075] (1) The present invention adopts an automated hand-eye calibration technology solution based on deep reinforcement learning, which solves the technical problems of traditional hand-eye calibration methods that are insufficiently accurate in dynamic environments and rely on complex calculations and external devices, and achieves the technical effects of improving calibration accuracy, enhancing robustness, and realizing automation and intelligence.

[0076] (2) The present invention adopts a strategy of automatically adjusting the viewing angle and distance, which solves the technical problems that the hand-eye calibration posture generation method based on K-means clustering is difficult to adjust the distance between the calibration plate and the center of the camera optical axis, the distance between the centers of the end effectors under different postures, and the calibration plate deviates from the camera field of view. This further improves the accuracy of the calibration process and achieves the technical effect of realizing high-precision calibration in intelligent manufacturing.

[0077] (3) The present invention adopts a technical solution that transforms the view-distance selection problem into a Markov decision process. In this process, the states of the robot's end effector and camera are defined as the state space in the Markov decision process. The viewpoint and distance adjustment actions selected by the robot constitute the action space. By designing a reward mechanism, the impact of each action on the calibration accuracy is evaluated and used as an immediate reward feedback. The view-distance convolutional neural network (VDCNN) is used to fit the view-distance action value function. , thereby predicting and optimizing the long-term calibration effect based on the current state, solving the technical problem of the demand for large-scale data sets based on deep learning methods, significantly improving the adaptability and calibration accuracy of the system, and achieving the technical effect of rapid adaptation and efficient calibration in different environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a flow chart of the robot automatic hand-eye calibration method based on deep reinforcement learning of the present invention;

[0079] Figure 2 Schematic diagram of the overall architecture of the perspective-distance convolutional neural network VDCNN of the present invention;

[0080] Figure 3 Schematic diagram of the training process of the perspective-distance value action function of the present invention;

[0081] Figure 4 A schematic diagram of the automatic hand-eye calibration environment of the robot of the present invention;

[0082] Figure 5 It is a box plot comparison diagram of the reprojection errors of the four methods, namely, the random generation method, the K-means algorithm, the improved K-means algorithm and the method of the present invention in a simulation environment;

[0083] Figure 6 It is a schematic diagram of box plot comparison of reprojection errors of the improved K-means algorithm and the method of the present invention under different position offsets in a simulation environment;

[0084] Figure 7 It is a schematic diagram of box plot comparison of reprojection errors of the improved K-means algorithm and the method of the present invention under different numbers of calibration postures in a simulation environment;

[0085] Figure 8 It is a schematic diagram of box plot comparison of reprojection errors of four methods, namely, the random generation method of the first calibration plate, the K-means algorithm, the improved K-means algorithm, and the method of the present invention in a real environment. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0087] Example 1

[0088] like Figure 1 As shown, this embodiment provides a robot automatic hand-eye calibration method based on deep reinforcement learning, comprising the following steps:

[0089] S1: First, the initial hand-eye matrix is ​​obtained through the manual hand-eye calibration method , the reprojection error of the initial hand-eye matrix is ​​within 5mm;

[0090] In this embodiment, during the automated hand-eye calibration process, the camera is mounted on the end effector of the robot (the eye is on the hand). This method provides higher flexibility compared to setting up the camera in a fixed environment.

[0091] In the eye-in-hand system, the core of hand-eye calibration is to determine the rigid body transformation relationship between the camera and the robot end effector. In order to improve the accuracy of the calibration results, four key criteria need to be obtained: maximizing the angle between the relative motion axes, maximizing the rotation angle of the relative motion, minimizing the distance from the calibration plate to the center of the camera optical axis, and minimizing the distance between the end effector centers under different postures. These key criteria are used to guide the direction of improving the accuracy of hand-eye calibration. First, the initial posture generated by the K-means clustering algorithm can ensure that the angle between the relative motion axes and the rotation angle of the relative motion can be maximized; then, the viewing angle is adjusted on the generated initial posture to minimize the distance from the calibration plate to the center of the optical axis; then, the distance adjustment is performed to minimize the distance between the calibration plate and the camera to minimize the distance between the end effector centers under different postures.

[0092] Since the camera is directly mounted on the end of the robot, the robot's position / posture can be defined by the camera's six-dimensional position (including position and orientation) in the robot's coordinate system. :

[0093] ;

[0094] in, , , is the position of the robot end in the base coordinate system, , , Respectively around , , The rotation angle of the axis.

[0095] S2: The robot is reset to its initial position. In order to generate an initial calibration posture with a large relative motion rotation angle, a set of postures of the end of the robot arm are randomly generated according to the preset rules, and the corresponding joint angles are solved using inverse kinematics. Then, the generated postures are screened to ensure that they meet the criteria for improving calibration accuracy. Finally, the K-means clustering algorithm is used to perform cluster analysis on the screened posture data to select the posture data that is most conducive to hand-eye calibration. a gesture;

[0096] This example randomly generates Then the K-means clustering algorithm is used to cluster the rotation vectors of these 10,000 postures, and finally the rotation vectors of the 10,000 postures are selected. The 10 poses closest to the cluster center.

[0097] When generating the random robot arm end posture, this embodiment constrains its rotation vector and requires it to point to the desktop projection of the initial position, which can greatly reduce the time for generating the random posture set.

[0098] Specifically, in a given space Get a point uniformly randomly , and then use Rodrigues' rotation formula to get the corresponding rotation matrix. The desktop projection of the known initial position is , then the vector from the random point to the projection point on the desktop is , the rotation matrix It can be calculated by the following formula:

[0099] ;

[0100] in, is the vector calculated by Rodriguez's formula and The rotation matrix between . and The rotation matrix is ​​the matrix corresponding to the vector and vector known by the rotation matrix. is (0, 0, -1), and its rotation matrix is ​​obtained through the teaching pendant. The robot Rotate it to achieve random posture pointing to the desktop projection. is a vector and vector The angle of represents an antisymmetric matrix.

[0101] After obtaining the end-arm posture, the posture is initially screened. The screening rules are as follows: (1) The generated posture is reachable and no collision occurs during the movement. (2) The angle between the calibration plate and the camera optical axis is less than ; The joint angles corresponding to the robot arm can be obtained by inverse solution of inverse kinematics. In this embodiment, the space For a cube with a side length of 0.6m, the preset maximum angle value .

[0102] K-means clustering is performed on the rotation part of the generated random robot arm pose set. Specifically, the rotation matrix Converted to the form of a rotation vector , using K-means clustering, in each class, select the class with the class center The closest vector As the representative of this class, retain The end-arm posture:

[0103] ;

[0104] in, is the rotation vector h and the rotation vector The Euclidean distance between Indicates cluster centers, h represents the rotation vector, Indicates the distance from the cluster center A total of 10,000 random postures were generated, and then the rotation matrices of these 10,000 random postures were converted into rotation vectors, that is, 10,000 , and This means that these 10,000 The and indivual, and It means to select any two different numbers from the defined set, that is, the natural number set [1, 10000] and , the inequality is established, that is, from these 10,000 postures, find The closest posture to the Euclidean distance is finally retained. The end posture of the robot arm is the generated initial calibration posture, and the generated initial calibration posture will be further adjusted through the deep reinforcement learning algorithm.

[0105] S3: The robot end effector moves in the order of the generated postures and takes RGB images of the calibration plate at each posture position. After preprocessing operations such as scaling, normalization, and grayscale conversion, these images are input into the viewpoint-distance convolutional neural network VDCNN (Viewpoint-Distance CNN) to predict the best view and the best distance, thereby obtaining the pixel coordinates of the best view. and the optimal adjustment range .

[0106] like Figure 2 As shown in Figure 2, the view-distance convolutional neural network (VDCNN) is a convolutional neural network based on an encoder-decoder structure, which is used to fit the view-distance action value function. Specifically, the function evaluates the expected value of future rewards that can be obtained by choosing different viewpoints and distances to adjust actions starting from the current state. By maximizing the view-distance action value function, the robot can autonomously learn and choose those viewpoints and distances that can bring the best calibration effect. This process involves estimating the view-distance value from the scene state image. The input of the network is , a preprocessed grayscale image captured by the robot at the pose determined by the K-means clustering algorithm. The network first encodes features through three convolutional layers, then decodes through three deconvolutional layers, and finally outputs a view value map with the same size as the input image. In addition, the network also includes a fully connected layer to output a 1×30 distance value vector.

[0107] Combination Figure 2 As shown, first, the preprocessed image of 320×240×1 is operated by the first convolution kernel with a size of 9×9, a number of 32, and a sliding step of 3 to obtain a feature map of size 104×78×32;

[0108] The feature map of size 104×78×32 is processed by the second convolution kernel of size 5×5, number 16, and sliding step size 2 to obtain a feature map of size 50×37×16;

[0109] The feature map of size 50×37×16 is processed by the third convolution kernel of size 3×3, number 8, and sliding step size 1 to obtain a feature map of size 48×35×8;

[0110] The feature map of size 48×35×8 is processed by the first deconvolution kernel of size 3×3, number 8, and sliding step size 1 to obtain a feature map of size 50×37×8;

[0111] The feature map of size 50×37×8 is processed by the second deconvolution kernel of size 5×5, number 16, and sliding step size 2 to obtain a feature map of size 103×77×16;

[0112] The feature map of size 103×77×16 is processed by the third deconvolution kernel with a size of 9×9, a number of 32, and a sliding step of 3 to obtain a feature map of size 315×237×32;

[0113] The feature map of 315×237×32 size is subjected to the fourth convolution kernel operation of size 4×4, number 1, sliding step 1, padding = [[4, 4], [3, 3]] to obtain a feature map of 320×240×1, namely the view value map. Each pixel value in the view value map represents the expected future reward after selecting the view represented by the pixel point; the feature map of 315×237×32 size is expanded into a 1×2388960 vector and mapped into a 1×30 output feature vector, namely the distance value vector, through the fully connected layer. The value in the distance value vector represents the expected future reward after selecting a specific distance adjustment amplitude. In this way, the view-distance convolutional neural network VDCNN can provide decision support for view-distance selection for robots in hand-eye calibration tasks.

[0114] The view-distance convolutional neural network VDCNN fits the view-distance action value function based on DQN reinforcement learning. The action with the highest value is the action performed in reinforcement learning. In the training process of DQN reinforcement learning, this embodiment introduces an experience replay mechanism to deal with the correlation between samples in reinforcement learning. This mechanism uses the characteristics of the robot's view selection task and optimizes the learning effect by adding the judgment of the end condition and experience enhancement during the training process. In addition, it adopts ε -greedy strategy to design the robot to select the next best view-distance adjustment action in the grasping scene The method is:

[0115] ;

[0116] ;

[0117] in, , Perspective Value Map Resolution, To explore the probability, Indicated in The uniform distribution of the interval. , then through the view-distance action value function Generate a view-distance adjustment action, otherwise generate a view action through 3 independent uniform distributions. In order to reduce the probability of exploration, the robot gradually tends to choose actions according to the trained strategy, while retaining the possibility of exploring better actions. As training progresses, the initial value Down to 0.1, that is:

[0118] ;

[0119] in, is the decreasing coefficient, is the training round, ε - In the greedy strategy, a random number is generated each time training. If the random number is greater than the exploration probability , then the action is selected by the maximum value of the view-distance action value function; otherwise, the action is explored (i.e., actions are randomly generated). The purpose of this is to explore actions as much as possible.

[0120] After moving to the predicted optimal viewing angle, the robot performs calibration plate detection and distance adjustment, and generates empirical data based on the calibration plate evaluation results after distance adjustment. ( Indicates whether the evaluation result of the calibration plate meets the requirements), enhances the obtained experience data, and then stores it in the experience pool. If the number of experiences in the experience pool is greater than the preset batch size , randomly drawn from the experience pool The experience data obtained by interacting with the environment contains a variable Used to determine whether the robot's view-distance adjustment task is completed, that is, whether the robot has moved to the ideal position.

[0121] award is defined as:

[0122] ;

[0123] ;

[0124] ;

[0125] in, Depends on the calibration plate detection center coordinates The image center coordinates If the Manhattan distance between No reward is given, preferably, . Depth value with the center of the calibration plate (in millimeters) is negatively correlated. When the depth exceeds the preset depth threshold No reward is given, preferably, .

[0126] In order to speed up the adjustment of the robot's perspective and distance, a reward function with a discount factor is used to guide its behavior to ensure that the task is completed in the shortest time, namely:

[0127] ;

[0128] The discount factor Represents the weight of future returns.

[0129] View-distance action-value function Indicates that the robot is Status at all times Next, according to the strategy Execute an action The expected return after is expressed as:

[0130] ;

[0131] like Figure 3 As shown, VDCNN is used to fit the view-distance action value function In the process of reinforcement learning training, in order to avoid To prevent overestimation of the value, an estimation network and a target network based on VDCNN are used. In each round of training, the parameters of the estimation network are updated, while the parameters of the target network are updated by regularly copying the parameters of the estimation network. In order to enhance the robustness of noise suppression, the loss function of the estimation network and the target network adopts Smooth L1Loss, and its loss function is defined as:

[0132] ;

[0133] in, represents the loss function, For the The estimated network parameters based on VDCNN in the iteration, is the target network parameter based on VDCNN, represents the view-distance action-value function corresponding to the estimated network parameters, Respectively represent The status of the iteration, Indicates The action of the iteration, Represents the view-distance action value function corresponding to the target network parameters. Each update only passes the gradient parameters of the reward element, and the losses of the corresponding elements at other positions are forced to zero. The optimization strategy uses the stochastic gradient descent algorithm, and the exploration strategy uses ε -greedy strategy, set exploration probability The initial value of is 0.5. .

[0134] Given , can be maximized by choosing The action of the function is used to determine the optimal action, thereby deriving the greedy optimal strategy ,Right now:

[0135] ;

[0136] in, yes The maximum value is calculated by the view-distance convolutional neural network VDCNN at each moment and selected The view-distance adjustment action corresponding to the value can be divided into view-adjustment action and distance-adjustment action. The view-adjustment action is obtained through the view-value graph, while the distance-adjustment action is obtained through the distance-value vector get.

[0137] S4: Using the camera model and coordinate transformation, the predicted optimal viewing angle pixel coordinates are converted into the optimal viewing angle and distance in the robot coordinate system. The robot can Move and obtain a partial image of the calibration plate to evaluate the calibration plate. If the evaluation result shows that the posture is ideal, record the posture of the end of the robot arm relative to the base at this time. And the calibration plate image taken by the camera.

[0138] View-distance adjustment action It is defined as the robot moving to the predicted next best viewing angle according to the viewing angle selection strategy, and then adjusting to the predicted best shooting distance according to the distance selection strategy. This embodiment simplifies the viewing angle-distance selection set into a finite viewing angle set with 4 degrees of freedom. That is, the environment state is represented according to the preprocessed grayscale image, and each pixel corresponds to a viewing angle, thereby generating a finite viewing angle set corresponding to each pixel, namely:

[0139] ;

[0140] Where: , Respectively represent the row coordinates and column coordinates of the pixel point adjusted by the viewing angle, Indicates the magnitude of the distance adjustment. Indicates the preprocessed image at pixel point The depth value corresponding to the depth map at the position, Indicates that the robot first pre-processes the pixels of the image The corresponding action of the perspective adjustment is performed based on the position, and then based on the amplitude Perform the action corresponding to the distance adjustment. Therefore, the dimension of the perspective adjustment action space is two-dimensional, while the dimension of the distance adjustment action space is one-dimensional.

[0141] The perspective adjustment action is to point the camera direction to the corresponding position of the pixel point to adjust the perspective. The purpose is to adjust the camera perspective to a suitable position so that the calibration plate is always in the camera field of view in the following distance adjustment action. At the same time, the posture generated by K-means clustering is adjusted and optimized to minimize the distance between the calibration plate and the camera optical axis, thereby improving the accuracy of hand-eye calibration.

[0142] Viewing Angle Adjustment Pixels In the perspective value diagram, The position with the largest value, that is:

[0143] ;

[0144] Through the camera pinhole model Convert pixel coordinates to camera coordinate system coordinates and combine them with the hand-eye transformation matrix obtained by the initial hand-eye calibration , the viewing angle can be adjusted by pixels Convert to coordinates expressed in the robot coordinate system:

[0145] ;

[0146] Where: Indicates that the pixel position in the preprocessing image Restore to the pixel position in the actual color image, Indicates action Through the camera pinhole model Converted action Position in the camera coordinate system;

[0147] Adjusting the robot's perspective actually involves rotating the robot's posture, so the corresponding rotation matrix needs to be calculated. The robot coordinates corresponding to the view adjustment point , we can get point to Vector , the corresponding rotation matrix can be obtained through the Rodrigues formula:

[0148] ;

[0149] Where: is the rotation matrix between two vectors calculated by Rodriguez formula. According to the rotation matrix The viewing angle can be adjusted by rotating the robot. and The rotation matrix is ​​the matrix corresponding to the vector and vector known by the rotation matrix. is (0, 0, -1), is a vector and vector Angle.

[0150] In this embodiment, the distance adjustment action is to reduce the distance between the center of the camera and the calibration plate as much as possible, so as to reduce the distance between the end effectors in different calibration postures as much as possible, thereby improving the accuracy of hand-eye calibration. If the reduced distance is too large, not all corner points can be displayed in the camera field of view, and calibration cannot be performed. Therefore, the reduced distance should be within an appropriate range.

[0151] Distance adjustment According to the 1×30 size distance value vector output by the fully connected layer In the calculation, the output prediction value is the largest position, that is:

[0152] ;

[0153] Adjust the distance of the robot-camera system, only modify the position of the robot without changing its orientation. The RGB image after adjusting the viewing angle is detected by the calibration plate to obtain the detection center position of the calibration plate , the detection center of the calibration plate is converted into the coordinate system of the robot as follows:

[0154] ;

[0155] Known initial hand-eye transformation matrix The robot coordinates corresponding to the current robot end , the robot coordinates corresponding to the current camera can be obtained , expressed as:

[0156] ;

[0157] Then the depth data is used to obtain the current camera pixel center position in the base coordinate system. , specifically expressed as:

[0158] ;

[0159] The depth data specifically refers to the depth map of the D-435 camera at the center of the current camera pixel. Depth value .

[0160] Combined with the robot coordinates corresponding to the calibration plate , get the coordinates of the robot end after the distance adjustment Should be:

[0161] ;

[0162] Where: Controls the distance adjustment. When it is smaller, the distance between the calibration plate and the camera center will increase to avoid the problem that the distance between the two is too close and the feature points are out of view.

[0163] This example implements reinforcement learning training in the Gazebo simulation environment and conducts a hand-eye calibration comparison experiment. At the same time, a physical experiment is conducted on a Yaskawa robot to compare and verify the effects of several robot automatic hand-eye calibration algorithms on improving calibration accuracy. The distance selection empirical enhancement algorithm framework is written in Pytorch without loss of generality, and the discount factor is set. , learning rate , experience pool capacity ,Target Network parameter synchronization steps .

[0164] In the hand-eye calibration process, each method performs 10 calibration experiments, and each group generates 10 postures. The hand-eye matrix of each group is calculated using the Tsai method. In the comparative experiment, 10 fixed postures were generated and the transformation matrix of each posture was recorded. By solving the PnP equation using the corner points of the calibration plate, we can obtain the transformation matrix from the camera to the calibration plate. Then, the theoretical transformation matrix of the calibration plate relative to the robot base is determined as:

[0165] ;

[0166] Therefore, each set of calibration postures can obtain a theoretical position of the center of the chessboard , the reprojection error is defined as:

[0167] ;

[0168] That is, the average value of the reprojection error calculated for each set of calibration postures is used as the error evaluation function of the hand-eye relationship matrix accuracy, where , is the actual position of the center of the calibration plate.

[0169] The following three typical robot automatic hand-eye calibration posture generation strategies are used to conduct comparative experiments with the NBP-CalibDRL automatic hand-eye calibration method based on deep reinforcement learning proposed in this paper. The specific scheme is as follows:

[0170] (1) Automatic hand-eye calibration method with random robot posture generation: In a given space Points are uniformly randomly generated inside the image, and their directions point to the initial positions. A total of 10 calibration postures are obtained for the final hand-eye calibration.

[0171] (2) Automatic hand-eye calibration method based on K-means clustering: The K-means algorithm is used to cluster the rotation components of several generated random poses, and finally 10 calibration poses are generated for the final hand-eye calibration.

[0172] (3) Improved automatic hand-eye calibration method based on K-means clustering

[0173] The specific improvement is: further screening the hand-eye calibration posture generated by the K-means clustering method.

[0174] The screening rule is offset from the center of the chessboard satisfy:

[0175] ;

[0176] in, and is a pre-set hyperparameter. This screening rule is used to control the distance between the chessboard and the camera center. The disadvantage of this method is that it is sensitive to position errors and angles. and When the difference is small, it takes a long time to screen, and when the difference is large, it is difficult to generate a good calibration posture. 0.35m, 0.45m;

[0177] As shown in Table 1 below, this embodiment uses the following indicators to evaluate the performance of the algorithm in the hand-eye calibration task:

[0178] Table 1 Hand-eye calibration performance evaluation index table

[0179]

[0180] like Figure 4 As shown in the figure, the Gazebo simulation platform is used to build a robot automatic hand-eye calibration environment. The simulation robot uses UniversalRobot: UR5, and the terminal integrates the color depth camera Realsense D435. The remote control interface of the Gazebo simulation platform is called to obtain the robot's motion state and color depth images during training.

[0181] The perspective-distance selection strategy is obtained through interactive learning between the robot and the environment. Based on this principle, the training steps are arranged as follows: a calibration plate is placed in the robot workspace, and after the robot moves to a random posture, the pre-processed grayscale image is input into the VDCNN for learning and updating, and experience data is continuously collected and stored in the experience pool during the task execution, until it moves to the ideal posture and then moves to the next random posture again. The perspective-distance selection strategy proposed in the present invention has achieved good convergence effect after 100 rounds of training in 500 rounds of posture adjustment task training.

[0182] As shown in Table 2 and Figure 5 As shown, the comparison results of the reprojection errors of the simulation environment are obtained;

[0183] Table 2 Comparison of reprojection errors of four methods under simulation environment (mm²)

[0184]

[0185] From Table 2 and Figure 5 It can be seen that the reprojection error of the method of the present invention is the smallest, and the calculated reprojection error variance is small. The improved K-means algorithm is second. The reprojection error of the random generation calibration posture method is the largest, and the variance is large. The K-means algorithm before the improvement has improved accuracy compared to the random calibration posture method, but the error is still larger than that of the method of the present invention and the improved K-means algorithm.

[0186] In practical applications, the center position of the chessboard will shift, and it is necessary to test whether the network can adapt to the shift of the chessboard position. At the same time, the center position of the chessboard calculated in the improved K-means algorithm may also produce errors, which can also be regarded as the shift of the chessboard position. Therefore, an experiment is set to verify the robustness of the improved K-means algorithm and the method of the present invention.

[0187] Three sets of experiments were conducted, with the chessboard offsets being (-10, -10, 0) mm, (-20, -20, 0) mm, and (-30, -30, 0) mm, respectively. The reprojection errors of the hand-eye calibration matrix obtained under different conditions were then calculated. The experimental results are shown in Tables 3 and Figure 6 shown.

[0188] Table 3 Comparison of reprojection errors of the two methods under different position offsets in the simulation environment (mm²)

[0189]

[0190] From Table 3 and Figure 6It can be seen that the reprojection error of the method of the present invention can be similar to or smaller than that of the improved K-means algorithm under different position offsets. The improved K-means algorithm is very sensitive to position offsets. When the position offset exceeds 30mm, the hand-eye calibration cannot be completed. Therefore, the center position of the chessboard is required to be calculated very accurately. However, the reprojection error of the method of the present invention is kept at a very small and similar level under position offsets of 10mm, 20mm, and 30mm, which proves the robustness of the method.

[0191] Reduce the number of calibration poses in each group and test the impact on the calibration results. The results are shown in Table 4 and Figure 7 shown.

[0192] Table 4 Reprojection error of the two methods under different calibration pose numbers in the simulation environment (mm²)

[0193]

[0194] From Table 4 and Figure 7 It can be seen that the reprojection error of the method of the present invention becomes significantly larger when there are 4 calibration postures in each group, while the performance of the reprojection error is not much different from that of 10 calibration postures when there are 6 calibration postures in each group. The improved K-means algorithm can also show better results when there are 4 calibration postures in each group. Similarly, when there are 6 calibration postures in each group, the reprojection error of the improved K-means algorithm is significantly greater than that of the method of the present invention.

[0195] In order to verify the effectiveness of the view-distance selection strategy trained in the simulation environment, the present invention carried out a hand-eye calibration experiment on a physical robot. The view predicted by the algorithm of the present invention from the image needs to be converted into the robot view through the hand-eye relationship, and in the physical robot experiment, the hand-eye calibration accuracy can be controlled to have a negligible impact on the algorithm of the present invention. In these experiments, the Shougang Motoman Yaskawa MOTOMAN-GP8 industrial robot was used. The robot is equipped with a remote control MOTOCOM32 API to complete the robot control, and an integrated Intel Realsense D435 depth camera is connected to the end effector.

[0196] Although the initial model already has high accuracy, this embodiment fine-tunes the model through a small number of training batches to ensure its robustness and accuracy in actual industrial environments. This additional training step helps the model better adapt to the specific dynamics and working environment of the physical robot.

[0197] The physical robot performs 10 groups of robot hand-eye calibration experiments consistent with the simulation for each method. Each group obtains 10 different robot end postures, and records the robot end posture data in each posture and the checkerboard image under the camera to calculate the hand-eye conversion matrix.

[0198] In the comparative experiment, the calibration plate selects a checkerboard with different sizes and corner points. In this embodiment, three calibration plates are preferably selected: the first type: the corner point distribution is 11×7, the size is 19.25 mm; the second type: the corner point distribution is 7×6, the size is 28.75 mm; the third type: the corner point distribution is 9×6, the size is 28 mm. After each calibration plate is placed in the robot hand-eye calibration workspace, 10 fixed randomly generated end posture data are recorded. The method consistent with the simulation is used to calculate the reprojection error by using each method to obtain the hand-eye matrix.

[0199] As shown in Table 5 below, the reprojection error comparison of various methods using the first type of calibration plate pattern as the calibration plate is shown, such as Figure 8 As shown, the corresponding box plot comparison is shown. Tables 6 and 7 compare the mean and standard deviation of the reprojection errors of various methods when using three different calibration plates. The variation in reprojection error is due to differences in the size and complexity of the calibration plates. The reprojection error obtained by adjusting the calibration posture using the method of the present invention is the smallest (16.699 mm², 17.917 mm², and 18.346 mm², respectively), and the variance is low. This proves that the method of the present invention achieves the highest accuracy in hand-eye calibration, while also ensuring highly stable calibration results.

[0200] Table 5 Comparison of reprojection errors of the four methods for the first calibration plate in real environment (mm²)

[0201]

[0202] Table 6 Comparison of average reprojection errors of four methods in real environment (mm²)

[0203]

[0204] Table 7 Comparison of standard deviation of reprojection error of four methods in real environment (mm)

[0205]

[0206] The method of the present invention innovatively applies deep reinforcement learning to industrial robot hand-eye calibration, improves calibration accuracy by adjusting the view angle and distance, and adopts the view-distance convolutional neural network VDCNN to fit the view-distance action value function fitting method, and improves calibration accuracy and robustness by utilizing autonomous learning and decision-making. In addition, the hand-eye calibration posture generation method based on K-means clustering is improved, which significantly improves the calibration accuracy.

[0207] Example 2

[0208] This embodiment provides a robot automatic hand-eye calibration system based on deep reinforcement learning, which is used to implement the robot automatic hand-eye calibration method based on deep reinforcement learning in Example 1. The system includes: an initial hand-eye matrix acquisition module, an initial calibration posture acquisition module, an image generation module, a network prediction module, a perspective adjustment pixel point acquisition module, a perspective adjustment module, and a distance adjustment module;

[0209] In this embodiment, the initial hand-eye matrix acquisition module is used to acquire the initial hand-eye matrix;

[0210] In this embodiment, the initial calibration posture acquisition module is used to generate and filter the postures of the end of the random robot arm, and perform cluster analysis on the filtered posture data to obtain multiple initial calibration postures;

[0211] In this embodiment, the image generation module is used to obtain the calibration plate image at each initial calibration posture and perform image preprocessing;

[0212] In this embodiment, the network prediction module is used to input the image after image preprocessing into the perspective-distance convolutional neural network VDCNN, the perspective-distance convolutional neural network VDCNN fits the perspective-distance action value function based on deep reinforcement learning, and predicts the optimal perspective-distance adjustment action, and the perspective-distance convolutional neural network VDCNN outputs the perspective value map and the distance value vector based on the encoder-decoder structure;

[0213] In this embodiment, the viewing angle adjustment pixel point acquisition module is used to calculate the position with the maximum predicted value in the viewing angle value map to obtain the viewing angle adjustment pixel point;

[0214] In this embodiment, the viewing angle adjustment module is used to convert the viewing angle adjustment pixel points into robot coordinates represented by the robot coordinate system based on the initial hand-eye matrix, obtain the corresponding rotation matrix, and rotate the robot according to the rotation matrix to achieve viewing angle adjustment;

[0215] In this embodiment, the distance adjustment module is used to calculate the position with the largest predicted value in the distance value vector, obtain the distance adjustment amplitude, perform distance adjustment on the robot, and obtain the final hand-eye calibration posture.

[0216] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A robot automated hand-eye calibration method based on deep reinforcement learning, characterized in that: The steps include: Get the initial hand-eye matrix; Generate and filter the postures of the end of the random robot arm, perform cluster analysis on the filtered posture data, and obtain multiple initial calibration postures; The calibration plate images under each initial calibration posture are obtained and image preprocessing is performed, and the images are input into the view-distance convolutional neural network VDCNN, which fits the view-distance action value function based on deep reinforcement learning and predicts the optimal view-distance adjustment action. The view-distance convolutional neural network VDCNN outputs a view value map and a distance value vector based on an encoder-decoder structure; Each pixel value in the perspective value map represents the expected future reward after selecting the perspective represented by the pixel; The values ​​in the distance value vector represent the future returns that can be expected after selecting the set distance adjustment amplitude; Calculate the position with the maximum predicted value in the perspective value map to obtain the perspective adjustment pixel point; Based on the initial hand-eye matrix, the view adjustment pixel points are converted into robot coordinates expressed in the robot coordinate system, and the corresponding rotation matrix is ​​obtained. The robot is rotated according to the rotation matrix to achieve the view adjustment; Calculate the position with the largest predicted value in the distance value vector, obtain the distance adjustment amplitude, adjust the distance of the robot, and obtain the final hand-eye calibration posture.

2. The robot automatic hand-eye calibration method based on deep reinforcement learning according to claim 1 is characterized in that: When generating the posture of the random end of the robot arm, its rotation vector is constrained to point to the desktop projection of the initial position, including: Get a random point in the set space , the coordinates of the desktop projection of the initial position are , the vector from the random point to the projection point on the desktop is , the corresponding rotation matrix is ​​obtained through the Rodriguez formula, expressed as: ； in, is a vector and vector The angle of represents an antisymmetric matrix, For vector With vector The rotation matrix between the robot Rotate it to achieve random posture pointing to the desktop projection. and The rotation matrix is ​​the matrix corresponding to the known unit vector and vector.

3. The robot automated hand-eye calibration method based on deep reinforcement learning according to claim 2, characterized in that: Perform cluster analysis on the filtered posture data to obtain multiple initial calibration postures, including: The rotation matrix Convert to rotation vector Based on the K-means clustering method, select the cluster center in each class The closest vector As a representative of this class, a set number of robot end-arm postures are retained, specifically expressed as: ； in, Indicates the distance from the cluster center The nearest vector, and Represents the rotation vector The and indivual, represents the Euclidean distance.

4. The robot automated hand-eye calibration method based on deep reinforcement learning according to claim 1, characterized in that: The view-distance convolutional neural network VDCNN includes multiple convolutional layers, multiple deconvolutional layers and fully connected layers. The calibration plate image after image preprocessing passes through multiple convolutional layers and multiple deconvolutional layers in sequence to obtain a view value map. The calibration plate image after image preprocessing passes through multiple convolutional layers, multiple deconvolutional layers and fully connected layers in sequence to obtain a distance value vector.

5. The robot automated hand-eye calibration method based on deep reinforcement learning according to claim 1, characterized in that: The view-distance convolutional neural network (VDCNN) is based on deep reinforcement learning to fit the view-distance action value function. ε -greedy strategy to select the next best view-distance adjustment action , specifically expressed as: ； ； ； ； ； ； ； in, , Perspective Value Map Resolution, To explore the probability, Indicated in The uniform distribution of the interval, represents the view-distance action-value function, Indicates the status, Indicates action, Indicates strategy, represents the reward function, Indicates reward, represents the discount factor, Indicates duration, Indicates the detection center coordinates of the calibration plate, represents the image center coordinates, Represents the preset Manhattan distance threshold, Indicates the preset depth threshold, Indicates the depth of the calibration plate detection center. express function.

6. The robot automated hand-eye calibration method based on deep reinforcement learning according to claim 1, characterized in that: The view-distance convolutional neural network VDCNN fits the view-distance action value function based on deep reinforcement learning, and the loss function is: ； in, represents the loss function, For the The estimated network parameters based on VDCNN in the iteration, is the target network parameter based on VDCNN, represents the view-distance action-value function corresponding to the estimated network parameters, Respectively represent The status of the iteration, Indicates The action of the iteration, Represents the view-distance action-value function corresponding to the target network parameters.

7. The robot automated hand-eye calibration method based on deep reinforcement learning according to claim 1, characterized in that: Calculate the position with the largest predicted value in the perspective value map to obtain the perspective adjustment pixel point, which is specifically expressed as: ； in, Indicates the viewing angle adjustment pixel point, Represents a perspective value graph; Based on the initial hand-eye matrix, the perspective adjustment pixel points are converted into robot coordinates expressed in the robot coordinate system, which is specifically expressed as: ； in, represents the initial hand-eye matrix, Indicates that the pixel position in the preprocessing image Restore to the pixel position in the actual color image, Representation of the optimal view-distance adjustment action through the camera pinhole model The position of the best view-distance adjustment action obtained by conversion in the camera coordinate system; Get the robot coordinates corresponding to the current robot end , we get the vector , the corresponding rotation matrix is ​​obtained through the Rodriguez formula, expressed as: ； in, To calculate the rotation matrix using the Rodriguez formula, and is the matrix corresponding to the unit vector and vector known in the rotation matrix, is a vector and vector Angle.

8. The robot automated hand-eye calibration method based on deep reinforcement learning according to claim 1, characterized in that: Calculate the position with the largest predicted value in the distance value vector and get the distance adjustment amplitude, which is specifically expressed as: ； in, Indicates the distance adjustment amplitude. Represents the distance value vector.

9. The robot automated hand-eye calibration method based on deep reinforcement learning according to claim 1, characterized in that: Adjust the distance of the robot to obtain the final hand-eye calibration posture, including: The calibration plate image after adjusting the viewing angle is detected by the calibration plate to obtain the center pixel coordinates , the robot coordinates corresponding to the calibration plate It is expressed as: ； in, Indicates the depth of the calibration plate detection center. represents the camera pinhole model, Indicates that the calibration plate will be tested at the center position Restore to the pixel position in the actual color image, represents the initial hand-eye matrix; Based on the initial hand-eye matrix and the robot coordinates corresponding to the current robot end, the robot coordinates corresponding to the current camera are obtained , specifically expressed as: ； in, Indicates the robot coordinates corresponding to the current robot end; Get the camera's depth map at the center of the current camera pixel Depth value , get the current camera pixel center position in the base coordinate system , specifically expressed as: ； Based on robot coordinates The coordinates of the robot end after the distance adjustment are: ； in, Indicates the coordinates of the robot end after distance adjustment.

10. A robot automated hand-eye calibration system based on deep reinforcement learning, characterized in that: The method for implementing the robot automatic hand-eye calibration based on deep reinforcement learning as described in any one of claims 1 to 9 comprises: an initial hand-eye matrix acquisition module, an initial calibration posture acquisition module, an image generation module, a network prediction module, a perspective adjustment pixel point acquisition module, a perspective adjustment module, and a distance adjustment module; The initial hand-eye matrix acquisition module is used to acquire the initial hand-eye matrix; The initial calibration posture acquisition module is used to generate and filter the postures of the end of the random robot arm, and perform cluster analysis on the filtered posture data to obtain multiple initial calibration postures; The image generation module is used to obtain the calibration plate image in each initial calibration posture and perform image preprocessing; The network prediction module is used to input the image after image preprocessing into the perspective-distance convolutional neural network VDCNN, the perspective-distance convolutional neural network VDCNN fits the perspective-distance action value function based on deep reinforcement learning, and predicts the optimal perspective-distance adjustment action. The perspective-distance convolutional neural network VDCNN outputs a perspective value map and a distance value vector based on an encoder-decoder structure; Each pixel value in the perspective value map represents the expected future reward after selecting the perspective represented by the pixel; The values ​​in the distance value vector represent the future returns that can be expected after selecting the set distance adjustment amplitude; The viewing angle adjustment pixel point acquisition module is used to calculate the position with the maximum predicted value in the viewing angle value map to obtain the viewing angle adjustment pixel point; The viewing angle adjustment module is used to convert the viewing angle adjustment pixel points into robot coordinates represented by the robot coordinate system based on the initial hand-eye matrix, obtain the corresponding rotation matrix, and rotate the robot according to the rotation matrix to achieve viewing angle adjustment; The distance adjustment module is used to calculate the position with the largest predicted value in the distance value vector, obtain the distance adjustment amplitude, perform distance adjustment on the robot, and obtain the final hand-eye calibration posture.

Images