Dual-arm robot collision estimation method, system, computer device and storage medium

Abstract

The invention discloses a dual-arm robot collision estimation method, system, computer equipment and storage medium. A dual-arm robot simulation model is built in a simulation platform. Paired gap data between different connecting rods of two mechanical arms and paired gap data between all connecting rods of the two mechanical arms and environmental obstacles are collected to construct two paired gap data sets; a paired gap learning neural network architecture is built, and the two constructed data sets are used for training respectively to construct a dual-arm robot self-collision estimation learning model and a dual-arm robot and environmental obstacle collision estimation learning model; at the same time, the joint angle vectors of the left and right mechanical arms of several groups of dual-arm robots are input to the two trained collision estimation models, and combined with the safety distance threshold, the dual-arm robot collision estimation results can be obtained in batches. The dual-arm robot collision estimation task is efficiently processed, thereby accelerating the dual-arm robot collision estimation process and providing effective support for subsequent efficient motion planning of the dual-arm robot.

Description

Technical Field

[0001] The present invention belongs to the technical field of dual-arm robot system collision estimation, and in particular relates to a dual-arm robot collision estimation method, system, computer equipment and storage medium. Background Art

[0002] In recent years, dual-arm robot systems have attracted extensive attention from researchers due to their advantages such as flexibility and collaboration. However, motion planning of dual-arm robots in complex environments remains a challenging problem. Compared with single-arm robot systems, dual-arm robots face exponentially increasing complexity in motion planning due to their higher degrees of freedom, which brings severe high-dimensional challenges under the curse of dimensionality. In addition, the shared workspace and complex dual-arm system configuration significantly increase the risk of mutual interference during coordinated motion. Therefore, the motion planning of dual-arm robots must not only solve the problem of collision-free trajectory generation with environmental obstacles, but also combine a robust self-collision avoidance mechanism to ensure smooth and efficient operation. Therefore, the premise for achieving efficient motion planning of dual-arm robots is to solve the problem of efficient collision estimation of dual-arm robots.

[0003] Regarding the collision estimation problem of robotic arms, researchers have proposed two types of collision estimation methods. One is a collision estimation method based on geometric models, including the bounding box method and its simplified variants. The other is a collision estimation method based on learning, including the kernel perceptron model, support vector machine (SVM) and neural network, which predicts whether a collision will occur by learning the collision distance (gap) between the robotic arm and the environment. Compared with the traditional collision estimation method based on geometric models, the collision estimation method based on learning has higher processing efficiency and has attracted the attention of more and more researchers. However, some learning-based methods have been proposed in the field to handle robotic arm collision detection, but they are still limited to a single robotic arm. Due to the complexity of the structure of a dual-arm robot, its collision distance function is complex and highly non-convex, and it is difficult to directly learn the collision distance function between the two robotic arms. Therefore, how to achieve efficient and high-precision collision estimation of a dual-arm robot system is more challenging.

[0004] In response to the above challenges, a dual-arm robot collision estimation method, system, computer device and storage medium are proposed. Summary of the invention

[0005] In response to the above technical problems, the present invention provides a dual-arm robot collision estimation method, system, computer device and storage medium, aiming to solve the problem that the existing dual-arm robot collision estimation method has low estimation accuracy and efficiency, which makes it difficult to achieve efficient motion planning of the dual-arm robot.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] A dual-arm robot collision estimation method, the method comprising the following steps:

[0008] S100: Building a dual-arm robot simulation model in the simulation platform, collecting paired gap data between different links of the two robotic arms and paired gap data between all links of the two robotic arms and environmental obstacles, and constructing paired gap data sets between the left robotic arm and the right robotic arm and paired gap data sets between the dual-arm robot and environmental obstacles;

[0009] S200: building a paired gap learning neural network, and using the paired gap learning neural network to learn the mapping relationship between the joint space of the left and right mechanical arms of the dual-arm robot and the paired gap space of different connecting rods, and the mapping relationship between the joint space of the left and right mechanical arms of the dual-arm robot and the paired gap space of different connecting rods and environmental obstacles;

[0010] S300: inputting the paired gap data set of the left robotic arm and the right robotic arm and the paired gap data set of the dual-arm robot and the environmental obstacles into the paired gap learning neural network respectively, obtaining the estimated values ​​of the paired gap values ​​of different links and the estimated values ​​of the paired gap values ​​of different links and the obstacles respectively based on the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot to the paired gap spaces of different links and the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot to the paired gap spaces of different links and the environmental obstacles, training is performed in combination with a preset loss function, and a dual-arm robot self-collision estimation learning model and a dual-arm robot and environmental obstacle collision estimation learning model are constructed;

[0011] S400: simultaneously inputting the joint angle vectors of the left and right robotic arms of several groups of dual-arm robots into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model, and combining the safety distance threshold to batch obtain the dual-arm robot collision estimation results.

[0012] Preferably, the dual-arm robot system consists of two robotic arms, a left robotic arm and a right robotic arm. Assuming that the degree of freedom of each robotic arm is N, each robotic arm contains N+1 connecting rods, and the N+1 connecting rods of the left robotic arm are expressed as , the N+1 links of the right robot are represented as ; In addition, the left robot arm joint space is represented as ,in represents the first joint angle of the left arm, The left arm joint angles; the joint space of the right robot arm is represented by ,in represents the first joint angle of the right arm, The right arm The S100 collects the paired clearance data between different links of the two robotic arms, including:

[0013] S110: Sample a set of joint angles in the joint space of the left and right robotic arms respectively and ,in, =1,2,…, , is the total number of sampling times, and simultaneously controls the two robotic arms to move to the corresponding joint positions;

[0014] S120: Use the distance measurement tool component of the simulation platform to measure the gap between any link of the left robotic arm and any link of the right robotic arm to obtain The pairwise clearance value of different connecting rods: Distance( , ), Distance( , ),…, Distance( , ), where Distance( , ) indicates connecting rod and connecting rod The gap value between

[0015] S130: Repeat the S120 process Second-rate, Different sets of dual-arm robot joint angles will be obtained accordingly There are different sets of connecting rod pair gap data, where the dimension of each set of pair gap data is ,in (N+1) (N+1), complete the construction of the paired gap dataset of the left and right robotic arms.

[0016] Preferably, it is assumed that the dual-arm robot is in a scenario including Obstacles , and the positions of all obstacles are fixed and known. S100 collects paired clearance data between all links of the two robotic arms and environmental obstacles, including:

[0017] S140: Sample a set of joint angles in the joint space of the left and right robotic arms respectively and ,in, =1,2,… , is the total number of sampling times, and simultaneously controls the two robotic arms to move to the corresponding joint positions;

[0018] S150: Use the distance measurement tool component of the simulation platform to measure the total Connecting rod to The gap between obstacles is obtained ( ) The pairwise gap value between each link and the obstacle: Distance( , ), Distance( , ),…, Distance( , ), Distance( , ), Distance( , ),…, Distance( , ), where Distance( , ) indicates connecting rod and obstacles The pairwise gap value between , ) points and obstacles The pairwise gap value between ;

[0019] S160: Repeat the S150 process Second-rate, Different sets of dual-arm robot joint angles will be obtained accordingly There are different sets of connecting rod pair gap data, where the dimension of each set of pair gap data is ,in ( ) , complete the construction of the paired gap dataset between the dual-arm robot and environmental obstacles.

[0020] Preferably, the paired gap learning neural network in S200 includes six parts in structure. The first part is an input layer, which converts the left robot arm of the dual-arm robot system into The joint angles of the right robot arm The joint angles are spliced ​​to form a dual-arm robot joint angle as the input vector , whose dimensions are , through the position encoding function For the input vector Encode, where is a hyperparameter that maps the input joint angles to a high-frequency space in order to capture more detailed motion patterns;

[0021] The second part consists of a fully connected layer and an activation function ReLU layer. After the output of the first part is processed by the fully connected layer, the output features will be mapped to a higher-dimensional representation space.

[0022] The third part includes multiple residual blocks. The residual block part introduces the residual learning mechanism through skip connections. The residual block consists of two layers of fully connected layers and activation function ReLU. In the main branch, the third part receives the high-dimensional features output by the second part, first maps them to the new feature space through the first fully connected layer, then uses the ReLU activation function for nonlinear transformation, and then further processes the features through the second fully connected layer. The skip connection directly adds the input features to the output of the main branch.

[0023] The fourth part consists of a fully connected layer, an activation function ReLU, and a DropOut layer. After the output of the third part is processed by the fully connected layer, the output features will be mapped to a higher-dimensional representation space;

[0024] The fifth part consists of a fully connected layer. After the output of the fourth part is processed by the fully connected layer of the fifth part, the output feature number is , the purpose is to achieve dimensional gap distance mapping.

[0025] Preferably, S300 includes:

[0026] choose The joint angles of the dual-arm robot are used as network input features. The gap values ​​of different connecting rod pairs are used as the network output features, that is, the output feature number Set to , , trained using the mean squared error between the pairwise gap estimates and actual values ​​between different links as the loss function, as follows:

[0027] ;

[0028] represents the optimal weight of the paired gap learning neural network under the current training set, represents the neural network weights, Indicates Actual value of the pairwise clearance between different connecting rods in a group, Represents a pairwise gap learning neural network with network weights The output of Estimated pairwise clearances between different links of the group;

[0029] After training, the optimal weights of the paired gap learning neural network can be obtained. , indicating that the mapping relationship between the joint space of the left and right arms of the dual-arm robot and the gap space of different connecting rod pairs has been learned, and the optimal weight Loaded into the paired gap learning neural network, thus constructing a dual-arm robot self-collision estimation learning model;

[0030] choose The joint angles of the dual-arm robot are used as network input features. The paired gap data between different connecting rods and obstacles are used as the network output features, that is, the output feature number Set to , , the mean squared error between the estimated and actual pairwise gaps between different links and obstacles is used as the loss function for training, as follows:

[0031] ;

[0032] represents the optimal weight of the paired gap learning neural network under the current training set, represents the neural network weights, Indicates The actual value of the pairwise clearance between the group link and the obstacle, Represents a pairwise gap learning neural network with network weights The output of pairwise gap estimates between group links and obstacles;

[0033] After training, the optimal weights of the paired gap learning neural network can be obtained. , indicating that the mapping relationship between the joint space of the left and right manipulators of the dual-arm robot and the paired gap space of different links and environmental obstacles has been learned, and the optimal weight Loaded into the paired gap learning neural network, a collision estimation learning model for a dual-arm robot and environmental obstacles is constructed.

[0034] Preferably, S400 includes:

[0035] S410: Batch sampling in the joint space of the left and right arms of the dual-arm robot system Group points: , , { , …, , ,in Indicates the first point collected in the left robot arm joint space. Indicates the first point collected in the joint space of the right robot arm, and the others are similar;

[0036] S420: For any set of dual-arm robot joint angles { , , , input it into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model at the same time, and make a judgment based on the safety distance threshold. If both models return the result of no collision, it means that there is no collision between the left and right arms, and the two robotic arms have no collision with the environment. , It can be used as a candidate path point for collision-free links in the motion path of the dual-arm robot. Otherwise, it is considered that the dual-arm robot has collided. , abandon;

[0037] S430: One-time adjustment of the joint space of the dual-arm robot Group sampling points: , , { , …, , After two collision estimation learning models are built, batch acquisition Group sampling point collision estimation results.

[0038] Preferably, S420 specifically includes:

[0039] For any set of joint angles sampled in the joint space of the left and right manipulators and , and input it into the dual-arm robot self-collision estimation learning model to obtain The estimated value of the pairwise gap between different links of the left and right robotic arms: D_es( , ), D_es ( , ),…, D_es ( , ), where D_es ( , ) represents the link output by the self-collision estimation learning model of the dual-arm robot and connecting rod An estimate of the gap between

[0040] The global shortest distance between the left and right arms Expressed as Minimum of pairwise clearance estimates between different links: {D_es( , ), D_es ( , ),…,D_es ( , )}

[0041] like Smaller than the safety distance set by the user , it means that there is a collision between the left and right robotic arms, and the final output of the dual-arm robot self-collision estimation learning model is set to 1. Otherwise, it is considered that there is no collision, and the final output of the dual-arm robot self-collision estimation learning model is 0;

[0042] For any set of joint angles sampled in the joint space of the left and right manipulators and , and input it into the dual-arm robot and environmental obstacle collision estimation learning model to obtain The estimated gap values ​​between different links and obstacles are: D_es ( , ), D_es ( , ),…, D_es ( , ), D_es( , ), D_es( , ),…, D_es ( , ), where D_es ( , ) and D_es ( , ) represents the link output by the learning model for collision estimation between the dual-arm robot and environmental obstacles and obstacles Between and connecting rod and obstacles An estimate of the gap between

[0043] The global shortest distance between all links and all obstacles Expressed as The minimum of the estimated pairwise gap values ​​between links and obstacles: {D_es ( , ), D_es ( , ),…,D_es ( , ), D_es( , ), D_es( , ),…, D_es ( , )};

[0044] like Smaller than the safety distance set by the user , it means that the dual-arm robot collides with the environmental obstacle, and the final output of the dual-arm robot and environmental obstacle collision estimation learning model is set to 1. Otherwise, it is considered that there is no collision, and the dual-arm robot and environmental obstacle collision estimation learning model finally outputs 0.

[0045] A dual-arm robot collision estimation system, including a simulation platform building and data set acquisition module, a neural network building module, a neural network training module and a collision result estimation module;

[0046] The simulation platform building and data set collection module is used to build a dual-arm robot simulation model in the simulation platform, collect the paired gap data between different connecting rods of the two robotic arms and the paired gap data between all connecting rods of the two robotic arms and environmental obstacles, and construct the paired gap data set between the left robotic arm and the right robotic arm and the paired gap data set between the dual-arm robot and environmental obstacles;

[0047] A neural network building module is used to build a paired gap learning neural network, and use the paired gap learning neural network to learn the mapping relationship between the joint space of the left and right mechanical arms of a dual-arm robot and the paired gap space of different connecting rods, as well as the mapping relationship between the joint space of the left and right mechanical arms of a dual-arm robot and the paired gap space of different connecting rods and environmental obstacles;

[0048] A neural network training module is used to input the paired gap data sets of the left and right robotic arms and the paired gap data sets of the dual-arm robot and the environmental obstacles into the paired gap learning neural network respectively, and obtain the estimated values ​​of the paired gap values ​​of different links and the estimated values ​​of the paired gap values ​​of different links and the obstacles respectively based on the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot and the paired gap space of different links and the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot and the paired gap space of different links and the environmental obstacles, and train with a preset loss function to construct a dual-arm robot self-collision estimation learning model and a dual-arm robot and environmental obstacle collision estimation learning model;

[0049] The collision result estimation module is used to simultaneously input the joint angle vectors of the left and right robotic arms of several groups of dual-arm robots into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model, and combine the safety distance threshold to batch obtain the dual-arm robot collision estimation results.

[0050] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the dual-arm robot collision estimation method when executing the computer program.

[0051] A computer-readable storage medium stores a computer program, which implements the steps of the dual-arm robot collision estimation method when executed by a processor.

[0052] The above-mentioned dual-arm robot collision estimation method, system, computer equipment and storage medium construct two paired gap learning models by collecting paired gap data between different links of the left and right robotic arms in the dual-arm robot system and paired gap data between all links of the left and right robotic arms and environmental obstacles, which are used to efficiently process the dual-arm robot collision estimation task, thereby accelerating the dual-arm robot collision estimation process and providing effective support for subsequent efficient motion planning of the dual-arm robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Flow chart of a dual-arm robot collision estimation method according to an embodiment of the present invention;

[0054] Figure 2 Schematic diagram of a dual-arm robot simulation model corresponding to an actual system in one embodiment of the present invention;

[0055] Figure 3 Schematic diagrams of the positions of the dual-arm robot under different joint configurations; wherein (a) is an example diagram of the position of the dual-arm robot system at dual-arm position 1, (b) is an example diagram of the position of the dual-arm robot system at dual-arm position 2, (c) is an example diagram of the position of the dual-arm robot system at dual-arm position 3, and (d) is an example diagram of the position of the dual-arm robot system at dual-arm position 4;

[0056] Figure 4 It is a diagram of the paired gap learning neural network architecture. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0058] In one embodiment, Figure 1 As shown, a dual-arm robot collision estimation method comprises the following steps:

[0059] S100: Building a dual-arm robot simulation model in the simulation platform, collecting paired gap data between different links of the two robotic arms and paired gap data between all links of the two robotic arms and environmental obstacles, and constructing paired gap data sets between the left robotic arm and the right robotic arm and paired gap data sets between the dual-arm robot and environmental obstacles;

[0060] S200: building a paired gap learning neural network, and using the paired gap learning neural network to learn the mapping relationship between the joint space of the left and right mechanical arms of the dual-arm robot and the paired gap space of different connecting rods, and the mapping relationship between the joint space of the left and right mechanical arms of the dual-arm robot and the paired gap space of different connecting rods and environmental obstacles;

[0061] S300: inputting the paired gap data set of the left robotic arm and the right robotic arm and the paired gap data set of the dual-arm robot and the environmental obstacles into the paired gap learning neural network respectively, obtaining the estimated values ​​of the paired gap values ​​of different links and the estimated values ​​of the paired gap values ​​of different links and the obstacles respectively based on the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot to the paired gap spaces of different links and the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot to the paired gap spaces of different links and the environmental obstacles, training is performed in combination with a preset loss function, and a dual-arm robot self-collision estimation learning model and a dual-arm robot and environmental obstacle collision estimation learning model are constructed;

[0062] S400: simultaneously inputting the joint angle vectors of the left and right robotic arms of several groups of dual-arm robots into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model, and combining the safety distance threshold to batch obtain the dual-arm robot collision estimation results.

[0063] The above-mentioned dual-arm robot collision estimation method, for any given dual-arm robot system, builds a dual-arm robot simulation model in a simulation platform, collects paired gap data between different links of the two robotic arms and paired gap data between all links of the two robotic arms and environmental obstacles, and constructs two paired gap data sets: a paired gap data set between the left robotic arm and the right robotic arm, and a paired gap data set between the dual-arm robot and environmental obstacles; builds a new paired gap learning neural network architecture, uses the two constructed data sets for training respectively, and combines the safety distance threshold to build two collision estimation learning models: a dual-arm robot self-collision estimation learning model and a dual-arm robot and environmental obstacle collision estimation learning model; at the same time, inputs several groups of joint angle vectors of the left and right robotic arms of the dual-arm robot into the two trained collision estimation models, so that the dual-arm robot collision estimation results can be obtained in batches.

[0064] Furthermore, for a given actual dual-arm robot system, the robot shape description file URDF (United Robotics Description Format) of the dual-arm robot system can be imported into any robot simulation platform (such as Gazebo, CoppeliaSim, Webots, etc.), and the corresponding dual-arm robot system simulation model can be obtained and displayed on the simulation platform, such as Figure 2As shown, it should be pointed out that the simulation model has exactly the same shape, structure, connecting rod and other information as the actual dual-arm robot system. The subsequent data set collection process is all based on the dual-arm robot system simulation model.

[0065] In one embodiment, the dual-arm robot system consists of two robotic arms, a left robotic arm and a right robotic arm. Assuming that the degree of freedom of each robotic arm is N, each robotic arm contains N+1 connecting rods. The N+1 connecting rods of the left robotic arm are expressed as , the N+1 links of the right robot are represented as ; In addition, the left robot arm joint space is represented as ,in represents the first joint angle of the left arm, The left arm joint angles; the joint space of the right robot arm is represented by ,in represents the first joint angle of the right arm, The right arm The S100 collects the paired clearance data between different links of the two robotic arms, including:

[0066] S110: Sample a set of joint angles in the joint space of the left and right robotic arms respectively and ,in, =1,2,…, , is the total number of sampling times, and simultaneously controls the two robotic arms to move to the corresponding joint positions; Figure 3 It shows four sets of examples of dual-arm robot system positions under different joint angle configurations;

[0067] S120: Use the distance measurement tool component of the simulation platform to measure the gap between any link of the left robotic arm and any link of the right robotic arm (that is, the shortest distance between any two links), so as to obtain The pairwise clearance value of different connecting rods: Distance( , ), Distance( , ),…, Distance( , ), where Distance( , ) indicates connecting rod and connecting rod The gap value between

[0068] S130: Repeat the S120 process Second-rate, Different sets of dual-arm robot joint angles will be obtained accordingly There are different sets of connecting rod pair gap data, where the dimension of each set of pair gap data is ,in (N+1) (N+1), complete the construction of the paired gap dataset of the left and right robotic arms.

[0069] In one embodiment, it is assumed that the dual-arm robot is in a scene including Obstacles , and the positions of all obstacles are fixed and known. S100 collects paired clearance data between all links of the two robotic arms and environmental obstacles, including:

[0070] S140: Sample a set of joint angles in the joint space of the left and right robotic arms respectively and ,in, =1,2,… , is the total number of sampling times, and simultaneously controls the two robotic arms to move to the corresponding joint positions;

[0071] S150: Use the distance measurement tool component of the simulation platform to measure the total Connecting rod to The gap between obstacles (that is, the shortest distance between any link and the obstacle) is obtained, thus obtaining ( ) The pairwise gap value between each link and the obstacle: Distance( , ), Distance( , ),…,Distance( , ), Distance( , ), Distance( , ),…,Distance( , ), where Distance( , ) indicates connecting rod and obstacles The pairwise gap value between , ) points and obstacles The pairwise gap value between ;

[0072] S160: Repeat the S150 process Second-rate, Different sets of dual-arm robot joint angles will be obtained accordingly There are different sets of connecting rod pair gap data, where the dimension of each set of pair gap data is ,in ( ) , complete the construction of the paired gap dataset between the dual-arm robot and environmental obstacles.

[0073] In one embodiment, Figure 4 As shown in the figure, the paired gap learning neural network in S200 consists of six parts in structure. The first part is the input layer, which converts the left robot arm of the dual-arm robot system into The joint angles of the right robot arm The joint angles are spliced ​​to form a dual-arm robot joint angle as the input vector , whose dimensions are , through the position encoding function For the input vector Encode, where is a hyperparameter that maps the input joint angles to a high-frequency space in order to capture more detailed motion patterns;

[0074] The second part consists of a fully connected layer and an activation function ReLU layer. After the output of the first part is processed by the fully connected layer, the output features will be mapped to a higher-dimensional representation space, which helps to capture richer and more complex feature information.

[0075] The third part includes multiple residual blocks to enhance the feature extraction capability and improve the network's expressiveness and robustness. The residual block introduces a residual learning mechanism through jump connections to avoid the gradient vanishing problem and effectively improve the model performance. The residual block consists of two fully connected layers and an activation function ReLU. In the main branch, the third part receives the high-dimensional features output by the second part, first maps them to a new feature space through the first fully connected layer, then uses the ReLU activation function for nonlinear transformation, and then further processes the features through the second fully connected layer. The jump connection directly adds the input features to the output of the main branch. Through this residual learning mechanism, the gradient vanishing problem is effectively alleviated, while the feature expression capability is enhanced, and the model performance and training stability are improved.

[0076] The fourth part consists of a fully connected layer, an activation function ReLU, and a DropOut layer. After the output of the third part is processed by the fully connected layer, the output features will be mapped to a higher-dimensional representation space, which helps capture richer and more complex feature information.

[0077] The fifth part consists of a fully connected layer. After the output of the fourth part is processed by the fully connected layer of the fifth part, the output feature number is , the purpose is to achieve In this embodiment, when the self-collision estimation learning model of the dual-arm robot is used, the output feature number is = , when learning a model for collision estimation between a dual-arm robot and environmental obstacles, the output feature number = .

[0078] During the training process, 80% of the data in the dataset is randomly selected as the training set, 10% as the validation set, and 10% as the test set.

[0079] In one embodiment, S300 includes:

[0080] The paired gap dataset of the left and right robotic arms constructed in S100 is used to train the paired gap learning neural network. The specific process is as follows: During the training process, the paired gap dataset of the left and right robotic arms (a total of 80% of the data in the group is used as the training data set, 10% is used as the validation data set, and 10% is used as the test data set. The dimension of the dual-arm robot joint angle (formed by the left robot arm joint angle and the right robot arm joint angle, each set of dual-arm robot joint angle data dimension is ) as the network input feature, The gap values ​​of different connecting rod pairs are used as the network output features, that is, the output feature number Set to , , trained using the mean squared error between the pairwise gap estimates and actual values ​​between different links as the loss function, as follows:

[0081] ;

[0082] represents the optimal weight of the paired gap learning neural network under the current training set, represents the neural network weights, Indicates Actual value of the pairwise clearance between different connecting rods in a group, Represents a pairwise gap learning neural network with network weights The output of Estimated pairwise clearances between different links of the group;

[0083] After training, the optimal weights of the paired gap learning neural network can be obtained. , indicating that the joint space of the left and right arms of the dual-arm robot (dimension is ) to the gap space between different connecting rod pairs (dimension is ) mapping relationship; the optimal weight Loaded into the paired gap learning neural network, thus constructing a dual-arm robot self-collision estimation learning model;

[0084] The paired gap dataset between the dual-arm robot and the environmental obstacles constructed in S100 is used to train the paired gap learning neural network. The specific process is as follows: During the training process, a random pair of gap datasets between the dual-arm robot and the environmental obstacles (a total of 80% of the data in the group is used as the training data set, 10% is used as the validation data set, and 10% is used as the test data set. The dimension of the dual-arm robot joint angle (formed by the left robot arm joint angle and the right robot arm joint angle, each set of dual-arm robot joint angle data dimension is ) as the network input feature, The paired gap data between different connecting rods and obstacles are used as the network output features, that is, the output feature number Set to , , the mean squared error between the estimated and actual pairwise gaps between different links and obstacles is used as the loss function for training, as follows:

[0085] ;

[0086] represents the optimal weight of the paired gap learning neural network under the current training set, represents the neural network weights, Indicates The actual value of the pairwise clearance between the group link and the obstacle, Represents a pairwise gap learning neural network with network weights The output of pairwise gap estimates between group links and obstacles;

[0087] After training, the optimal weights of the paired gap learning neural network can be obtained. , indicating that the joint space of the left and right arms of the dual-arm robot (dimension is ) to the gap space between different links and environmental obstacles (dimension is ) mapping relationship, the optimal weight Loaded into the paired gap learning neural network, a collision estimation learning model for a dual-arm robot and environmental obstacles is constructed.

[0088] In one embodiment, S400 includes:

[0089] S410: Batch sampling in the joint space of the left and right arms of the dual-arm robot system Group points: , , { , …, , ,in Indicates the first point collected in the left robot arm joint space. Indicates the first point collected in the joint space of the right robot arm, and the others are similar;

[0090] S420: For any set of dual-arm robot joint angles { , , , input it into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model at the same time, and make a judgment based on the safety distance threshold. If both models return the result of no collision, it means that there is no collision between the left and right arms, and the two robotic arms have no collision with the environment. , It can be used as a candidate path point for collision-free links in the motion path of the dual-arm robot. Otherwise, it is considered that the dual-arm robot has collided. , abandon;

[0091] S430: One-time adjustment of the joint space of the dual-arm robot Group sampling points: , , { , …, , After two collision estimation learning models are built, batch acquisition Group sampling point collision estimation results.

[0092] In one embodiment, S420 specifically includes:

[0093] For any set of joint angles sampled in the joint space of the left and right manipulators and (The joint angles are not repeated in the previous dataset) and are input into the dual-arm robot self-collision estimation learning model to obtain The estimated value of the pairwise gap between different links of the left and right robotic arms: D_es( , ), D_es ( , ),…, D_es ( , ), where D_es ( , ) represents the link output by the self-collision estimation learning model of the dual-arm robot and connecting rod An estimate of the gap between

[0094] The global shortest distance between the left and right arms Expressed as Minimum of pairwise clearance estimates between different links: {D_es( , ), D_es ( , ),…,D_es ( , )};

[0095] like Smaller than the safety distance set by the user , it means that there is a collision between the left and right robotic arms, and the final output of the dual-arm robot self-collision estimation learning model is set to 1. Otherwise, it is considered that there is no collision, and the final output of the dual-arm robot self-collision estimation learning model is 0;

[0096] For any set of joint angles sampled in the joint space of the left and right manipulators and , and input it into the dual-arm robot and environmental obstacle collision estimation learning model to obtain The estimated gap values ​​between different links and obstacles are: D_es ( , ), D_es ( , ),…, D_es ( , ), D_es( , ), D_es( , ),…, D_es ( , ), where D_es ( , ) and D_es ( , ) represents the link output by the learning model for collision estimation between the dual-arm robot and environmental obstacles and obstacles Between and connecting rod and obstacles An estimate of the gap between

[0097] The global shortest distance between all links and all obstacles Expressed as The minimum of the estimated pairwise gap values ​​between links and obstacles: {D_es ( , ), D_es ( , ),…,D_es ( , ), D_es( , ), D_es( , ),…, D_es ( , )};

[0098] like Smaller than the safety distance set by the user , it means that the dual-arm robot collides with the environmental obstacle, and the final output of the dual-arm robot and environmental obstacle collision estimation learning model is set to 1. Otherwise, it is considered that there is no collision, and the dual-arm robot and environmental obstacle collision estimation learning model finally outputs 0.

[0099] A dual-arm robot collision estimation system, including a simulation platform building and data set acquisition module, a neural network building module, a neural network training module and a collision result estimation module;

[0100] The simulation platform building and data set collection module is used to build a dual-arm robot simulation model in the simulation platform, collect the paired gap data between different connecting rods of the two robotic arms and the paired gap data between all connecting rods of the two robotic arms and environmental obstacles, and construct the paired gap data set between the left robotic arm and the right robotic arm and the paired gap data set between the dual-arm robot and environmental obstacles;

[0101] A neural network building module is used to build a paired gap learning neural network, and use the paired gap learning neural network to learn the mapping relationship between the joint space of the left and right mechanical arms of a dual-arm robot and the paired gap space of different connecting rods, as well as the mapping relationship between the joint space of the left and right mechanical arms of a dual-arm robot and the paired gap space of different connecting rods and environmental obstacles;

[0102] A neural network training module is used to input the paired gap data sets of the left and right robotic arms and the paired gap data sets of the dual-arm robot and the environmental obstacles into the paired gap learning neural network respectively, and obtain the estimated values ​​of the paired gap values ​​of different links and the estimated values ​​of the paired gap values ​​of different links and the obstacles respectively based on the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot and the paired gap space of different links and the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot and the paired gap space of different links and the environmental obstacles, and train with a preset loss function to construct a dual-arm robot self-collision estimation learning model and a dual-arm robot and environmental obstacle collision estimation learning model;

[0103] The collision result estimation module is used to simultaneously input the joint angle vectors of the left and right robotic arms of several groups of dual-arm robots into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model, and combine the safety distance threshold to batch obtain the dual-arm robot collision estimation results.

[0104] For the specific definition of the dual-arm robot collision estimation system, please refer to the definition of the dual-arm robot collision estimation method in the above text, which will not be repeated here. Each module in the above dual-arm robot collision estimation system can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0105] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a dual-arm robot collision estimation method when executing the computer program.

[0106] A computer-readable storage medium stores a computer program, which implements the steps of a dual-arm robot collision estimation method when executed by a processor.

[0107] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0108] Beneficial effects of the present invention:

[0109] (1) Compared with the existing method of directly learning the shortest gap value between the left and right mechanical arms of the dual-arm robot system, due to the complex structure of the dual-arm robot, its collision distance function is complex and highly non-convex, resulting in the existing method having problems of poor learning effect and accuracy. The present invention learns the paired gaps of any two connecting rods between the two arms, and then obtains the shortest gap between all connecting rods, realizes the global collision estimation of the dual-arm robot itself, and decomposes the left and right arm gap learning problem in the complex dual-arm robot system into several different connecting rod paired gap learning problems, which not only reduces the learning complexity, but also improves the accuracy of collision estimation in the results, which is conducive to further improving the planning performance of the dual-arm robot in the future. Similarly, the above beneficial effects are also reflected in the collision learning of the dual-arm robot and environmental obstacles, which will not be repeated.

[0110] (2) Compared with the existing method that uses one model to learn the gap mapping relationship between the robot arm and all obstacles (the other robot arm is also regarded as an obstacle), the present invention separates and decomposes the gap mapping relationship between the robot arms and the gap mapping relationship between the robot arms and environmental obstacles in the above process, and uses two independent models for learning and training respectively, which can further reduce the learning difficulty of the dual-arm robot system and improve the accuracy of collision estimation.

[0111] The dual-arm robot collision estimation method, system, computer device and storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A dual-arm robot collision estimation method, characterized in that: The method comprises the following steps: S100: Building a dual-arm robot simulation model in the simulation platform, collecting paired gap data between different links of the two robotic arms and paired gap data between all links of the two robotic arms and environmental obstacles, and constructing paired gap data sets between the left robotic arm and the right robotic arm and paired gap data sets between the dual-arm robot and environmental obstacles; S200: building a paired gap learning neural network, and using the paired gap learning neural network to learn the mapping relationship between the joint space of the left and right mechanical arms of the dual-arm robot and the paired gap space of different connecting rods, and the mapping relationship between the joint space of the left and right mechanical arms of the dual-arm robot and the paired gap space of different connecting rods and environmental obstacles; S300: inputting the paired gap data set of the left robotic arm and the right robotic arm and the paired gap data set of the dual-arm robot and the environmental obstacles into the paired gap learning neural network respectively, obtaining the estimated values ​​of the paired gap values ​​of different links and the estimated values ​​of the paired gap values ​​of different links and the obstacles respectively based on the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot to the paired gap spaces of different links and the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot to the paired gap spaces of different links and the environmental obstacles, training is performed in combination with a preset loss function, and a dual-arm robot self-collision estimation learning model and a dual-arm robot and environmental obstacle collision estimation learning model are constructed; S400: simultaneously inputting the joint angle vectors of the left and right robotic arms of several groups of dual-arm robots into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model, and combining the safety distance threshold to batch obtain the dual-arm robot collision estimation results.

2. The method according to claim 1, characterized in that The dual-arm robot system consists of two robotic arms, the left robotic arm and the right robotic arm. Assuming that the degree of freedom of each robotic arm is N, each robotic arm contains N+1 connecting rods. The N+1 connecting rods of the left robotic arm are expressed as , the N+1 links of the right robot arm are represented as ; In addition, the left robot arm joint space is represented as ,in represents the first joint angle of the left arm, The left arm joint angles; The joint space of the right robot arm is represented as ,in represents the first joint angle of the right arm, The right arm The S100 collects the paired clearance data between different links of the two robotic arms, including: S110: Sample a set of joint angles in the joint space of the left and right robotic arms respectively and ,in, =1,2,…, , is the total number of sampling times, and simultaneously controls the two robotic arms to move to the corresponding joint positions; S120: Use the distance measurement tool component of the simulation platform to measure the gap between any link of the left robotic arm and any link of the right robotic arm to obtain The pairwise clearance value of different connecting rods: Distance( , ), Distance( , ),…, Distance( , ), where Distance( , ) indicates connecting rod and connecting rod The gap value between S130: Repeat the S120 process Second-rate, Different sets of dual-arm robot joint angles will be obtained accordingly There are different sets of connecting rod pair gap data, where the dimension of each set of pair gap data is ,in (N+1) (N+1), complete the construction of the paired gap dataset of the left and right robotic arms.

3. The method according to claim 2, characterized in that Assume that the dual-arm robot is in a scenario including Obstacles , and the positions of all obstacles are fixed and known. S100 collects paired clearance data between all links of the two robotic arms and environmental obstacles, including: S140: Sample a set of joint angles in the joint space of the left and right robotic arms respectively and ,in, =1,2,… , is the total number of sampling times, and simultaneously controls the two robotic arms to move to the corresponding joint positions; S150: Use the distance measurement tool component of the simulation platform to measure the total Connecting rod to The gap between obstacles is obtained ( ) The pairwise gap value between each link and the obstacle: Distance( , ), Distance( , ),…, Distance( , ), Distance( , ), Distance( , ),…, Distance( , ), where Distance( , ) indicates connecting rod and obstacles The pairwise gap value between , ) indicates connecting rod and obstacles The pairwise gap value between ; S160: Repeat the S150 process Second-rate, Different sets of dual-arm robot joint angles will be obtained accordingly There are pairs of gap data between different links and obstacles, where the dimension of each pair of gap data is ,in ( ) , complete the construction of the paired gap dataset between the dual-arm robot and environmental obstacles.

4. The method according to claim 3, characterized in that The paired gap learning neural network in S200 consists of six parts in structure. The first part is the input layer, which converts the left arm of the dual-arm robot system into The joint angles of the right robot arm The joint angles are spliced ​​to form a dual-arm robot joint angle as the input vector , whose dimensions are , through the position encoding function For the input vector Encode, where is a hyperparameter that maps the input joint angles to a high-frequency space in order to capture more detailed motion patterns; The second part consists of a fully connected layer and an activation function ReLU layer. After the output of the first part is processed by the fully connected layer, the output features will be mapped to a higher-dimensional representation space. The third part includes multiple residual blocks. The residual block part introduces the residual learning mechanism through skip connections. The residual block consists of two layers of fully connected layers and activation function ReLU. In the main branch, the third part receives the high-dimensional features output by the second part, first maps them to the new feature space through the first fully connected layer, then uses the ReLU activation function for nonlinear transformation, and then further processes the features through the second fully connected layer. The skip connection directly adds the input features to the output of the main branch. The fourth part consists of a fully connected layer, an activation function ReLU, and a DropOut layer. After the output of the third part is processed by the fully connected layer, the output features will be mapped to a higher-dimensional representation space; The fifth part consists of a fully connected layer. After the output of the fourth part is processed by the fully connected layer of the fifth part, the output feature number is , the purpose is to achieve dimensional gap distance mapping.

5. The method according to claim 4, characterized in that S300 includes: choose The joint angles of the dual-arm robot are used as network input features. The gap values ​​of different connecting rod pairs are used as the network output features, that is, the output feature number Set to , , trained using the mean squared error between the pairwise gap estimates and actual values ​​between different links as the loss function, as follows: ； represents the optimal weight of the paired gap learning neural network under the current training set, represents the neural network weights, Indicates Actual value of the pairwise clearance between different connecting rods in a group, Represents a pairwise gap learning neural network with network weights The output of Estimated pairwise clearances between different links of the group; After training, the optimal weights of the paired gap learning neural network can be obtained. , indicating that the mapping relationship between the joint space of the left and right arms of the dual-arm robot and the gap space of different connecting rod pairs has been learned, and the optimal weight Loaded into the paired gap learning neural network, thus constructing a dual-arm robot self-collision estimation learning model; choose The joint angles of the dual-arm robot are used as network input features. The paired gap data between different connecting rods and obstacles are used as the network output features, that is, the output feature number Set to , , the mean squared error between the estimated and actual pairwise gaps between different links and obstacles is used as the loss function for training, as follows: ； represents the optimal weight of the paired gap learning neural network under the current training set, represents the neural network weights, Indicates The actual value of the pairwise clearance between the group link and the obstacle, Represents a pairwise gap learning neural network with network weights The output of pairwise gap estimates between group links and obstacles; After training, the optimal weights of the paired gap learning neural network can be obtained. , indicating that the mapping relationship between the joint space of the left and right manipulators of the dual-arm robot and the paired gap space of different links and environmental obstacles has been learned, and the optimal weight Loaded into the paired gap learning neural network, a collision estimation learning model for a dual-arm robot and environmental obstacles is constructed.

6. The method according to claim 5, characterized in that S400 includes: S410: Batch sampling in the joint space of the left and right arms of the dual-arm robot system Group points: , , { , …, , ,in Indicates the first point collected in the left robot arm joint space. Indicates the first point collected in the joint space of the right robot arm, and the others are similar; S420: For any set of dual-arm robot joint angles { , , , input it into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model at the same time, and make a judgment based on the safety distance threshold. If both models return the result of no collision, it means that there is no collision between the left and right arms, and the two robotic arms have no collision with the environment. , It can be used as a candidate path point for collision-free links in the motion path of the dual-arm robot. Otherwise, it is considered that the dual-arm robot has collided. , abandon; S430: One-time adjustment of the joint space of the dual-arm robot Group sampling points: , , { , …, , After two collision estimation learning models are built, batch acquisition Group sampling point collision estimation results.

7. The method according to claim 6, characterized in that S420 is specifically: For any set of joint angles sampled in the joint space of the left and right manipulators and , and input it into the dual-arm robot self-collision estimation learning model to obtain The estimated value of the pairwise gap between different links of the left and right robotic arms: D_es( , ), D_es ( , ),…, D_es ( , ), where D_es ( , ) represents the link output by the self-collision estimation learning model of the dual-arm robot and connecting rod An estimate of the gap between The global shortest distance between the left and right arms Expressed as Minimum of pairwise clearance estimates between different links: {D_es( , ), D_es ( , ),…,D_es ( , )} like Smaller than the safety distance set by the user , it means that there is a collision between the left and right robotic arms, and the final output of the dual-arm robot self-collision estimation learning model is set to 1. Otherwise, it is considered that there is no collision, and the final output of the dual-arm robot self-collision estimation learning model is 0; For any set of joint angles sampled in the joint space of the left and right manipulators and , and input it into the dual-arm robot and environmental obstacle collision estimation learning model to obtain The estimated gap values ​​between different links and obstacles are: D_es ( , ), D_es ( , ),…, D_es ( , ), D_es( , ), D_es( , ),…, D_es ( , ), where D_es ( , ) and D_es ( , ) represents the link output by the learning model for collision estimation between the dual-arm robot and environmental obstacles and obstacles Estimated clearance between the connecting rod and obstacles An estimate of the gap between The global shortest distance between all links and all obstacles Expressed as The minimum of the estimated pairwise gap values ​​between links and obstacles: {D_es ( , ), D_es ( , ),…,D_es ( , ), D_es( , ), D_es( , ),…,D_es( , )}; like Smaller than the safety distance set by the user , it means that the dual-arm robot collides with the environmental obstacle, and the final output of the dual-arm robot and environmental obstacle collision estimation learning model is set to 1. Otherwise, it is considered that there is no collision, and the dual-arm robot and environmental obstacle collision estimation learning model finally outputs 0.

8. A dual-arm robot collision estimation system, characterized in that: It includes simulation platform building and data set acquisition module, neural network building module, neural network training module and collision result estimation module; The simulation platform building and data set collection module is used to build a dual-arm robot simulation model in the simulation platform, collect the paired gap data between different connecting rods of the two robotic arms and the paired gap data between all connecting rods of the two robotic arms and environmental obstacles, and construct the paired gap data set between the left robotic arm and the right robotic arm and the paired gap data set between the dual-arm robot and environmental obstacles; A neural network building module is used to build a paired gap learning neural network, and use the paired gap learning neural network to learn the mapping relationship between the joint space of the left and right mechanical arms of a dual-arm robot and the paired gap space of different connecting rods, as well as the mapping relationship between the joint space of the left and right mechanical arms of a dual-arm robot and the paired gap space of different connecting rods and environmental obstacles; A neural network training module is used to input the paired gap data sets of the left and right robotic arms and the paired gap data sets of the dual-arm robot and the environmental obstacles into the paired gap learning neural network respectively, and obtain the estimated values ​​of the paired gap values ​​of different links and the estimated values ​​of the paired gap values ​​of different links and the obstacles respectively based on the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot and the paired gap space of different links and the mapping relationship between the joint space of the left and right robotic arms of the dual-arm robot and the paired gap space of different links and the environmental obstacles, and train with a preset loss function to construct a dual-arm robot self-collision estimation learning model and a dual-arm robot and environmental obstacle collision estimation learning model; The collision result estimation module is used to simultaneously input the joint angle vectors of the left and right robotic arms of several groups of dual-arm robots into the dual-arm robot self-collision estimation learning model and the dual-arm robot and environmental obstacle collision estimation learning model, and combine the safety distance threshold to batch obtain the dual-arm robot collision estimation results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images