Robot grabbing decision learning and determining method, equipment and medium

Abstract

The embodiment of the invention provides a robot grabbing decision learning and determining method and device and a medium, and relates to the technical field of robot control. The method comprises the following steps: extracting interlayer features of a sample three-dimensional point cloud by using a plurality of different intelligent agents to obtain interlayer features corresponding to the intelligent agents, mapping the interlayer features corresponding to the intelligent agents into grabbing decisions by using the intelligent agents to serve as target grabbing decisions corresponding to the intelligent agents, and in addition, aiming at each intelligent agent, obtaining target grabbing decisions corresponding to the intelligent agents; and the intelligent agent is trained by using the decision loss, the feature loss and the network training loss corresponding to the intelligent agent, so that the accuracy of predicting the grabbing pose of the dexterous hand by the intelligent agent is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to methods, equipment and media for learning and determining robot grasping decisions. Background Technology

[0002] In the field of robotic grasping, relevant information from the scene, such as point clouds, is typically collected and mapped to grasping parameters using an end-to-end mapping method to obtain the robot's grasping pose. However, this method easily degenerates into mean or median solutions in complex multimodal grasping distributions, making it difficult to handle the diverse poses required for dexterous hand grasping. Furthermore, while existing generative models are applicable to complex multimodal scenarios, they are only suitable for predicting grasping poses in gripper structures. For scenarios involving a large number of parameters, such as dexterous hands, accurately predicting the grasping pose using generative models is extremely difficult. Therefore, a solution capable of accurately predicting the grasping pose of dexterous hands is urgently needed. Summary of the Invention

[0003] The purpose of this invention is to provide a method, device, and medium for learning and determining robot grasping decisions, so as to improve the accuracy of dexterous hand grasping pose prediction. The specific technical solution is as follows:

[0004] This application provides a learning method for robot grasping decisions, the method comprising:

[0005] For each of the multiple different agents, perform the following steps:

[0006] The intermediate layer features of the sample 3D point cloud are extracted using the intelligent agent to obtain the intermediate layer features corresponding to the intelligent agent, wherein the sample 3D point cloud is a 3D point cloud of the sample scene including the object to be grasped.

[0007] The intelligent agent is used to map the corresponding intermediate layer features into grasping decisions, which serve as the target grasping decisions for the intelligent agent. The target grasping decisions include the grasping pose of the dexterous hand grasping the object to be grasped in the sample scene.

[0008] The agent is trained using the decision loss, feature loss, and network training loss corresponding to the agent. The decision loss of the agent represents the difference between the target grasping decision of the agent and the target grasping decision of other agents. The feature loss of the agent represents the difference between the intermediate layer features of the agent and the intermediate layer features of other agents. The network training loss of the agent represents the difference between the target grasping decision of the agent and the ground truth. The ground truth includes the actual pose of the dexterous hand when grasping the object to be grasped in the sample scene.

[0009] In one possible implementation, the decision loss corresponding to the agent is obtained by means of:

[0010] Calculate the similarity between the target grasping decision of the aforementioned agent and the target grasping decision of other agents, and use this similarity as the similarity of each of the other agents.

[0011] The target grasping decisions corresponding to each of the other intelligent agents are weighted and summed to obtain the first grasping decision corresponding to the intelligent agent, wherein the weight of the target grasping decision corresponding to the same intelligent agent among the other intelligent agents is positively correlated with its corresponding similarity;

[0012] The difference between the target grasping decision of the agent and the first grasping decision is taken as the decision loss of the agent.

[0013] In one possible implementation, the feature loss corresponding to the agent is obtained by means of:

[0014] Calculate the similarity between the target grasping decision of the agent and the target grasping decision of other agents, and use this similarity as the similarity of each agent.

[0015] The intermediate layer features corresponding to each of the other intelligent agents are weighted and summed to obtain the first intermediate layer feature corresponding to the intelligent agent, wherein the weight of the intermediate layer feature corresponding to the same intelligent agent among the other intelligent agents is positively correlated with its corresponding similarity;

[0016] The difference between the intermediate layer features of the agent and the first intermediate layer features is used as the feature loss corresponding to the agent.

[0017] In one possible implementation, the step of extracting intermediate layer features of the sample 3D point cloud using the agent to obtain the intermediate layer features corresponding to the agent includes:

[0018] The agent is used to predict the graspable score and object score of each spatial point in the sample 3D point cloud. The graspable score represents the probability that the spatial point can be grasped, and the object score represents the probability that the spatial point belongs to the object to be grasped.

[0019] Based on the object scores predicted by the agent, spatial points whose object scores meet preset filtering conditions are selected, and the selected spatial points and their graspable scores are used as intermediate layer features corresponding to the agent.

[0020] In one possible implementation, the step of using the agent to map the corresponding intermediate layer features into a grasping decision, which serves as the grasping decision for the agent, includes:

[0021] Based on the spatial points included in the intermediate layer features corresponding to the intelligent agent, multiple candidate grasping decisions are generated.

[0022] The grabbing decision with the highest target score is determined from the multiple candidate grabbing decisions generated by the agent, and is taken as the target grabbing decision corresponding to the agent, wherein the target score is positively correlated with the grabbable score of each point grabbed by the candidate strategy.

[0023] In one possible implementation, the decision loss is calculated in the following manner:

[0024]

[0025] Among them, L kl Let P represent the decision loss calculated using KL divergence, and let P represent the total number of joints in the dexterous hand. p This represents the rotation angle of the p-th joint of the dexterous hand in the grasping decision of the agent. This represents the target grasping decision of a certain intelligent agent. This represents the target-grabbing decision of other intelligent agents.

[0026] In one possible implementation, the feature loss is calculated in the following manner:

[0027]

[0028] Among them, L mse Represents feature loss, This represents the intermediate layer features corresponding to the agent. This represents the intermediate layer features corresponding to other agents.

[0029] This application also provides a method for determining robot grasping decisions, the method comprising:

[0030] The target intelligent agent and the target 3D point cloud are obtained. The target intelligent agent is trained according to the learning method of robot grasping decision as described above. The target 3D point cloud is a 3D point cloud obtained by the robot from the target scene including the object to be grasped.

[0031] The target 3D point cloud is input into the target intelligent agent to determine the grasping decision of the target scene. The grasping decision is used to instruct the robot's dexterous hand to assume the pose corresponding to the robot's hand when the robot grasps the object to be grasped from the target scene.

[0032] This application also provides a learning device for robot grasping decisions, the device comprising:

[0033] The feature extraction module is used to extract intermediate layer features of the sample 3D point cloud using multiple different agents to obtain the intermediate layer features corresponding to each agent. The sample 3D point cloud is a 3D point cloud of the sample scene including the object to be grasped.

[0034] The decision generation module is used to map the corresponding intermediate layer features of each of the intelligent agents into grasping decisions, which are used as the target grasping decisions of each of the intelligent agents. The target grasping decisions include the grasping pose of the dexterous hand grasping the object to be grasped in the sample scene.

[0035] The training module is used to train each agent using its corresponding decision loss, feature loss, and network training loss. The decision loss represents the difference between the target grasping decision of the agent and the target grasping decisions of other agents. The feature loss represents the difference between the intermediate layer features of the agent and the intermediate layer features of other agents. The network training loss represents the difference between the target grasping decision of the agent and the ground truth. The ground truth includes the actual pose of the dexterous hand when grasping the object in the sample scenario.

[0036] In one possible implementation, the decision loss corresponding to the agent is obtained by means of:

[0037] Calculate the similarity between the target grasping decision of the aforementioned agent and the target grasping decision of other agents, and use this similarity as the similarity of each of the other agents.

[0038] The target grasping decisions corresponding to each of the other intelligent agents are weighted and summed to obtain the first grasping decision corresponding to the intelligent agent, wherein the weight of the target grasping decision corresponding to the same intelligent agent among the other intelligent agents is positively correlated with its corresponding similarity;

[0039] The difference between the target grasping decision of the agent and the first grasping decision is taken as the decision loss of the agent.

[0040] In one possible implementation, the feature loss corresponding to the agent is obtained by means of:

[0041] Calculate the similarity between the target grasping decision of the agent and the target grasping decision of other agents, and use this similarity as the similarity of each agent.

[0042] The intermediate layer features corresponding to each of the other intelligent agents are weighted and summed to obtain the first intermediate layer feature corresponding to the intelligent agent, wherein the weight of the intermediate layer feature corresponding to the same intelligent agent among the other intelligent agents is positively correlated with its corresponding similarity;

[0043] The difference between the intermediate layer features of the agent and the first intermediate layer features is used as the feature loss corresponding to the agent.

[0044] In one possible implementation, the feature extraction module includes:

[0045] The score prediction submodule is used to predict the graspable score and object score of each spatial point in the sample 3D point cloud for each agent. The graspable score is used to represent the probability that the spatial point can be grasped, and the object score is used to represent the probability that the spatial point belongs to the object to be grasped.

[0046] The spatial point filtering submodule is used to select spatial points whose object scores meet preset filtering conditions for each agent according to the object scores predicted by the agent, and use the selected spatial points and their grabbable scores as the intermediate layer features corresponding to the agent.

[0047] In one possible implementation, the decision generation module is specifically used for:

[0048] For each agent, multiple candidate grasping decisions are generated based on the spatial points included in the intermediate layer features corresponding to the agent.

[0049] For each agent, the grasping decision with the highest target score is determined from the multiple candidate grasping decisions generated by the agent, and is taken as the target grasping decision corresponding to the agent, wherein the target score is positively correlated with the graspable score of each point grasped by the candidate strategy.

[0050] In one possible implementation, the decision loss is calculated in the following manner:

[0051]

[0052] Among them, L kl Let P represent the decision loss calculated using KL divergence, and let P represent the total number of joints in the dexterous hand. p This represents the rotation angle of the p-th joint of the dexterous hand in the grasping decision of the agent. This represents the target grasping decision of a certain intelligent agent. This represents the target-grabbing decision of other intelligent agents.

[0053] In one possible implementation, the feature loss is calculated in the following manner:

[0054]

[0055] Among them, L mse Represents feature loss, This represents the intermediate layer features corresponding to the agent. This represents the intermediate layer features corresponding to other agents.

[0056] This application also provides a device for determining robot grasping decisions, the device comprising:

[0057] The data acquisition module is used to acquire the target intelligent agent and the target 3D point cloud. The target intelligent agent is trained according to the learning method of robot grasping decision as described above. The target 3D point cloud is a 3D point cloud obtained by the robot from the target scene including the object to be grasped.

[0058] The decision determination module is used to input the target 3D point cloud into the target intelligent agent and determine the grasping decision of the target scene. The grasping decision is used to instruct the robot's dexterous hand to assume the pose corresponding to the robot's hand when the robot grasps the object to be grasped from the target scene.

[0059] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0060] Memory, used to store computer programs;

[0061] The processor, when executing the program stored in the memory, implements any of the above-described learning methods and decision-making methods for robot grasping decisions.

[0062] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the learning method for robot grasping decision and the determination method for robot grasping decision described above.

[0063] This invention also provides a computer program product containing instructions that, when run on a computer, causes the computer to execute any of the above-described robot grasping decision learning methods and robot grasping decision determination methods.

[0064] Beneficial effects of the embodiments of the present invention:

[0065] This invention employs multiple different agents to extract intermediate-layer features from a 3D point cloud of a sample scene containing the object to be grasped. Since different agents focus on different features when extracting intermediate-layer features, the grasping decisions obtained from the intermediate-layer feature mappings of different agents will also differ. Furthermore, since each agent's decision loss represents the difference between its grasping decision and those of other agents, its feature loss represents the difference between its intermediate-layer features and those of other agents, and the network training loss represents the difference between the agent's target grasping decision and the ground truth, training each agent using their decision and feature losses can constrain the diversity of grasping decisions and intermediate-layer features, ensuring that the grasping decisions of each agent do not differ too much. Moreover, training each agent using the network training loss ensures that the final converged solution of each agent is close to the ground truth, guaranteeing the accuracy of the predicted grasping decisions. This effectively combines the advantages of consistency constraints and ground truth supervision, enabling the trained agents to accurately predict the grasping pose of a dexterous hand.

[0066] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0068] Figure 1 This is a first schematic diagram of a learning method for robot grasping decision-making provided in an embodiment of this application;

[0069] Figure 2 This is a second schematic diagram of the learning method for robot grasping decision-making provided in an embodiment of this application;

[0070] Figure 3 A third schematic diagram illustrating the learning method for robot grasping decision-making provided in this application embodiment;

[0071] Figure 4 A schematic diagram illustrating the determination of decision loss provided in an embodiment of this application;

[0072] Figure 5 A schematic diagram illustrating the determination of feature loss as provided in an embodiment of this application;

[0073] Figure 6 A schematic diagram illustrating the framework of the robot grasping decision-making learning method provided in the embodiments of this application;

[0074] Figure 7 A schematic diagram illustrating the method for determining robot grasping decisions provided in an embodiment of this application;

[0075] Figure 8 A schematic diagram of the learning device for robot grasping decision-making provided in an embodiment of this application;

[0076] Figure 9 A schematic diagram of the structure of the robot grasping decision determination device provided in the embodiments of this application;

[0077] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0078] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0079] Knowledge distillation is a model compression technique typically used to transfer knowledge from a large model (teacher model) to a smaller model (student model), thereby reducing computational resources while maintaining performance. It has been widely applied in the field of robotic grasping, where it can improve the target agent's operational capabilities and adaptability to different tasks and environments by transferring the knowledge of one or more meta-agents in a refined form to the target agent. However, currently, knowledge distillation technology in robotic grasping is only applicable to gripper scenarios. When grasping tasks expand from grippers to dexterous hands, the degrees of freedom and parameter dimensions of dexterous hands increase significantly, making existing methods difficult to apply directly.

[0080] For example, existing technologies that construct large-scale gripper grasping datasets using sampling and sorting methods are primarily designed for grippers with low degrees of freedom and relatively simple structures. These methods typically involve sampling large-scale grasping postures within a limited parameter space and then selecting feasible grasping methods based on certain physical constraints or heuristic rules to construct a training dataset. However, this approach is difficult to directly apply to high-degree-of-freedom dexterous hands. Dexterous hands possess far greater degrees of freedom and joint flexibility than grippers, with extremely high dimensionality in their grasping space. Grasping decisions are no longer simple gripping actions but involve multi-finger coordination, complex contact point distributions, and dynamic adjustments to diverse hand shapes. Traditional sampling and sorting methods struggle to efficiently cover all potential grasping possibilities in such a high-dimensional action space, failing to guarantee a high success rate. Therefore, a solution capable of accurately predicting the grasping posture of dexterous hands needs to be developed.

[0081] To improve the accuracy of pose prediction for dexterous hand grasping, embodiments of this application provide a learning method for robot grasping decisions, see [link to relevant documentation]. Figure 1 , Figure 1 This is a first schematic diagram of a learning method for robot grasping decision-making provided in an embodiment of this application. The method includes: for each of a plurality of different agents, performing the following steps S10 to S30:

[0082] Step S10: Use the agent to extract the intermediate layer features of the sample 3D point cloud respectively, and obtain the intermediate layer features corresponding to the agent.

[0083] Among them, the sample 3D point cloud is a 3D point cloud of the sample scene including the object to be grasped;

[0084] Step S20: Use the agent to map the corresponding intermediate layer features into grasping decisions, which are then used as the target grasping decisions for the agent.

[0085] Among them, the target grasping decision includes the grasping pose of the dexterous hand in grasping the object to be grasped in the sample scene;

[0086] Step S30: Train the agent using the decision loss, feature loss and network training loss corresponding to the agent;

[0087] Among them, the decision loss corresponding to the agent is used to represent the difference between the target grasping decision of the agent and the target grasping decision of other agents, the feature loss corresponding to the agent is used to represent the difference between the intermediate layer features corresponding to the agent and the intermediate layer features corresponding to other agents, and the network training loss corresponding to the agent is used to represent the difference between the target grasping decision of the agent and the ground truth, where the ground truth is the actual pose of the dexterous hand when grasping the object to be grasped in the sample scene.

[0088] In this embodiment of the application, multiple different agents are employed to extract intermediate layer features from the 3D point cloud of a sample scene containing the object to be grasped. Since different agents have varying understanding capabilities, and intermediate layer features can be considered as information understood by the agents from the sample scene, the intermediate layer features extracted by different agents will differ. Because different agents often focus on different features when extracting intermediate layer features, the grasping decisions obtained from the intermediate layer feature mappings of different agents will also differ. Furthermore, since the decision loss of each agent represents the difference between its grasping decision and those of other agents, the feature loss of each agent represents the difference between its intermediate layer features and those of other agents, and the network training loss represents the difference between the agent's target grasping decision and the ground truth, training each agent using its decision loss and feature loss can constrain the diversity of grasping decisions and intermediate layer features, allowing each agent to learn from each other's understanding capabilities and focus areas. Furthermore, training each agent using the network training loss ensures that the final converged solution of each agent is close to the true value, guaranteeing the accuracy of the predicted grasping decisions. This effectively combines the advantages of consistency constraints and truth-based supervision, enabling the trained agent to accurately predict the grasping pose of the dexterous hand.

[0089] The following is a detailed explanation of steps S10 to S30:

[0090] In step S10 above, the sample scene refers to a three-dimensional space containing the object to be grasped and its surrounding environment. The sample scene should closely resemble the robot's actual working environment. For example, if the robot is a domestic service robot, the sample scene could be "a coffee cup on a table"; if the robot is used to retrieve goods for customers in a supermarket, the sample scene could be "one or more goods on a shelf". The sample 3D point cloud can be the original 3D point cloud data obtained by scanning the sample scene with a depth camera or LiDAR, or it can be the 3D point cloud data obtained after preprocessing the original 3D point cloud data. Preprocessing can be any processing that can improve the quality of the 3D point cloud or reduce the computational load of subsequent feature extraction, including but not limited to one or more of the following methods: downsampling, noise reduction, outlier removal, etc.

[0091] Different agents are pre-built, and each agent contains a feature extraction network. Extracting intermediate layer features of sample 3D point clouds using agents means inputting the sample 3D point cloud into the agent's feature extraction network, and having each agent's feature extraction network output its corresponding intermediate layer features.

[0092] Different agents use different feature extraction networks, so the features extracted by these networks are not entirely the same. For example, suppose there are three agents: agent 1 focuses on extracting the shape features of the object to be grasped from the sample 3D point cloud, agent 2 focuses on extracting the position features of the object to be grasped from the sample 3D point cloud, and agent 3 focuses on extracting the surface features of the object to be grasped.

[0093] In step S20 above, each agent also includes a grasping decision prediction network. The intermediate layer features extracted by each agent are input into the corresponding grasping decision prediction network, thus mapping different grasping decisions. For example, assuming agent 1 extracts intermediate layer feature 1, agent 2 extracts intermediate layer feature 2, and agent 3 extracts intermediate layer feature 3, then intermediate layer feature 1 is input into agent 1's grasping decision prediction network, outputting grasping decision 1; intermediate layer feature 2 is input into agent 2's grasping decision prediction network, outputting grasping decision 2; and intermediate layer feature 3 is input into agent 3's grasping decision prediction network, outputting grasping decision 3.

[0094] The grasping pose of the dexterous hand in the sample scenario refers to the grasping pose when the object is successfully grasped. It is understood that the grasping pose of the dexterous hand is usually a multi-dimensional pose, with the specific dimensions depending on the sensor configuration and functional requirements of the dexterous hand. In one possible implementation, the grasping pose of the dexterous hand is the three-dimensional coordinates (x, y, z), three-dimensional direction (r, p, y), and / or opening width of each joint of the dexterous hand when it successfully grasps the object.

[0095] It is understandable that if the dexterous hand collides with the object to be grasped or the grasping path planned for the dexterous hand may lead the dexterous hand close to a singularity, the dexterous hand will not be able to successfully reach the object to be grasped and perform the grasping task. Therefore, in one possible implementation, the grasping decision may also include the grasping path in the sample scenario in which the dexterous hand successfully grasps the object to be grasped, and the probability of the dexterous hand colliding or reaching a singularity in the grasping path is less than a preset probability threshold.

[0096] In step S30 above, the decision loss for each agent is used to measure the difference between the agent's predicted grasping decision and the grasping decisions predicted by other agents. When calculating the decision loss for each agent, the difference between the grasping decisions of that agent and each of the other agents can be calculated separately, and then the average of these differences can be used as the decision loss for that agent. For example, assuming the difference between the grasping decisions of agent 1 and agent 2 is difference 1, and the difference between the grasping decisions of agent 1 and agent 3 is difference 2, and the average of difference 1 and difference 2 is difference 3, then difference 3 can be used as the decision loss for agent 1. Alternatively, the average of the grasping decisions of the other agents can be calculated first, and then the difference between the grasping decision of that agent and the average can be calculated as the decision loss for that agent. For example, the average of the grasping decisions of agents 2 and 3 (value 1) can be calculated first, and then the difference between the grasping decision of agent 1 and the average (value 1) can be calculated as the decision loss for agent 1. Specifically, the decision loss of each agent can be represented by the mean square error of its grasping decision compared to that of other agents, or by the KL (Kullback-Leibler) divergence.

[0097] Understandably, since the dimensions of the intermediate layer features output by different agents may differ, it's advisable to map these different dimensions of intermediate layer features to the same feature space before calculating the feature loss. Similar to calculating the decision loss, the feature loss of each agent can also be calculated in the same way. The difference lies in that the decision loss is calculated based on the grasping decisions of each agent, while the feature loss is calculated based on the intermediate layer features of each agent. The specific calculation process for representing the decision loss (or feature loss) of each agent using the mean squared error or KL divergence between the grasping decisions (or intermediate layer features) of each agent and the grasping decisions (or intermediate layer features) of other agents is detailed below and will not be repeated here.

[0098] The network training loss is the difference between the grasping decision predicted by each agent and the actual grasping pose. For example, assuming the actual grasping pose is pose A, agent 1 predicts grasping decision 1, and agent 2 predicts grasping decision 2, then the difference between pose A and grasping decision 1 is calculated as the network training loss for agent 1, and the difference between pose A and grasping decision 2 is calculated as the network training loss for agent 2. Specifically, for each agent, the mean squared error between the agent's predicted grasping decision and the actual grasping pose can be used as the agent's network training loss.

[0099] After training each agent, in practical applications, the grasping decisions output by each trained agent can be averaged and used as the final grasping decision. Alternatively, a single agent can be selected, and its grasping decision can be used as the final grasping decision. Furthermore, the agent with the smallest decision loss can be selected, or an agent can be randomly selected.

[0100] To further improve the accuracy of predictions, in one possible implementation, after step S30, it can be determined whether the decision loss, feature loss, and network training loss of each agent meet a preset convergence condition. If not, the process returns to steps S10 to S30; if they do meet, training is complete. The preset convergence condition indicates that the decision loss, feature loss, and network training loss no longer change significantly.

[0101] Understandably, because the sample 3D point cloud in step S10 contains a large amount of data, the intermediate layer features extracted by the agent may include features unrelated to the object to be grasped, resulting in a large computational load in subsequent calculations. To reduce the computational load, see [link to relevant documentation]. Figure 2 , Figure 2 This is a second schematic diagram of a learning method for robot grasping decisions provided in an embodiment of this application. The method includes: for each of a plurality of different agents, performing the following steps S101 to S30:

[0102] Step S101: Use the intelligent agent to predict the captureable score and object score of each spatial point in the sample 3D point cloud.

[0103] The grabbable score represents the probability that a spatial point can be grabbed, while the object score represents the probability that a spatial point belongs to an object to be grabbed. The higher the grabbable score, the greater the probability that the spatial point can be grabbed; the higher the object score, the greater the probability that the spatial point belongs to an object to be grabbed.

[0104] Step S102: Based on the object score predicted by the agent, select spatial points whose object scores meet the preset screening conditions, and use the selected spatial points and their captureable scores as the intermediate layer features corresponding to the agent.

[0105] The preset filtering conditions can be spatial points where the object score is greater than a preset score threshold, or a preset number of spatial points with the highest object score. The preset number is any integer greater than 1. Other filtering conditions are also possible, but this application does not limit them.

[0106] Step S20: Use the agent to map the corresponding intermediate layer features into grasping decisions, which are then used as the target grasping decisions for the agent.

[0107] Step S30: Train the agent using the decision loss, feature loss and network training loss corresponding to the agent.

[0108] Among them, the above steps S101 to S102 are Figure 1 The detailed steps of step S10, steps S20 and S30 are described above and will not be repeated here.

[0109] By using the embodiments of this application, each spatial point is filtered by object score, effectively removing interference points such as background, focusing the attention on the object to be grasped, significantly reducing the computational load of subsequent mapping grasping decisions and loss calculation. Furthermore, the graspable score allows the agent to pay more attention to areas with high grasping probability, which helps to generate grasping poses with higher success rate.

[0110] In determining the intermediate-layer features of each agent using object scores and graspable scores, in one possible implementation, step S20 may specifically include the following steps, see [link to relevant documentation]. Figure 3 , Figure 3 A third schematic diagram of the robot grasping decision learning method provided in the embodiments of this application includes: for each of a plurality of different agents, performing the following steps S101 to S30:

[0111] Step S101: Use the intelligent agent to predict the captureable score and object score of each spatial point in the sample 3D point cloud.

[0112] Step S102: Based on the object score predicted by the agent, select spatial points whose object scores meet the preset screening conditions, and use the selected spatial points and their captureable scores as the intermediate layer features corresponding to the agent.

[0113] Step S201: Generate multiple candidate grasping decisions based on the spatial points included in the intermediate layer features corresponding to the agent.

[0114] It is understandable that the intermediate layer features of the same agent include multiple spatial points. Therefore, multiple grasping decisions can be generated based on the intermediate layer features of each agent. For example, suppose the intermediate layer features of agent 1 include two spatial points (denoted as P1 to P2) selected from the 3D point cloud of the sample scene. Each spatial point contains 3D coordinates (x, y, z) and a corresponding graspable score s (s∈[0,1], representing the probability that the point can be grasped). The coordinates of P1 are (0.1, 0.2, 0.05), and the graspable score s1 = 0.85; the coordinates of P2 are (0.15, 0.18, 0.06), and the score s2 = 0.78. Then, grasping decision M1 can be generated based on spatial point P1, and grasping decision M2 can be generated based on spatial point P2.

[0115] Step S202: Determine the grabbing decision with the highest target score from the multiple candidate grabbing decisions generated by the agent, and use it as the target grabbing decision corresponding to the agent;

[0116] Among them, the target score is positively correlated with the captureable score of each point captured by the candidate strategy.

[0117] Understandably, since the target score is positively correlated with the capture score of the points captured by the candidate strategy, the higher the capture score of a spatial point, the higher the target score of the candidate capture decision generated based on that point. Taking the spatial points P1 and P2 above as an example, the capture score of P1 is greater than that of P2. Therefore, the candidate capture decision generated based on P1 will be used as the target capture decision for agent 1.

[0118] Step S30: Train the agent using the decision loss, feature loss and network training loss corresponding to the agent.

[0119] Among them, the above steps S201 and S202 are Figure 2 The detailed steps of step S20, steps S101 to S102 and step S30 are described above and will not be repeated here.

[0120] In the embodiment of this application, the candidate crawling decision with the highest target score is selected as the target crawling decision. Since the target score is positively correlated with the crawlable score of the points crawled by the candidate crawling decision, the crawlable score of the points crawled by the target candidate strategy is the highest, that is, the probability of the points crawled by the target candidate strategy being crawled is high, which significantly improves the crawling success rate.

[0121] The following explains the process of calculating the mean squared error or KL divergence between the grasping decisions (or intermediate layer features) of each agent and the grasping decisions (or intermediate layer features) of other agents:

[0122] Taking the decision loss of each agent as represented by KL divergence as an example, the decision loss of the i-th agent can be calculated according to the following formula 1:

[0123]

[0124] Among them, L kl Let P represent the decision loss calculated using KL divergence, and let a represent the total number of joints in the dexterous hand. p Let represent the rotation angle of the p-th joint of the dexterous hand in the grasping decision of the agent. This represents the target grasping decision of the i-th agent. This represents the target-grabbing decision of other intelligent agents.

[0125] If the feature loss of each agent is represented by mean squared error, then the feature loss corresponding to the i-th agent can be calculated according to the following formula 2:

[0126]

[0127] Among them, L mse Represents feature loss, This represents the intermediate layer features corresponding to the i-th agent. This represents the intermediate layer features corresponding to other agents.

[0128] By using the embodiments of this application, the decision loss and feature loss can be quickly calculated by calculating the KL divergence or mean square error of the grasping decisions or intermediate layer features of other intelligent agents, thereby improving the learning rate of grasping decisions.

[0129] Based on this, in one possible implementation, for each agent, the decision loss corresponding to that agent is determined according to the following... Figure 4 The method shown is used to determine this. Figure 4 The schematic diagram for determining decision loss provided in the embodiments of this application includes the following steps:

[0130] Step S401: Calculate the similarity between the target grasping decision of the agent and the target grasping decision of other agents, and use it as the similarity of each agent.

[0131] The calculation of the similarity between the target grasping decision of the agent and the target grasping decisions of other agents can be achieved by directly calculating the similarity between the grasping poses included in the target grasping decision of the agent and the grasping poses included in the target grasping decisions of other agents, or by calculating the Euclidean distance between the points grasped by the target grasping decisions of the agent and the other agents to represent the similarity between the target grasping decisions, or by other methods to calculate the similarity between different grasping decisions. The embodiments of this application do not limit this.

[0132] Step S402: Perform a weighted summation of the target grasping decisions of each of the other intelligent agents to obtain the first grasping decision for each intelligent agent;

[0133] Among these, the weight of the target grasping decision for the same agent among other agents is positively correlated with its corresponding similarity. For example, suppose the similarity between target grasping decision 1 of agent 1 and target grasping decision 2 of agent 2 is A, the similarity between target grasping decision 1 of agent 1 and target grasping decision 3 of agent 3 is B, and the similarity between target grasping decision 1 of agent 1 and target grasping decision 4 of agent 4 is C. The first grasping decision = a * target grasping decision 2 + b * target grasping decision 3 + c * target grasping decision 4. If A > B > C, then a > b > c.

[0134] Step S403: The difference between the agent's target grasping decision and the first grasping decision is taken as the decision loss of the agent.

[0135] By introducing an attention mechanism, when calculating the decision loss of each agent, different weights are assigned to the grasping decisions of different agents based on their similarity to other agents. This allows the agents to focus more on learning the grasping decisions of agents with high similarity during the training process, ensuring the reliability of knowledge distillation.

[0136] Similarly, when calculating feature loss, attention mechanisms can be used to assign different weights to the intermediate layer features of different agents based on their similarity to other agents. This allows the agents to focus more on learning the intermediate layer features extracted by agents with high similarity during training, thus avoiding the transmission of noisy features among multiple agents.

[0137] In this case, for each agent, the feature loss corresponding to that agent is calculated as follows: Figure 5 The method shown is used to determine this. Figure 5 The schematic diagram for determining feature loss provided in the embodiments of this application includes the following steps:

[0138] Step S501: Calculate the similarity between the target grasping decision of the agent and the target grasping decision of other agents, and use it as the similarity of each agent.

[0139] Step S502: Perform a weighted summation of the intermediate layer features corresponding to each of the other agents to obtain the first intermediate layer features corresponding to the agent.

[0140] Among them, the weights of intermediate layer features corresponding to the same agent in other agents are positively correlated with their corresponding similarity;

[0141] Step S503: The difference between the intermediate layer features of the agent and the first intermediate layer features is used as the feature loss corresponding to the agent.

[0142] Step S501 is the same as step S401 above, step S502 is similar to step S402 above, and step S503 is similar to step S403 above. The only difference is that in step S402, the grasping decisions corresponding to each agent are weighted and summed, in step S503, the intermediate layer features corresponding to each agent are weighted and summed, in step S403, the difference between the target grasping decision and the first grasping decision of the agent is calculated, and in step S503, the difference between the intermediate layer features and the first intermediate layer features of the agent is calculated. Steps S501 to S503 will not be described again here.

[0143] Understandably, due to differences in network structure, training data, or initialization, the quality of target grasping decisions output by different agents may vary. For example, agent 1 may be better at grasping regular objects, agent 2 at grasping irregular objects, and agent 3 at grasping regular objects with smooth surfaces. Furthermore, if the object to be grasped is an irregular object with a rough surface, the grasping decisions output by agents 1 through 3 will differ significantly. Therefore, when training agents using decision loss, to enable the agents to learn high-quality grasping decisions, an attention mechanism can be used to calculate the decision loss for each agent. Specifically, the current agent's grasping decision is used as the query, and the grasping decisions of other agents are used as keys / values. Attention weights are calculated, thereby weighted and fused from each grasping decision or intermediate layer feature to achieve knowledge distillation.

[0144] In one possible implementation, the weights of different agents can be calculated using the following formula 3:

[0145]

[0146] Where W is the agent's weight, Softmax is the Softmax function, Q is the agent's expected reward, K is the expected reward of other agents for the key, V is the expected reward of other agents for the value, and d k The dimension of the key.

[0147] The learning method for robot grasping decisions of this application will be described in detail below with reference to specific embodiments.

[0148] Taking two intelligent agents as an example, such as Figure 6 The diagram shows a framework of the robot grasping decision learning method provided in this application embodiment. First, the 3D point cloud of the sample scene is input into two different agents. The two agents extract the intermediate layer features of the sample scene respectively, and obtain their respective grasping decisions based on the extracted intermediate layer features. Then, the grasping decisions output by each agent are used as keys / values ​​to calculate their respective attention weights, and feature loss and decision loss are calculated using the attention weights of each agent.

[0149] Taking three agents as an example, the intermediate layer features extracted by agent 1 are designated as feature 1, those extracted by agent 2 as feature 2, and those extracted by agent 3 as feature 3 (corresponding to step S10 above). Then, agent 1 maps feature 1 to grasping decision 1, agent 2 maps feature 2 to grasping decision 2, and agent 3 maps feature 3 to grasping decision 3 (corresponding to step S20 above). If agent 1 is to be trained, the decision loss L of agent 1 is determined based on the differences between grasping decision 1 and grasping decision 2 and grasping decision 3, respectively. kl1 The feature loss L of agent 1 is determined based on the differences between feature 1 and features 2 and 3, respectively. mse1 Calculate the network training loss l1 of agent 1 based on the truth value, and calculate the total loss L1 = l1 + L kl1 +L mse1 The agent 1 is trained using the total loss (corresponding to step S30 above).

[0150] When calculating the decision loss and feature loss of agent 1, the similarity s1 between grasping decision 1 and grasping decision 2 is first calculated as the similarity of agent 2, and the similarity s2 between grasping decision 1 and grasping decision 3 is calculated as the similarity of agent 3 (corresponding to the above steps S401 / S502). Then, based on the positive correlation between similarity and weight, the weight 1 of agent 1 and the weight 2 of agent 2 are determined, and the grasping decision 2 of agent 2 and the grasping decision 3 of agent 3 are weighted and summed (corresponding to the above step S402) to obtain the first grasping decision 1 = weight 1 * grasping decision 2 + weight 2 * grasping decision 3; at the same time, the intermediate layer feature 2 of agent 2 and the intermediate layer feature 3 of agent 3 are weighted and summed (corresponding to the above step S502) to obtain the first intermediate layer feature 1 = weight 1 * intermediate layer feature 2 + weight 2 * intermediate layer feature 3.

[0151] Then, the difference between the grasping decision 1 of agent 1 and the first grasping decision is calculated as the decision loss of agent 1 (corresponding to the aforementioned step S403), and the difference between the intermediate layer feature 1 of agent 1 and the first intermediate layer feature is calculated as the feature loss of agent 1 (corresponding to the aforementioned step S503).

[0152] Similarly, when training agent 2, the decision loss L of agent 2 is determined based on the differences between agent 2's grasping decision 2 and grasping decisions 1 and 3, respectively. kl2 The feature loss L of agent 2 is determined based on the differences between feature 2 and features 1 and 3, respectively. mse2 The network training loss l2 of agent 2 is calculated based on the truth value, and the total loss of agent 2 is calculated as L2 = l2 + L. kl2 +L mse2 Agent 2 is trained using the total loss; when training agent 3, the decision loss L of agent 3 is determined based on the difference between agent 3's grasping decision 3 and grasping decisions 1 and 2, respectively. kl3 The feature loss L of agent 3 is determined based on the differences between feature 3 and features 1 and 2, respectively. mse3 The network training loss l3 of agent 3 is calculated based on the truth value, and the total loss of agent 3 is calculated as L3 = l3 + L kl3 +L mse3 The total loss is used to train agent 3.

[0153] As can be seen from the above, after training each agent, the grasping decisions output by each trained agent can be averaged and the average value can be used as the final grasping decision. Alternatively, the grasping decision output by the agent with the smallest decision loss can be used as the final grasping decision.

[0154] Based on this, embodiments of this application also provide a method for determining robot grasping decisions, such as... Figure 7 The diagram shown is a schematic representation of a robot grasping decision determination method provided in an embodiment of this application. The method includes the following steps:

[0155] Step S701: Obtain the target intelligent agent and the target 3D point cloud;

[0156] Among them, the target intelligent agent is trained according to the learning method of robot grasping decision as described above, and the target 3D point cloud is the 3D point cloud obtained by the robot from the target scene including the object to be grasped.

[0157] Step S702: Input the target 3D point cloud into the target intelligent agent to determine the grasping decision of the target scene;

[0158] The grasping decision is used to instruct the robot's dexterous hand on the pose of the robot when it grasps the object to be grasped from the target scene.

[0159] By using the embodiments of this application, since the target agent is trained by utilizing the decision loss, feature loss and network training loss of each agent, the advantages of consistency constraints and truth supervision can be effectively integrated, so that the grasping decisions of each agent will not differ too much and are close to the truth. Therefore, the target agent can accurately predict the grasping decision of the target scene.

[0160] This application also provides a learning device for robot grasping decisions, such as... Figure 8 The diagram shown is a structural schematic of a robot grasping decision-making learning device provided in an embodiment of this application. The device includes:

[0161] The feature extraction module 801 is used to extract intermediate layer features of the sample 3D point cloud using multiple different intelligent agents to obtain the intermediate layer features corresponding to each intelligent agent. The sample 3D point cloud is a 3D point cloud of the sample scene including the object to be grasped.

[0162] The decision generation module 802 is used to map the corresponding intermediate layer features of each of the intelligent agents into grasping decisions, which are used as the target grasping decisions of each of the intelligent agents. The target grasping decisions include the grasping pose of the dexterous hand grasping the object to be grasped in the sample scene.

[0163] The training module 803 is used to train each agent using the decision loss, feature loss, and network training loss corresponding to that agent. The decision loss represents the difference between the target grasping decision of the agent and the target grasping decisions of other agents. The feature loss represents the difference between the intermediate layer features of the agent and the intermediate layer features of other agents. The network training loss represents the difference between the target grasping decision of the agent and the ground truth. The ground truth includes the actual pose of the dexterous hand when grasping the object to be grasped in the sample scenario.

[0164] In this embodiment, multiple agents are employed to extract intermediate-layer features from the 3D point cloud of a sample scene containing the object to be grasped. Since different agents focus on different features when extracting intermediate-layer features, the grasping decisions obtained from the intermediate-layer feature mappings of different agents will also differ. Furthermore, since each agent's decision loss represents the difference between its grasping decision and those of other agents, its feature loss represents the difference between its intermediate-layer features and those of other agents, and the network training loss represents the difference between the agent's target grasping decision and the ground truth, training each agent using its decision loss and feature loss can constrain the diversity of grasping decisions and intermediate-layer features, ensuring that the grasping decisions of each agent do not differ too much. Moreover, training each agent using the network training loss ensures that the final converged solution of each agent is close to the ground truth, guaranteeing the accuracy of the predicted grasping decisions. This effectively combines the advantages of consistency constraints and ground truth supervision, enabling the trained agents to accurately predict the grasping pose of a dexterous hand.

[0165] In one possible implementation, the decision loss corresponding to the agent is obtained by means of:

[0166] Calculate the similarity between the target grasping decision of the aforementioned agent and the target grasping decision of other agents, and use this similarity as the similarity of each of the other agents.

[0167] The target grasping decisions corresponding to each of the other intelligent agents are weighted and summed to obtain the first grasping decision corresponding to the intelligent agent, wherein the weight of the target grasping decision corresponding to the same intelligent agent among the other intelligent agents is positively correlated with its corresponding similarity;

[0168] The difference between the target grasping decision of the agent and the first grasping decision is taken as the decision loss of the agent.

[0169] In one possible implementation, the feature loss corresponding to the agent is obtained by means of:

[0170] Calculate the similarity between the target grasping decision of the agent and the target grasping decision of other agents, and use this similarity as the similarity of each agent.

[0171] The intermediate layer features corresponding to each of the other intelligent agents are weighted and summed to obtain the first intermediate layer feature corresponding to the intelligent agent, wherein the weight of the intermediate layer feature corresponding to the same intelligent agent among the other intelligent agents is positively correlated with its corresponding similarity;

[0172] The difference between the intermediate layer features of the agent and the first intermediate layer features is used as the feature loss corresponding to the agent.

[0173] In one possible implementation, the feature extraction module includes:

[0174] The score prediction submodule is used to predict the graspable score and object score of each spatial point in the sample 3D point cloud for each agent. The graspable score is used to represent the probability that the spatial point can be grasped, and the object score is used to represent the probability that the spatial point belongs to the object to be grasped.

[0175] The spatial point filtering submodule is used to select spatial points whose object scores meet preset filtering conditions for each agent according to the object scores predicted by the agent, and use the selected spatial points and their grabbable scores as the intermediate layer features corresponding to the agent.

[0176] In one possible implementation, the decision generation module is specifically used for:

[0177] For each agent, multiple candidate grasping decisions are generated based on the spatial points included in the intermediate layer features corresponding to the agent.

[0178] For each agent, the grasping decision with the highest target score is determined from the multiple candidate grasping decisions generated by the agent, and is taken as the target grasping decision corresponding to the agent, wherein the target score is positively correlated with the graspable score of each point grasped by the candidate strategy.

[0179] In one possible implementation, the decision loss is calculated in the following manner:

[0180]

[0181] Among them, Lkl Let P represent the decision loss calculated using KL divergence, and let P represent the total number of joints in the dexterous hand. p This represents the rotation angle of the p-th joint of the dexterous hand in the grasping decision of the agent. This represents the target grasping decision of a certain intelligent agent. This represents the target-grabbing decision of other intelligent agents.

[0182] In one possible implementation, the feature loss is calculated in the following manner:

[0183]

[0184] Among them, L mse Represents feature loss, This represents the intermediate layer features corresponding to the agent. This represents the intermediate layer features corresponding to other agents.

[0185] This application also provides a device for determining robot grasping decisions, such as... Figure 9 The diagram shown is a structural schematic of a robot grasping decision determination device provided in an embodiment of this application. The device includes:

[0186] The data acquisition module 901 is used to acquire the target intelligent agent and the target 3D point cloud. The target intelligent agent is trained according to the learning method of robot grasping decision as described above. The target 3D point cloud is a 3D point cloud obtained by the robot from the target scene including the object to be grasped.

[0187] The decision determination module 902 is used to input the target three-dimensional point cloud into the target intelligent agent and determine the grasping decision of the target scene. The grasping decision is used to instruct the robot's dexterous hand to take the pose of the robot when it grasps the object to be grasped from the target scene.

[0188] By using the embodiments of this application, since the target agent is trained by utilizing the decision loss, feature loss and network training loss of each agent, the advantages of consistency constraints and truth supervision can be effectively integrated, so that the grasping decisions of each agent will not differ too much and are close to the truth. Therefore, the target agent can accurately predict the grasping decision of the target scene.

[0189] This invention also provides an electronic device, such as... Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0190] Memory 1003 is used to store computer programs;

[0191] When the processor 1001 executes the program stored in the memory 1003, it performs the following steps for each of the multiple different intelligent agents:

[0192] The intermediate layer features of the sample 3D point cloud are extracted using the intelligent agent to obtain the intermediate layer features corresponding to the intelligent agent, wherein the sample 3D point cloud is a 3D point cloud of the sample scene including the object to be grasped.

[0193] The intelligent agent is used to map the corresponding intermediate layer features into grasping decisions, which serve as the target grasping decisions for the intelligent agent. The target grasping decisions include the grasping pose of the dexterous hand grasping the object to be grasped in the sample scene.

[0194] The agent is trained using the decision loss, feature loss, and network training loss corresponding to the agent. The decision loss of the agent represents the difference between the target grasping decision of the agent and the target grasping decision of other agents. The feature loss of the agent represents the difference between the intermediate layer features of the agent and the intermediate layer features of other agents. The network training loss of the agent represents the difference between the target grasping decision of the agent and the ground truth. The ground truth includes the actual pose of the dexterous hand when grasping the object to be grasped in the sample scene.

[0195] This invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0196] Memory, used to store computer programs;

[0197] When a processor executes a program stored in memory, it performs the following steps:

[0198] The target intelligent agent and the target 3D point cloud are obtained. The target intelligent agent is trained according to the learning method of robot grasping decision as described above. The target 3D point cloud is a 3D point cloud obtained by the robot from the target scene including the object to be grasped.

[0199] The target 3D point cloud is input into the target intelligent agent to determine the grasping decision of the target scene. The grasping decision is used to instruct the robot's dexterous hand to assume the pose corresponding to the robot's hand when the robot grasps the object to be grasped from the target scene.

[0200] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0201] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0202] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0203] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0204] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the learning method for robot grasping decision and the determination method for robot grasping decision described above.

[0205] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the robot grasping decision learning methods and robot grasping decision determination methods in the above embodiments.

[0206] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0207] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0208] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0209] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A learning method for robot grasping decisions, characterized in that, The method includes: For each of the multiple different agents, perform the following steps: The intermediate layer features of the sample 3D point cloud are extracted using the intelligent agent to obtain the intermediate layer features corresponding to the intelligent agent, wherein the sample 3D point cloud is a 3D point cloud of the sample scene including the object to be grasped. The intelligent agent is used to map the corresponding intermediate layer features into grasping decisions, which serve as the target grasping decisions for the intelligent agent. The target grasping decisions include the grasping pose of the dexterous hand grasping the object to be grasped in the sample scene. The agent is trained using the decision loss, feature loss, and network training loss corresponding to the agent. The decision loss of the agent represents the difference between the target grasping decision of the agent and the target grasping decision of other agents. The feature loss of the agent represents the difference between the intermediate layer features of the agent and the intermediate layer features of other agents. The network training loss of the agent represents the difference between the target grasping decision of the agent and the ground truth. The ground truth includes the actual pose of the dexterous hand when grasping the object to be grasped in the sample scene.

2. The method according to claim 1, characterized in that, The decision loss corresponding to the intelligent agent is obtained through the following methods: Calculate the similarity between the target grasping decision of the aforementioned agent and the target grasping decision of other agents, and use this similarity as the similarity of each of the other agents. The target grasping decisions corresponding to each of the other intelligent agents are weighted and summed to obtain the first grasping decision corresponding to the intelligent agent, wherein the weight of the target grasping decision corresponding to the same intelligent agent among the other intelligent agents is positively correlated with its corresponding similarity; The difference between the target grasping decision of the agent and the first grasping decision is taken as the decision loss of the agent.

3. The method according to claim 1 or 2, characterized in that, The feature loss corresponding to the agent is obtained through the following methods: Calculate the similarity between the target grasping decision of the agent and the target grasping decision of other agents, and use this similarity as the similarity of each agent. The intermediate layer features corresponding to each of the other intelligent agents are weighted and summed to obtain the first intermediate layer feature corresponding to the intelligent agent, wherein the weight of the intermediate layer feature corresponding to the same intelligent agent among the other intelligent agents is positively correlated with its corresponding similarity; The difference between the intermediate layer features of the agent and the first intermediate layer features is used as the feature loss corresponding to the agent.

4. The method according to any one of claims 1 to 3, characterized in that, The step of extracting intermediate layer features from the sample 3D point cloud using the intelligent agent to obtain the intermediate layer features corresponding to the intelligent agent includes: The agent is used to predict the graspable score and object score of each spatial point in the sample 3D point cloud. The graspable score represents the probability that the spatial point can be grasped, and the object score represents the probability that the spatial point belongs to the object to be grasped. Based on the object scores predicted by the agent, spatial points whose object scores meet preset filtering conditions are selected, and the selected spatial points and their graspable scores are used as intermediate layer features corresponding to the agent.

5. The method according to claim 4, characterized in that, The step of using the agent to map the corresponding intermediate layer features into grasping decisions, which serve as the grasping decisions for the agent, includes: Based on the spatial points included in the intermediate layer features corresponding to the intelligent agent, multiple candidate grasping decisions are generated. The grabbing decision with the highest target score is determined from the multiple candidate grabbing decisions generated by the agent, and is taken as the target grabbing decision corresponding to the agent, wherein the target score is positively correlated with the grabbable score of each point grabbed by the candidate strategy.

6. The method according to any one of claims 1 to 5, characterized in that, The decision loss is calculated in the following way: Among them, L kl Let P represent the decision loss calculated using KL divergence, and let P represent the total number of joints in the dexterous hand. p This represents the rotation angle of the p-th joint of the dexterous hand in the grasping decision of the agent. This represents the target grasping decision of a certain intelligent agent. This represents the target-grabbing decision of other intelligent agents.

7. The method according to any one of claims 1 to 6, characterized in that, The feature loss is calculated in the following way: Among them, L mse Represents feature loss, This represents the intermediate layer features corresponding to the agent. This represents the intermediate layer features corresponding to other agents.

8. A method for determining robot grasping decisions, characterized in that, The method includes: The target intelligent agent and the target 3D point cloud are obtained. The target intelligent agent is trained according to the learning method for robot grasping decision as described in any one of claims 1 to 7. The target 3D point cloud is a 3D point cloud obtained by the robot from the target scene including the object to be grasped. The target 3D point cloud is input into the target intelligent agent to determine the grasping decision of the target scene. The grasping decision is used to instruct the robot's dexterous hand to assume the pose corresponding to the robot's hand when the robot grasps the object to be grasped from the target scene.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1 to 7 or claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7 or claim 8.

Images