Gripping execution method and device of mechanical hand, computer equipment and storage medium

Abstract

The application discloses a gripping execution method and device of a mechanical hand, computer equipment and a storage medium. The gripping execution task of the mechanical hand is divided into two stages. In the first stage, the initial gripping posture of the mechanical hand is determined to approach and reach the motion trajectory of the target gripping posture of the mechanical hand. In the second stage, the observation space data corresponding to the environment in which the mechanical hand is located at the current time is acquired, the corresponding action space data is generated based on the observation space data, the action sequence for moving the target object to the target position is obtained, and the gripping task of the mechanical hand can be accurately guided to be autonomously executed on the basis of the target gripping posture.

Description

Technical Field

[0001] This invention relates to the field of dexterous hand control technology, and in particular to a gripping execution method, device, computer equipment, and storage medium for a robotic hand. Background Technology

[0002] With the development of robotic grasping technology, traditional two-finger grasping methods have made great progress, but they have certain limitations in handling complex objects. Therefore, dexterous hand grasping, especially five-finger grasping, has emerged, providing a richer and more diverse range of grasping methods.

[0003] Related technologies can obtain static grasping postures through algorithms that generate dexterous posture data. However, the dexterous hand in these technologies needs to dynamically complete the grasping task. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, this invention proposes a gripping execution method, apparatus, computer device, and storage medium for a robotic arm.

[0005] This invention provides a gripping execution method for a robotic arm, the method comprising: In the first stage, the trajectory of the robot arm's initial gripping posture approaching and reaching the target gripping posture is determined; wherein, the target gripping posture is the gripping posture in which the robot arm can stably grip the target object; In the second stage, the observation space data of the environment in which the robot is located at the current moment is acquired, and corresponding motion space data is generated based on the observation space data; wherein, the observation space data is used to characterize the observation space corresponding to the robot after passing through the motion trajectory; the motion space data is used to characterize the robot's decision action on the target object, so as to move the target object to the target position.

[0006] In one embodiment, determining the motion trajectory from the initial gripping posture of the robotic hand to the target gripping posture of the robotic hand includes: Obtain the initial posture data corresponding to the initial gripping posture, and the target posture data corresponding to the target gripping posture; Motion planning is performed based on the initial posture data and the target posture data to obtain the motion trajectory.

[0007] In one embodiment, the action space data is obtained by predicting the observation space data using a target neural network; the target neural network is obtained in the following manner: The initial neural network is trained based on the grip posture data corresponding to several grip postures to obtain multiple state-based expert models. Distillation is performed on the multiple state-based expert models to obtain the target neural network based on visual input.

[0008] In one embodiment, the expert model corresponds to a cluster of gripping postures; the state-based expert model is obtained in the following manner: Clustering is performed on the grasping posture data corresponding to the several grasping postures to obtain at least one grasping posture cluster; The initial neural network is trained using the grip posture data in the grip posture clusters to obtain a state-based expert model corresponding to each grip posture cluster.

[0009] In one embodiment, the step of clustering the grasping posture data corresponding to the plurality of grasping postures to obtain at least one grasping posture cluster includes: Based on the geometry and placement of the target object, the grasping posture data corresponding to several grasping postures are clustered to obtain the grasping posture cluster.

[0010] In one embodiment, the action space data is obtained by predicting the observation space data using a target neural network; the target neural network is obtained in the following manner: Acquire first type of gripping posture data, second type of gripping posture data, and third type of gripping posture data; wherein, the first matching degree between the first type of gripping posture data and the sample object is higher than the second matching degree between the second type of gripping posture data and the sample object, and the second matching degree between the second type of gripping posture data and the sample object is higher than the third matching degree between the third type of gripping posture data and the sample object. The initial neural network is trained sequentially using the first type of grip posture data, the second type of grip posture data, and the third type of grip posture data to obtain the target neural network.

[0011] In one embodiment, the action space data is obtained by predicting the observation space data using a target neural network; the target neural network is trained using the following reward function:

[0012] in, It is an object movement reward item, used to encourage the robotic arm to move the target object towards the target location; It is a posture maintenance reward item, which is used to take at least a portion of the parameter items involved in the initial grip posture as a maintenance target; It is an additional success reward when the target object moves to the target position; , , These are the corresponding weight parameters.

[0013] In one embodiment, the target pose data is obtained by any of the following methods: The point cloud data of the target object is input into the target grasping posture generation model to obtain the target posture data; The target posture data is generated based on the force closure error of the robotic arm on the target object and the mold penetration depth data corresponding to the target object.

[0014] In one embodiment, the target object includes multiple objects.

[0015] This invention provides a gripping execution device for a robotic arm, the device comprising: The motion trajectory determination module is used in the first stage to determine the motion trajectory of the robot hand approaching and reaching the target gripping posture of the robot hand; wherein, the target gripping posture is the gripping posture in which the robot hand can stably grip the target object; The motion data generation module is used in the second stage to acquire the observation space data of the environment where the robot is located at the current moment, and generate corresponding motion space data based on the observation space data; wherein, the observation space data is used to characterize the observation space corresponding to the robot after passing through the motion trajectory; the motion space data is used to characterize the decision action of the robot on the target object, so as to move the target object to the target position.

[0016] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0017] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods described above.

[0018] In this invention, the gripping task of the robotic arm is divided into two stages. In the first stage, the initial gripping posture of the robotic arm is determined to approach and reach the target gripping posture. In the second stage, the observation space data of the environment in which the robotic arm is located at the current moment is acquired, and the corresponding motion space data is generated based on the observation space data to obtain the motion sequence for moving the target object to the target position. This enables the robotic arm to be accurately guided to autonomously perform the gripping task based on the target gripping posture.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating an application scenario of the gripping execution method of the robotic arm provided according to an embodiment of this specification. Figure 2 This is a flowchart of a gripping execution method for a robotic arm according to an embodiment of this specification; Figure 3 This is a flowchart for determining a motion trajectory according to an embodiment of this specification; Figure 4 A flowchart for obtaining a target neural network according to an embodiment of this specification; Figure 5 A flowchart for obtaining an expert model according to an embodiment of this specification; Figure 6 This is a flowchart illustrating the two-stage gripping execution method of the robotic arm provided according to an embodiment of this specification. Figure 7 This is a schematic diagram of the first stage of the process of a robotic arm performing a grasping task according to an embodiment of this specification; Figure 8 This is a schematic diagram of the second stage of the robotic arm grasping and performing a task according to an embodiment of this specification; Figure 9 This is a schematic diagram of the gripping actuator of a robotic arm provided according to an embodiment of this specification. Detailed Implementation

[0021] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0022] The gripping execution method of the robotic arm provided in the embodiments of this specification can be applied to... Figure 1 In the application scenarios shown. Please refer to [link / reference]. Figure 1The robotic arm's gripping execution system includes a depth camera 110 and a robotic arm 120. The control unit of the robotic arm is communicatively connected to the depth camera. The depth camera acquires spatial data of the robotic arm's surrounding environment. Taking object lifting as an example, the gripping execution process of the robotic arm is divided into two stages: the approach and gripping stage and the object lifting stage. In the approach and gripping stage, motion planning can effectively plan a collision-free trajectory from the initial gripping posture to the target gripping posture, that is, the motion trajectory of the robotic arm approaching and reaching the target gripping posture from its initial gripping posture. Under the target gripping posture, the robotic arm can stably grip the target object. Therefore, in the approach and gripping stage, the robotic arm has changed from its initial gripping posture to the target gripping posture, which stably grips the target object.

[0023] Table 1

[0024] Table 2

[0025] To stably lift the target object to its target position, after the robotic arm achieves the target grasping posture, it is necessary to further determine the lifting or control strategy. A target neural network for the object lifting phase is learned through reinforcement learning and can be deployed in the robotic arm's control unit. Please refer to Table 1, which shows the parameters of the observation space with the control strategy. During the object lifting phase, the observation space data corresponding to the current moment of the robotic arm's environment is acquired. The target neural network then predicts the corresponding motion space data based on this observation space data (see Table 2). The motion space data allows the robotic arm to be controlled to lift the target object to its target position.

[0026] It's important to note that reinforcement learning (RL) is a branch of machine learning that involves an agent exploring its environment and learning how to achieve certain goals through interaction. The agent takes actions and receives rewards or penalties from the environment based on those actions. The goal of reinforcement learning is to find a policy that maximizes the agent's cumulative reward in the long run. For robots, reinforcement learning provides a method that allows them to autonomously learn to perform a task (such as grasping or lifting an object) through interaction with their environment. The core components of reinforcement learning include: 1. Agent: The entity that performs the action; in this case, it is the robotic arm. 2. Environment: The external world in which the intelligent agent interacts; in this case, it is a simulated physical environment. 3. Reward: Feedback given to the agent for each action, which can be positive (reward) or negative (punishment). 4. Policy: Defines the rules or methods by which an agent chooses actions in a given situation. In the embodiments of this specification, it is used to provide a control policy for the robotic arm based on the observation of the current environmental state.

[0027] Please see Figure 2 This specification provides a gripping execution method for a robotic arm, which may include the following steps: S210. In the first stage, determine the motion trajectory of the robot arm from its initial gripping posture to its target gripping posture.

[0028] The target gripping posture refers to the gripping posture in which the robotic arm can stably grasp the target object. The target object can be one, two, or three objects. The geometry of the target object must be such that the robotic arm can grasp it in one go. If the distance between any two adjacent objects in the initial object set exceeds a preset distance threshold, one of those adjacent objects can be moved to bring them sufficiently close to obtain the target object. The initial gripping posture can be determined by initializing the robotic arm to its correct position.

[0029] Specifically, in some implementations, to effectively plan a collision-free approach trajectory from the initial gripping posture to the target gripping posture, linear interpolation can be performed between the initial gripping posture and the target gripping posture to obtain the motion trajectory from the initial gripping posture to the target gripping posture of the robot. In other implementations, the initial motion trajectory is obtained through linear interpolation, and then optimized to obtain the motion trajectory from the initial gripping posture to the target gripping posture, for example, by optimizing the objective of removing clipping while maintaining temporal smoothness.

[0030] S220. In the second stage, acquire the observation space data of the environment where the robot is located at the current moment, and generate the corresponding motion space data based on the observation space data.

[0031] Among them, the observation space data is used to characterize the observation space corresponding to the robot after passing through the motion trajectory; the motion space data is used to characterize the robot's decision-making actions on the target object in order to move the target object to the target position.

[0032] The robotic arm's grasping task is divided into two stages: Stage 1 and Stage 2. In Stage 1, the robotic arm needs to approach and reach the target grasping posture from its initial grasping posture to achieve a stable grasp of the target object. In Stage 2, since the robotic arm has a stable grasp of the target object, it can control the object's movement. The robotic arm can move the target object horizontally or vertically.

[0033] Specifically, after the robotic arm grasps the target object in the first stage, it needs to move the object in the second stage. This could involve lifting the object to a target height (e.g., 15cm) or translating it to a target position. In the second stage, the robotic arm's movement trajectory can be either a straight line or a curve. The system acquires the observation space data of the robotic arm's environment at the current moment and generates corresponding motion space data based on this data. The target object is then moved using this motion space data. In the next moment, the system acquires the observation space data for that moment and generates corresponding motion space data based on it. This process is repeated to control the movement of the target object to its target position.

[0034] In the above implementation, the gripping task of the robotic arm is divided into two stages. In the first stage, the initial gripping posture of the robotic arm is determined to approach and reach the target gripping posture. In the second stage, the observation space data of the environment where the robotic arm is located at the current moment is acquired, and the corresponding motion space data is generated based on the observation space data to obtain the motion sequence of moving the target object to the target position. This enables the robotic arm to be accurately guided to perform the gripping task autonomously based on the target gripping posture.

[0035] In some implementations, please refer to Figure 3 Determining the motion trajectory of the robotic arm from its initial gripping posture to its target gripping posture can include the following steps: S310. Obtain the initial posture data corresponding to the initial grip posture, and the target posture data corresponding to the target grip posture.

[0036] The initial posture data can include the initial position, initial orientation, and initial joint angles of the robotic arm in its initial grasping posture. The target grasping posture data can include the target position, target orientation, and target joint angles of the robotic arm when it grasps the target object. Specifically, the initial and target grasping postures can be observed using sensing devices such as depth cameras and LiDAR to obtain point cloud data corresponding to the initial and target grasping postures. Data extraction is performed based on the point cloud data corresponding to the initial grasping posture to obtain the initial posture data. Similarly, data extraction is performed based on the point cloud data corresponding to the target grasping posture to obtain the initial posture data corresponding to the target grasping posture.

[0037] S320. Motion planning is performed based on initial posture data and target posture data to obtain the motion trajectory.

[0038] Motion planning, also known as path planning, is an important technique in robotics. It mainly studies how to control a robot or manipulator to move from an initial state (such as an initial grasping posture) to a target state (such as a target grasping posture) while avoiding collisions with objects in the environment. The motion planning process typically requires consideration of the robot's physical constraints, dynamic characteristics, and the geometric properties of the environment.

[0039] Specifically, this embodiment uses motion planning to obtain the execution trajectory or running trajectory of the robotic arm from the initial grasping posture to the target grasping posture, and avoids the target object being grasped from touching other positions. The running trajectory corresponds to the action sequence, which can be understood as each frame of the motion trajectory containing the position, orientation (rotation angle), and joint angle of the robotic arm in that frame. The initial grasping posture can be specified by the user, and the target grasping posture can be a grasping posture generated by a grasping posture generation model.

[0040] In the above embodiments, motion planning is performed using the initial posture data corresponding to the initial gripping posture and the target posture data corresponding to the target gripping posture to obtain the motion trajectory. This enables the robotic arm to approach and reach the target gripping posture without collision by motion planning, so that the robotic arm can stably grip the target object and complete the first stage of the gripping task.

[0041] In some implementations, action space data is obtained by predicting observation space data using a target neural network. See also... Figure 4 The target neural network can be obtained in the following ways: S410. Train the initial neural network based on the grip posture data corresponding to several grip postures to obtain multiple state-based expert models.

[0042] S420. Distill multiple state-based expert models to obtain a target neural network based on visual input.

[0043] In some cases, a state-based expert model can first be trained in a simulated physical environment. Then, to apply the gripping execution method in real-world scenarios, the strategy from the simulated physical environment needs to be refined into a vision-based strategy. This vision-based strategy observes the robot's state information and perceives point cloud data from devices such as depth cameras.

[0044] The initial neural network can be pre-built to acquire the decision-making actions of the robotic arm. It can be trained in a simulated physical environment used for reinforcement learning research. The input to the initial neural network can be observational spatial information from the simulated physical environment, which can be data observed from the environment to provide decision-making actions. Observational spatial information can include the initial state of the sample object, the point cloud features of the sample object, the target state of the sample object, the target state error of the sample object, the initial state of the robotic arm, the point cloud features of the robotic arm, the target state of the sample object, the target error of the robotic arm, the target error of the robotic arm joints, the target rotation error of the robotic arm, and the target position error of the robotic arm joints. It should be noted that, to provide geometric perception, a pre-trained segmentation network (such as PointNet) can be used to extract the point cloud features of the robotic arm and the sample object. It should also be noted that, to provide continuous spatial observation information throughout the second stage, both features and object states are calculated in the coordinates of the robotic arm. The sample object can be a single object or a collection of multiple objects.

[0045] Specifically, once the robotic arm reaches the target grasping posture, a control policy can be obtained through reinforcement learning (RL) to precisely control the joint torques as the robotic arm moves the object. For example, the policy can be trained in multiple parallel simulated physical environments; exemplified by training the policy in 512 parallel simulated physical environments within IsaacGym. Isaac Gym is a physical environment for reinforcement learning research. The Isaac Gym environment allows developers to use GPUs (Graphics Processing Units) to run environment simulations and store spatial observation information and rewards as GPU tensors, directly feeding them into the neural network for computation. This offers two advantages: First, Isaac Gym avoids the conversion from CPU simulation to GPU network training inherent in traditional interactive environments, thus accelerating the entire training process. Second, because it uses GPUs to simulate the physical environment, the Isaac Gym environment can simultaneously compute thousands of simulation environments in parallel, quickly providing a large number of training samples for the current policy.

[0046] In each simulated physical environment, a combination of a robotic arm and a target object to be grasped is set up. This combination can be initialized with the grasping position, posture, and joint angles (pose) of the robotic arm. Information observed from the simulated physical environment is determined based on the grasping posture data corresponding to the grasping posture. The initial neural network to be trained predicts a decision action (i.e., motion space data) based on the information currently observed from the simulated physical environment. The robotic arm in the simulated physical environment begins to execute the predicted decision action, and the reward (or incentive) for this decision action can be calculated. Furthermore, the PPO algorithm can be used to optimize the parameters of the initial neural network to obtain a state-based expert model. It should be noted that when the robotic arm successfully completes the grasping task (e.g., raising the target object above a certain height) or fails (the target object is more than a certain distance from the robotic arm), the simulated physical environment can be reset, i.e., initialized.

[0047] In some cases, a policy describes how an agent chooses actions given environmental observations. A state-based policy refers to an observation that includes state information about the environment, such as the position and velocity of objects. Using this information can accelerate training. However, this information is often unavailable in real-world scenarios, or its measurement and estimation are difficult, making it unusable directly. To make it usable in such scenarios, we need to obtain a vision-based policy through distillation. This can directly use data captured by sensors such as cameras as environmental observations to provide better actions. Specifically, after obtaining multiple state-based expert models, to adapt to the real-world environment, these models need to be distilled to obtain a target neural network based on visual input. For example, to apply this algorithm in a real-world scenario, the DAgger algorithm can be used to refine the policy into a vision-based policy that observes the state information of the hand and perceives the point cloud captured by a depth camera.

[0048] In some implementations, the expert model corresponds to a cluster of gripping postures. See [link to relevant documentation]. Figure 5 State-based expert models can be obtained in the following ways: S510. Cluster the grasping posture data corresponding to several grasping postures to obtain at least one grasping posture cluster. S520. Train the initial neural network using the grasping posture data in the grasping posture cluster to obtain a state-based expert model corresponding to each grasping posture cluster.

[0049] In some cases, the enhancement strategy changes significantly when the object configuration and pre-grasping pose change. For example, the grasping pose for two spheres differs from that for two cylinders. Similarly, the grasping pose may differ when the target object is placed parallel to the forearm versus perpendicular to it. Therefore, to improve the generalization of the grasping task execution scheme to out-of-distribution object configurations and obtain a universal strategy that can generalize to different situations, the grasping pose data corresponding to several grasping poses can be clustered into different grasping pose clusters. For each grasping pose cluster, the grasping pose data in that cluster corresponds to one possible combination of target objects. If an initial neural network is trained based on each grasping pose data in each grasping pose cluster, a state-based expert model adapted to that situation can be obtained.

[0050] For example, clustering is performed on the grasping posture data corresponding to several grasping postures to obtain grasping posture cluster A for case A, grasping posture cluster B for case B, and grasping posture cluster C for case C. Based on each grasping posture data in grasping posture cluster A, the corresponding observation space information is determined and input into an initial neural network. The corresponding reward data is calculated, and the parameters of the initial neural network are updated to obtain a state-based expert model A adapted to case A. Similarly, a state-based expert model B adapted to case B and a state-based expert model C adapted to case C can be obtained. It can be seen that by training corresponding expert models separately using different grasping posture clusters, different expert models are adapted to different cases. To obtain a general policy that can generalize to different cases, all expert models can be distilled into a general target neural network based on visual input. That is, in this embodiment, for each grasping posture cluster, a state-based expert policy can be trained separately. During policy distillation, at least some expert policies are distilled into a generalist policy based on visual input. In some implementations, clustering is performed on the grasping posture data corresponding to several grasping postures to obtain at least one grasping posture cluster, including: clustering the grasping posture data corresponding to several grasping postures according to the geometry and placement of the target object to obtain a grasping posture cluster.

[0051] Specifically, the gripping posture of the robotic arm corresponds to a target object, and some target objects may have the same or similar geometric shapes and placements. Therefore, the gripping posture data corresponding to several gripping postures can be clustered according to the geometric shape and placement of the target object, placing the gripping posture data corresponding to target object objects with similar or identical geometric shapes and placements into the same gripping posture cluster, resulting in at least one gripping posture cluster. In some implementations, the elements in each gripping posture cluster are the robotic arm posture and the corresponding target object.

[0052] In the above embodiments, by using the grasping posture data in the grasping posture cluster to train the initial neural network, a state-based expert model corresponding to each grasping posture cluster is obtained, which provides the prerequisite for obtaining a general strategy that can generalize to different situations.

[0053] In some implementations, the motion space data is obtained by predicting the observation space data using a target neural network. The target neural network is obtained by acquiring first-type grasping posture data, second-type grasping posture data, and third-type grasping posture data; wherein the first-type grasping posture data has a higher first-matching degree with the sample object than the second-matching degree with the sample object, and the second-matching degree with the sample object is higher than the third-matching degree with the sample object; the initial neural network is trained sequentially using the first-type grasping posture data, the second-type grasping posture data, and the third-type grasping posture data to obtain the target neural network.

[0054] In some cases, there may be slight clipping or finger float relative to the target object in the target grasping posture. Due to numerical precision limitations, these issues are computationally difficult to avoid. Furthermore, during the initial approach to the target grasping posture along the trajectory, collisions may occur, disrupting the grasp of the target object. Therefore, training a neural network solely with high-quality grasping posture data has limitations, such as its inability to effectively handle inaccurate grasping postures. To enable the neural network to adapt to inaccurate grasping postures, a curriculum plan can be used during training. Thus, it is necessary to construct three types of grasping posture data: Type 1, Type 2, and Type 3. Different grasping posture data have different degrees of matching with the sample object; the quality of the grasping posture data depends on this matching degree. A higher matching degree between the grasping posture data and the sample object indicates higher quality grasping posture data.

[0055] In a simulated physical environment, three types of grip posture data are acquired: Type I, Type II, and Type III. Since the quality of Type I grip posture data is higher than Type II, and the quality of Type II is higher than Type III, the initial neural network can be trained first using Type I grip posture data. This allows the neural network to acquire the basic ability to perform the second-stage task. Then, Type II grip posture data from a portion of the simulated physical environment is used for training, enabling the neural network to handle less precise grip postures during the second-stage task. Finally, Type III grip posture data from a portion of the simulated physical environment is used for training, enabling the neural network to handle even less precise grip postures during the second-stage task.

[0056] In this embodiment, when training or distilling the policy, initially, all pre-grasping postures are selected from the highest quality grasping posture data. After the policy acquires the basic ability to improve object handling, in some environments, the target is replaced with inaccurate grasping posture data caused by lower generation quality or collisions between the hand and object upon approach; the proportion of inaccurate grasping posture data gradually increases from 0% to 60%, during which time the policy gradually learns to handle inaccurate postures. For example, the proportion of training using second-type grasping posture data can gradually increase from 0% to 60%. Similarly, the proportion of training using third-type grasping posture data can gradually increase from 0% to 60%.

[0057] In the above embodiments, the initial neural network is trained sequentially using the first type of grip posture data, the second type of grip posture data, and the third type of grip posture data to obtain the target neural network. Through this training process, the trained neural network gradually learns to handle inaccurate grip postures, thereby improving the robustness of the execution scheme to poor-quality grip postures.

[0058] In some implementations, the action space data is obtained by predicting the observation space data using a target neural network. The target neural network is trained using the following reward function:

[0059] in, It is an object movement reward, used to encourage the robotic arm to move the target object to the target location. It is a posture maintenance reward, which is used to take at least a portion of the parameters involved in the initial grip posture as the maintenance target. It is an additional success reward when the target object moves to the target position. , , These are the corresponding weight parameters.

[0060] In some implementations, the target object includes multiple objects; the object movement reward is determined based on the position of a specified object within the target object. Parameters involved in the initial gripping posture include the joint angles of the robotic arm, the orientation of the robotic arm, and the position and orientation of the target object in the robotic arm's coordinate system.

[0061] For example, if a robotic arm lifts a target object, the object movement reward is the object lifting reward. The specified object can be the object closest to the support surface of the target object. Since the target object includes multiple objects, the nearest distance to the support surface of the target object is determined, and the object lifting reward is determined based on this nearest distance.

[0062] For example, if a robotic arm translates a target object, the object movement reward is the object translation reward. The specified object can be the object closest to the initial position of the target object. Since the target object includes multiple objects, the closest distance to the initial position of the target object is determined, and the object translation reward is determined based on this closest distance.

[0063] For example, if a robotic arm moves a target object in a specified direction, and the object's position data in that direction meets a preset position condition, the object movement reward is determined based on that position data. The preset position condition can be being closest to the target object's initial position or being closest to the target object's support surface.

[0064] For example, the goal of the control policy is to lift all objects off the table. To achieve this, the target neural network can be trained using the following reward function:

[0065] in, It is an object lifting reward item, used to encourage the robotic arm to lift the target object to the target position; It is a posture maintenance reward item, which is used to take at least a portion of the parameters involved in the initial grip posture as the maintenance target; It is an additional success reward when the target object moves to the target position; , , These are the corresponding weight parameters.

[0066] For example, the target neural network can also be trained using the following reward function:

[0067] in, Encourage hands to lift all objects. and Provide guidance and intensive rewards to encourage the hand to remain near the pre-grasping posture; Encourage the hand to maintain the object in its original posture within the hand's local coordinate system. When both objects are lifted to 15cm, Additional rewards are provided and the environment is reset. Furthermore, failing to complete a gripping action for 75 consecutive steps, timing out, or the object moving too far from the robotic arm will trigger an early reset of the simulated physical environment.

[0068] In some implementations, target pose data is obtained by any of the following methods: inputting point cloud data of the target object into a target grasping pose generation model to obtain target pose data; or, generating target pose data based on the force closure error of the robot arm on the target object and the mold penetration depth data corresponding to the target object.

[0069] The target grasping posture generation model can employ a conditional diffusion model (DDPM). The target grasping posture data output by the model can include the target position, target orientation, and target joint angles corresponding to the state in which the robot grips the target object. Penetration depth data is used to characterize penetration energy.

[0070] Specifically, in some implementations, the target grasping posture generation model can be deployed on the controller of the robotic arm. The controller can be connected to a sensing unit (such as LiDAR, camera, etc.) to collect data from the target object, obtaining point cloud data of the target object. The acquired point cloud data is input into the target grasping posture generation model, which can directly output the corresponding target grasping posture data based on the input point cloud data, enabling the robotic arm to grasp the target object. In this implementation, the target grasping posture generation model can quickly output the grasping posture data corresponding to the target object, with the entire process taking only a few seconds or even less than ten seconds. This method of rapidly generating grasping postures can be applied to real-time robotic systems.

[0071] Specifically, in some embodiments, the initial gripping energy of the robot arm on the target object is determined based on the sum of the force closure error and the mold penetration depth data corresponding to the target object. By optimizing the robot arm's posture data with the goal of minimizing the gripping energy, the gripping posture data that the robot arm can grasp the target object is determined, thereby obtaining the target gripping posture that the robot arm can grasp the target object. In this embodiment, the method of generating target posture data based on the sum of the force closure error and the mold penetration depth data takes tens of minutes, but the quality of the generated target posture data is reliable.

[0072] In some implementations, the target object includes multiple objects, and the gripping posture data is determined using the following formula. Grasp target object Grasping energy:

[0073] in, For the target object The number of objects in the middle. Used to represent the surface of a robotic arm. For the surface of the robotic arm The above is aimed at the first an object The contact point, User-represented contact points For the an object Force closure error, Used to represent the target object The corresponding molding depth data, This data is used to indicate joint over-limit data for a robotic arm, representing the angle beyond the joint's limit. , These are the weighting coefficients.

[0074] For example, please refer to Figure 6 Target pose data can be generated using either of two parallel methods. Figure 6 In this context, A refers to the generation of target posture data based on the force closure error of the robotic arm on the target object and the corresponding molding depth data of the target object. Figure 6 In this context, B refers to the target grasping pose generation model. The point cloud data of the target object is input into the model to obtain the target pose data. Target pose data can be generated in any way. Figure 6The "C" in this context refers to motion planning based on the generated target posture data and the initial posture data corresponding to the initial grasping posture. This "Planned Reaching" involves planning a motion trajectory to approach and reach the target grasping posture, completing the first stage of the robotic arm's grasping task. This allows the user to proceed to the second stage, which is... Figure 6 D involves learning a sequence of actions to lift an object off a table using a goal-based reinforcement learning method.

[0075] State-based policies refer to observations that include state information from the environment, such as the position and velocity of objects. Using this information can accelerate training. However, in many scenarios, this information is not directly available, or its measurement and estimation are difficult, making direct use impossible. To use this information in such situations, distillation is required. Figure 6 The system obtains a vision-based policy, which can directly use data captured by sensors such as cameras as environmental observations to provide better action.

[0076] Furthermore, target pose data can be determined through a target grasping pose generation model. After obtaining the trained target neural network, please refer to [link / reference needed]. Figure 7 and Figure 8 The robotic arm's grasping task is divided into a two-stage grasping posture execution scheme. In the first stage (e.g. Figure 7 As shown), the robotic arm approaches and achieves the target grasping posture through motion planning control. In the second stage (as shown... Figure 8 As shown, the robotic arm learns grasping and execution strategies through reinforcement learning.

[0077] In some implementations, the target object comprises multiple objects. The surface of the robotic arm has contact points with the target object. The force closure error is determined by: S710. Determine the target object where the contact point is located within the target object object.

[0078] S720. Based on the summation of the force closure error of the contact point to the target object, the sum of the force closure error of the robot arm to the target object is obtained.

[0079] In the field of robotics, force closure error refers to the inability of a robot's manipulator to maintain a force closure state between itself and an object. This means the manipulator cannot apply sufficient contact force to hold the object in place, leading to displacement or slippage during grasping or manipulation. This error affects the robot's accuracy and stability; therefore, reducing force closure estimation error is crucial for improving a robot's grasping and manipulation capabilities.

[0080] Specifically, in some implementations, when the target object is a single object, that object can be identified as the target object where the contact point is located. The force closure error of the robot arm on the target object is the force closure error of the contact point on that object.

[0081] In some embodiments, the target object can be a stack of multiple objects. At least some of the objects in the target object group come into contact with the surface of the robot arm, creating contact points. The sample objects where the contact points are located are denoted as the target objects. For any given target object, there are multiple contact points between the robot arm and that target object. With the help of friction, the resultant force provided by the multiple contact points can resist external forces or torques in any direction, enabling the robot arm to stably grasp the contacted object. In this state, there is a force closure error of the contact points to the target object. For a target object group comprising multiple objects, for each target object, it is necessary to integrate the force closure error of the contact points to the target object using any of the following methods: summation, weighted summation, product, etc., to obtain the sum of the force closure errors of the robot arm to the target object group, so as to determine whether the robot arm can stably grasp the set of sample objects, that is, whether it can resist external forces or torques in any direction.

[0082] For example, for each contact point, each contact point has coordinates, and each contact point has a corresponding normal vector on the surface of the object being contacted. Based on the coordinates of each contact point and the corresponding normal vector, it is possible to estimate whether the object is in a force-closed state, thus obtaining the force-closure error.

[0083] In some implementations, the target object includes multiple objects. The molding depth data is determined by: the molding depth between two adjacent objects within the target object, the molding depth between the robot and objects within the target object, and the molding depth between the robot and the bearing surface of the target object.

[0084] Specifically, the target object consists of several pairs of adjacent objects. The first clipping depth between two adjacent objects is calculated. When the robotic arm grasps the target object, clipping also occurs between the robotic arm and at least some of the objects; therefore, the second clipping depth between the robotic arm and the objects is calculated. When the target object is placed on a support surface, and the robotic arm grasps the target object from the support surface, clipping also occurs between the robotic arm and the support surface; therefore, the third clipping depth between the robotic arm and the support surface is calculated. The first, second, and third clipping depths are summed or weighted summed to obtain the clipping depth data. Furthermore, the robotic arm's self-clipping data can also be considered; then, the clipping depth data is determined based on the robotic arm's self-clipping data, the first, second, and third clipping depths.

[0085] In this embodiment, by determining the penetration depth data, a data basis is provided for accurately determining the gripping energy of the robotic arm in grasping the target object, and further, accurate target gripping posture data can be generated.

[0086] This specification provides a gripping execution method for a robotic arm, wherein the robotic arm grips a target object, which may include multiple objects. The surface of the robotic arm has contact points with the target objects. The method may include the following steps: S802, Obtain the initial posture data corresponding to the initial grip posture.

[0087] S804. Determine the target object where the contact point is located within the target object object; S806. Based on the summation of the force closure error of the contact point to the target object, the sum of the force closure error of the robot arm to the target object is obtained.

[0088] S808. Determine the molding depth data based on the molding depth between two adjacent objects in the target object, the molding depth between the robot and an object in the target object, and the molding depth between the robot and the bearing surface of the target object.

[0089] S810. Based on the force closure error of the robot arm on the target object and the mold penetration depth data corresponding to the target object, generate target posture data corresponding to the target gripping posture.

[0090] Among them, the target gripping posture is the gripping posture in which the robotic arm can stably grip the target object.

[0091] S812. In the first stage, motion planning is performed based on the initial posture data and the target posture data to obtain the motion trajectory.

[0092] S814. In the second stage, acquire the observation space data corresponding to the environment where the robot is located at the current moment.

[0093] S816. Predict the observed spatial data through the target neural network to generate the corresponding action spatial data.

[0094] Among them, the observation space data is used to characterize the observation space corresponding to the robot after passing through the motion trajectory; the motion space data is used to characterize the robot's decision-making actions on the target object in order to move the target object to the target position.

[0095] This process involves clustering grip posture data corresponding to several grip postures based on the geometry and placement of the target object, resulting in grip posture clusters. The initial neural network is then trained using the grip posture data from these clusters to obtain a state-based expert model for each cluster. Finally, multiple state-based expert models are distilled to obtain a target neural network based on visual input.

[0096] This specification provides a gripping actuator 900 for a robotic arm. Please refer to [link / reference]. Figure 9 The gripping actuator 900 includes: The motion trajectory determination module 910 is used in the first stage to determine the motion trajectory of the robot hand approaching and reaching the target gripping posture of the robot hand; wherein, the target gripping posture is the gripping posture in which the robot hand can stably grip the target object. The motion data generation module 920 is used in the second stage to acquire the observation space data corresponding to the environment where the robot is located at the current moment, and generate corresponding motion space data based on the observation space data; wherein, the observation space data is used to characterize the observation space corresponding to the robot after passing through the motion trajectory; the motion space data is used to characterize the decision action of the robot on the target object, so as to move the target object to the target position.

[0097] For detailed specifications of the gripping actuator of the robotic arm, please refer to the description of the gripping execution method of the robotic arm above.

[0098] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to perform the method described in any of the above embodiments.

[0099] This specification also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods described in any of the above embodiments.

[0100] This specification also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0101] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0102] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0103] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0104] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0105] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0106] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A gripping execution method for a robotic arm, characterized in that, The method includes: In the first stage, the trajectory of the robot arm's initial gripping posture approaching and reaching the target gripping posture is determined; wherein, the target gripping posture is the gripping posture in which the robot arm can stably grip the target object; In the second stage, the observation space data of the environment in which the robotic arm is located at the current moment is acquired, and corresponding motion space data is generated based on the observation space data; wherein, the observation space data is used to characterize the observation space of the robotic arm after passing through the motion trajectory; the motion space data is used to characterize the decision action of the robotic arm on the target object, so as to move the target object to the target position; The action space data is obtained by predicting the observation space data using a target neural network; the target neural network is obtained in the following way: Acquire first type of gripping posture data, second type of gripping posture data, and third type of gripping posture data; wherein, the first matching degree between the first type of gripping posture data and the sample object is higher than the second matching degree between the second type of gripping posture data and the sample object, and the second matching degree between the second type of gripping posture data and the sample object is higher than the third matching degree between the third type of gripping posture data and the sample object. The initial neural network is trained sequentially using the first type of grip posture data, the second type of grip posture data, and the third type of grip posture data to obtain the target neural network.

2. The method according to claim 1, characterized in that, The process of determining the motion trajectory from the initial gripping posture of the robotic arm to the target gripping posture of the robotic arm includes: Obtain the initial posture data corresponding to the initial gripping posture, and the target posture data corresponding to the target gripping posture; Motion planning is performed based on the initial posture data and the target posture data to obtain the motion trajectory.

3. The method according to claim 1, characterized in that, The action space data is obtained by predicting the observation space data through a target neural network; The target neural network is obtained in the following way: The initial neural network is trained based on the grip posture data corresponding to several grip postures to obtain multiple state-based expert models. Distillation is performed on the multiple state-based expert models to obtain the target neural network based on visual input.

4. The method according to claim 3, characterized in that, The expert model corresponds to a cluster of gripping postures; The state-based expert model is obtained in the following way: Clustering is performed on the grasping posture data corresponding to the several grasping postures to obtain at least one grasping posture cluster; The initial neural network is trained using the grip posture data in the grip posture clusters to obtain a state-based expert model corresponding to each grip posture cluster.

5. The method according to claim 4, characterized in that, The step of clustering the grip posture data corresponding to the plurality of grip postures to obtain at least one grip posture cluster includes: Based on the geometry and placement of the target object, the grasping posture data corresponding to several grasping postures are clustered to obtain the grasping posture cluster.

6. The method according to claim 1, characterized in that, The action space data is obtained by predicting the observation space data using a target neural network; the target neural network is trained using the following reward function: in, It is an object movement reward item, used to encourage the robotic arm to move the target object towards the target location; It is a posture maintenance reward item, which is used to take at least a portion of the parameter items involved in the initial grip posture as a maintenance target; It is an additional success reward when the target object moves to the target position; , , These are the corresponding weight parameters.

7. The method according to claim 6, characterized in that, The target object includes multiple objects; the object movement reward is determined based on the position of a specified object within the target object. The parameters involved in the initial gripping posture include the joint angles of the robotic arm, the orientation of the robotic arm, and the position and orientation of the target object in the coordinate system of the robotic arm.

8. The method according to claim 2, characterized in that, The target pose data is obtained through any of the following methods: The point cloud data of the target object is input into the target grasping posture generation model to obtain the target posture data; The target posture data is generated based on the force closure error of the robotic arm on the target object and the mold penetration depth data corresponding to the target object.

9. A gripping actuator for a robotic arm, used to implement the steps of the method according to any one of claims 1 to 8, characterized in that, The device includes: The motion trajectory determination module is used in the first stage to determine the motion trajectory of the robot hand approaching and reaching the target gripping posture of the robot hand; wherein, the target gripping posture is the gripping posture in which the robot hand can stably grip the target object; The motion data generation module is used in the second stage to acquire the observation space data of the environment where the robot is located at the current moment, and generate corresponding motion space data based on the observation space data; wherein, the observation space data is used to characterize the observation space corresponding to the robot after passing through the motion trajectory; the motion space data is used to characterize the decision action of the robot on the target object, so as to move the target object to the target position.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images