Underdriven dexterous hand grabbing pose generation method based on deep learning

Abstract

The invention discloses an under-driven dexterous hand grabbing pose generation method based on deep learning, and belongs to the technical field of robot operation and intelligent grabbing. According to the method, the prior knowledge of a large language model LLM to real capture is introduced into a data set, and a reasonable capture pose is realized by using object semantic-geometric joint features of a point cloud encoder Point Transform and the LLM. According to the method, three types of grabbing strategy classification models for the under-actuated five-finger dexterous hand are constructed, including two-finger kneading, three-finger kneading and five-finger grabbing, and are used for restraining and guiding generation of subsequent fine grabbing postures. According to the selected grabbing strategy, a network is generated to predict six-dimensional poses and angles of all joints of the dexterous hand, and a grabbing posture matched with the shape of the object is obtained. According to the method, two constraint loss items are additionally added to realize grabbing quality optimization. The method can be suitable for intelligent grabbing tasks of industrial robots, service robots and precision operation equipment under the condition that negative grabbing of samples and complex post-processing optimization are not needed.

Description

Technical Field

[0001] This invention relates to the field of intelligent robots, and in particular to a method for generating grasping poses using an underactuated dexterous hand based on deep learning. Background Technology

[0002] With the development of robotics technology, dexterous hands have received widespread attention in industrial manufacturing, intelligent services, and precision operations. Compared to traditional two- or three-finger mechanical grippers, five-fingered dexterous hands have more joint degrees of freedom and more complex hand structures, enabling more flexible grasping and manipulation capabilities. However, the complex structure and high control dimensions of dexterous hands make the planning and generation of their grasping poses one of the core challenges in robotic manipulation research.

[0003] Existing dexterous hand grasping methods mainly include physical simulation-based search optimization methods, geometry-heuristic grasping planning methods, and deep learning-based grasping generation methods. Among these, deep learning-based grasping methods have developed rapidly in recent years. Most of these methods generate executable grasping postures for different types of robotic hands by learning from a large amount of successful and failed grasping data. However, these methods generally suffer from the following problems:

[0004] (1) High dependence on data: Many methods require a large dataset containing failed crawling samples, and crawling evaluation is carried out through binary classification or scoring networks, resulting in high data collection costs, difficult labeling, and complex training process.

[0005] (2) Problems exist such as the grasping posture not conforming to the characteristics of the object: After generating the initial grasping posture, some methods need to rely on adversarial networks, multi-stage optimization or complex post-processing modules to obtain the final grasping posture that can be executed, making it difficult to deploy the system and apply it in real time.

[0006] (3) Lack of semantic-level prior knowledge for grasping: Traditional deep learning grasping generation methods usually rely only on visual or point cloud features, ignoring the important influence of object semantic attributes on grasping strategies. For example, categories such as "bottle", "scissors" and "sphere" have natural differences in grasping methods, but existing methods rarely use high-level semantic information for strategy guidance.

[0007] Therefore, there is an urgent need for a grasping pose generation method that can integrate geometric and semantic features, reduce training data requirements, and is applicable to underactuated dexterous hand structures, in order to improve the rationality, stability, and versatility of grasping postures. The underactuated dexterous hand grasping pose generation method proposed in this invention, based on deep learning and a large language model, can effectively overcome the above-mentioned shortcomings and achieve more reasonable, stable, and executable grasping posture generation for target objects. Summary of the Invention

[0008] This invention relates to the field of intelligent robots, and in particular to a deep learning-based method for generating grasping poses of an underactuated dexterous hand. The purpose is to solve the problems of unstable grasping and lack of flexibility in existing robots equipped with underactuated five-fingered dexterous hands when grasping and manipulating unknown target objects, and to improve the robot's intelligent decision-making and operational capabilities when facing complex grasping geometric objects and usage scenarios.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a method for generating grasping poses using an underactuated dexterous hand based on deep learning, comprising the following steps:

[0010] Step S1: Create the crawling dataset. This method constructs a crawling dataset based on the YCB object dataset. First, the RGB-D images of the objects in the dataset are preprocessed and converted into point cloud data. The obtained point cloud data is input into a point cloud encoder (Point Transformer) to obtain the geometric features of the objects. Then, prompt words are input into a large language model according to the object category. Finally, the responses are text-encoded to obtain semantic feature vectors.

[0011] Step S2: The geometric features and semantic features obtained in step S1 are spliced ​​and fused to obtain joint features. The grasping strategy classification network is used to predict the most suitable grasping type for the target object. The grasping strategy types include two-finger pinch, three-finger pinch and five-finger grasp.

[0012] Step S3: Input the grasping strategy category and joint features obtained in step S2 into the grasping generation network to predict the six-dimensional pose parameters and joint angles of the underactuated dexterous hand, and generate a grasping posture that matches the shape of the target object.

[0013] Step S4: During the training phase, contact constraint loss and orientation constraint loss are applied to the grasping pose generated in step S3 to improve the executability and stability of the grasping pose, and generate the final six-DOF palm pose and fine joint angles of the fingers.

[0014] Step S5: To further verify the effectiveness of the constraint loss function proposed in this method, ablation experiments were conducted for different loss function configurations, and the proposed method was compared with existing methods, demonstrating the feasibility and effectiveness of the proposed method.

[0015] Furthermore, the specific steps of step S1 are as follows:

[0016] Step S1-1: The YCB object dataset is collected from RGB-D images of real objects. This method selects 30 items from it. The depth map is converted into point cloud data through preprocessing to fully reflect the three-dimensional structure of the object's surface;

[0017] Step S1-2: Based on the point cloud data collected in step S1-1, manually annotate the reasonable grasping pose of the target object. The annotation includes the rotation of the dexterous hand during grasping. Peaceful relocation and the angles of each joint Each grasping gesture will be divided into one of three basic grasping types;

[0018] Step S1-3: Input the point cloud data obtained in step S1-1 into the point cloud encoder Point Transformer, and extract the three-dimensional geometric feature vector of the target object through the point cloud self-attention mechanism. Size is .

[0019] Steps S1-4: Based on the category name and appearance attributes of the target object, construct semantic prompt words for the large language model, and input the prompt words into the large language model to obtain a text response containing object function information and human grasping intuition;

[0020] Step S1-5: Encode the text response output by the large language model in Step S1-4. This will yield a semantic feature vector containing rich prior knowledge from the large language model. Size is .

[0021] Furthermore, the specific steps of step S2 are as follows:

[0022] Step S2-1: Apply the geometric features obtained in step S1 and semantic features By splicing, we get a size of Joint feature input This includes both the object's three-dimensional structural information and semantic functional information;

[0023] Step S2-2: Combine the joint features obtained in step S2-1 The grasping strategy classification module obtains the grasping type from the input. This module uses an MLP classification network with stacked fully connected layers and ReLU activation functions to reduce the dimensionality and encode the joint features, extracting deep feature representations to distinguish different grasping strategies. A Sigmoid activation function is used as the output layer of the classification network to output independent confidence values ​​for the three grasping strategies (two-finger pinch, three-finger pinch, and five-finger grasp) to reflect the feasibility of each strategy on the current object. Finally, the grasping type with the highest confidence is selected as the output. Proceed to the next stage of the network. During the training phase, binary cross-entropy loss is used. Calculate the error between the prediction result and the true label for each crawling strategy. The mathematical expression is as follows:

[0024]

[0025] in Indicates the number of samples. This indicates that the quality label was retrieved; 1 indicates success, and 0 indicates failure. This represents the grabbing quality score predicted by the model. A multi-label grabbing strategy classification network is trained, enabling the network to simultaneously determine the applicability of multiple grabbing methods to the target object.

[0026] Furthermore, the specific steps of step S3 are as follows:

[0027] Step S3-1: Based on the crawling strategy category obtained in step S2 Read the rough hand configuration parameters corresponding to the crawling type from the preset crawling template library. The coarse hand configuration includes preset joint angles for each joint of the underactuated dexterous hand. This is based on the point cloud-semantic information of the target object. Calculate the center position of the object's point cloud. This is used as the initial translation vector for grasping. During the training phase, zero-mean Gaussian noise is added to the rotation parameters of the actual grasping posture to obtain the initial rotation parameters. During the inference phase, since there are no real samples, this method samples rotation vectors from a uniform distribution as... ;

[0028] Step S3-2: Configure the rough hand type The center position of the object Initial rotation parameters and the joint features obtained in step S2 Together, they form the input vector for the grasping generation network, which is then fed into the grasping pose generation network. The grasping generation network encodes the input vector through a fully connected residual structure and outputs the joint angle correction. And obtain the fine hand shape configuration according to the following formula:

[0029]

[0030] The fine hand configuration is used to describe the fine joint angles of an underactuated dexterous hand.

[0031] The capture generator network simultaneously outputs translation corrections. The final capture translation vector is obtained according to the following formula:

[0032]

[0033] The capture-generating network further outputs rotation corrections. The final grasping rotation vector is obtained according to the following formula:

[0034]

[0035] The resulting translation vector Rotation vector and finger joint angle Combined into a complete underactuated dexterous hand grasping pose information :

[0036]

[0037] Furthermore, the specific steps of step S4 are as follows:

[0038] Step S4-1: Obtain the grasping pose generated in step S3 The corresponding set of hand mesh vertices First, import the URDF model of the underactuated dexterous hand used in this method into the network, and select the set of fingertip vertices that frequently come into contact with the target object during the grasping process. For each contact vertex Calculate the point cloud set from it to the target object. Find the minimum Euclidean distance and calculate the contact loss according to the following formula. :

[0039]

[0040] By minimizing this loss, the fingertips in the generated hand pose are brought close to and conform to the surface of the target object, thus improving the stability of the grasp.

[0041] Determine the unit vector of the robotic arm's grasping approach direction. Calculate the unit vector of direction between the center of the object and the center of the palm. Orientation loss is calculated using the following formula. :

[0042]

[0043] This constrains the hand's orientation to align with the center of the object, thereby ensuring the robotic arm maintains a reasonable grasping direction when approaching the object and improving the physical feasibility of generating a grasping posture.

[0044] Step S4-2: Apply the binary cross-entropy loss obtained in step S2-2 The two types of geometric losses obtained in step S4-1 are weighted and combined according to preset weights to form the grasping optimization loss function during the training phase:

[0045]

[0046] The crawling generation network is optimized through backpropagation, making the generated crawling poses more consistent with physical properties.

[0047] Furthermore, the specific steps of step S5 are as follows:

[0048] Step S5-1: In the simulated grasping experiment, this method uses the YCB object dataset for the experiment and uses two metrics to evaluate the grasping results: grasping ε-mass, which represents the radius of the largest six-dimensional sphere centered at the origin that can be enclosed within the convex hull of the torque space; and cross volume, which represents the penetration between the finger and the object. The goal is to minimize the cross volume between the object and the finger to avoid grasping failure.

[0049] Step S5-2: To verify the effectiveness of the two additional losses added to the network by this method, the following ablation experiment was designed: one of the loss functions was removed respectively ( And keep the remaining losses unchanged. For each configuration (remove contact loss, remove orientation loss), retrain the model and evaluate it using the same training and test sets;

[0050] Step S5-3: To verify the performance difference between this method and existing methods, a comparative experiment was conducted in a simulation environment. GraspIt! was selected as the benchmark. Specifically, in the simulation environment, 800 grasping attempts were performed using both this method and the GraspIt! method, and the quality and stability of the grasps generated by each method were statistically evaluated. Through ablation experiments and comparative experiments, the method of this invention demonstrated superior performance compared to the control method in terms of accuracy, speed, and stability of grasping posture generation, proving the effectiveness of this method.

[0051] Compared with the prior art, the present invention has at least the following advantages:

[0052] First, this invention incorporates LLM semantic features into the grasping dataset, leveraging the rich prior knowledge of large language models to help the network generate more reasonable grasping poses that better conform to human grasping postures. Second, this invention constructs a three-class grasping strategy classification model for underactuated dexterity hands, including two-finger pinch, three-finger pinch, and five-finger grasping. This reduces the difficulty of generating the initial grasping pose and decreases network complexity. Third, this method adds two additional constraint loss terms (contact constraint and orientation constraint) to optimize grasping, making the grasping poses generated by the network more consistent with physical characteristics and improving the feasibility of the model-generated gestures in the real world. Attached Figure Description

[0053] Figure 1 This is a diagram of the overall architecture of the grasping pose generation network for underactuated five-fingered dexterous hands according to the present invention.

[0054] Figure 2 Examples of RGB and depth images of three items from the YCB dataset used in this invention.

[0055] Figure 3 This is a schematic diagram illustrating the coarse grasping postures classified according to three underactuated dexterous hand grasping strategies used in this invention.

[0056] Figure 4 This is a schematic diagram of the underactuated dexterous hand grasping pose for three different objects generated by the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and beneficial effects of this invention clearer and more apparent, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0058] like Figure 1 As shown, a deep learning-based underactuated dexterous hand grasping pose generation method includes the following steps:

[0059] Step S1: Create the crawling dataset;

[0060] S1-1: The dataset for this invention is selected from 30 items in the YCB object dataset. This object dataset is collected from RGB-D images of real objects. The depth map is converted into point cloud data through preprocessing to fully reflect the three-dimensional structure of the object surface.

[0061] like Figure 2 As shown, Figure 2 -(a), 2-(b) and 2-(c) are RGB images and point cloud images of a bottle, a cup and a banana sampled from the real world, respectively;

[0062] S1-2: Based on the point cloud data collected in step S1-1, manually annotate the reasonable grasping pose of the target object. The annotation includes the rotation of the dexterous hand's palm during grasping. Peaceful relocation and the angles of each joint Each grasping gesture will be divided into one of three basic grasping types;

[0063] S1-3: Input the point cloud obtained in step S1-1 into the point cloud encoder Point Transformer, and extract the three-dimensional geometric feature vector of the target object through the point cloud self-attention mechanism. Size is ,in Indicates the number of groups defined by PointTransformer;

[0064] S1-4: Based on the target object's category name and appearance attributes, construct semantic cue words for the large language model. Use the cue word: "I want to grab a [object label]. First, provide its object category, then describe its shape in detail to successfully grasp it." Input the cue word into the large language model to obtain a textual response that includes object functional information and human grasping intuition.

[0065] S1-5: Encode the text response output by the large language model in step S1-4 using a pre-trained BERT-large-uncased model, extract the tokens from each sentence and perform max pooling on them. This will yield a semantic feature vector containing rich prior knowledge from the large language model. Size is .

[0066] Step S2: Capture the strategy classification network;

[0067] S2-1: Apply the geometric features obtained in step S1 and semantic features By splicing, we get a size of Joint feature input This includes both the object's three-dimensional structural information and semantic functional information;

[0068] S2-2: Combine the joint features obtained in step S2-1 The grasping strategy classification module obtains the grasping type from the input. This module uses an MLP classification network with stacked fully connected layers and ReLU activation functions to reduce the dimensionality and encode the joint features, extracting deep feature representations to distinguish different grasping strategies. A Sigmoid activation function is used as the output layer of the classification network to output independent confidence values ​​for the three grasping strategies (two-finger pinch, three-finger pinch, and five-finger grasp) to reflect the feasibility of each strategy on the current object. Finally, the grasping type with the highest confidence is selected as the output. Proceed to the next layer. During the training phase, binary cross-entropy loss is used. Calculate the error between the prediction result and the true label for each crawling strategy. The mathematical expression is as follows:

[0069]

[0070] in Indicates the number of samples; This indicates the capture quality label; 1 indicates success and 0 indicates failure. This represents the model's predicted grasping quality score, output after passing through the Sigmoid function. During training, the network uses backpropagation to optimize parameters, learning the mapping relationship between input features and grasping strategies. First, the loss is calculated, and the weights are updated using gradient descent. Backpropagation then uses the chain rule to propagate the error forward from the output layer, adjusting parameters layer by layer to minimize the loss function. Through iterative training, the model gradually optimizes its classification decisions, improving classification accuracy.

[0071] like Figure 3 As shown, Figure 3 -(a), 3-(b) and 3-(c) respectively demonstrate the coarse grasping postures of three grasping strategy types: two-finger pinch, three-finger pinch, and five-finger grip.

[0072] Step S3: Capture pose to generate network;

[0073] S3-1: Crawling strategy category obtained in step S2 Read the rough hand configuration parameters corresponding to the crawling type from the preset crawling template library. The coarse hand configuration includes preset joint angles for each joint of the underactuated dexterous hand. Based on the point cloud geometry of the target object, the center position of the object's point cloud or object mesh is calculated. This is used as the initial translation vector for grasping. During the training phase, zero-mean Gaussian noise is added to the rotation parameters of the actual grasping posture to obtain the initial rotation parameters. During the inference phase, rotation vectors are sampled from a uniform distribution as... ;

[0074] S3-2: Configure the rough hand type The center position of the object Initial rotation parameters and the joint features obtained in step S2 Together, they form the input vector for the grasping generation network, which is then fed into the grasping pose generation network. The grasping generation network encodes the input vector through a fully connected residual structure and outputs the joint angle correction. And obtain the fine hand shape configuration according to the following formula:

[0075]

[0076] The fine hand configuration is used to describe the fine joint angles of an underactuated dexterous hand.

[0077] The capture generator network simultaneously outputs translation corrections. The final capture translation vector is obtained according to the following formula:

[0078]

[0079] The capture-generating network further outputs rotation corrections. The final grasping rotation vector is obtained according to the following formula:

[0080]

[0081] The resulting translation vector Rotation vector and finger joint angle Combined into a complete grasping pose information of an underactuated dexterous hand:

[0082]

[0083] like Figure 4 Show, Figure 4 -(a), 4-(b) and 4-(c) respectively demonstrate the fine-grasping poses generated by the network model for three different grasping strategy types for three objects with different shapes.

[0084] Step S4: Additional loss term constraint;

[0085] S4-1: Obtain the grasping pose generated in step S3 The corresponding set of hand mesh vertices During network training, the URDF model (Uniform Robot Description Format) of the underactuated five-fingered dexterous hand used in this method is added to describe the connection relationships of the joints and links of the five-fingered dexterous hand robot. Importing this model into the simulation environment allows the robot's appearance and functions to be displayed on the computer. Based on the hand shape strategy s output in step S2-2, the set of fingertip vertices that frequently come into contact with the target object during grasping is selected. For each contact vertex Calculate the point cloud set from it to the target object. Find the minimum Euclidean distance and calculate the contact loss according to the following formula. :

[0086]

[0087] By minimizing this loss, the fingertips in the generated hand pose are brought close to and conform to the surface of the target object, thus improving the stability of the grasp.

[0088] Based on the rotation vector output in step S3-2 Determine the unit vector of the robotic arm's grasping approach direction. Calculate the unit vector of direction between the center of the object and the center of the palm. Orientation loss is calculated using the following formula. :

[0089]

[0090] This constrains the hand's orientation to align with the center of the object, thereby ensuring the robotic arm maintains a reasonable grasping direction when approaching the object and improving the physical feasibility of generating a grasping posture.

[0091] S4-2: Apply the binary cross-entropy loss obtained in step S2-2 The two types of geometric losses obtained in step S4-1 are weighted and combined according to preset weights to form the grasping optimization loss function during the training phase:

[0092]

[0093] In the formula These represent the weights of each loss function in the total loss. The crawling generation network is optimized through backpropagation to make the generated crawling poses more consistent with physical characteristics.

[0094] Step S5: Ablation Experiment and Comparative Experiment

[0095] S5-1: This method uses 50 objects from the YCB object dataset for experiments, including objects used in model training and new objects never seen by the model before. Two metrics are used to evaluate the grasping results: grasping ε-mass, which represents the radius of the largest six-dimensional sphere centered at the origin that can be enclosed within the convex hull of the moment space; this metric evaluates the stability and feasibility of the generated grasp. Cross-volume represents the penetration between the finger and the object; the goal is to minimize the cross-volume between the object and the finger to avoid grasping failure. During network training, a batch size of 32 is used, the initial learning rate is set to 0.005, and the total number of training epochs is 400. The weights of the loss function are set as follows: =1, =100, =1;

[0096] S5-2: To verify the effectiveness of the two additional losses added to the network by this method, the following ablation experiment was conducted: one of the loss functions was removed respectively ( The remaining losses were kept unchanged. For each configuration (removing contact loss, removing orientation loss), the model was retrained and evaluated using the same training and test sets. The experimental results are shown in the table below:

[0097]

[0098] The experimental results show that removing contact loss ( This leads to a significant decrease in gripping quality and an increase in cross-volume, indicating that contact loss is crucial to gripping accuracy. While removing other loss terms has a smaller impact, it still results in a decrease in gripping performance.

[0099] S5-3: To verify the performance difference between our proposed method and existing methods, a comparative experiment was conducted in a simulation environment. GraspIt! was chosen as the benchmark. Specifically, in the simulation environment, 800 grasping attempts were performed using both our proposed method and the GraspIt! method, and the quality and stability of the grasps generated by each method were statistically evaluated. The experimental results of the two grasping pose detection algorithms on the YCB test set are shown in the table below:

[0100]

[0101] The comparison results show that the method of this invention has significant advantages over GraspIt! in simulated environments. The model proposed by this method not only improves the quality of grasp generation but also effectively reduces the penetration area between the finger and the object, proving the effectiveness of this invention in practical applications.

[0102] Through ablation experiments and comparative experiments, the method of the present invention showed superior performance to the control method in terms of accuracy and stability of grasping posture generation, proving the effectiveness of the present method.

Claims

1. A method for generating grasping pose using an underactuated dexterous hand based on deep learning, characterized in that, It includes the following steps: Step S1: Create a crawling dataset. A crawling dataset is constructed based on the YCB object dataset. First, the RGB-D images of the objects in the dataset are preprocessed and converted into point cloud data. The obtained point cloud data is input into the Point Transformer to obtain the geometric features of the objects. Then, prompt words are input into the large language model according to the object category. Finally, the answers are text-encoded to obtain semantic feature vectors. Step S2: The geometric and semantic feature vectors obtained in step S1 are concatenated and fused to obtain joint features. The grasping strategy classification network is then used to predict the most suitable grasping type for the target object. The grasping strategy types include two-finger pinch, three-finger pinch, and five-finger grasp. Step S3: Input the grasping strategy category and joint features obtained in step S2 into the grasping generation network to predict the six-dimensional pose parameters and joint angles of the underactuated dexterous hand, and generate a grasping posture that matches the shape of the target object. Step S4: During the training phase, contact constraint loss and orientation constraint loss are applied to the grasping pose generated in step S3 to improve the executability and stability of the grasping pose, and generate the final six-DOF palm pose and fine joint angles of the fingers. Step S5 involves conducting ablation experiments for different loss function configurations and comparing them to demonstrate their feasibility and effectiveness.

2. The method for generating pose for grasping with an underactuated dexterous hand based on deep learning as described in claim 1, wherein step S1 is specifically as follows: Step S1-1: The YCB object dataset is collected from RGB-D images of real objects, and 30 items are selected from them. The depth map is converted into point cloud data through preprocessing to fully reflect the three-dimensional structure of the object's surface; Step S1-2: Based on the point cloud data collected in step S1-1, manually annotate the grasping pose of the target object. The annotation includes the rotation of the dexterous hand's palm during grasping. Peaceful relocation and the angles of each joint Each grasping gesture will be divided into one of three basic grasping types; Step S1-3: Input the point cloud data obtained in step S1-1 into the point cloud encoder Point Transformer, and extract the three-dimensional geometric feature vector of the target object through the point cloud self-attention mechanism. Size is ; Steps S1-4: Based on the category name and appearance attributes of the target object, construct semantic prompt words for the large language model, and input the prompt words into the large language model to obtain a text response containing object function information and human grasping intuition; Step S1-5: Encode the text response output by the large language model in Step S1-4. This will yield a semantic feature vector containing the prior knowledge of the large language model. Size is .

3. The method for generating pose for grasping with an underactuated dexterous hand based on deep learning as described in claim 1, wherein step S2 is specifically as follows: Step S2-1: Apply the geometric features obtained in step S1 and semantic features By splicing, we get a size of Joint feature input This includes both the object's three-dimensional structural information and semantic functional information; Step S2-2: Combine the joint features obtained in step S2-1 The grasping strategy classification module obtains the grasping type from the input. This module uses an MLP classification network with stacked fully connected layers and ReLU activation functions to reduce the dimensionality and encode the joint features, extracting deep feature representations to distinguish different grasping strategies. A Sigmoid activation function is used as the output layer of the classification network to output independent confidence values ​​for the three grasping strategies: two-finger pinch, three-finger pinch, and five-finger grasp, reflecting the feasibility of each strategy on the current object. Finally, the grasping type with the highest confidence is selected as the output. The network is then used in subsequent iterations; during the training phase, binary cross-entropy loss is employed. Calculate the error between the prediction result and the true label for each crawling strategy. The mathematical expression is as follows: ； in, Indicates the number of samples. This indicates that the quality label was retrieved; 1 indicates success, and 0 indicates failure. This represents the crawling quality score predicted by the model.

4. The method for generating pose for grasping with an underactuated dexterous hand based on deep learning as described in claim 1, wherein step S3 is as follows: Step S3-1: Based on the crawling strategy category obtained in step S2 Read the rough hand configuration parameters corresponding to the crawling type from the preset crawling template library. The rough hand configuration includes preset joint angles for each joint of the underactuated dexterous hand; based on the point cloud-semantic information of the target object. Calculate the center position of the object's point cloud. This is used as the initial translation vector for grasping; during the training phase, zero-mean Gaussian noise is added to the rotation parameters of the actual grasping posture to obtain the initial rotation parameters. During the inference phase, rotation vectors are sampled from a uniform distribution as... ; Step S3-2: Configure the rough hand type The center position of the object Initial rotation parameters and the joint features obtained in step S2 Together, they form the input vector of the grasping generation network and are fed into the grasping pose generation network; the grasping generation network encodes the input vector through a fully connected residual structure and outputs the joint angle correction. And obtain the fine hand shape configuration according to the following formula: ; The fine hand configuration is used to describe the fine joint angles of an underactuated dexterous hand; The capture generator network simultaneously outputs translation corrections. The final capture translation vector is obtained according to the following formula: ； The capture-generating network further outputs rotation corrections. The final grasping rotation vector is obtained according to the following formula: ； The resulting translation vector Rotation vector and finger joint angle Combined into a complete underactuated dexterous hand grasping pose information : .

5. The method for generating pose for grasping with an underactuated dexterous hand based on deep learning as described in claim 1, wherein step S4 is specifically as follows: Step S4-1: Obtain the grasping pose generated in step S3 The corresponding set of hand mesh vertices First, import the unified robot description format model of the underactuated dexterous hand into the network, and select the set of fingertip vertices that frequently come into contact with the target object during grasping. For each contact vertex Calculate the point cloud set from it to the target object. Find the minimum Euclidean distance and calculate the contact loss according to the following formula. : ； By minimizing this contact loss, the fingertips in the generated hand pose are brought close to and adhere to the surface of the target object, thus improving the stability of the grasp. Determine the unit vector of the robotic arm's grasping approach direction. Calculate the unit vector of direction between the center of the object and the center of the palm. Orientation loss is calculated using the following formula. : ； This constrains the hand's orientation to align with the center of the object, thereby ensuring the robotic arm maintains a reasonable grasping direction when approaching the object and improving the physical feasibility of generating a grasping posture. Step S4-2: Apply the binary cross-entropy loss obtained in step S2-2 The two types of geometric losses obtained in step S4-1 are weighted and combined according to preset weights to form the grasping optimization loss function during the training phase: ； The crawling generation network is optimized through backpropagation, making the generated crawling poses more consistent with physical properties.

6. The method for generating pose for grasping with an underactuated dexterous hand based on deep learning as described in claim 1, wherein step S5 is as follows: Step S5-1: In the simulated grasping experiment, the YCB object dataset was selected for the experiment, and two metrics were used to evaluate the grasping results: grasping ε-mass, which represents the radius of the largest six-dimensional sphere centered at the origin that can be enclosed within the convex hull of the torque space; and cross volume, which represents the penetration between the finger and the object. The goal is to minimize the cross volume between the object and the finger to avoid grasping failure. Step S5-2: To verify the effectiveness of the two additional losses added to the network, an ablation experiment is designed: one of the loss functions is removed respectively. ), and keep the remaining loss unchanged; for each configuration, retrain the model and evaluate it using the same training set and test set; Step S5-3: Select a benchmark and conduct 800 crawling attempts in a simulation environment for comparison, and statistically evaluate the quality and stability of the crawls generated by each method.

Images