Three-dimensional object double-dexterous-hand grabbing generation method based on depth diffusion model

Abstract

The invention provides a three-dimensional object double-dexterous-hand grabbing generation method based on a deep diffusion model, relates to the technical field of robot grabbing, and automatically realizes generation of double-hand grabbing gestures facing dexterous hands by utilizing a deep learning technology. The method specifically comprises the steps of object three-dimensional point cloud input, depth VAE generation model two-hand grabbing parameter prediction, VAE feature division decoding, diffusion model feature denoising training, high feature quality two-hand grabbing gesture acquisition and the like, and can automatically generate two-hand grabbing gestures based on three-dimensional object point cloud data and a deep learning model. And the purposes of precision, intelligence and high efficiency of the grabbing process of the double dexterous hands are achieved.

Description

TECHNICAL FIELD

[0001] The application relates to the field of robot grasping technology, and in particular to a three-dimensional object double-dexterous hand grasping generation method based on a deep diffusion model. BACKGROUND

[0002] With the rapid development of technology, robot technology has been widely applied in various fields such as industry, agriculture, medical treatment and home service, which has significantly promoted the improvement of production efficiency and the improvement of life quality. Among them, the grasping technology of the robot is the key link to realize the autonomous operation of the robot and complete complex tasks. However, most of the traditional robot grasping methods are based on pre-set models and fixed grasping schemes, and for different types and shapes of objects, the grasping effect often cannot achieve a relatively ideal effect. Therefore, how to realize more accurate and stable grasping for objects has become an important problem in the research of robot grasping technology. In recent years, deep learning technology has achieved a dominant position in many research fields, including computer vision, speech recognition and natural language processing. Diffusion model is one of the latest generation models in deep learning technology, which has reached the most advanced level in generation effect. With the continuous development of deep learning, computer vision and other technologies in the field of robot application, the robot grasping generation method based on deep learning model has gradually become a research hotspot. However, the existing grasping generation method based on deep model and the related large-scale data set mainly serve the single-hand grasping task, and the research and data of double-hand grasping are relatively less. In actual application scenarios, compared with double-hand grasping, single-hand grasping often cannot provide enough stable and balanced grasping posture, especially for heavy or irregular-shaped objects, single hand may be difficult to stably grasp the object, and the object may easily slip or be unstable. In addition, the range and flexibility of single-hand grasping are also limited. Some objects may need to be grasped and operated better by two hands, and single-hand grasping may not achieve ideal results. Double-hand grasping has higher flexibility and stability than single-hand grasping, and double-hand grasping can utilize the synergistic effect of two hands to realize more stable grasping and more delicate operation of the object. Therefore, combined with deep diffusion model and double-hand grasping technology, it is expected to realize higher-level robot grasping operation, and the research of robot double-hand grasping method based on deep diffusion model has important theoretical value and practical significance.

[0003] The current robot double-hand grasp generation method has the following problems: (1) The large-scale grasp database is collected in the case of single-hand grasp, and lacks double-hand grasp data; due to the lack of true value data, the existing double-hand grasp method can only be based on an optimization model, and the calculation efficiency is low and cannot be widely used. (2) The existing deep grasp generation method is only suitable for single-hand grasp, and directly extending the single-hand grasp generation method to double hands will cause the left and right hands to overlap, that is, there is a serious collision between the two hands. The core reason for this problem is that the single-hand model only models the interaction between the single dexterous hand and the object, and directly extending the single-hand generation model to the double-hand generation model will generate the double-hand grasp parameters as a single high-dimensional vector, which cannot distinguish the independent motion characteristics of the left and right hands, resulting in the generated left and right hand grasp parameters being prone to position interference in the three-dimensional space, for example, the distance between the left hand palm center coordinate and the right hand palm center coordinate is less than the mechanical structure safety distance, or the finger joint motion trajectory intersects, which ultimately causes a serious collision between the two hands. This also leads to poor quality of the double-hand deep features extracted by the single-hand grasp generation method, which cannot directly serve the actual robot double-hand grasp generation task. SUMMARY

[0004] The application provides a three-dimensional object double-dexterous hand grasp generation method based on a deep diffusion model, which aims to solve the problems in the existing robot double-hand grasp generation method, such as the lack of double-hand grasp data, the collision between the left and right hands when directly extending the single-hand grasp generation method to double hands, and the poor quality of the double-hand deep features extracted, which cannot directly serve the actual robot double-hand grasp generation task.

[0005] The application provides a three-dimensional object double-dexterous hand grasp generation method based on a deep diffusion model, which includes the following steps: Step S1: Obtain a double-dexterous hand grasp true value data set; Step S2: Input the three-dimensional object point cloud and the corresponding double-hand grasp true value into a double-hand variational autoencoder generation VAE model, extract double-hand grasp features through a neural network, and split the feature space according to the left and right hands, and output predicted double-hand grasp parameters based on the left and right hand features; Step S3: Take the double-hand grasp feature distribution learned by the double-hand VAE model as the target, and use the diffusion model of the feature space to sample from Gaussian noise to generate denoised double-hand grasp features; Step S4: Split the denoised double-hand grasp features according to the rules of step S2, and input them into the decoder of the double-hand VAE model to obtain optimized double-hand grasp parameters; Step S5: Input the three-dimensional object point cloud and the optimized double-hand grasp parameters into a classifier network to output a label indicating whether the grasp is successful or not; Step S6: input the three-dimensional object grid and the bimanual grasping parameters labeled as grasping success into the grasping optimization module, take the hand-object penetration depth and contact distance as the objective function, fine-tune the grasping parameters through gradient descent method, and output the final optimized bimanual grasping result.

[0006] Further, the specific method of obtaining the bimanual grasping ground truth data set in step S1 comprises: inputting the object three-dimensional grid model in the bimanual data set into the optimization module of the left and right hand dexterous manipulator differentiable model, optimizing the hand-object penetration and force closure energy function as the objective function, and obtaining the bimanual grasping ground truth , , wherein is the manipulator spatial displacement parameter, is the manipulator spatial rotation quaternion parameter, is the manipulator joint value parameter; and using the physical simulation environment screening to calculate the grasping success label , specifically comprising: Step S11: inputting the object three-dimensional grid model in the bimanual data set into the optimization module of the left and right hand dexterous manipulator differentiable model, randomly taking a set of right hand grasping parameters from the existing single hand grasping data set, and taking random parameters as the optimization initial value; Step S12: optimizing the left hand grasping parameters, adjusting the left hand grasping parameters, and iterating times to end the optimization and obtain the bimanual grasping ground truth; Step S13: inputting the object three-dimensional grid model and the optimized bimanual grasping parameters into the physical simulation environment, picking up the object based on the imported grasping gesture, and if the object does not fall, it is grasped successfully, and is recorded.

[0007] Further, the specific method of inputting the three-dimensional object point cloud and the corresponding bimanual grasping ground truth into the bimanual VAE model through the encoder to extract the bimanual grasping features and split the feature space according to the left and right hands in step S2 comprises: Step S21: inputting the three-dimensional point cloud and the corresponding bimanual grasping ground truth parameters into the bimanual VAE model, using an encoder composed of a neural network to extract dimensional bimanual grasping features related to the object; Step S22: split the bimanual grasping features , the first dimensional features are left hand grasping features , and the last left hand grasp feature ; Step S23: using the decoder composed of two neural networks, the left hand grasp feature input into the left hand grasp parameter decoder, output the predicted left hand grasp parameter , right hand grasp feature input into the right hand grasp parameter decoder, output the predicted right hand grasp parameter .

[0008] Further, the specific method of generating the denoised double-hand grasp feature by taking the double-hand grasp feature distribution learned by the double-hand VAE model in step S3 as the target and sampling from Gaussian noise using the diffusion model of the feature space includes: Step S31: take out the double-hand distribution extracted by the trained VAE, calculate the difference between the double-hand distribution and the standard Gaussian distribution based on the set hyperparameters, and obtain the noise true value ; Step S32: extract a feature from the double-hand distribution and input it into a neural network to predict the noise value that needs to be removed from the standard Gaussian noise to restore the feature ; Step S33: use the Gaussian noise true value to constrain the diffusion model training, and the loss function of the training is: wherein, is the norm.

[0009] Further, the specific method of cutting the denoised double-hand grasp feature according to the rules of step S2 and inputting it into the decoder of the double-hand VAE model to obtain the optimized double-hand grasp parameter in step S4 includes: Step S41: extract a random feature from the standard Gaussian noise and input it into the diffusion model to obtain the double-hand grasp feature restored by removing noise ; Step S42: cut into left hand grasp feature and right hand grasp feature , and input them into the left hand and right hand grasp parameter decoders and respectively to obtain the denoised grasp parameter .

[0010] Further, the specific method of inputting the three-dimensional object point cloud and the optimized double-hand grasp parameter into the classifier network to output the label of successful or failed grasping in step S5 includes: Step S51: input the three-dimensional object point cloud and the two-hand grasp parameters in series, the classifier composed of the residual network, and output the predicted grasp success label ; Step S52: with the grasp success label true value constraint the classifier network training, and the loss function of the training is: .

[0011] Further, the specific method of inputting the three-dimensional object grid and the two-hand grasp parameters labeled as grasp success into the grasp optimization module in step S6, taking the hand-object penetration depth and contact distance as the objective function, and fine-tuning the grasp parameters through the gradient descent method to output the final optimized two-hand grasp result includes: Step S61: input the three-dimensional object grid and the predicted grasp parameters , calculate the two-hand-object penetration depth, left-hand and right-hand penetration depth, and contact distance between the two hands and the object; Step S62: calculate the optimized two-hand grasp parameters using the gradient descent method; Step S63: after optimization M times, output the final two-hand grasp parameters .

[0012] Compared with the prior art, the present application has the following beneficial effects: 1. The present application and the previous two-hand grasp method can only be calculated based on the optimization method, and the present application proposes a large-scale two-hand grasp dataset, which can use a deep model to directly predict two-hand grasp parameters.

[0013] 2. Based on the data, the present application designs a two-hand generation model, which can distinguish left and right hands in the feature space, and solves the problem of two-hand collision when a single-hand grasp generation model is applied to a two-hand grasp task.

[0014] (3) The present application designs a diffusion model in the feature space, uses a deep diffusion model to improve the quality of two-hand grasp features, and realizes high-precision two-hand grasp generation. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the exemplary embodiments of the present application will be readily understood through reading the detailed description of the exemplary embodiments of the present application below, with reference to the accompanying drawings. In the drawings, several embodiments of the present application are shown by way of example and not limitation, in which the same or corresponding reference numbers indicate the same or corresponding parts, wherein: Figure 1 is the overall structure flowchart designed by the present application; Figure 2 is a two-handed grasp generation model flowchart designed by the present application; Figure 3 is a two-handed grasp feature denoising diffusion model flowchart designed by the present application; Figure 4 is an encoder and decoder structure flowchart of the two-handed grasp generation model designed by the present application; Figure 5 is a grasp quality prediction model flowchart designed by the present application; Figure 6 is a visualization result map of the test instance. DETAILED DESCRIPTION

[0016] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application can be more thoroughly understood and so that the scope of the present application can be conveyed to those skilled in the art. If not specifically stated, the technical means used in the examples are conventional means known to those skilled in the art.

[0017] The present application provides a three-dimensional object two-handed dexterous grasp generation method based on a deep diffusion model, the steps are as follows, and the following will supplement the "implementation purpose and technical effect" for each step and detail the solving process of the "left and right hand collision" problem: Step S1: generating a three-dimensional mesh model of an object in a two-handed dataset and optimizing the left and right hand dexterous robot micromodel input optimization module to obtain two-handed grasp true values with hand-object penetration and force-closed energy functions as objective functions wherein , is a robot spatial displacement parameter, is a robot spatial rotation quaternion parameter, is a robot joint value parameter; and using a physical simulation environment to filter and calculate a grasp success label indicates that the grasp is successful, indicates that the grasp fails; Purpose of step S1: existing two-handed grasp datasets are scarce and of uneven quality (e.g., containing a large amount of invalid data that does not meet physical constraints), and direct use will cause subsequent model training bias. Step S1 builds a high-quality two-handed dataset through "optimization generation + simulation verification" to provide reliable true value reference for subsequent models.

[0018] ​Step S1 technical effect: Compared with real scene collection, generating data through simulation environment can reduce hardware loss and safety risk, and can generate more than 100,000 groups of data (10,000 groups are constructed in this embodiment, including 5,000 groups of positive samples); fixing the right hand parameters and adjusting the left hand parameters during optimization can avoid left and right hand collision in the initial stage and reduce invalid optimization; physical simulation verification ensures that the grabbing parameters in the data set meet the physical constraints of "no hand-object penetration, no double-hand collision, and no object falling", and improves the physical feasibility of the data.

[0019] Further, the step S1 is specifically: Step S11: input the object three-dimensional mesh model M in the two-hand data set and the left and right hand dexterous manipulator differentiable model into the optimization module, randomly take out a group of right hand grabbing parameters from the existing single-hand grabbing data set, and take random parameters as the optimization initial value; Step S12: optimize with the hand-object penetration and force closed energy function as the objective function, adjust the left hand grabbing parameters, and end the optimization after K iterations to obtain the two-hand grabbing true value; Step S13: import the object three-dimensional mesh model M and the optimized two-hand grabbing parameters into the physical simulation environment, grab the object based on the imported grabbing gesture, if the object does not fall, it is considered as a successful grabbing, and , if the object falls, it is considered as a failed grabbing, and ; Step S2: as shown in Figure 2 , input the three-dimensional point cloud of the object and the corresponding two-hand grabbing true value parameters into the two-hand VAE model, and cut the feature space to respectively output the predicted two-hand grabbing parameters; Step S2 implementation purpose: solve the problem of "left and right hand collision caused by directly extending single-hand model to two hands": the single-hand model does not distinguish left and right hand features, and generates two-hand parameters as a single high-dimensional vector, which is easy to cause collision between two hands; this step solves the collision problem caused by the proximity of left and right hand parameters by "feature cutting + independent decoding".

[0020] Step S2 technical effect: after cutting the feature space, the left and right hand feature spaces are completely independent, and the model can learn the motion characteristics of the left and right hands (such as left hand assisting balance and right hand dominating grabbing) respectively, avoiding confusion of two-hand features; the independent decoder outputs parameters, ensuring that the left hand parameters are determined only by the left hand features, and the right hand parameters are determined only by the right hand features, further solving the problem of two-hand collision.

[0021] Further, the step S2 is specifically: Step S21: input the three-dimensional point cloud and the corresponding two-hand grasping true value parameters into the two-hand VAE model, and use a 4-layer neural network module to extract the object-related two-hand grasping features ; ; Step S22: split the two-hand grasping features , the front dimension as the left-hand grasping features , and the rear dimension as the right-hand grasping features ; Step S23: use a decoder composed of two neural network modules, the left-hand grasping features input into the left-hand grasping parameter decoder, and the output is the predicted left-hand grasping parameters , and the right-hand grasping features input into the right-hand grasping parameter decoder, and the output is the predicted right-hand grasping parameters ; Step S3: as shown in Figure 3 , the two-hand grasping features learned by the VAE are taken out, and based on the diffusion model of the feature space, the de-noised two-hand grasping features learned from the Gaussian noise are obtained. Step S3 implementation purpose: the VAE model has a “prior distribution deviation”, and the generated primary features are prone to contain noise, which causes slight position deviation of the decoded two-hand parameters (such as abnormal finger joint angles), and long-term cumulative deviation may cause two-hand parameter generation errors; this step de-noises through the diffusion model to optimize the feature quality and ensure that the features meet the true value two-hand grasping distribution.

[0022] Step S3 technical effect: after de-noising through the diffusion model for multiple iterations, the feature noise error can be significantly reduced, and the accuracy of the subsequent decoded two-hand parameters is improved.

[0023] Further, the step S3 specifically comprises: Step S31: take out the primary two-hand distribution extracted by the trained VAE, calculate the difference between the primary two-hand distribution and the standard Gaussian distribution based on the set hyperparameters, and obtain the noise true value ; Step S32: extract the features from the primary two-hand distribution, input the features into the diffusion model network module, and predict the noise value that needs to be removed from the standard Gaussian noise for the features ; Step S33: use the Gaussian noise true value to constrain the diffusion model training, and the loss function of the training is: wherein For norm, is the true value of the Gaussian noise, is the predicted value of the Gaussian noise output by the diffusion model.

[0024] Step S4: the de-noised double-hand grasping features are input into the trained double-hand decoder VAE respectively to obtain the final double-hand grasping parameters; The purpose of step S4 is to convert the de-noised high-quality features into specific double-hand grasping parameters, while continuing the "feature segmentation" strategy of step S2 to ensure that the de-noised features still maintain the independence of the left and right hands, avoiding collision problems caused by feature fusion.

[0025] The technical effect of step S4 is that the hand-object penetration depth of the de-noised parameters is significantly reduced compared to the parameters directly output by the VAE, and the distance between the two hands meets the mechanical safety requirements.

[0026] Further, the step S4 specifically comprises: Step S41: a random feature is extracted from the standard Gaussian noise , and the de-noised double-hand grasping features are obtained by inputting the diffusion model ; Step S42: the de-noised double-hand grasping features are segmented into left-hand grasping features and right-hand grasping features, and are respectively input into the left-hand and right-hand grasping parameter decoders to obtain the de-noised grasping parameters ; Step S5: as shown in Figure 5 , the three-dimensional object point cloud and the double-hand grasping parameters are input into the classifier network to output the labels of successful or failed grasping to predict the grasping quality; The purpose of step S5 is to filter out physically feasible double-hand grasping parameters. Some de-noised parameters may still have problems such as insufficient hand-object contact and joint angle out-of-bounds, which may fail directly. This step filters high-success-rate parameters through the classifier to reduce invalid calculations in subsequent optimization.

[0027] The technical effect of step S5 is that the classifier network can effectively filter parameters that "appear to have no collision but actually fail to grasp", and reduce invalid inputs for the subsequent optimization module, improving the overall efficiency of the algorithm.

[0028] Further, the step S5 specifically comprises: Step S51: the three-dimensional object point cloud and the double-hand grasping parameters are concatenated and input into the classifier network to output the predicted grasping success label ; Step S52: with the success of grabbing the label true value The constraint classifier network training, the loss function of training is: Wherein, The success of grabbing the label true value, The success of grabbing the label prediction value output by the classifier network.

[0029] Step S6: input the three-dimensional object grid And the predicted double-hand grabbing parameters Into the grabbing optimization module, taking the double-hand object penetration depth, left-hand and right-hand penetration depth and the contact distance between the double hands and the object as the objective function, fine-tuning the predicted double-hand grabbing parameters, and outputting the final double-hand grabbing result ; The purpose of step S6 is to further eliminate potential physical motion failure problems: even after the previous steps, the parameters may still have slight deviations (such as the left-hand little finger joint angle being slightly abnormal), and this step fine-tunes through gradient descent to ensure that the final parameters fully meet the physical constraints.

[0030] The technical effect of step S6 is that after optimization, various physical indicators of the result meet the standard requirements, and the success rate of the final result is improved.

[0031] Further, the step S6 is specifically: Step S61: input the three-dimensional object grid And the predicted grabbing parameters Calculate the double-hand object penetration depth, left-hand and right-hand penetration depth, and the contact distance between the double hands and the object; Step S62: use gradient descent method to calculate the optimized double-hand grabbing parameters; Step S63: after optimization M times, output the final double-hand grabbing parameters .

[0032] The VAE and classifier in steps S2 and S5 described above are expansion models based on FFHNet, which are only preferred embodiments of the present application. Any equivalent changes and modifications made within the scope of the patent application of the present application shall be included in the scope of the present application. Embodiment

[0033] (I) Experimental purpose Verify the superiority of the double-hand grabbing generation method based on the deep diffusion model proposed in the present application in terms of grabbing quality, diversity and physical feasibility, focus on solving the collision problem when the traditional method is expanded to the double-hand task, and quantitatively evaluate the generalization ability of the method to different categories of objects.

[0034] (II) Experimental Environment and Data Preparation Hardware environment: NVIDIA RTX 4090 GPU, ensuring efficient execution of large-scale data processing and model training and inference.

[0035] Software environment: PyTorch deep learning framework, Isaac Gym physics simulation platform (used for verifying the success rate of the capture) and Python 3.9 code environment.

[0036] Dataset: The dual-dexterity hand grasping dataset constructed using this invention contains 300 everyday objects (covering 6 major categories including tools, containers, and electronic devices), totaling 100,000 sets of two-hand grasping data (50,000 positive samples + 50,000 negative samples), divided into a training set (240 objects) and a test set (60 objects) at a 4:1 ratio. All object sizes are normalized to 20-30cm to suit the operational range of dual-dexterity hands.

[0037] (III) Comparison Methods and Evaluation Indicators Four mainstream crawling generation methods were selected and extended to two-handed tasks as comparison methods to ensure fairness in the comparison (all methods were retrained on the dataset of this invention): GraspTTA: A single-hand generation method based on CVAE, which directly extends to a two-hand feature space; GraspLDM: A one-handed method that integrates CVAE and diffusion models, without a two-handed feature splitting mechanism; Ø UGG: A pure diffusion model for generating grasping data, which directly diffuses within the parameter space of both hands; Ø DDG: A discriminative single-hand generation method based on Transformer, extended to bi-handed parallel prediction.

[0038] This invention sets five evaluation indicators from three dimensions: grasping quality, physical feasibility, and diversity. Ø Scraping success rate ( The percentage of successful samples in physical simulations where the object does not fall and the maximum penetration depth is <0.5cm is as high as possible. Ø Penetration depth of hand object ( ): The maximum penetration depth between the hand contact point and the object surface (unit: cm), the smaller the better; Ø Penetration depth of both hands ( ): Average penetration depth between the contact points of the left and right hands (unit: cm), the smaller the better; Ø Postural diversity: Quantified by the standard deviation of parameters obtained from left and right hand grasping, including: Spatial location standard deviation ( Rotational Quaternion Standard Deviation Joint Angle Standard Deviation The greater the value, the stronger the diversity.

[0039] (IV) Experimental steps Test steps of the present application: Ø The three-dimensional model point cloud of the object is first input into the double-hand VAE network to obtain initial double-hand features; Ø The initial double-hand features are input into the feature diffusion model to obtain high-quality denoised double-hand features; Ø Based on the denoised double-hand features, the double-hand grasping posture is decoded and input into the optimization module for fine-tuning; Ø All fine-tuned double-hand grasping results are input into the grasping quality classifier to retain the successful grasping results.

[0040] Comparative experiment test process: For each test object, each method generates 100 groups of double-hand grasping parameters, a total of 5000 test samples; the generated parameters are imported into the Isaac Gym simulation environment to perform grasping verification (6 gravity directions, object lifting 15cm, lasting 100 simulation steps); record each evaluation index, and perform visual analysis on abnormal results (such as collision, penetration).

[0041] Table 1 and Figure 6 are the comparative numerical results and visual results of the test examples: Table 1 is the comparative numerical results of the test examples The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.​​

Claims

1. A three-dimensional object dual dexterous hand grasping generation method based on a deep diffusion model, the dual dexterous hand grasping generation method comprising the following steps: Step S1: obtaining a dual dexterous hand grasping ground truth dataset; Step S2: inputting a three-dimensional object point cloud and corresponding dual hand grasping ground truth into a dual hand variational autoencoder to generate a VAE model, extracting dual hand grasping features through a neural network, and dividing the feature space according to left and right hands, and outputting predicted dual hand grasping parameters based on left and right hand features; Step S3: taking the dual hand grasping feature distribution learned by the dual hand VAE model as a target, sampling from Gaussian noise using a diffusion model of the feature space, and generating denoised dual hand grasping features; Step S4: dividing the denoised dual hand grasping features according to the rules of step S2, inputting them into the decoder of the dual hand VAE model, and obtaining optimized dual hand grasping parameters; Step S5: inputting the three-dimensional object point cloud and the optimized dual hand grasping parameters into a classifier network, and outputting a label indicating whether the grasping is successful or not; Step S6: inputting a three-dimensional object mesh and dual hand grasping parameters with a label indicating successful grasping into a grasping optimization module, taking hand-object penetration depth and contact distance as objective functions, and fine-tuning the grasping parameters through gradient descent method to output the final optimized dual hand grasping result.

2. The method of claim 1, wherein the method further comprises: The specific method for obtaining the dual dexterous hand grasping ground truth dataset in step S1 comprises: Object three-dimensional mesh model in double-hand dataset Optimize the left and right hand dexterous manipulator with the input optimization module of the differentiable model, and take the hand object penetration and force closure energy function as the objective function to obtain the true value of the double-hand grabbing , wherein , is the spatial displacement parameter of the manipulator, is the spatial rotation quaternion parameter of the manipulator, is the joint value parameter of the manipulator; and a physical simulation environment is used for screening and calculating the grabbing success label , specifically comprising: Step S11: the object three-dimensional mesh model in the two-hand dataset With the left and right hand dexterous manipulator differentiable model input optimization module, a set of right hand grasping parameters is randomly taken out from the existing single hand grasping dataset, and the left hand takes random parameters as the optimization initial value; Step S12: optimize the left hand grasp parameters by taking the hand object penetration and force closure energy function as the objective function, iterate After that, the optimization is finished, and the double-hand grasp true value is obtained. Step S13: importing the optimized two-hand grasping parameters into the physical simulation environment, and grasping the object based on the imported grasping gesture , and if the object does not fall, the grasping is successful, and the grasping success is recorded , and if the object falls, the grasping fails, and the grasping failure is recorded .

3. The method of claim 1, wherein the method further comprises: The specific method for inputting the three-dimensional object point cloud and corresponding dual hand grasping ground truth into the dual hand VAE model to extract dual hand grasping features and divide the feature space according to left and right hands in step S2 comprises: Step S21: input the three-dimensional point cloud and the corresponding double-hand grasping ground truth parameters into the double-hand VAE model, use an encoder composed of a neural network to extract the double-hand grasping features related to the object double-hand grasping features ; Step S22: Segmentation of two-handed grasp features , front dimension as left-hand grasp feature , back dimension as right-hand grasp feature ; Step S23: Decoder composed of two neural networks, left-hand grasp feature Input left-hand grasp parameter decoder, output predicted left-hand grasp parameters , right-hand grasp feature Input right-hand grasp parameter decoder, output predicted right-hand grasp parameters .

4. The method of claim 1, wherein the method further comprises: The specific method for taking the dual hand grasping feature distribution learned by the dual hand VAE model as a target, sampling from Gaussian noise using a diffusion model of the feature space, and generating denoised dual hand grasping features in step S3 comprises: Step S31: The bimanual distribution extracted by the trained VAE is taken out, and the difference between the bimanual distribution and the standard Gaussian distribution is calculated based on the set hyperparameters, to obtain the noise true value ; Step S32: extracting features from the two-hand distribution , inputting a neural network, predicting the noise value to be removed for the feature to be restored from the standard Gaussian noise ; Step S33: Using the Gaussian noise true value The loss function of the constraint diffusion model training is: wherein is norm.

5. The method of claim 1, wherein the method further comprises: The specific method for dividing the denoised dual hand grasping features according to the rules of step S2, inputting them into the decoder of the dual hand VAE model, and obtaining optimized dual hand grasping parameters in step S4 comprises: Step S41: extracting a random feature from standard Gaussian noise , inputting the diffusion model to obtain the double-hand grasping feature after removing noise and restoration ; Step S42: inputting the left-hand and right-hand grasp parameters into the left-hand and right-hand grasp parameter decoders, respectively Split into left-hand grasp features and right-hand grasp features , respectively input left-hand and right-hand grasp parameter decoders and , to obtain the denoised grasp parameters .

6. The method of claim 1, wherein the method further comprises: The specific method for inputting the three-dimensional object point cloud and the optimized dual hand grasping parameters into the classifier network, and outputting a label indicating whether the grasping is successful or not in step S5 comprises: Step S51: obtaining a three-dimensional object point cloud and two-hand grasp parameters in series, inputting a classifier composed of a residual network, and outputting a predicted grasp success label ; Step S52: label ground truth with successful grasp The constraint classifier network training, the loss function of training is: 。 7. The method of claim 1, wherein the method further comprises: The specific method for inputting a three-dimensional object mesh and dual hand grasping parameters with a label indicating successful grasping into a grasping optimization module, taking hand-object penetration depth and contact distance as objective functions, and fine-tuning the grasping parameters through gradient descent method to output the final optimized dual hand grasping result in step S6 comprises: Step S61: grid the three-dimensional object and predicted grasp parameters input, compute the two-hand pair object penetration depth of the two hands and the object, the left hand and right hand penetration depths, and the contact distance of the two hands and the object; Step S62: calculating the optimized dual hand grasping parameters using the gradient descent method; Step S63: After the M times of optimization, output the final two-hand grasping parameters .

Images