Dexterous hand grasping pose generation method and system based on CVAE and ball query algorithm

Abstract

This invention discloses a method and system for generating grasping postures of a multi-fingered dexterous hand based on CVAE and Ball Query algorithms, belonging to the field of robot grasping control technology. The method includes: (1) a data sampling step; (2) a data preprocessing step: standardizing the collected data to generate enhanced point cloud data and constructing a training dataset; (3) a model training step: using a multi-scale feature extraction module combined with global semantics and local geometric features extracted by the Ball Query algorithm, generating grasping postures using a conditional variational autoencoder, and optimizing model parameters through reconstruction loss and KL divergence; (4) a real-time deployment step: integrating the trained model into a physical platform, filtering candidate grasping postures based on parallel collision detection, and realizing real-time grasping control by combining inverse kinematics verification. This invention effectively solves the problem of insufficient generalization ability of traditional methods in grasping complex objects, and achieves adaptive grasping of unknown objects while ensuring grasping stability.

Description

Technical Field

[0001] This invention relates to the field of robot grasping control technology, specifically to a method and system for generating grasping poses of a dexterous hand based on CVAE and Ball Query algorithms, which is particularly suitable for adaptive grasping operations of unknown objects in scenarios such as factories and homes. Background Technology

[0002] Current deep learning-based grasping generation methods face two major challenges: First, variational autoencoders (VAEs) rely on unconstrained random sampling to generate candidate grasping poses, leading to a large number of samples failing due to collisions or kinematic inaccessibility by the robotic arm. This necessitates computationally expensive post-processing for filtering, severely limiting real-time performance. Second, existing methods typically use local point clouds of the object as input, failing to explicitly model the object's global pose in the robot's base coordinate system. This results in spatial misalignment of the generated grasping poses and a significant decrease in robustness in partially occluded scenarios due to missing geometric information. Traditional solutions, while partially mitigating these issues through iterative optimization via physical simulation or post-processing correction, suffer from high computational complexity and decoupling from the generative model, making efficient end-to-end collaboration difficult. Summary of the Invention

[0003] To address the aforementioned problems, this invention proposes a method and system for generating poses for dexterous hand grasping based on CVAE and Ball Query algorithms. This effectively solves the problem of insufficient generalization ability of traditional methods in grasping complex objects, and achieves adaptive grasping of unknown objects while ensuring grasping stability.

[0004] The technical solution of the present invention is as follows:

[0005] (1) Data sampling steps: Configure the models of the dexterous hand and the target object in the simulation environment, plan the grasping posture of the dexterous hand through the circumferential sampling path, trigger the grasping action and verify the grasping stability, and record the posture, joint angle and contact force data of successful grasping.

[0006] Load the multi-joint degree-of-freedom model of the dexterous hand and the target object model into the simulation environment, and bind the Mocap point to the palm of the dexterous hand to control the global motion;

[0007] Plan a circular sampling path for the dexterous hand around the Z-axis of the target object, and generate 100 sampling poses at fixed angle intervals;

[0008] When the contact force of the dexterous hand's palm exceeds the threshold, a grasping action is triggered, controlling the thumb to close to a preset angle, while the other four fingers apply a constant torque to wrap the object;

[0009] After grasping, the dexterous hand is vertically lifted and the object's displacement is monitored to determine whether the grasp was successful. Valid data is recorded and all sampling points are iterated over in a loop.

[0010] (2) Data preprocessing steps: Standardize the collected data to generate enhanced point cloud data and construct a training dataset;

[0011] The position coordinates of the dexterous hand and the object are normalized and scaled to the range of [-1,1].

[0012] The original point cloud of the target object was randomly downsampled to 2048 key points, and three-dimensional Gaussian noise was superimposed and some areas were randomly erased to simulate occlusion.

[0013] The normalized capture parameters are bound to the enhanced point cloud, divided into training, validation and test sets according to the proportions, and stored in a layered compressed HDF5 format.

[0014] (3) Model training steps: The grasping posture is generated by combining the global semantics and the local geometric features extracted by the Ball Query algorithm through the multi-scale feature extraction module, and the model parameters are optimized by the reconstruction loss and KL divergence.

[0015] The multi-scale feature extraction module specifically includes:

[0016] Global semantic features of point clouds are extracted through hierarchical set abstraction operations;

[0017] Local geometric features are extracted based on the Ball Query algorithm, including the coordinate offset of the center point, the angle between the normal vectors, and the curvature difference when querying the spherical neighborhood.

[0018] The global and local features are concatenated into a fused feature, which is then input into the conditional variational autoencoder.

[0019] The conditional variational autoencoder specifically includes:

[0020] Encoder: It concatenates the fused features, normalized grasping pose, and object pose as input, and generates the mean and variance of latent variables through a multilayer perceptron;

[0021] Decoder: It concatenates the latent variables with the conditional features mapped to the object pose, reconstructs the grasping pose through a multilayer perceptron, and outputs the position constraint to the interval [-1,1] and the unit quaternion;

[0022] Loss function: combines position error, quaternion direction cosine difference and KL divergence to constrain the potential spatial distribution.

[0023] (4) Real-time deployment steps: Integrate the trained model into the physical platform, filter candidate grasping postures based on parallel collision detection, and realize real-time grasping control by combining inverse kinematics verification.

[0024] The point cloud of the target object is acquired by a depth camera, and after preprocessing, it is input into the model to generate multiple candidate grasping poses.

[0025] Based on parallel computing, the collision between candidate poses and the environment is quickly detected, and the effective poses without collisions are selected.

[0026] Candidate postures are sorted by force closure index, and the reachability of the robotic arm is verified by combining inverse kinematics, and the optimal grasping command is output.

[0027] The present invention also discloses a dexterous hand grasping system based on the above method, characterized in that it includes:

[0028] Simulation platform module: Configure the MuJoCo physics engine, load the dexterous hand and object model, and perform data acquisition and automated loop testing;

[0029] Data preprocessing module: Implements data cleaning, point cloud enhancement, and dataset construction functions;

[0030] Model training module: Deploys a multi-scale feature extraction network and CVAE architecture, and combines the Ball Query algorithm to extract local geometric features;

[0031] Real-time control module: Integrates depth camera and robotic arm, and realizes real-time grasping decision through parallel collision detection and inverse kinematics calculation.

[0032] The simulation platform module includes:

[0033] Dexterous hand model: Defines 15 joint degrees of freedom and dynamic parameters, and controls global pose through Mocap points;

[0034] Contact detection unit: Real-time monitoring of palm contact force, triggering grasping action and recording contact force distribution;

[0035] Anomaly handling unit: terminates the action and marks invalid data when a collision or joint over-limit occurs.

[0036] The present invention also discloses a computer-readable storage medium storing program code that executes the above-described method, wherein the code implements the entire process of data acquisition, model training and real-time capture control during runtime.

[0037] The beneficial effects of this invention are as follows:

[0038] The innovation of this invention lies in: embedding the real-time 6D pose of the object as a conditional variable into a conditional variational autoencoder (CVAE), constraining the spatial consistency of generated poses through a pose matrix projection mechanism, and reducing invalid samples; combining the BallQuery algorithm to extract local geometric features of the point cloud (such as differences in normal vectors and curvature changes) to enhance the model's adaptability to occluded scenes; and introducing implicit constraints on physical feasibility during the training phase to optimize the potential spatial distribution and generate highly available candidate poses. Experiments show that this method significantly improves the efficiency of pose generation and the success rate in occluded scenes, and meets real-time requirements without complex post-processing. Attached Figure Description

[0039] Figure 1 This is a flowchart of the data sampling steps;

[0040] Figure 2 A flowchart of the data preprocessing steps;

[0041] Figure 3 This is a flowchart for model training. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0043] A method for generating pose for dexterous hand grasping based on CVAE and Ball Query algorithms includes the following steps:

[0044] Step 1: Data Sampling Steps: (e.g.) Figure 1 As shown, a model of a dexterous hand and a target object is configured in the simulation environment. The grasping posture of the dexterous hand is planned by the circular sampling path, the grasping action is triggered and the grasping stability is verified. The posture, joint angle and contact force data of successful grasping are recorded.

[0045] Step 1: Simulation Environment Configuration:

[0046] 1) Model Loading: Import the mechanical structure model (XML file) of the multi-fingered dexterous hand into MuJoCo, and define the 15 joint degrees of freedom and dynamic parameters of the dexterous hand. The target object model is placed at the center of the simulation scene, with its coordinate system Z-axis pointing vertically upwards.

[0047] 2) Mocap Point Binding: A Mocap point is set at the palm of the dexterous hand as a global motion control reference. This point is rigidly bound to the dexterous hand base, and the overall movement of the dexterous hand within the operating space is achieved by adjusting the Mocap pose.

[0048] Step 2: Initial Posture and Movement Planning

[0049] 1) Initial posture: The starting posture of the dexterous hand is set with the palm facing down, the four fingers together and extended forward, and the initial height is 10cm above the object.

[0050] 2) Circular sampling path: Control the dexterous hand to perform a 360° circular motion around the object's Z-axis, setting a sampling point every 3.6°, for a total of 100 evenly distributed poses. Each sampling point corresponds to a specific orientation of the dexterous hand in the horizontal plane.

[0051] Step 3: Grabbing Action Triggering Mechanism

[0052] 1) Contact detection: When the contact force between the palm area of ​​the dexterous hand and the surface of the object exceeds 0.1N, the grasping action sequence is triggered.

[0053] 2) Finger control mode: The thumb is used for position control and is driven to the preset closing angle (80% of the maximum stroke); the other four fingers apply a constant torque so that the fingers adapt to wrap the shape of the object within 2 seconds.

[0054] Step 4: Crawling stability verification

[0055] After the grasping action is completed, control the dexterous hand to rise at a constant speed of 5cm along the Z-axis, and simultaneously monitor the object's motion status: Success judgment condition: vertical displacement of the object ≥3cm and attitude angle change <5°; Failure judgment condition: the object falls off or deviates beyond the threshold.

[0056] Step 5: Data Storage and Iteration

[0057] 1) Data recording: Save the 6D pose (3D coordinates + quaternions), joint angles, contact force distribution and object pose transformation matrix of the successfully grasped dexterous hand.

[0058] 2) Automated loop: After each test, reset the dexterous hand to the initial pose, traverse all 100 sampling points, until multi-angle coverage is completed.

[0059] Step 6: Quality Control

[0060] 1) Anomaly handling: Real-time monitoring of collisions between the dexterous hand and the scene boundary. When a collision is triggered, the current action is immediately terminated and recorded as a failure sample.

[0061] 2) Data cleaning: Remove invalid data that exceeds joint limits (angle exceeds hardware constraints) or has abnormal contact force direction (deviation from surface normal vector > 30°).

[0062] Step 2: Data preprocessing steps: such as Figure 2 As shown, the collected data is standardized to generate enhanced point cloud data and construct a training dataset;

[0063] Step 1: Data Loading and Validation: Read the CSV file (29 columns). The data in the CSV file are as follows: 1-3 are the gripping positions; 4-7 are the gripping postures; 8-22 are the 15 joint positions; 23-25 ​​are the object positions; and 26-29 are the object postures.

[0064] Step 2: Standardization: Position normalization: Scaling the position coordinates of the dexterous hand and the object to the range of [-1, 1].

[0065] Step 3: Point Cloud Data Generation: First, the original 3D point cloud data of the target object is loaded (acquisition resolution 2mm). Then, multi-stage enhancement processing is performed: the point cloud density is normalized to 2048 key points through random downsampling; on this basis, 3D Gaussian noise with a standard deviation σ = 2mm is superimposed to simulate sensor measurement error; a spherical mask is used to randomly erase 20% of the point cloud area to construct a local occlusion effect. Finally, a matching enhanced point cloud is generated for each effective grasping posture, realizing a one-to-one correspondence between the grasping sample and the perceived data, and enhancing the model's robustness to noise and occlusion.

[0066] Step 4: Dataset Construction: Normalized grasping parameters (including pose, joint angles, etc.) are key-value bound to the augmented point cloud using unique identifiers to ensure data consistency. A stratified random sampling strategy is employed, dividing the training, validation, and test sets in an 8:1:1 ratio to maintain a uniform distribution of object categories and grasping success rates. The final data is stored in HDF5 format. Leveraging its efficient hierarchical data model, independent data groups are created within the file to store the point cloud matrix (shape: N×2048×3), grasping parameter tensors (shape: N×29), and metadata tags (including normalization coefficients, augmentation parameters, and physics simulation environment configuration). Data compression technology reduces storage volume, supporting high-speed parallel read / write and incremental updates.

[0067] Step 3: Model training steps: as follows Figure 3 As shown, by combining global semantics and local geometric features extracted by the Ball Query algorithm through a multi-scale feature extraction module, the grasping posture is generated using a conditional variational autoencoder, and the model parameters are optimized by reconstruction loss and KL divergence.

[0068] 1. Multi-scale point cloud feature extraction

[0069] (1) Global Feature Extraction (PointNet++ Architecture)

[0070] Input point cloud P∈RN×3 and perform hierarchical feature extraction:

[0071]

[0072] Where P is the input point cloud, containing the 3D coordinates of N points; SA(L) This represents the abstract operation of the l-th layer set, which includes farthest point sampling, neighborhood feature extraction and max pooling. When L=4, the output feature dimension of each layer is 64→128→256→512 respectively. This is a function compound operator that indicates sequential processing between levels.

[0073] (2) Local feature extraction (Ball Query enhancement)

[0074] For each center point p i The neighborhood N(p) i Computational geometric features:

[0075]

[0076] Where, p i Given the coordinates of the center point, a spherical neighborhood with radius r = 0.1m centered at pi, and a maximum number of points k = 32; For point p i The normal vector is estimated through eigenvalue decomposition of the covariance matrix; κ(p i Let p be a point. i Gaussian curvature; Δn ij Represents the neighborhood point p j With center point p i The angle between the normal vectors (rad); Δκ ij Represents the neighborhood point p j With center point p i The absolute value of the curvature difference.

[0077]

[0078] in, For the neighborhood point p j For the center point p i The coordinate difference; MLP is a 3-layer perceptron.

[0079] (3) Feature fusion

[0080] Global and local feature concatenation:

[0081]

[0082] in, For the concatenation operation, it means combining global features. With local features The features are merged into 640-dimensional fusion features.

[0083] 2. Conditional Variational Autoencoder (CVAE)

[0084] (1) Encoder

[0085] The input consists of three parts concatenated:

[0086]

[0087] in, To normalize the grasping pose, the position [x,y,z]∈[-1,1] is included. 3 And the unit quaternion [qw,qx,qy,qz]; The pose of the object in the world coordinate system, including the original position and quaternion (unnormalized).

[0088] Transformation process of encoder MLP:

[0089] h1=LeakReLU(W1h in +b1) (6)

[0090] h2=LeakReLU(W2h1+b2) (7)

[0091] h3=LeakReLU(W3h2+b3) (8)

[0092] μ=W4h3+b4 (9)

[0093] logσ=Softplus(W5h3+b5) (10)

[0094] z=μ+σ⊙∈,∈~ξ(0,I) (11)

[0095] Among them, W i b i Here, represents the weight matrix and bias vector of the encoder's i-th layer, respectively; ReLU is the modified linear unit activation function, defined as ReLU(x) = max(0,x); Softplus is the smooth activation function, defined as Softplus(x) = ln(1 + e^(-1 / x)). x ), ensuring variance σ > 0; ⊙ represents the element-wise multiplication operator; The noise vector sampled from a standard normal distribution; Let N be a latent variable, and let N be a distribution N(μ,σ). 2 I).

[0096] (2) Decoder

[0097] Conditional feature injection and decoding process:

[0098] c = MLP(T) (12)

[0099]

[0100] in, The conditional features represent the object's pose T after mapping; Tanh is the hyperbolic tangent function, which constrains the position output to [-1, 1]. In practical applications, it is mapped to the robot's workspace through linear scaling; L2Norm is the L2 normalization operation, ensuring that the output quaternion [q w ,q x ,q y ,q z Satisfying unit constraints

[0101] 3. Loss Function

[0102] The total loss consists of the reconstruction loss and the KL divergence:

[0103] Φ=λ recon Φ recon +λ KL Φ KL (17)

[0104] Where, λ recon λ represents the reconstruction loss weight; KL =0.01 is the KL divergence weight.

[0105] (1) Reconstruction loss:

[0106]

[0107] in, This represents the actual grasping posture; α = 3.0 is the weighting coefficient for the quaternion direction cosine loss; The dot product of the predicted quaternion and the true quaternion is used to measure the consistency of attitude orientation.

[0108] (2) KL divergence:

[0109]

[0110] Where tr(σ) 2 ) is the trace of the variance matrix. logdet(σ 2 ) is the logarithmic determinant of the variance matrix; d z =256 represents the dimensions of the latent variables.

[0111] (4) Real-time deployment steps: Integrate the trained model into the physical platform, filter candidate grasping postures based on parallel collision detection, and realize real-time grasping control by combining inverse kinematics verification.

[0112] Example:

[0113] Data acquisition and simulation configuration

[0114] Step 1.1 Setting up the simulation environment

[0115] 1) Computing platform: A workstation equipped with a high-performance GPU (such as an NVIDIA RTX 3090) and MuJoCo 2.3.7 installed.

[0116] It features a physics engine and integrates ROS (Robot Operating System) for hardware communication.

[0117] 2) Model Loading: Dexterous Hand Model: Import the XML file of the multi-fingered dexterous hand, defining 15 joint degrees of freedom and dynamic parameters (such as mass, friction coefficient, joint limits). Target Object: Load the target model from the YCB object library and place it at the center of the simulation scene, ensuring its coordinate system Z-axis is vertically upward. Mocap Point Configuration: Set a Mocap point at the palm position of the dexterous hand, connecting it to the dexterous hand base via rigid binding for global pose control.

[0118] Step 1.2 Motion Path Planning

[0119] 1) Initial posture setting: The initial posture of the dexterous hand is with the palm facing down, four fingers extended forward and together, and the initial height is above the object surface.

[0120] 10cm.

[0121] 2) Circular sampling path: A circular path with a radius of 15cm is generated with the object's Z-axis as the center, and 100 sampling points are evenly divided at 3.6° intervals. Each sampling point corresponds to the azimuth angle of the dexterous hand's end in the horizontal plane, and the joint angle is calculated through inverse kinematics.

[0122] 3) Motion Control: A linear interpolation algorithm is used to control the dexterous hand to move to each sampling point at a constant speed of 3 cm / s. Step 1.3 Triggering and Execution of Grasping Action

[0123] 1) Contact detection: Define a palm contact sensor in MuJoCo. When the sensor detects a normal contact force exceeding 0.1N, a gripping action is triggered.

[0124] 2) Finger Control Mode: Thumb: Position control mode is used, with the target joint angle set to 80% of the maximum stroke, and precise positioning is achieved through a PID controller. Other four fingers: A constant torque is applied, and the force control mode adaptively wraps around the object surface for 2 seconds.

[0125] 3) Grasping stability verification: After grasping, control the dexterous hand to lift vertically 5cm along the Z-axis at a speed of 2cm / s. Success criteria: Vertical displacement of the object ≥ 3cm and attitude angle change (roll angle, pitch angle) < 5°.

[0126] Step 1.4 Data Recording and Quality Control

[0127] 1) Data storage format:

[0128] a) Dexterous hand pose: 6-DOF pose (3D coordinates + unit quaternion).

[0129] b) Joint angles: Real-time angle values ​​(rad) for 15 joints.

[0130] c) Object pose: The 6-DOF pose of the object in the world coordinate system.

[0131] d) Contact force data: Three-dimensional vector and resultant force amplitude of the contact force at the tips of the four fingers.

[0132] 2) Exception handling:

[0133] a) Monitor the collision between the dexterous hand and the scene boundary in real time. If the minimum distance is less than 1cm, immediately terminate the current action and record it as a failed sample.

[0134] b) Remove invalid data where the joint angle exceeds the limit (e.g., thumb joint > 1.2 rad) or the contact force direction deviates from the surface normal vector by more than 30°.

[0135] 2. Data Preprocessing and Augmentation

[0136] Step 2.1 Data Cleaning and Standardization

[0137] 1) Validity verification: Remove samples with quaternion norms exceeding [0.95, 1.05]; filter out abnormal data with total contact force <1N or single finger force >5N.

[0138] 2) Normalization process:

[0139] a) Position coordinates: Linearly map the position of the dexterous hand and the object to the interval [-1,1], and retain the scaling factor for subsequent inverse transformation.

[0140] b) Joint angles: linearly normalized to [-1, 1] according to the range of motion of the hardware.

[0141] Step 2.2 Point Cloud Enhancement and Alignment

[0142] Point cloud augmentation process:

[0143] a) Random downsampling: 2048 points are randomly selected from the original point cloud, retaining the main geometric features.

[0144] b) Noise injection: Gaussian noise is added to each point coordinate to simulate sensor measurement error.

[0145] Step 2.3 Dataset Construction

[0146] 1) Data storage format:

[0147] a) HDF5 file structure: / train / pointclouds: augmented point cloud data (dimension: number of samples × 2048 × 3); / train / grasp_params: normalized capture parameters (dimension: number of samples × 29); / metadata: stores normalization coefficients, noise parameters and object attributes (JSON format).

[0148] b) Dataset partitioning: The training set, validation set, and test set are divided in an 8:1:1 ratio to ensure a uniform distribution of object categories and capture success rate.

[0149] 3. Model Training and Deployment

[0150] Step 3.1 Network Architecture Configuration

[0151] 1) Multi-scale feature extraction:

[0152] a) Global features: Based on the improved PointNet++ architecture, a 4-layer set abstraction module outputs 256-dimensional global semantic features.

[0153] b) Local features: Using Ball Query neighborhood (radius 0.1m, maximum number of points 32), the difference between the angle of the normal vector and the curvature is extracted, and 256-dimensional local geometric features are output.

[0154] c) Feature fusion: Combine global and local features into a fused feature.

[0155] 2) CVAE structure:

[0156] a) Encoder: 5-layer fully connected network (hidden layer dimensions 1024→512→256→128, outputting 128-dimensional latent variables).

[0157] b) Decoder: 4-layer fully connected network (hidden layer dimensions 512→256→128, output 7-dimensional grasping posture).

[0158] Step 3.2 Training Strategy

[0159] 1) Loss function:

[0160] a) Reconstruction loss: Calculate the position error between the predicted pose and the true value, and the quaternion direction cosine difference.

[0161] b) KL divergence: constrains the distribution of latent variables to approximate a standard normal distribution, with a weighting coefficient of 0.1.

[0162] 2) Optimize settings:

[0163] a) Optimizer: AdamW, initial learning rate 1e-2, Cosine decay to 1e-5.

[0164] b) Regularization: Dropout (ratio 0.2) and gradient clipping (threshold 1.0).

[0165] Step 3.3 Real-time Deployment and Optimization

[0166] 1) Hardware platform:

[0167] a) Robotic arm: UR5, with a dexterous hand at the end.

[0168] 2) Online reasoning process:

[0169] a) Point cloud preprocessing: Downsample to 2048 points, add noise and transform the coordinate system;

[0170] b) Candidate pose generation: CVAE generates 30 candidate poses, and valid poses are selected through collision detection;

[0171] c) Optimal pose selection: Select the 5 most stable candidate grasping poses based on energy optimization and verify their reachability through inverse kinematics to select the final grasping gesture.

Claims

1. A method for generating pose for dexterous hand grasping based on CVAE and Ball Query algorithms, characterized in that, Includes the following steps: (1) Data sampling steps: Configure the models of the dexterous hand and the target object in the simulation environment, plan the grasping posture of the dexterous hand through the circumferential sampling path, trigger the grasping action and verify the grasping stability, and record the posture, joint angle and contact force data of successful grasping. (2) Data preprocessing steps: Standardize the collected data to generate enhanced point cloud data and construct a training dataset; (3) Model training steps: The grasping posture is generated by combining the global semantics and the local geometric features extracted by the Ball Query algorithm through the multi-scale feature extraction module, and the model parameters are optimized by the reconstruction loss and KL divergence. The multi-scale feature extraction module specifically includes: Global semantic features of point clouds are extracted through hierarchical set abstraction operations; Input point cloud Hierarchical feature extraction operation: Where P is the input point cloud, containing the 3D coordinates of N points; SA (L) The l-th layer represents the abstract operation of the set, which includes farthest point sampling, neighborhood feature extraction, and max pooling. When L = 4, the output feature dimensions of each layer are 64→128→256→512 respectively; ∘ is the function composition operator, which represents the sequential processing between layers. Local geometric features are extracted based on the Ball Query algorithm, including the coordinate offset of the center point, the angle between the normal vectors, and the curvature difference when querying the spherical neighborhood. For each center point neighborhood Computational geometric features: in, Given the coordinates of the center point, a spherical neighborhood with radius r = 0.1m centered at pi, and a maximum number of points k = 32; For point The normal vector is estimated through the eigenvalue decomposition of the covariance matrix; For point Gaussian curvature; Representing neighborhood points With the center point The angle between the normal vectors (rad); Representing neighborhood points With the center point The absolute value of the curvature difference; in, For neighborhood points For the center point The coordinate difference; MLP is a 3-layer perceptron; The global and local features are concatenated into a fused feature, which is then input into the conditional variational autoencoder. Global and local feature concatenation: Where ⊕ represents the concatenation operation, indicating that global features are concatenated. With local features Merged into 640-dimensional fusion features; (4) Real-time deployment steps: Integrate the trained model into the physical platform, filter candidate grasping postures based on parallel collision detection, and realize real-time grasping control by combining inverse kinematics verification.

2. The method according to claim 1, characterized in that, The data sampling step specifically includes: Load the multi-joint degree-of-freedom model of the dexterous hand and the target object model into the simulation environment, and bind the Mocap point to the palm of the dexterous hand to control the global motion; Plan a circular sampling path for the dexterous hand around the Z-axis of the target object, and generate 100 sampling poses at fixed angle intervals; When the contact force of the dexterous hand's palm exceeds the threshold, a grasping action is triggered, controlling the thumb to close to a preset angle, while the other four fingers apply a constant torque to wrap the object; After grasping, the dexterous hand is vertically lifted and the object's displacement is monitored to determine whether the grasp was successful. Valid data is recorded and all sampling points are iterated over in a loop.

3. The method according to claim 1, characterized in that, The data preprocessing steps specifically include: The position coordinates of the dexterous hand and the object are normalized and scaled to the range of [-1,1]. The original point cloud of the target object was randomly downsampled to 2048 key points, and three-dimensional Gaussian noise was superimposed and some areas were randomly erased to simulate occlusion. The normalized capture parameters are bound to the enhanced point cloud, divided into training, validation and test sets according to the proportions, and stored in a layered compressed HDF5 format.

4. The method according to claim 1, characterized in that, The conditional variational autoencoder specifically includes: Encoder: It concatenates the fused features, normalized grasping pose, and object pose as input, and generates the mean and variance of latent variables through a multilayer perceptron; Decoder: It concatenates the latent variables with the conditional features mapped to the object pose, reconstructs the grasping pose through a multilayer perceptron, and outputs the position constraint to the interval [-1,1] and the unit quaternion; Loss function: combines position error, quaternion direction cosine difference and KL divergence to constrain the potential spatial distribution.

5. The method according to claim 1, characterized in that, The real-time deployment steps specifically include: The point cloud of the target object is acquired by a depth camera, and after preprocessing, it is input into the model to generate multiple candidate grasping poses. Based on parallel computing, the collision between candidate poses and the environment is quickly detected, and the effective poses without collisions are selected. Candidate postures are sorted by force closure index, and the reachability of the robotic arm is verified by combining inverse kinematics, and the optimal grasping command is output.

6. A dexterous hand grasping system based on the method of any one of claims 1-5, characterized in that, include: Simulation platform module: Configure the MuJoCo physics engine, load the dexterous hand and object model, and perform data acquisition and automated loop testing; Data preprocessing module: Implements data cleaning, point cloud enhancement, and dataset construction functions; Model training module: Deploys a multi-scale feature extraction network and CVAE architecture, and combines the Ball Query algorithm to extract local geometric features; Real-time control module: Integrates depth camera and robotic arm, and realizes real-time grasping decision through parallel collision detection and inverse kinematics calculation.

7. The system according to claim 6, characterized in that, The simulation platform module includes: Dexterous hand model: Defines 15 joint degrees of freedom and dynamic parameters, and controls global pose through Mocap points; Contact detection unit: Real-time monitoring of palm contact force, triggering grasping action and recording contact force distribution; Anomaly handling unit: terminates the action and marks invalid data when a collision or joint over-limit occurs.

8. A computer-readable storage medium, characterized in that, The system stores program code that performs the method of any one of claims 1-5, wherein the code implements the entire process of data acquisition, model training and real-time capture control during runtime.

Images