Dexterous hand grabbing pose generation method and system based on CVAE and Ball Query algorithms

Abstract

The invention discloses a multi-fingered dexterous hand grabbing posture generation method and system based on CVAE and a Ball Query algorithm, and belongs to the technical field of robot grabbing control. The method comprises the following steps: (1) a data sampling step; (2) a data preprocessing step: carrying out standardization processing on the collected data, generating enhanced point cloud data and constructing a training data set; (3) a model training step: through a multi-scale feature extraction module, combining global semantics and local geometric features extracted by a Ball Query algorithm, generating a grabbing attitude by using a conditional variation auto-encoder, and optimizing model parameters through reconstruction loss and KL divergence; and (4) real-time deployment: integrating the trained model to a physical platform, screening candidate grabbing postures based on parallel collision detection, and realizing real-time grabbing control in combination with inverse kinematics verification. The problem that a traditional method is insufficient in generalization ability in complex object grabbing is effectively solved, and self-adaptive grabbing of unknown objects is achieved while grabbing stability is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of robot grasping control, and particularly relates to a dexterous hand grasping pose generation method and system based on the CVAE and Ball Query algorithms, which are particularly suitable for adaptive grasping operations of unknown objects in scenarios such as factories and families. Background Art

[0002] Currently, the grasping generation methods based on deep learning face two core challenges: First, the variational autoencoder (VAE) relies on unconstrained random sampling to generate candidate grasping postures, resulting in a large number of samples being invalid due to collisions or kinematic inaccessibility of the robotic arm, and relying on computationally expensive post-processing screening, which severely restricts real-time performance; Second, existing methods usually use the local point cloud of the object as input, without explicitly modeling the global pose of the object in the robot base coordinate system, resulting in the generated grasping postures being prone to spatial misalignment, and the robustness significantly decreasing due to the lack of geometric information in local occlusion scenarios. Although traditional solutions can partially alleviate the problems through physical simulation iterative optimization or post-processing correction, they have high computational complexity and are decoupled from the generation model, making it difficult to achieve end-to-end efficient collaboration. Summary of the Invention

[0003] In view of the above problems, the present invention proposes a dexterous hand grasping pose generation method and system based on the CVAE and Ball Query algorithms, which effectively solves the problem of insufficient generalization ability of traditional methods in grasping complex objects, and realizes adaptive grasping of unknown objects while ensuring grasping stability.

[0004] The technical solution of the present invention is as follows:

[0005] (1) Data sampling step: Configure the models of the dexterous hand and the target object in the simulation environment, plan the grasping poses of the dexterous hand through a circular sampling path, trigger the grasping action and verify the grasping stability, and record the poses, joint angles, and contact force data of successful grasps;

[0006] Load the multi-joint degree-of-freedom model of the dexterous hand and the target object model in the simulation environment, and bind the Mocap point to the palm center of the dexterous hand to control the global movement;

[0007] Plan a circular sampling path of the dexterous hand around the Z-axis of the target object, and generate 100 sampling poses at fixed angular intervals;

[0008] When the contact force of the dexterous hand palm exceeds the threshold, trigger the grasping action, control the thumb to close to a preset angle, and the other four fingers apply a constant torque to wrap the object;

[0009] After grasping, vertically lift the dexterous hand and monitor the displacement of the object to determine whether the grasp is successful, record the valid data, and loop through all sampling points.

[0010] (2) Data preprocessing steps: Standardize the collected data, generate enhanced point cloud data, and construct a training dataset;

[0011] Normalize the position coordinates of the dexterous hand and the object, and scale them to the interval [-1, 1];

[0012] Randomly downsample the original point cloud of the target object to 2048 key points, add three-dimensional Gaussian noise, and randomly erase some areas to simulate occlusion;

[0013] Bind the normalized grasping parameters to the enhanced point cloud, and divide them into training set, validation set and test set according to a certain proportion, and store them in the hierarchical compressed HDF5 format.

[0014] (3) Model training steps: Combine the global semantics and local geometric features extracted by the Ball Query algorithm through the multi-scale feature extraction module, generate the grasping pose using the conditional variational autoencoder, and optimize the model parameters through the reconstruction loss and KL divergence;

[0015] The multi-scale feature extraction module specifically includes:

[0016] Extract the global semantic features of the point cloud through hierarchical set abstraction operations;

[0017] Extract local geometric features based on the Ball Query algorithm, including the coordinate offset, normal vector angle, and curvature difference of the spherical neighborhood query center point;

[0018] Concatenate the global features and local features into a fused feature, and input it into the conditional variational autoencoder.

[0019] The conditional variational autoencoder specifically includes:

[0020] Encoder: Concatenate the fused feature, normalized grasping pose, and object pose as the input, and generate the mean and variance of the latent variable through a multi-layer perceptron;

[0021] Decoder: Concatenate the latent variable and the conditional feature mapped by the object pose, reconstruct the grasping pose through a multi-layer perceptron, and output the position constraint to the interval [-1, 1] and the unit quaternion;

[0022] Loss function: Combine the position error, quaternion direction cosine difference, and KL divergence to constrain the latent space distribution.

[0023] (4) Real-time deployment steps: Integrate the trained model into the physical platform, screen the candidate grasping poses based on parallel collision detection, and combine inverse kinematics verification to achieve real-time grasping control.

[0024] The target object point cloud is obtained through a depth camera, and after preprocessing, it is input into the model to generate multiple candidate grasping postures;

[0025] Based on parallel computing, quickly detect the collision between the candidate postures and the environment, and filter out the valid postures without collision;

[0026] Sort the candidate postures according to the force closure index, and combine inverse kinematics to verify the reachability of the robotic arm, and output the optimal grasping instruction.

[0027] The present invention also discloses a dexterous hand grasping system based on the above method, which is characterized by including:

[0028] Simulation platform module: Configure the MuJoCo physics engine, load the dexterous hand and object models, and execute data acquisition and automated loop testing;

[0029] Data preprocessing module: Implement functions of data cleaning, point cloud enhancement, and dataset construction;

[0030] Model training module: Deploy a multi-scale feature extraction network and a CVAE architecture, and combine the Ball Query algorithm to extract local geometric features;

[0031] Real-time control module: Integrate a depth camera and a robotic arm, and realize real-time grasping decision-making through parallel collision detection and inverse kinematics calculation.

[0032] The simulation platform module includes:

[0033] Dexterous hand model: Define 15 joint degrees of freedom and dynamic parameters, and control the global pose through Mocap points;

[0034] Contact detection unit: Real-time monitor the palm contact force, trigger the grasping action and record the contact force distribution;

[0035] Abnormal handling unit: Terminate the action when there is a collision or joint limit, and mark invalid data.

[0036] The present invention also discloses a computer-readable storage medium, storing program codes for executing the above method, and the codes realize the whole process of data acquisition, model training, and real-time grasping control when running.

[0037] The beneficial effects of the present invention are as follows:

[0038] The innovation of the present invention lies in: embedding the real-time 6D pose of an object as a conditional variable into a conditional variational autoencoder (CVAE), constraining the spatial consistency of the generated poses through a pose matrix projection mechanism to reduce invalid samples; combining the BallQuery algorithm to extract local geometric features of the point cloud (such as normal vector differences and curvature changes) to enhance the model's adaptability to occluded scenes; at the same time, introducing physical feasibility implicit constraints in the training stage to optimize the latent space distribution to generate highly available candidate poses. Experiments show that this method significantly improves the efficiency of grasping pose generation and the success rate in occluded scenes, and can meet the real-time requirements without complex post-processing. Description of the Drawings

[0039] Figure 1 It is a flowchart of the data sampling step;

[0040] Figure 2 It is a flowchart of the data preprocessing step;

[0041] Figure 3 It is a flowchart of model training. Detailed Implementation Manner

[0042] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0043] A dexterous hand grasping pose generation method based on CVAE and Ball Query algorithm includes the following steps:

[0044] Step 1: Data sampling step: As Figure 1 shown, configure the models of the dexterous hand and the target object in the simulation environment, plan the grasping poses of the dexterous hand through circular sampling paths, trigger the grasping action and verify the grasping stability, and record the poses, joint angles and contact force data of successful grasps;

[0045] The first step: Simulation environment configuration:

[0046] 1) Model loading: Import the mechanical structure model (XML file) of the multi-fingered dexterous hand into MuJoCo, and define the 15 joint degrees of freedom and dynamic parameters of the dexterous hand. The target object model is placed at the center of the simulation scene, and the Z axis of its coordinate system is vertically upward.

[0047] 2) Mocap point binding: Set a Mocap point at the palm center of the dexterous hand as the global motion control reference. This point is rigidly bound to the base of the dexterous hand, and the overall movement of the dexterous hand in the operating space is achieved by adjusting the Mocap pose.

[0048] The second step: Initial pose and motion planning

[0049] 1) Initial posture: The starting posture of the dexterous hand is set with the palm facing down, four fingers together and extended forward, and the initial height is 10 cm above the object.

[0050] 2) Circular sampling path: Control the dexterous hand to perform 360° circular motion around the Z axis of the object, set a sampling point every 3.6°, and a total of 100 evenly distributed poses. Each sampling point corresponds to a specific position of the dexterous hand in the horizontal plane.

[0051] Step 3: Grab Action Trigger Mechanism

[0052] 1) Contact detection: When the contact force between the palm area of the dexterous hand and the surface of the object exceeds 0.1N, the grasping action sequence is triggered.

[0053] 2) Finger control mode: The thumb is position-controlled and driven to a preset closing angle (80% of maximum travel); the other four fingers apply a constant torque so that the fingers can adapt to the shape of the wrapped object within 2 seconds.

[0054] Step 4: Crawl stability verification

[0055] After the grasping action is completed, the dexterous hand is controlled to rise 5 cm at a uniform speed along the Z axis, and the movement state of the object is monitored simultaneously: the success judgment condition is: the vertical displacement of the object is ≥ 3 cm and the posture angle change is < 5°; the failure judgment condition is: the object falls off or the offset exceeds the threshold.

[0056] Step 5: Data storage and iteration

[0057] 1) Data recording: Save the 6D pose (3D coordinates + quaternion), joint angles, contact force distribution and object pose transformation matrix of the successfully grasped dexterous hand.

[0058] 2) Automatic cycle: After each test, the dexterous hand is reset to the initial position and all 100 sampling points are traversed until multi-angle coverage is completed.

[0059] Step 6: Quality Control

[0060] 1) Exception handling: The collision between the dexterous hand and the scene boundary is monitored in real time. When a collision is triggered, the current action is terminated immediately and recorded as a failure sample.

[0061] 2) Data cleaning: Eliminate invalid data with joint out-of-limit (angle exceeds hardware constraint) or abnormal contact force direction (deviation from surface normal vector>30°).

[0062] Step 2: Data preprocessing steps: Figure 2 As shown, the collected data is standardized to generate enhanced point cloud data and construct a training data set;

[0063] Step 1: Data Loading and Verification: Read a CSV file (with 29 columns). The data in the CSV file are as follows: columns 1 - 3 are the grasping positions; columns 4 - 7 are the grasping postures; columns 8 - 22 are the positions of 15 joints; columns 23 - 25 are the object positions; columns 26 - 29 are the object postures.

[0064] Step 2: Standardization Processing: Position Normalization: Scale the position coordinates of the dexterous hand and the object to the interval [-1, 1].

[0065] Step 3: Point Cloud Data Generation: First, load the original three-dimensional point cloud data of the target object (acquisition resolution 2mm), and then perform multi-stage enhancement processing: Regularize the point cloud density to 2048 key points through random downsampling; on this basis, superimpose three-dimensional Gaussian noise with a standard deviation σ = 2mm to simulate sensor measurement errors; randomly erase 20% of the point cloud area using a spherical mask to construct a local occlusion effect. Finally, generate a matching enhanced point cloud for each valid grasping posture, realizing a one-to-one correspondence between the grasping samples and the perception data, and enhancing the model's robustness to noise and occlusion.

[0066] Step 4: Dataset Construction: Bind the normalized grasping parameters (including pose, joint angles, etc.) and the enhanced point cloud through a unique identifier to ensure data consistency. Adopt a stratified random sampling strategy to divide the training set, validation set, and test set in a ratio of 8:1:1, maintaining a uniform distribution of object categories and grasping success rates. The final data is stored in the HDF5 format. Using its efficient hierarchical data model, establish independent data groups in the file to store the point cloud matrix (shape: N×2048×3), the grasping parameter tensor (shape: N×29), and the metadata tags (including normalization coefficients, enhancement parameters, physical simulation environment configurations). Reduce the storage volume through data compression technology, supporting high-speed parallel reading and writing and incremental updates.

[0067] Step 3: Model Training Steps: As Figure 3 shown, extract local geometric features through the multi-scale feature extraction module combined with global semantics and the Ball Query algorithm, generate grasping postures using the conditional variational autoencoder, and optimize the model parameters through reconstruction loss and KL divergence;

[0068] 1. Multi-scale Point Cloud Feature Extraction

[0069] (1) Global Feature Extraction (PointNet++ Architecture)

[0070] The input point cloud P ∈ RN×3 undergoes hierarchical feature extraction operations:

[0071]

[0072] where P is the input point cloud, containing the three-dimensional coordinates of N points; SA(L) Denote the set abstraction operation of the $l$-th layer, which includes farthest point sampling, neighborhood feature extraction, and max pooling. When $L = 4$, the output feature dimensions of each layer are $64 \to 128 \to 256 \to 512$ in sequence; It is a function composition operator, representing sequential processing between layers.

[0073] (2) Local Feature Extraction (Enhanced by Ball Query)

[0074] For each center point $p$ i 's neighborhood $N(p$ i ), calculate geometric features:

[0075]

[0076] where $p$ i is the center point coordinate. A spherical neighborhood with center $p_i$ and radius $r = 0.1m$, and the maximum number of points $k = 32$; is the normal vector of point $p$ i , estimated by eigenvalue decomposition of the covariance matrix; $\kappa(p$ i ) is the Gaussian curvature of point $p$ i ; $\Delta n$ ij represents the angle (rad) between the normal vector of the neighborhood point $p$ j and the center point $p$ i ; $\Delta\kappa$ ij represents the absolute value of the curvature difference between the neighborhood point $p$ j and the center point $p$ i .

[0077]

[0078] where is the coordinate difference of the neighborhood point $p$ j for the center point $p$ i ; MLP is a 3-layer perceptron.

[0079] (3) Feature Fusion

[0080] Global and local feature concatenation:

[0081]

[0082] where is the concatenation operation, indicating that the global feature and the local feature are combined into a 640-dimensional fused feature.

[0083] 2. Conditional Variational Autoencoder (CVAE)

[0084] (1) Encoder

[0085] The input is composed of three parts spliced together:

[0086]

[0087] Among them, is the normalized grasping pose, including the position [x, y, z] ∈ [-1, 1] 3 and the unit quaternion [qw, qx, qy, qz]; is the pose of the object in the world coordinate system, including the original position and the quaternion (not normalized).

[0088] The transformation process of the encoder MLP:

[0089] h1 = LeakReLU(W1h in + b1) (6)

[0090] h2 = LeakReLU(W2h1 + b2) (7)

[0091] h3 = LeakReLU(W3h2 + b3) (8)

[0092] μ = W4h3 + b4 (9)

[0093] logσ = Softplus(W5h3 + b5) (10)

[0094] z = μ + σ ⊙ ∈, ∈ ~ ξ(0, I) (11)

[0095] Among them, W i , b i are the weight matrix and bias vector of the i-th layer of the encoder respectively; ReLU is the rectified linear unit activation function, defined as ReLU(x) = max(0, x); Softplus is the smooth activation function, defined as Softplus(x) = ln(1 + e x ), ensuring that the variance σ > 0; ⊙ represents the element-wise multiplication operator; is the noise vector sampled from the standard normal distribution; is the latent variable, following the distribution N(μ, σ 2 I).

[0096] (2) Decoder

[0097] Conditional feature injection and decoding process:

[0098] c = MLP(T) (12)

[0099]

[0100] Among them, Represents the conditional features after the object pose T is mapped; Tanh is the hyperbolic tangent function that constrains the position output to [-1, 1], and in practical applications, it is linearly scaled and mapped to the robot's workspace; L2Norm is the L2 normalization operation to ensure that the output quaternion [q w , q x , q y , q z satisfies the unit constraint

[0101] 3. Loss Function

[0102] The total loss consists of the reconstruction loss and the KL divergence:

[0103] Φ = λ recon Φ recon + λ KL Φ KL (17)

[0104] where λ recon represents the reconstruction loss weight; λ KL = 0.01 is the KL divergence weight.

[0105] (1) Reconstruction Loss:

[0106]

[0107] where represents the true grasping pose; α = 3.0 is the weighting coefficient of the quaternion direction cosine loss; is the dot product of the predicted quaternion and the true quaternion, used to measure the pose direction consistency.

[0108] (2) KL Divergence:

[0109]

[0110] where tr(σ 2 ) is the trace of the variance matrix, logdet(σ 2 ) is the logarithm of the determinant of the variance matrix; d z = 256 is the latent variable dimension.

[0111] (4) Real-time Deployment Steps: Integrate the trained model into the physical platform, screen candidate grasping poses based on parallel collision detection, and combine inverse kinematics verification to achieve real-time grasping control.

[0112] Example:

[0113] Data Collection and Simulation Configuration

[0114] Step 1.1 Simulation Environment Setup

[0115] 1) Computing platform: A workstation equipped with a high-performance GPU (such as NVIDIA RTX 3090) and MuJoCo 2.3.7 installed

[0116] Physics engine, and integrates ROS (Robot Operating System) to achieve hardware communication.

[0117] 2) Model loading: Dexterous hand model: Import the XML file of the multi-finger dexterous hand, define 15 joint degrees of freedom and dynamic parameters (such as mass, friction coefficient, joint limit). Target object: Load the target model from the YCB object library and place it in the center of the simulation scene, ensuring that its coordinate system Z axis is vertically upward. Mocap point configuration: Set the Mocap point at the palm of the dexterous hand and connect it to the dexterous hand base through rigid binding for global posture control.

[0118] Step 1.2 Motion path planning

[0119] 1) Initial posture setting: The initial posture of the dexterous hand is with the palm facing down, four fingers together and extended forward, and the initial height is

[0120] 10cm.

[0121] 2) Circular sampling path: With the Z axis of the object as the center, a circular path with a radius of 15 cm is generated, and 100 sampling points are evenly divided at 3.6° intervals. Each sampling point corresponds to the azimuth angle of the end of the dexterous hand in the horizontal plane, and the joint angle is calculated by inverse kinematics.

[0122] 3) Motion control: Use a linear interpolation algorithm to control the dexterous hand to move to each sampling point at a constant speed of 3 cm / s. Step 1.3 Triggering and executing the grasping action

[0123] 1) Contact detection: Define the palm contact sensor in MuJoCo. When the sensor detects that the normal contact force exceeds 0.1N, the grasping action is triggered.

[0124] 2) Finger control mode: Thumb: adopts position control mode, the target joint angle is set to 80% of the maximum stroke, and precise positioning is achieved through PID controller. The other four fingers: apply constant torque and wrap the object surface adaptively through force control mode for 2s.

[0125] 3) Grasping stability verification: After grasping, control the dexterous hand to vertically lift 5 cm along the Z axis at a speed of 2 cm / s. Success judgment conditions: the vertical displacement of the object is ≥ 3 cm and the attitude angle change (roll angle, pitch angle) is < 5°.

[0126] Step 1.4 Data recording and quality control

[0127] 1) Data storage format:

[0128] a) Dexterous hand pose: 6 - degree - of - freedom pose (3D coordinates + unit quaternion).

[0129] b) Joint angles: Real - time angle values (rad) of 15 joints.

[0130] c) Object pose: 6 - degree - of - freedom pose of the object in the world coordinate system.

[0131] d) Contact force data: Three - dimensional vectors of the contact forces at the four fingertip ends and the resultant force magnitude.

[0132] 2) Abnormality handling:

[0133] a) Real - time monitor the collision between the dexterous hand and the scene boundary. If the minimum distance < 1 cm, immediately terminate the current action and record it as a failed sample.

[0134] b) Eliminate invalid data where the joint angle exceeds the limit (e.g., the thumb joint > 1.2 rad) or the deviation between the contact force direction and the surface normal vector > 30°.

[0135] 2. Data pre - processing and augmentation

[0136] Step 2.1 Data cleaning and standardization

[0137] 1) Validity check: Eliminate samples with quaternion norms outside the range [0.95, 1.05]; filter abnormal data where the total contact force < 1 N or the force of a single finger > 5 N.

[0138] 2) Normalization processing:

[0139] a) Position coordinates: Linearly map the positions of the dexterous hand and the object to the interval [-1, 1], and retain the scaling coefficient for subsequent inverse transformation.

[0140] b) Joint angles: Linearly normalize to [-1, 1] according to the hardware motion range.

[0141] Step 2.2 Point cloud augmentation and alignment

[0142] Point cloud augmentation process:

[0143] a) Random down - sampling: Randomly extract 2048 points from the original point cloud to retain the main geometric features.

[0144] b) Noise injection: Add Gaussian noise to each point coordinate to simulate sensor measurement errors.

[0145] Step 2.3 Dataset construction

[0146] 1) Data storage format:

[0147] a) HDF5 file structure: / train / pointclouds: enhanced point cloud data (dimension: number of samples × 2048 × 3); / train / grasp_params: normalized grasping parameters (dimension: number of samples × 29); / metadata: stores normalization coefficients, noise parameters, and object attributes (JSON format).

[0148] b) Dataset division: The training set, validation set, and test set are divided in a ratio of 8:1:1 to ensure a uniform distribution of object categories and grasping success rates.

[0149] 3. Model Training and Deployment

[0150] Step 3.1 Network Architecture Configuration

[0151] 1) Multi-scale feature extraction:

[0152] a) Global features: Based on the improved PointNet++ architecture, with 4 layers of set abstraction modules, outputting 256-dimensional global semantic features.

[0153] b) Local features: Using the Ball Query neighborhood (radius 0.1m, maximum number of points 32), extracting the angle difference of normal vectors and curvature differences, and outputting 256-dimensional local geometric features.

[0154] c) Feature fusion: Concatenating global and local features into fused features.

[0155] 2) CVAE structure:

[0156] a) Encoder: A 5-layer fully connected network (hidden layer dimensions 1024 → 512 → 256 → 128, outputting 128-dimensional latent variables).

[0157] b) Decoder: A 4-layer fully connected network (hidden layer dimensions 512 → 256 → 128, outputting 7-dimensional grasping poses).

[0158] Step 3.2 Training Strategy

[0159] 1) Loss function:

[0160] a) Reconstruction loss: Calculate the position error between the predicted pose and the true value and the quaternion direction cosine difference.

[0161] b) KL divergence: Constrain the latent variable distribution to be close to the standard normal distribution, with a weight coefficient of 0.1.

[0162] 2) Optimization settings:

[0163] a) Optimizer: AdamW, with an initial learning rate of 1e-2, decaying to 1e-5 using Cosine annealing.

[0164] b) Regularization: Dropout (ratio 0.2) and gradient clipping (threshold 1.0).

[0165] Step 3.3 Real-time Deployment and Optimization

[0166] 1) Hardware Platform:

[0167] a) Robot Arm: UR5, with a dexterous hand mounted at the end.

[0168] 2) Online Inference Process:

[0169] a) Point Cloud Preprocessing: Downsample to 2048 points, add noise and transform the coordinate system;

[0170] b) Candidate Pose Generation: CVAE generates 30 candidate poses, and valid poses are screened through collision detection;

[0171] c) Optimal Pose Selection: Select the 5 most stable candidate grasping poses according to the minimum energy and verify the reachability through inverse kinematics to select the final grasping gesture.

Claims

1. A method for generating grasping poses of a dexterous hand based on the CVAE and Ball Query algorithms, characterized in that, It includes the following steps: (1) Data sampling step: Configure the models of the dexterous hand and the target object in the simulation environment, plan the grasping poses of the dexterous hand through a circular sampling path, trigger the grasping action and verify the grasping stability, and record the poses, joint angles and contact force data of successful grasps; (2) Data preprocessing step: Standardize the collected data, generate enhanced point cloud data and construct a training data set; (3) Model training step: Extract local geometric features combined with global semantics and the Ball Query algorithm through a multi-scale feature extraction module, generate grasping postures using a conditional variational autoencoder, and optimize the model parameters through reconstruction loss and KL divergence; (4) Real-time deployment step: Integrate the trained model into the physical platform, screen candidate grasping postures based on parallel collision detection, and combine inverse kinematics verification to achieve real-time grasping control.

2. The method according to claim 1, characterized in that The data sampling step specifically includes: Load the multi-joint degree-of-freedom model of the dexterous hand and the target object model in the simulation environment, and bind Mocap points to the palm center of the dexterous hand to control the global movement; Plan a circular sampling path of the dexterous hand around the Z-axis of the target object, and generate 100 sampling poses at fixed angular intervals; When the contact force of the palm center of the dexterous hand exceeds the threshold, trigger the grasping action, control the thumb to close to a preset angle, and apply a constant torque to the other four fingers to wrap the object; After grasping, vertically lift the dexterous hand and monitor the displacement of the object, determine whether the grasp is successful, record the valid data and loop through all sampling points.

3. The method according to claim 1, characterized in that The data preprocessing step specifically includes: Normalize the position coordinates of the dexterous hand and the object, and scale them to the interval [-1,1]; Randomly downsample the original point cloud of the target object to 2048 key points, superimpose three-dimensional Gaussian noise and randomly erase some areas to simulate occlusion; Bind the normalized grasping parameters to the enhanced point cloud, divide them into a training set, a validation set and a test set according to a ratio, and store them in a hierarchical compressed HDF5 format.

4. The method according to claim 1, characterized in that, The multi-scale feature extraction module specifically includes: Extract the global semantic features of the point cloud through hierarchical set abstraction operations; Extract local geometric features based on the Ball Query algorithm, including the coordinate offset, normal vector angle and curvature difference of the spherical neighborhood query center point; Concatenate the global features and local features into a fused feature and input it into the conditional variational autoencoder.

5. The method according to claim 1, wherein The conditional variational autoencoder specifically includes: Encoder: Concatenate the fused feature, the normalized grasping pose and the object pose as the input, and generate the mean and variance of the latent variable through a multi-layer perceptron; Decoder: Concatenate the latent variable and the conditional feature mapped by the object pose, reconstruct the grasping pose through a multi-layer perceptron, and output the position constraint to the interval [-1,1] and the unit quaternion; Loss function: Combine the position error, the quaternion direction cosine difference and the KL divergence to constrain the latent space distribution.

6. The method according to claim 1, characterized in that, The real-time deployment step specifically includes: Obtain the point cloud of the target object through a depth camera, preprocess it and input it into the model to generate multiple candidate grasping postures; Based on parallel computing, quickly detect the collision between the candidate postures and the environment, and screen out the valid postures without collision; Sort candidate postures according to the force closure index, verify the reachability of the robotic arm by combining inverse kinematics, and output the optimal grasping instruction.

7. A dexterous hand grasping system based on the method according to any one of claims 1-6, characterized in that, Including: Simulation platform module: Configure the MuJoCo physics engine, load the dexterous hand and object models, and perform data collection and automated loop testing; Data preprocessing module: Implement functions of data cleaning, point cloud enhancement, and dataset construction; Model training module: Deploy a multi-scale feature extraction network and a CVAE architecture, and extract local geometric features by combining the Ball Query algorithm; Real-time control module: Integrate a depth camera and a robotic arm, and achieve real-time grasping decisions through parallel collision detection and inverse kinematics calculations.

8. The system according to claim 7, wherein The simulation platform module includes: Dexterous hand model: Define 15 joint degrees of freedom and dynamic parameters, and control the global pose through Mocap points; Contact detection unit: Monitor the palm contact force in real time, trigger the grasping action, and record the contact force distribution; Abnormal handling unit: Terminate the action when a collision or joint limit is exceeded, and mark invalid data.

9. A computer-readable storage medium, characterized in that, Store program code for executing the method according to any one of claims 1-6, and the code realizes the entire process of data collection, model training, and real-time grasping control when running.

Images