Manipulator grasping point training method and device based on diffusion model, and prediction method and device

Abstract

The application provides a mechanical hand grabbing point training method and device based on a diffusion model, and a prediction method, wherein the process of the training method is as follows: sample object point clouds and marked label point clouds are acquired, and are recorded as label data; the label data, diffusion time and effective point indication are input into a diffusion model, and sample grabbing point groups are generated on the sample object point clouds; the label point clouds are taken as label samples, the sample object point clouds are taken as a current candidate region, and the sample grabbing point groups are taken as generated samples, which are input into a contrast learning model to perform label sample contrast and generated sample contrast, and optimal sample grabbing regions are screened out; and a group of grabbing points that are most similar to the label sample features in the optimal sample grabbing regions are taken as optimal sample grabbing points. The application has the beneficial effect that the grabbing success rate of the mechanical hand for objects with the same structure but different structures, the same shape but different materials and the like in a complex environment is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of robot target grasping, and in particular relates to a mechanical hand grasping point training method and device based on a diffusion model. BACKGROUND

[0002] With the continuous development of artificial intelligence and robot technology, mechanical hands play an increasingly important role in many fields such as industrial manufacturing, logistics sorting, and home service. Realizing the precise grasping of various objects by mechanical hands is one of the core links to promote the intelligent level of robots. Traditional grasping point prediction methods mostly rely on geometric rule modeling of objects or feature analysis based on two-dimensional images. These methods often show great limitations when facing objects with complex and diverse shapes, partial occlusion, or significant changes in illumination in real environments. With the popularity of 3D sensors and point cloud reconstruction technology, three-dimensional point clouds have gradually become an important data basis for object representation and operation tasks, bringing more abundant and detailed spatial information for mechanical hand grasping.

[0003] However, grasping point prediction based on three-dimensional point clouds still faces many technical challenges. First, point cloud data itself has problems such as sparsity, irregularity, and noise interference, which directly affect the accuracy of feature extraction and grasping area modeling. Second, the distribution of mechanical hand grasping points often has high local correlation and diversity, and the spatial position and physical characteristics of the optimal grasping point differ greatly under different objects and different scenarios. Currently, deep neural network methods based on regression or classification are mainly used for grasping point prediction of three-dimensional point clouds, which has improved the grasping point prediction effect to some extent, but still has significant shortcomings in the spatial distribution and fine-grained distinction of object surface grasping points, making it difficult to fully meet the needs of complex real-world environments. The specific defects are as follows: point cloud sparsity and occlusion lead to grasping failure; when the point cloud is sparse or partially severely occluded, the ability to analyze the geometry of the contact area is significantly reduced when evaluating the grasping quality from the point cloud through an end-to-end network, resulting in a decrease in grasping success rate; limited generalization, training samples are dependent, and in the existing technology, large-scale labeled samples are usually used for training, and the optimization is mostly for specific object categories, the model performance decreases significantly in unknown object or deformed object scenarios, traditional grasping methods have weak adaptability to new objects, and it is difficult to ensure the grasping robustness on unseen objects; insufficient modeling of grasping point spatial distribution, current mainstream grasping prediction methods are mostly regression or classification strategies, only scoring each candidate point or pose, without modeling the spatial structure of the grasping point distribution, lacking a comparison and recognition mechanism for subtle differences between candidate grasping regions, which makes the grasping points in clustered regions easy to blur or overlap, affecting the decision quality. SUMMARY

[0004] Therefore, the present application aims to overcome the above-mentioned problems in the prior art, and proposes a mechanical hand grasping point training method and device based on a diffusion model.

[0005] To achieve the above-mentioned purposes, the technical scheme of the present application is as follows:

[0006] The present application provides a mechanical hand grasping point training method based on a diffusion model, comprising the following steps:

[0007] S1. Obtain a sample object point cloud of a training sample body, obtain a marked point cloud marked on the training sample body, and record the sample object point cloud and the marked point cloud as marked data, wherein when the marked point cloud is grasped as a sample grasping point, the training sample body can be stably grasped;

[0008] S2. Input the marked data, diffusion time and effective point indication into a diffusion model, and perform reverse denoising processing after step-by-step noise adding processing, to generate a region set of candidate sample grasping points on the sample object point cloud, recorded as a sample grasping point group;

[0009] S3. Input the marked point cloud as a marked sample, the sample object point cloud as a current candidate region, and the sample grasping point group as a generated sample into a contrast learning model, to obtain corresponding sample features, and then perform marked sample contrast and generated sample contrast, and minimize the marked sample contrast result and maximize the generated sample contrast result, to screen out an optimal sample grasping region;

[0010] S4. Take a group of grasping points most similar to the marked sample features in the optimal sample grasping region as optimal sample grasping points.

[0011] Further, the sample object point cloud set in step S1 is recorded as P, and wherein represents the position of the i-th point in the point cloud in space, N represents the number of points in the point cloud;

[0012] The marked point cloud set is recorded as A, and wherein represents the position of the i-th point in the point cloud in space, j represents the position of the i-th point in the point cloud in space, represents the contact normal vector, represents the contact surface parameter, and M represents the number of fingers of the mechanical hand.

[0013] Further, in step S2, the reverse denoising process is: the marked point cloud and the current marked point cloud after adding noise are input into a diffusion prediction unit together with the features, wherein the diffusion prediction unit includes a plurality of diffusion prediction modules with the same structure, each diffusion prediction module includes two encoding layers with the same structure and two decoding layers with the same structure,

[0014] In the same diffusion prediction module, the marked point cloud is input into the first encoding layer for diffusion time-aware dynamic attention mechanism processing, the output data of the first encoding layer are input into the second encoding layer and the first decoding layer respectively, the current marked point cloud after adding noise is input into the first decoding layer for diffusion time-aware dynamic attention mechanism processing, and then processed through the cross-dimensional conditional interaction module, the obtained output data are input into the second encoding layer and the second decoding layer respectively, the output data of the first encoding layer and the output data of the first decoding layer after cross-dimensional conditional interaction module processing are fused and then processed through diffusion time-aware dynamic attention mechanism in the second encoding layer, the output data of the first encoding layer and the first decoding layer after diffusion time-aware dynamic attention mechanism processing are fused and then processed through the cross-dimensional conditional interaction module in the first decoding layer, the output data of the first decoding layer are input into the second decoding layer for diffusion time-aware dynamic attention mechanism processing, the obtained output data are fused with the output data of the second encoding layer and then processed through the cross-dimensional conditional interaction module in the second decoding layer, and the output data of the second decoding layer are sequentially processed through linear transformation and normalization and then data splitting to form first point cloud features and second point cloud features,

[0015] and the first point cloud features in each diffusion prediction module are fused with the features of the next diffusion time and then input into the next diffusion prediction module; the second point cloud features in each diffusion prediction module are fused together through skip connection, and sample grabbing point groups are output.

[0016] Further, in step S2, the diffusion adaptive encoding module is used in the noise adding process, and the adaptive diffusion time encoding expression is,

[0017] ,

[0018] wherein Gate is a gating function; t is the stage of the diffusion process, which is a scalar identifier of the diffusion time; sin-cos(t) is a trigonometric function operation combination with t as the variable; W g is a learnable weight matrix, and represents element-level multiplication; Embedding ( t ) is the original t-related encoding;

[0019] The diffusion time-aware dynamic attention mechanism adjusts the self-attention weight by introducing a diffusion time embedding, and the diffusion time-aware dynamic attention mechanism expression is,

[0020] ,

[0021] wherein: is diffusion encoding, γ is a learnable weight, Q, K, V are query, key, value in the attention mechanism respectively, t is diffusion time, d is feature dimension, W q 、W k 、W v Q, K, V are the corresponding learnable weight matrix respectively; soft max(·) is an activation function;

[0022] The expression of the cross-dimensional conditional interaction module is:

[0023] ,

[0024] ,

[0025] wherein, in the attention mechanism, is the query, key, value in the crawling data, is the query, key, value of the current noisy crawling point state, and flows bidirectionally between the crawling data and the current noisy crawling point state.

[0026] Further, in step S3, a feature encoder is arranged in the contrast learning model, and the labeled point cloud, the sample object point cloud and the sample crawling point group are input into the feature encoder, wherein the labeled sample is denoted as , the current candidate region is denoted as , and the generated sample is denoted as .

[0027] The process of obtaining the sample feature in the contrast learning model is that the feature encoder extracts the point cloud feature through parameters , and maps , , to the labeled sample feature , the current candidate region feature , and the generated sample feature respectively.

[0028] And the labeled sample comparison process is to calculate the distance between the current candidate region feature and the labeled sample feature by a distance function, wherein

[0029] ,

[0030] The generation sample comparison process is to calculate the distance between the current candidate region feature and the generation sample feature by a distance function wherein,

[0031] .

[0032] Further, the optimal sample grasping point is denoted as , and wherein represents the preferred sample grasping region, is the feature expression of , and is the feature expression of the labeled sample; is a similarity measurement function; is a function for obtaining the interval where the maximum value of is located.

[0033] The second aspect of the present application provides a grasping point prediction method based on the above-mentioned model training method, comprising the following steps:

[0034] s1. obtaining an object point cloud of a target object;

[0035] s2. inputting the object point cloud of the target object into the trained diffusion model, predicting and outputting the best grasping point through the diffusion model as the grasping target of the end effector of the robot hand.

[0036] Further, the object point cloud includes point coordinates, normal vectors or point sets with color information.

[0037] The third aspect of the present application provides a model training device, comprising:

[0038] a data acquisition module for acquiring a sample object point cloud of a training sample body and acquiring a labeled point cloud labeled on the training sample body;

[0039] a first processing module for inputting the labeled data combined with diffusion time and effective point indication into the diffusion model, and performing reverse denoising processing after step-by-step noise adding processing to generate a sample grasping point group on the sample object point cloud;

[0040] a second processing module for inputting the labeled point cloud as a labeled sample, the sample object point cloud as a current candidate region, and the sample grasping point group as a generation sample into a contrast learning model to obtain corresponding sample features, and then performing labeled sample comparison and generation sample comparison, and performing minimum processing on the labeled sample comparison result and maximum processing on the generation sample comparison result to screen out a preferred sample grasping region;​​

[0041] The confirmation module is configured to confirm a set of grasping points in the preferred sample grasping area that are most similar to the marked sample features as optimal sample grasping points.

[0042] Compared with the prior art, the present application has the following advantages:

[0043] The method of the present application improves the modeling capability of the spatial distribution of grasping points in the point cloud by using the diffusion model, and introduces contrast learning to enhance the feature distinction between different regions. The method fully combines the generative modeling capability of the diffusion model and the high discriminative characteristics of contrast learning, and performs adaptive modeling and optimization selection on the key regions in the grasping process of the robot hand. By introducing noise disturbance and contrast training mechanism of positive and negative region samples, the method can effectively improve the distinguishing ability of the model for different grasping points, and enhance the robustness to the sparsity and noise of the point cloud. The method not only improves the grasping success rate of the robot hand for objects with the same type but different structures, and the same shape but different materials in complex environments, but also provides a solid technical support for the generalization ability of intelligent robots in multiple scene applications, and has important theoretical significance and application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the illustrative embodiments of the present application and their descriptions, and do not constitute improper limitations on the present application. In the drawings:

[0045] Figure 1 The training method flowchart of the embodiment one of the present application;

[0046] Figure 2 The acquisition method schematic diagram of the marked data of the embodiment one of the present application;

[0047] Figure 3 The diffusion model training flowchart of the embodiment one of the present application;

[0048] Figure 4 The diffusion model structure schematic diagram of the embodiment one of the present application;

[0049] Figure 5 The diffusion prediction unit structure schematic diagram of the embodiment one of the present application;

[0050] Figure 6 The contrast learning model processing flowchart of the embodiment one of the present application;

[0051] Figure 7 The prediction method flowchart of the embodiment two of the present application. DETAILED DESCRIPTION

[0052] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0053] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0054] Embodiment one

[0055] As Figure 1 shown, the mechanical hand grasping point training method based on the diffusion model, that is, the model training method, includes the following steps:

[0056] S1. As Figure 2 shown, sample object point clouds of a training sample body are obtained, and labeled point clouds labeled on the training sample body are obtained. The sample object point clouds and the labeled point clouds are recorded as labeled data. When the labeled point clouds are grasped as sample grasping points, the training sample body can be stably grasped;

[0057] S2. As Figure 3 shown, the labeled data is input into the diffusion model combined with diffusion time and effective point indication. After step-by-step noise adding processing and reverse denoising processing, a region set of candidate sample grasping points is generated on the sample object point clouds, which is recorded as a sample grasping point group;

[0058] S3. As Figure 6 shown, the labeled point clouds are taken as labeled samples, the sample object point clouds are taken as current candidate regions, and the sample grasping point group is taken as generated samples, which are input into a contrast learning model to obtain corresponding sample features. Then, labeled sample comparison and generated sample comparison are performed, and the labeled sample comparison result is minimized, and the generated sample comparison result is maximized. The preferred sample grasping region is screened out, that is, a plurality of preferred sample grasping points obtained by further optimization based on the sample grasping point group (not shown in Figure 3 In this embodiment, the labeled sample comparison result is minimized to make the contrast learning model learn that similar point cloud features are close to each other. The generated sample comparison result is maximized to make the contrast learning model learn that different point cloud features are far away from each other. Through the optimization goal of minimizing the distance between the labeled samples and maximizing the distance between the generated samples, the feature encoder learns point cloud features with higher distinguishability;

[0059] S4. The set of grasping points most similar to the labeled sample features in the preferred sample grasping region is taken as the optimal sample grasping point, that is, the result obtained by further convergence processing in the preferred sample grasping region.

[0060] The set of sample object point clouds in step S1 is recorded as P, and wherein represents the position of the i-th point in the point cloud in space, N represents the number of points in the point cloud;

[0061] A is a set of marked point clouds, and wherein represents the first j the position of the grasping point formed by the root finger in space, represents the contact normal vector, represents the contact surface parameter, M represents the number of fingers of the manipulator, that is, the number of points in the marked point cloud. In this embodiment, , represents that the manipulator adopts a grasping mode of 5 fingers.

[0062] As shown in Figure 4 and Figure 5 , the diffusion model in step S2 includes a multi-scale feature fusion module, a diffusion adaptive coding module and a diffusion prediction unit. The diffusion adaptive coding module is used for step-by-step noise adding processing, and the diffusion prediction unit is used for reverse denoising processing. Wherein,

[0063] The noise adding process is:

[0064] The sample object point cloud, the marked point cloud and the diffusion time are input into the multi-scale feature fusion module to obtain the fusion features. After the fusion features are removed with the effective point indication (Mask), the effective data is processed to obtain the effective fusion features. The diffusion time is input into the diffusion adaptive coding module to dynamically control the diffusion time, and the adaptive diffusion time coding result is obtained. The coding can adapt to different denoising steps, so as to effectively improve the generation quality and denoising performance. The adaptive diffusion time coding result and the effective fusion features are integrated by addition to obtain the coding processed marked point cloud (point cloud features, including space, features, effectiveness, diffusion time coding) and the state of adding noise to the current marked point cloud (that is, the noise data of the marked point cloud at a certain time in the diffusion process).

[0065] The reverse denoising process is:

[0066] The marked point cloud and the features of the current marked point cloud after adding noise are input into the diffusion prediction unit. The diffusion prediction unit includes a plurality of diffusion prediction modules with the same structure. Each diffusion prediction module includes two coding layers with the same structure and two decoding layers with the same structure. The diffusion time perception dynamic attention mechanism and the cross-dimension condition interaction module are used to realize the bidirectional interaction between the condition branch and the noisy branch. Specifically,

[0067] In the same diffusion prediction module, the marked point cloud is input into a first encoding layer for diffusion time-aware dynamic attention mechanism processing, the output data of the first encoding layer are respectively input into a second encoding layer and a first decoding layer, the state of the current marked point cloud added with noise is input into the first decoding layer for diffusion time-aware dynamic attention mechanism processing, and then the output data are processed through a cross-dimensional conditional interaction module, the obtained output data are respectively input into the second encoding layer and a second decoding layer, the output data of the first encoding layer and the output data of the first decoding layer processed through the cross-dimensional conditional interaction module are fused and then processed through diffusion time-aware dynamic attention mechanism in the second encoding layer, the output data of the first encoding layer and the output data of the first decoding layer processed through diffusion time-aware dynamic attention mechanism are fused and then processed through the cross-dimensional conditional interaction module in the first decoding layer, the output data of the first decoding layer are input into the second decoding layer for diffusion time-aware dynamic attention mechanism processing, the output data obtained are fused with the output data of the second encoding layer and then processed through the cross-dimensional conditional interaction module in the second decoding layer, the output data of the second decoding layer are sequentially processed through linear transformation and normalization, and then data splitting is performed, to form a first point cloud feature and a second point cloud feature, and the first point cloud feature in each diffusion prediction module is fused with the feature of the next diffusion time and then input into the next diffusion prediction module; the second point cloud features in each diffusion prediction module are fused together through skip connection, and a sample grabbing point group is output.

[0068] As Figure 4 and Figure 5As shown, in this embodiment, the number of diffusion prediction modules is 3, in the same diffusion prediction module, in each encoding layer, the input data is processed by the diffusion time perception dynamic attention mechanism of the current encoding layer, and then subjected to local convolution (Local Conv), the local convolution result is combined with the input data of the current encoding layer to perform the first residual normalization (Add&Norm) processing of the current encoding layer, and then subjected to the feed forward network (Feed Forward) processing of the current encoding layer, the feed forward network processing result of the current encoding layer is combined with the first residual normalization result of the current encoding layer to perform the second residual normalization processing of the current encoding layer; in each decoding layer, the input data is processed by the diffusion time perception dynamic attention mechanism of the current decoding layer, and then combined with the input data of the current decoding layer to perform the first residual normalization processing of the current decoding layer, the first residual normalization result of the current decoding layer is combined (i.e. the first residual normalization result of the first decoding layer is added to the second residual normalization result output by the first encoding layer, and the first residual normalization result of the second decoding layer is added to the second residual normalization result output by the second encoding layer) with the second residual normalization result output by the corresponding level of encoding layer, and then processed by the cross-dimension conditional interaction module of the current decoding layer, combined with the first residual normalization result of the current decoding layer to perform the second residual normalization processing of the current decoding layer, and then processed by the feed forward network of the current decoding layer, combined with the second residual normalization result of the current decoding layer to perform the third residual normalization processing of the current decoding layer; the input data of the first encoding layer is the marker point cloud; the input data of the first decoding layer is the state of the current marker point cloud with noise, and the input data of the second decoding layer is the third residual normalization result output by the first decoding layer; the output data of the cross-dimension conditional interaction module in the first decoding layer is fed back to the diffusion time perception dynamic attention mechanism of the second encoding layer, combined (i.e. added) with the second residual normalization result output by the first encoding layer as the input data of the second encoding layer; the third residual normalization result output by the second decoding layer is sequentially processed by linear transformation (Linear) and normalization (Softmax) to perform data splitting (Split), forming the first point cloud feature and the second point cloud feature, and the first point cloud feature in each diffusion prediction module is input into the next diffusion prediction module after being fused with the feature of the next diffusion time, completing the recursion of the diffusion chain, and the second point cloud features in each diffusion prediction module are fused together through skip connection, outputting the sample grabbing point group.

[0069] In this embodiment, the main function of the encoding layer part is to globally model the conditional data features and integrate the context information.

[0070] In this embodiment, the sample object point cloud, the marker point cloud and the diffusion time are input into the multi-scale feature fusion module, and then different scale convolution kernels are used for feature screening to obtain multi-scale local features. The multi-scale convolution results are spliced (Concat) and then dimensionally reduced or aligned through linear projection (Linear Projection) to obtain the fused features. Among them, the different scale convolution kernels are 3x3, 5x5 and 7x7 convolution kernels, which can extract local features at different scales, thereby significantly improving the ability to capture details; here the normal vector can use PCA neighborhood = 20, the Adam optimizer can be selected in the training stage, the initial learning rate is 1e -4 , the batch size is 16, and the training is 200 epochs (the above values can be adjusted according to the specific scene requirements). In this embodiment, Mask is a binary flag, mask = 1 indicates that the data point is valid (actual observation value), and mask = 0 indicates that the data point is invalid (the position is missing data). In the subsequent calculation process, multiplying the fused features by the Mask can effectively eliminate invalid points.

[0071] In step S2, the diffusion adaptive coding module is used in the noise adding process, and the adaptive diffusion time coding expression is,

[0072] ,

[0073] Among them, Gate is a gating function; t is the stage of the diffusion process, which is a scalar identifier of the diffusion time; sin-cos(t) is a trigonometric function operation combination with t as the variable, which uses the sine function value minus the cosine function value as an input of the gate function Gate. The purpose is to map the diffusion time t to the trigonometric function feature space, use the periodicity, smooth change and other characteristics of the trigonometric function to provide a regular input signal related to the diffusion time t for the gating function, and let the gating effect present the corresponding change mode with t (different diffusion stages); W g is a learnable weight matrix, and represents element-level multiplication; Embedding ( t ) is the original t-related coding;

[0074] The diffusion time-aware dynamic attention mechanism adjusts the self-attention weight by introducing diffusion time embedding, and the diffusion time-aware dynamic attention mechanism expression is,

[0075]

[0076] Among them, For diffusion encoding, gamma is a learnable weight used to regulate the strength of attention, Q, K, and V are query, key, and value in the attention mechanism, t is the diffusion time, and d is the feature dimension, W q 、W k 、W v are the learnable weight matrices corresponding to Q, K, and V, respectively; soft max(·) is an activation function; in this embodiment, the diffusion time-aware dynamic attention mechanism of the encoding layer and the decoding layer is consistent;

[0077] The expression of the cross-dimensional conditional interaction module is:

[0078]

[0079]

[0080] In the attention mechanism, is the query, key, and value in the captured data, is the query, key, and value of the current noisy capture point state, and flows bidirectionally between the labeled point cloud data and the current noisy capture point state. In this embodiment, the condition vector of the cross-dimensional conditional interaction module not only unidirectionally inputs Decoder as a condition reference, but also reversely interacts with the intermediate layer output of Encoder for cross-dimensional feature interaction, mainly used to realize more rich bidirectional interaction between the condition feature and the denoising target feature, thereby significantly improving the expression ability and reasoning ability of the diffusion prediction module.

[0081] In step S3, a feature encoder is provided in the contrast learning model to extract the feature representation of the capture region. The labeled point cloud, sample object point cloud, and sample capture point group are input to the feature encoder, wherein the labeled sample is denoted as , which is the actual capture success region, and the known target of the same kind; the current candidate region is denoted as , which is the original sample; and the generated sample is denoted as ;

[0082] The process of obtaining the sample feature in the contrast learning model is that the feature encoder extracts the point cloud feature through the parameters , and maps , , to the labeled sample feature , the current candidate region feature , and the generated sample feature , respectively;

[0083] ​​Furthermore, the process of comparing labeled samples involves calculating the features of the current candidate region using a distance function. Features of labeled samples distance ,in,

[0084] ,

[0085] The process of generating and comparing samples involves calculating the features of the current candidate region using a distance function. Features of generated samples distance ,in,

[0086] .

[0087] In step S4, the optimal sample capture point is denoted as... ,and ,in Indicates the preferred sample capture area. for Feature representation, To represent the features of the labeled samples; For the similarity measurement function, preferably, Cosine similarity; To obtain The function whose maximum value lies in the interval. Here It is the optimal sample grasping point for the end effector of the robot arm, and can be directly used for the grasping action control and execution when the robot arm grasps training samples.

[0088] Example 2

[0089] like Figure 7 As shown, the prediction method, implemented using the training method described in Example 1, includes the following steps:

[0090] s1. Obtain the object point cloud of the target object;

[0091] s2. Input the point cloud of the target object into the trained diffusion model. The diffusion model predicts and outputs the optimal grasping point, including spatial coordinates, normal vector, confidence score, etc., as the grasping target of the robot's end effector. In this embodiment, the grasping point is actually a group of points, consistent with the number of fingers of the robot. For example, if the robot to be used has 5 fingers, the predicted optimal grasping point is actually a group of points including 5 point positions, corresponding to the final grasping position of each finger. Figure 7 The best grab point is only shown as one point location (the location of the red dot), while the other point locations are obscured behind the target object.

[0092] An object point cloud includes point coordinates, normal vectors, or a set of points with color information.

[0093] In this embodiment, objects with different structures in the same category (such as cups with different shapes and sizes) and objects with the same shape but different weights and materials can be automatically adapted, and more accurate and robust spatial modeling of the grasping area is achieved. In the experiment, on a dataset containing multiple different cups (regular cups, irregular handle cups, curved edge cups, etc.) and materials (ceramic, plastic, glass, stainless steel, etc.), the accuracy of grasping area prediction is improved by 8-15 percentage points compared to existing mainstream algorithms such as PointNet++ and GPD. The specific evaluation method uses public datasets such as GraspNet to separately count the Top-1 grasping point prediction accuracy and the actual robot grasping success rate for each type of object. The evaluation standard is that the robot performs multiple grasping experiments with the predicted grasping point as the target, and the object is grasped and held steadily for 3 seconds to be considered successful. Experimental data shows that in the context of irregular cups, the average grasping success rate of the present application can reach 92%, while the existing method is 78-83%; in the context of objects with the same shape but different materials, the grasping success rate of the present application is 89%, while the existing method is 75-80%.

[0094] Secondly, the present application can effectively improve the prediction stability and anti-interference ability when dealing with low-quality, sparse and noisy point cloud data, significantly reducing the probability of grasping failure caused by data occlusion and missing. After testing with standard point cloud downsampling and Gaussian noise simulation methods, it was found that the present application method still maintains an accuracy rate of 85% or above in grasping area prediction when the point cloud data is sparse to 30% of the original number of points and superimposes 10% noise, which is significantly better than existing mainstream models.

[0095] Thirdly, the present application can achieve good generalization performance without relying on massive amounts of high-quality human-labeled data, greatly reducing the model deployment and migration cost, and improving the efficiency of application in industrial scenarios. Compared with existing products, the present application improves the average grasping success rate by about 12 percentage points compared to PointNet++ in the transfer experiment between different object categories, and simplifies the model structure and deployment process, shortening the system online period.

[0096] In summary, the present application not only improves the accuracy of grasping point prediction and the success rate of robot grasping, but also enhances the adaptability to diverse objects and robustness in complex environments, simplifying the model training and deployment process. The above test results show that the present application has significant advantages over existing technologies and can meet the application requirements of intelligent manufacturing, automatic sorting, complex assembly and other diverse high-demand scenarios.

[0097] Embodiment Three

[0098] The model training device comprises:

[0099] The data acquisition module is configured to acquire a sample object point cloud of a training sample body and acquire a marked point cloud marked on the training sample body, and record the sample object point cloud and the marked point cloud as marked data.

[0100] The first processing module is configured to input the marked data, diffusion time and effective point indication into a diffusion model, perform step-by-step noise adding processing, and then perform reverse denoising processing to generate a sample grasping point group on the sample object point cloud.

[0101] The second processing module is configured to input the marked point cloud as a marked sample, the sample object point cloud as a current candidate region, and the sample grasping point group as a generated sample into a contrast learning model to obtain corresponding sample features, perform marked sample contrast and generated sample contrast, minimize the marked sample contrast result, maximize the generated sample contrast result, and screen out an optimal sample grasping region.

[0102] The confirmation module is configured to take a group of grasping points most similar to the marked sample features in the optimal sample grasping region as optimal sample grasping points.

[0103] The above merely describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a gripping point of a robot hand based on a diffusion model, characterized in that, The method comprises the following steps: S1. Obtain a sample object point cloud of a training sample body, obtain a marked point cloud marked on the training sample body, and record the sample object point cloud and the marked point cloud as marked data, wherein the marked point cloud can stably grasp the training sample body when the sample grasping point is grasped; S2. Input the marked data, diffusion time and effective point indication into a diffusion model, perform step-by-step noise adding processing, and then perform reverse denoising processing to generate a region set of candidate sample grasping points on the sample object point cloud, recorded as a sample grasping point group; The reverse denoising processing process is that the marked point cloud and the feature of the current marked point cloud after adding noise are input into a diffusion prediction unit, wherein the diffusion prediction unit comprises a plurality of diffusion prediction modules with the same structure, and each diffusion prediction module comprises two encoding layers with the same structure and two decoding layers with the same structure, In the same diffusion prediction module, the marked point cloud is input into the first encoding layer for diffusion time perception dynamic attention mechanism processing, the output data of the first encoding layer are input into the second encoding layer and the first decoding layer respectively, the current marked point cloud after adding noise is input into the first decoding layer for diffusion time perception dynamic attention mechanism processing, and then the output data are processed through a cross-dimension conditional interaction module to obtain output data which are input into the second encoding layer and the second decoding layer respectively, the output data of the first encoding layer and the output data of the first decoding layer after the cross-dimension conditional interaction module processing are fused and then subjected to diffusion time perception dynamic attention mechanism processing in the second encoding layer, the output data of the first encoding layer and the output data after the diffusion time perception dynamic attention mechanism processing in the first decoding layer are fused and then processed through the cross-dimension conditional interaction module in the first decoding layer, the output data of the first decoding layer are input into the second decoding layer for diffusion time perception dynamic attention mechanism processing, the output data obtained after the processing and the output data of the second encoding layer are fused and then processed through the cross-dimension conditional interaction module in the second decoding layer, and the output data of the second decoding layer are subjected to linear transformation and normalization processing in sequence and then subjected to data splitting to form first point cloud features and second point cloud features, and the first point cloud features in each diffusion prediction module are fused with the features of the next diffusion time and then input into the next diffusion prediction module; the second point cloud features in each diffusion prediction module are fused through skip connection to output the sample grasping point group; The diffusion time perception dynamic attention mechanism adjusts the self-attention weight by introducing diffusion time embedding, and the diffusion time perception dynamic attention mechanism expression is ； wherein, is diffusion encoding, γ is a learnable weight, Q, K, V are query, key, value in attention mechanism respectively, t is diffusion time, d is feature dimension, W q 、W k 、W v are learnable weight matrices corresponding to Q, K, V respectively; soft max(·) is an activation function; The expression of the cross-dimension conditional interaction module is ， ， wherein, in the attention mechanism, for queries, keys, values in the data to be grabbed, for queries, keys, values in the current noisy grab point state, and bidirectional flow between the landmark point cloud data and the current noisy grab point state; S3. Input the marked point cloud as a marked sample, the sample object point cloud as a current candidate region, and the sample grasping point group as a generated sample into a contrast learning model to obtain corresponding sample features, perform marked sample contrast and generated sample contrast, minimize the marked sample contrast result, maximize the generated sample contrast result, and screen out an optimal sample grasping region. S4. The set of grasping points in the preferred sample grasping area that is most similar to the labeled sample feature is taken as the optimal sample grasping point.

2. The method of claim 1, wherein the method further comprises: The set of sample object point clouds in step S1 is denoted by P, and wherein represents the position of the i-th point in the point cloud in space, N represents the number of points in the point cloud; The set of labeled point clouds is denoted as A, and wherein denotes the j the position of the grasp point formed by the root finger in space, denotes the contact normal vector, denotes the touch surface parameter, and M represents the number of fingers of the robot hand.

3. The method of claim 1, wherein the method further comprises: In step S2, the diffusion adaptive coding module is used in the noise adding process, and the adaptive diffusion time coding expression is, ， wherein Gate is a gating function; t is a stage of a diffusion process, which is a scalar identifier of diffusion time; sin-cos(t) is a trigonometric function operation combination with t as a variable; W g is a learnable weight matrix, and denotes element-wise multiplication; Embedding(t) is an original, t-related code. 4.The method of claim 1, wherein: In step S3, the feature encoder is arranged in the contrast learning model, and the marked point cloud, the sample object point cloud and the sample grasping point cloud are input into the feature encoder, wherein the marked sample is denoted as , the current candidate region is denoted as , and the generated sample is denoted as . In the process of obtaining the sample feature in the contrast learning model, the feature encoder maps the input image to the feature by parameters Extract the point cloud feature, and map , , The labeled sample feature , the current candidate region feature , and the generated sample feature , respectively. and the marked sample pair comparison process is to calculate the distance function of the current candidate region feature and the marked sample feature distance wherein, ， The process of generating and comparing samples involves calculating the features of the current candidate region using a distance function. Features of generated samples distance ,in, 。 5. The method of claim 1, wherein: In step S4, the optimal sample grabbing point is denoted as , and , wherein represents the preferred sample grabbing region, is the feature expression of , and is the feature expression of the marked sample; is a similarity measure function; is a function for obtaining the interval where the maximum value of is located.

6. A prediction method implemented using the robot gripper point training method based on a diffusion model according to any one of claims 1 to 5, characterized in that, The method comprises the following steps: s1. Obtain the object point cloud of the target object; s2. Input the object point cloud of the target object into the trained diffusion model, and output the best grasping point predicted by the diffusion model as the grasping target of the end effector of the manipulator.

7. A model training device for implementing the method for training a gripping point of a robot hand based on a diffusion model according to any one of claims 1 to 5, characterized in that, Comprise: A data acquisition module is configured to acquire sample object point clouds of training sample objects and acquire labeled point clouds on the training sample objects; A first processing module is configured to input the labeled data, diffusion time and effective point indication into the diffusion model, perform step-by-step noise adding processing, and then perform reverse denoising processing to generate a sample grasping point group on the sample object point clouds; A second processing module is configured to input the labeled point cloud as a labeled sample, the sample object point cloud as a current candidate area, and the sample grasping point group as a generated sample into a contrast learning model to obtain corresponding sample features, perform labeled sample comparison and generated sample comparison, minimize the labeled sample comparison result, maximize the generated sample comparison result, and screen out a preferred sample grasping area; A confirmation module is configured to take a set of grasping points in the preferred sample grasping area that is most similar to the labeled sample feature as the optimal sample grasping point.

Images