A six-degree-of-freedom grasping detection method and system based on physical prior knowledge

Abstract

A six-degree-of-freedom grasping detection method and system based on physical prior knowledge, the method comprising: 1, obtaining a plurality of scene point clouds containing target objects, and labeling grasping poses for each scene point cloud; 2, building a six-degree-of-freedom grasping detection neural network model; 3, training the six-degree-of-freedom grasping detection neural network model using a training set, and updating model parameters during training; 4, evaluating the six-degree-of-freedom grasping detection neural network using a test set, and saving the best performing model as the model used for subsequent inference; 5, obtaining real grasping task scene point clouds and inputting them into the six-degree-of-freedom grasping detection neural network model to obtain the grasping pose of the target object in the real task scene, and the robot execution system completes the grasping and transportation task according to the grasping pose of the target object. The present application alleviates the dependence of the grasping detection model on the target object category information involved in the training to some extent, and improves the generalization of grasping detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot grasping, in particular to a six-degree-of-freedom grasping detection method and system based on physical prior knowledge. BACKGROUND

[0002] Precise object grasping is a key task of robot operation, which is to obtain a grasping pose that can form an effective contact and stable lifting of the target object from the image collected by the robot perception system, and then execute the grasping pose by the robot execution system equipped with a specific gripper to complete the grasping task. With the increasing automation in the field of intelligent manufacturing, six-degree-of-freedom grasping detection methods and systems are widely used in industrial manufacturing autonomous assembly, express logistics automatic sorting and other application scenarios.

[0003] In the face of different types and sizes of target objects, how to design a general six-degree-of-freedom grasping detection method is an important challenge. The six-degree-of-freedom grasping detection method based on deep learning breaks through the traditional way of searching for a grasping pose in physical space using fixed logic, and trains a grasping detection neural network model through a large amount of labeled data, greatly improving the success rate of generating effective grasping poses. However, the data-driven deep learning method inevitably leads to the dependence of the six-degree-of-freedom grasping detection model on training data, which is manifested in that when the inherent physical characteristics such as shape or size of the target object are significantly different from the training data, the grasping pose generation effect will show a cliff-like decline. Although existing means such as continuous learning and incremental learning can improve this cliff-like decline, the high training hardware cost hinders general production enterprises from considering and deploying the corresponding detection algorithm. Therefore, how to low-cost solve the dependence of the six-degree-of-freedom grasping detection model on training data, so that it can complete effective grasping pose detection of new categories of target objects without continuous investment in training costs is a challenging research content. SUMMARY

[0004] The present application provides a six-degree-of-freedom grasping detection method and system based on physical prior knowledge to solve the technical problems mentioned in the background.

[0005] To achieve the above purpose, the technical solution of the present application is as follows:

[0006] The present application provides a six-degree-of-freedom grasping detection method based on physical prior knowledge, comprising the following steps:

[0007] S1, acquire a plurality of scene point clouds containing target objects, label grasping poses for each scene point cloud, and divide the training set and the test set according to a predetermined proportion;

[0008] S2, a six-degree-of-freedom grasping detection neural network model based on physical prior knowledge is built, which comprises a point cloud sparse feature extraction module, a graspable region optimization regression module, a grasping parameter estimation module and a grasping pose refinement module based on physical prior knowledge connected in sequence;

[0009] S3, the six-degree-of-freedom grasping detection neural network model is trained by using the training set, a loss function is set, the difference between the output of the six-degree-of-freedom grasping detection neural network model and the real grasping pose is calculated, and the parameters of the six-degree-of-freedom grasping detection neural network are updated according to the gradient condition;

[0010] S4, the trained six-degree-of-freedom grasping detection neural network is evaluated by using the test set, and the model with the best performance in the evaluation link is saved as the six-degree-of-freedom grasping detection neural network model used for subsequent reasoning;

[0011] S5, the real grasping scene point cloud is obtained and input into the six-degree-of-freedom grasping detection neural network model obtained in S4 to obtain the grasping pose of the target object in the real working scene, and the robot execution system completes the grasping and transportation task according to the grasping pose of the target object.

[0012] Another aspect of the present application also provides a six-degree-of-freedom grasping detection system based on physical prior knowledge, comprising a robot execution system which performs grasping detection according to the six-degree-of-freedom grasping detection method.

[0013] Compared with the prior art, the present application has the following advantages:

[0014] 1. High grasping detection accuracy

[0015] The present method realizes the screening of the grasping candidate points in the grasping detection process by fitting the grasping candidate point confidence score to a three-dimensional Gaussian distribution, so that the confidence score distribution of the generated grasping candidate points is fitted to the real label (i.e. the grasping pose), reducing the possibility of misjudgment, not only accelerating the capture of the effective grasping region by the six-degree-of-freedom grasping detection neural network model, but also improving the reliability of the subsequent grasping parameter generation, and further improving the accuracy of the grasping detection.

[0016] 2. Good grasping detection generalization

[0017] The existing data-driven grasping detection method inevitably makes the neural network dependent on the training data, and often cannot ignore the influence of the inherent properties such as the size, shape and texture of the target object unrelated to grasping. The grasping pose parameter optimization method based on physical prior knowledge proposed by the present method further optimizes the grasping parameters based on the grasping effective flatness and force closure score, so that the optimized grasping parameters are more consistent with the real world evaluation standard, and to some extent, the dependence on the training data is eliminated. Attached Figure Description

[0018] Figure 1 This is a flowchart of the six-degree-of-freedom grasping and detection method in this invention. Detailed Implementation

[0019] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0020] This invention aims to propose a six-degree-of-freedom grasping detection method and system based on physical prior knowledge. It addresses the following issues in tasks such as autonomous assembly in industrial manufacturing and automatic sorting in express logistics under the field of intelligent manufacturing: (1) the accuracy of the grasping posture generated by the existing grasping detection model is not high, resulting in a low success rate of the robot execution system in completing robust grasping; (2) when the newly added target object is significantly different from the object used for training, the grasping detection model cannot transfer the learned knowledge to the grasping detection of the new object, and additional training costs are required to adapt to the new category. (1) A point-level graspable region optimization method based on the distribution trend of effective grasping posture is proposed. The effective grasping posture is represented by a three-dimensional Gaussian distribution, which shows that the effective grasping posture is clustered in space and the grasping quality score is negatively correlated with the difference between the local optimal grasping posture parameters. The score representation of the graspable region is optimized to solve the problem that the grasping detection network cannot effectively locate the graspable region in complex scenes, resulting in inaccurate grasping posture generation. (2) A grasping posture parameter optimization method based on physical prior knowledge is proposed. In order to solve the problem that the grasping detection model is inevitably affected by the inherent physical properties such as the shape, size and texture of the target object during the training process, the flatness and force closure evaluation criteria in the physical prior knowledge are used to provide optimization direction for the grasping posture parameters. The optimized grasping posture parameters can better fit the real-world grasping posture evaluation criteria, which to a certain extent alleviates the dependence of the grasping detection model on the target object category information participating in the training and improves the generalization of grasping detection.

[0021] The invention is described in detail below:

[0022] Reference Figure 1 This application provides a six-degree-of-freedom grasping and detection method based on prior physical knowledge, including the following steps:

[0023] S1. Obtain several scene point clouds containing the target object, label the capture pose for each scene point cloud, and divide the training set and test set according to a preset ratio (e.g., 9:1).

[0024] S2. Construct a six-degree-of-freedom grasping detection neural network model based on physical prior knowledge. This model includes a point cloud sparse feature extraction module, a graspable region optimization and regression module, a grasping parameter estimation module, and a grasping posture refinement module based on physical prior knowledge, which are connected in sequence.

[0025] S3. Train the six-degree-of-freedom grasping detection neural network model using the training set, set the loss function, calculate the difference between the output of the six-degree-of-freedom grasping detection neural network model and the actual grasping posture, and update the parameters of the six-degree-of-freedom grasping detection neural network according to the gradient.

[0026] S4. Use the test set to evaluate the trained six-degree-of-freedom grasping and detection neural network, and save the best-performing model in the evaluation stage as the six-degree-of-freedom grasping and detection neural network model for subsequent inference.

[0027] S5. Obtain the point cloud of the real grasping operation scene and input it into the six-degree-of-freedom grasping detection neural network model obtained in S4 to obtain the grasping posture of the target object in the real operation scene. The robot execution system completes the grasping and transfer operation tasks based on the grasping posture of the target object.

[0028] In some embodiments, S1 specifically includes the following steps:

[0029] S11. Obtain several scene point clouds containing the target object;

[0030] S12. For each scene point cloud, annotate the grasping posture of the target object when each point in the scene point cloud is used as the gripper center point. The grasping posture includes the grasping posture quality score, grasping candidate points, gripper opening width, gripper advance depth along the approach direction, and rotation matrix. The rotation matrix includes the approach direction and planar rotation. The specific formula is as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] in, Indicates the grasping posture for grabbing the target object; These represent the grasping quality score, grasping candidate points, gripper opening width, gripper advance depth along the approach direction, approach direction, and planar rotation angle, respectively; and represent the shape as... The real number field space, where This indicates the number of candidate points obtained from sampling; Represents the set of real numbers;

[0035] S13, divide the several annotated scene point clouds into a training set and a test set according to a preset ratio.

[0036] In some embodiments, the point cloud sparse feature extraction module is configured to extract feature information of a set dimension in the scene point cloud; in this embodiment, the set dimension refers to a high dimension; the point cloud sparse feature extraction module takes the scene point cloud perceived by the robot perception module as input and outputs a high-dimensional feature of the scene point cloud; the point cloud sparse feature extraction module is composed of MinkUNet18, the input channel is , and the output channel is In this example , ; as shown in Figure 1 , the point cloud sparse feature extraction module includes an encoder and a decoder connected in sequence.

[0037] The graspable region optimization regression module is configured to calculate and optimize the confidence score of the graspable region of the grasp candidate point; the graspable region optimization regression module takes the high-dimensional feature of the scene point cloud as input and outputs the confidence score of the graspable region of the grasp candidate point; the graspable region optimization regression module includes a graspable region confidence score regression head and a graspable region optimization block connected in sequence, wherein the graspable region confidence score regression head includes a convolution layer , a normalization layer one, a Relu activation layer one, and a convolution layer ; the graspable region optimization block is configured to optimize the confidence score of the graspable region output by the graspable region confidence score regression head; wherein the input dimension of the convolution layer is , and the output dimension is In this example , ; the input dimension of the convolution layer is , and the output dimension is In this example . The first two dimensions in the output of the convolution layer are used as the classification result of background segmentation, and the third dimension is used as the confidence score of the grasp region.

[0038] The grasp parameter estimation module is configured to estimate the grasp pose of the grasp candidate point as the center of the gripper; the grasp parameter estimation module takes the neighborhood feature of the grasp candidate point as input and outputs the grasp pose including the grasp pose quality score, the gripper opening width, the gripper advancing depth along the approaching direction, and the rotation matrix value; the grasp parameter estimation module includes a local point cloud cropping block, a feature fusion block, and a grasp parameter regression block; the feature fusion block includes a convolution layer ; the convolution layer is configured to perform feature fusion on the neighborhood point set feature obtained by the local point cloud cropping block; the convolution layer The input dimension of the convolution layer is , and the output dimension is ; the grasp parameter regression block comprises sequentially connected convolution layers , normalization layers two, Relu activation layers two, convolution layers , normalization layers three, Relu activation layers three, and convolution layers ; wherein the input dimension of the convolution layer is , and the output dimension is ; the input dimension of the convolution layer is , and the output dimension is ; the input dimension of the convolution layer is , and the output dimension is ; the output of the convolution layer is a first dimension of a confidence score, a second dimension of an approaching direction, a third dimension of a plane rotation angle, a fourth dimension of an opening width, and a fifth dimension of a gripper advancing depth along the approaching direction.

[0039] The grasp pose refinement module based on physical prior knowledge is used to optimize the grasp pose generated by the grasp parameter estimation module, and takes the scene point cloud and the grasp pose as input, and outputs the optimized grasp pose; the grasp pose refinement module comprises a contact point calculation block, a flatness calculation block, and a force closure calculation block, wherein the contact point calculation block is used to calculate the point closest to the gripper connecting line in the scene point cloud according to the scene point cloud and the grasp pose output by the grasp parameter estimation module, and the closest point is taken as the gripper contact point; the flatness calculation block and the force closure calculation block are used to calculate the neighborhood-related scores of the contact point.

[0040] In some embodiments, S3 specifically comprises the following steps:

[0041] S31, randomly selecting a scene point cloud from the training set, sampling sampling points from the scene point cloud as input to obtain an input point cloud ; in this example ;

[0042] S32, inputting the input point cloud into the point cloud sparse feature extraction module, the point cloud sparse feature extraction module performs feature clustering in a step-by-step sampling manner to obtain grasp candidate points and features with a dimension of , which are specifically expressed by the following formula:

[0043] ;

[0044] ;

[0045] wherein, represents a point cloud sparse feature extraction module; in this example ;

[0046] S33, the grasp candidate points and features are sent to the graspable region optimization regression module for candidate point quality evaluation, and the graspable region confidence score regression head predicts the confidence score of each grasp candidate point . The graspable region optimization block optimizes the confidence score of each candidate point , and obtains the confidence score of the candidate point after multiple optimizations ;

[0047] S34, the local point cloud cropping block in the grasp parameter estimation module crops the input point cloud with four groups of different radius cylinders . In this embodiment, the four groups of different radii are , , , in this example ;

[0048] After cropping, a local point cloud point set (i.e., neighborhood point set feature) is obtained, in this example ; the feature fusion block extracts and fuses the local point cloud point set to obtain the candidate point feature ; the candidate point feature is input into the grasp parameter regression block, and the grasp parameter regression block regresses the gripper approaching direction , the plane rotation angle , the gripper opening width , the forward depth along the approaching direction , and the quality score to obtain the estimated grasp pose;

[0049] The gripper approaching direction , the plane rotation angle , the gripper opening width , the forward depth along the approaching direction , and the quality score are all outputs of the grasp parameter estimation module;

[0050] S35, the grasp pose refinement module based on physical prior knowledge optimizes the predicted grasp pose to obtain the optimized grasp pose.

[0051] In some embodiments, the S33 specifically includes the following steps:

[0052] S331, the candidate points of grabbing are selected according to the feature and characteristics into the graspable area optimization regression module;

[0053] S332, the graspable area confidence score regression head in the graspable area optimization regression module calculates the confidence score of each candidate point of grabbing according to the feature ;

[0054] S333, for each candidate point of grabbing , the graspable area optimization block calculates the average Euclidean distance between it and the k neighbor points, and calculates the distance standard deviation according to the average Euclidean distance;

[0055] S334, the confidence score of each candidate point of grabbing is cleaned and optimized according to the average Euclidean distance and the distance standard deviation;

[0056] S335, after the confidence score cleaning and optimization of the candidate points, the candidate points of grabbing are grouped into graspable areas by using KNN clustering, and then a three-dimensional Gaussian distribution is fitted to the graspable areas, so as to realize further optimization of the confidence score of the candidate points, and obtain the confidence score of the candidate points after multiple optimizations .

[0057] In some embodiments, the calculation formula of the average Euclidean distance and the distance standard deviation in S333 is as follows:

[0058] ;

[0059] ;

[0060] ;

[0061] wherein, represents the average distance between the i th candidate point and other candidate points, represents the i th candidate point, represents the average Euclidean distance; represents the current i th neighbor point; represents the distance standard deviation; The cleaning and optimization in S334 is specifically as follows:

[0062] ;

[0063] ;

[0064] wherein, represents the i th candidate point,​​​​​ a confidence score of the candidate point, a confidence score of the cleaned candidate point; a weighting coefficient;

[0065] the confidence score of the candidate point optimized multiple times in the S335 is specifically as follows:

[0066]

[0067] wherein, represents a fitted three-dimensional Gaussian distribution, represents a mean generated after fitting, represents a covariance matrix generated after fitting; represents an exponential function with the natural constant e as the base.

[0068] In some embodiments, the S35 specifically comprises the following steps:

[0069] S351, input the input point cloud and the estimated grasp pose into a grasp pose refinement module based on physical prior knowledge, and the contact point calculation block in the grasp pose refinement module based on physical prior knowledge restores the jaw position according to the input point cloud and the estimated grasp pose, and then calculates the contact point neighborhood point set from the input point cloud and the jaw position.

[0070] S352, input the contact point neighborhood point set into a flatness calculation block, calculate the mean and the skewness matrix of each neighborhood point in the contact point neighborhood point set , then obtain three eigenvalues that decrease in turn from the skewness matrix, and calculate the flatness score according to the three eigenvalues;

[0071] S353, obtain the jaw line vector , input the jaw line vector into a force closure calculation block to calculate the force closure score , and then calculate the score of the estimated grasp pose according to the force closure score and the flatness score;

[0072] S354, then use the score gradient trend of the estimated grasp pose to adjust the estimated grasp pose of the grasp parameter estimation module to obtain the optimal grasp pose in the grasp pose parameter space.

[0073] In some embodiments, the contact point neighborhood point set ​The calculation formula of the mean value, the skewness variance matrix, and the flatness score of each neighborhood point is as follows:

[0074]

[0075]

[0076]

[0077] wherein, denotes the i-th neighborhood point, denotes the mean value of the neighborhood point, denotes the skewness variance matrix of the neighborhood point set; T denotes the transpose; denotes the neighborhood point set of the contact point in turn reduced three eigenvalues of the covariance matrix; denotes the flatness score; the gradient of the flatness score will provide an optimization suggestion for each parameter of the grasping pose as the optimization direction. The force closure score in the S353

[0078] The calculation formula of the score of the estimated grasping pose is as follows:

[0079]

[0080]

[0081] wherein, denotes the weighted coefficient; denotes the two-norm calculation.

[0082] In some embodiments, the loss function in the S3 is specifically as follows:

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] ;​​​​​​​​​​​​​​

[0092] ;

[0093] In the formula, The mean square error loss function is represented by MSE; The cross-entropy loss function is represented by CE; The respective values of the grasp candidate point score loss, the approach vector loss, the plane rotation loss, the grasp quality loss, the gripper advancement depth along the approach direction loss, the gripper opening width loss, and the confidence score loss are represented by L G, L A, L P, L Q, L D, L W, and L C, respectively. The respective values of the grasp candidate point score prediction, the approach vector prediction, the plane rotation prediction, the grasp quality prediction, the gripper advancement depth along the approach direction prediction, the gripper opening width prediction, and the confidence score prediction are represented by G, A, P, Q, D, W, and C, respectively. The respective values of the grasp candidate point score ground truth, the approach vector ground truth, the plane rotation ground truth, the grasp quality ground truth, the gripper advancement depth along the approach direction ground truth, the gripper opening width ground truth, and the confidence score ground truth are represented by G, A, P, Q, D, W, and C, respectively. The respective values of the weights of the respective losses are represented by w G, w A, w P, w Q, w D, w W, and w C, respectively.

[0094] Compared with the prior art, the present application has the following advantages:

[0095] 1. High accuracy of grasp detection

[0096] The present method optimizes the grasp candidate point screening link in the grasp detection process based on the characteristics of the effective grasp pose aggregation distribution, so that the confidence score distribution of the generated grasp candidate points conforms to the real label (i.e. the grasp pose), reducing the possibility of misjudgment, not only accelerating the capture of the effective grasp area by the six-degree-of-freedom grasp detection neural network model, but also improving the reliability of the subsequent grasp parameter generation, and thus improving the accuracy of the grasp detection.

[0097] ​​​​​​​​​​​​​​​​​​​​​​​2. Good generalization of grasping detection

[0098] The existing data-driven grasping detection method inevitably makes the neural network dependent on the training data, and often cannot ignore the influence of the inherent attributes such as the size, shape and texture of the target object unrelated to grasping. The grasping posture parameter optimization method based on physical prior knowledge provided by the method takes the effective flatness and force closure degree score of grasping as the basis to further optimize the grasping parameters, so that the optimized grasping parameters are more consistent with the real world evaluation standard, and to a certain extent, the dependence on the training data is eliminated.

[0099] Another aspect of the present application also provides a six-degree-of-freedom grasping detection system based on physical prior knowledge, comprising a robot execution system, and the robot execution system performs grasping detection according to the six-degree-of-freedom grasping detection method.

[0100] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Moreover, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the realization of the ordinary skilled person in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope required by the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A six-degree-of-freedom grasping and detection method based on prior physical knowledge, characterized in that, Includes the following steps: S1. Obtain several scene point clouds containing the target object, label the capture pose for each scene point cloud, and divide the training set and test set according to a preset ratio. S2. Construct a six-degree-of-freedom grasping detection neural network model based on physical prior knowledge. This model includes a point cloud sparse feature extraction module, a graspable region optimization and regression module, a grasping parameter estimation module, and a grasping posture refinement module based on physical prior knowledge, which are connected in sequence. S3. Train the six-degree-of-freedom grasping detection neural network model using the training set, set the loss function, calculate the difference between the output of the six-degree-of-freedom grasping detection neural network model and the actual grasping posture, and update the parameters of the six-degree-of-freedom grasping detection neural network according to the gradient. S4. Use the test set to evaluate the trained six-degree-of-freedom grasping and detection neural network, and save the best-performing model in the evaluation stage as the six-degree-of-freedom grasping and detection neural network model for subsequent inference. S5. Obtain the point cloud of the real grasping operation scene and input it into the six-degree-of-freedom grasping detection neural network model obtained in S4 to obtain the grasping posture of the target object in the real operation scene. The robot execution system completes the grasping and transfer operation tasks based on the grasping posture of the target object. The point cloud sparse feature extraction module is used to extract feature information of a set dimension from the scene point cloud; The crawlable region optimization regression module is used to calculate and optimize the confidence scores of crawlable candidate points; it includes a crawlable region confidence score regression head and a crawlable region optimization block connected in sequence, wherein the crawlable region confidence score regression head contains convolutional layers connected in sequence. One layer, one ReLU activation layer, and one convolutional layer. The crawlable region optimization block is used to optimize the confidence score of the crawlable region output by the crawlable region confidence score regression head. The grasping parameter estimation module is used to estimate the grasping posture using candidate grasping points as gripper centers. It includes a local point cloud cropping block, a feature fusion block, and a grasping parameter regression block. The feature fusion block includes convolutional layers. The parameter-capturing regression block contains sequentially connected convolutional layers. Normalization layer 2, ReLU activation layer 2, Convolutional layer Normalization layer 3, ReLU activation layer 3, and convolutional layer ; The grasping posture refinement module based on physical prior knowledge includes a contact point calculation block, a flatness calculation block, and a force closure calculation block. The contact point calculation block is used to calculate the closest point in the scene point cloud to the gripper line based on the scene point cloud and the grasping posture output by the grasping parameter estimation module, and uses the closest point as the gripper contact point. The flatness calculation block and the force closure calculation block are used to calculate the neighborhood correlation score of the contact point.

2. The six-degree-of-freedom grasping and detection method based on prior physical knowledge according to claim 1, characterized in that, S1 specifically includes the following steps: S11. Obtain several scene point clouds containing the target object; S12. For each scene point cloud, annotate the grasping posture of the target object when each point in the scene point cloud is used as the gripper center point. The grasping posture includes the grasping posture quality score, grasping candidate points, gripper opening width, gripper advance depth along the approach direction, and rotation matrix. The rotation matrix includes the approach direction and planar rotation. The specific formula is as follows: ； ； ； in, Indicates the grasping posture for grabbing the target object; , , , , , These represent the grasping quality score, grasping candidate point, gripper opening width, gripper advance depth along the approach direction, approach direction, and plane rotation angle, respectively. Indicates shape as The real number field space, where This indicates the number of candidate points obtained from sampling; Represents the set of real numbers; S13. Divide several labeled scene point clouds into training set and test set according to the preset ratio.

3. The six-degree-of-freedom grasping and detection method based on prior physical knowledge according to claim 2, characterized in that, S3 specifically includes the following steps: S31. Randomly select a scene point cloud from the training set, and sample from the scene point cloud. Using 10 sampling points as input, we obtain the input point cloud. ; S32, Input point cloud The input is fed into the point cloud sparse feature extraction module, which performs feature clustering using a step-by-step sampling method to obtain a point count of... Candidate points to be captured and dimensions are Features ; S33, Capture candidate points and characteristics The data is fed into the crawlable region optimization regression module for candidate point quality assessment, and the crawlable region confidence score regression head predicts the confidence score of each crawlable candidate point. It can capture the confidence score of each candidate point for the region optimization block. Optimization was performed to obtain the confidence scores of the candidate points after multiple optimizations. ; S34. Capture local point cloud clipping blocks within the parameter estimation module, using cylinders of different radii to clip the input point cloud. Cut to get Local point cloud point set Feature fusion blocks for local point cloud point sets Feature extraction and fusion are performed to obtain candidate point features. ; Features of candidate points The input is fed into the grasping parameter regression block, which in turn feeds the gripper approach direction output by the grasping parameter estimation module. Planar rotation angle Grip opening width Depth of advance along the approach direction and quality score Regression is performed to obtain the predicted grasping posture; S35. The grasping posture refinement module based on physical prior knowledge optimizes the predicted grasping posture to obtain the optimized grasping posture.

4. The six-degree-of-freedom grasping and detection method based on physical prior knowledge according to claim 3, characterized in that, S33 specifically includes the following steps: S331, Capture candidate points and characteristics The data is fed into the crawlable region optimization regression module. S332, The crawlable region confidence score regression head based on features in the crawlable region optimization regression module. Calculate each crawl candidate point confidence score ; S333, For each crawling candidate point It can capture an optimized area block, calculate the average Euclidean distance between it and its k neighboring points, and calculate the standard deviation of the distance based on the average Euclidean distance; S334. Based on the mean Euclidean distance and the standard deviation of the distance, for each candidate point to be crawled... confidence score Perform cleaning and optimization; S335. After the confidence scores are cleaned and optimized, KNN clustering is used to group the candidate points into crawlable regions. Then, a three-dimensional Gaussian distribution is fitted to the crawlable regions to further optimize the confidence scores of the candidate points, resulting in multiple optimizations of the candidate point confidence scores. .

5. The six-degree-of-freedom grasping and detection method based on prior physical knowledge according to claim 4, characterized in that, The specific formulas for calculating the mean Euclidean distance and the standard deviation of the distance in S333 are as follows: ； ； ； in, Indicates the first The average distance between each candidate point and other candidate points Indicates the first 1 candidate point Indicates the average Euclidean distance; Indicates the current number One neighboring point; Indicates the distance from the standard deviation; The cleaning optimization in S334 is expressed by the following formula: ； in, Indicates the first Confidence scores of each candidate point Indicates the first The confidence score after cleaning optimization; Indicates the weighting coefficient; The confidence scores of the candidate points after multiple optimizations in S335 The specific calculation formula is as follows: ； in, Represents the fitted three-dimensional Gaussian distribution. This represents the mean value generated after fitting. This represents the covariance matrix generated after fitting. This represents an exponential function with the natural constant e as its base.

6. The six-degree-of-freedom grasping and detection method based on physical prior knowledge according to claim 5, characterized in that, S35 specifically includes the following steps: S351, Input point cloud The estimated grasping posture is input into the grasping posture refinement module based on physical prior knowledge. The contact point calculation block in the grasping posture refinement module calculates the contact point according to the input point cloud. The gripper position is reconstructed from the predicted grasping posture, and then the input point cloud is used to determine the gripper position. The neighborhood set of the contact point is calculated based on the position of the gripper. ; S352, Set the neighborhood points of the contact point The input is fed into the flatness calculation block to calculate the neighborhood point set of the contact point. The mean and oblique variance matrix of each neighboring point are obtained; then, three eigenvalues ​​that decrease sequentially are obtained from the oblique variance matrix, and the flatness score is calculated based on the three eigenvalues. S353, Obtain the clamp connection vector Connect the gripper vectors Input into the force closure calculation block to calculate the force closure fraction. Then, based on the force closure fraction The flatness score is used to calculate the score of the estimated grasping posture; S354. Then, the grasping posture predicted by the grasping parameter estimation module is adjusted using the fractional gradient tendency of the predicted grasping posture to obtain the optimal grasping posture in the grasping posture parameter space.

7. The six-degree-of-freedom grasping and detection method based on physical prior knowledge according to claim 6, characterized in that, The set of neighboring points of the contact point in S352 The formulas for calculating the mean, oblique variance matrix, and flatness score of each neighboring point are as follows: ； ； ； in, Indicates the first Neighboring points, Represents the mean of the neighborhood points. represents the oblique variance matrix of the neighborhood point set; T represents the transpose; , , Represents the successively decreasing neighborhood set of the contact point. The three eigenvalues ​​of the covariance matrix; Represents the flatness fraction; The force closure fraction in S353 The formulas for calculating the estimated grasping posture score are as follows: ； ； in, Indicates the weighting coefficient; This indicates the calculation of the L2 norm.

8. The six-degree-of-freedom grasping and detection method based on physical prior knowledge according to claim 7, characterized in that, The loss function in S3 is as follows: ； ； ； ； ； ； ； ； ； ； In the formula, This represents the mean squared error loss function; Represents the cross-entropy loss function; , , , , , , These represent the following losses: candidate point capture score loss, proximity vector loss, planar rotation loss, capture quality loss, gripper advance depth loss along the proximity direction, gripper opening width loss, and confidence score loss, respectively. , , , , , These represent the predicted scores for grabbing candidate points, proximity vectors, planar rotation, grabbing quality, gripper advance depth along the proximity direction, gripper opening width, and confidence score, respectively. , , , , , , These represent the true values ​​of the candidate point capture score, the proximity vector, the planar rotation, the capture quality, the gripper advance depth along the proximity direction, the gripper opening width, and the confidence score, respectively. , , , , , , These represent the weights of each loss.

9. A six-degree-of-freedom grasping and detection system based on prior physical knowledge, characterized in that, The invention includes a robot execution system that performs grasping and detection according to the six-degree-of-freedom grasping and detection method described in any one of claims 1 to 8.

Images