6-dof grasping method for robot arm based on shape features

Abstract

The 6-DoF grabbing method of a mechanical arm based on shape features belongs to the technical fields of robots and artificial intelligence, and solves the problems of low grabbing precision and poor generalization in the existing autonomous intelligent grabbing process of robots.The point cloud completion network and the grabbing network are established;the target object point cloud is extracted from the depth map input by a camera, the target object point cloud is input into the point cloud completion network to generate a target object completion point cloud;the obtained completion point cloud is spliced with the original target object point cloud to generate a spliced point cloud;the spliced point cloud is input into the grabbing network, the grabbing network predicts a grabbing posture, and a grabbing posture base parameter is acquired;in the grabbing posture base parameter, the maximum value of a confidence score is extracted, the corresponding elements of a gripper clamping orientation base vector, a gripper clamping baseline direction base vector and a clamping width matrix of the gripper are acquired, and the optimal grabbing posture parameter is generated for robot grabbing.The present application is suitable for robot intelligent grabbing.

Description

Technical Field

[0001] This invention belongs to the field of robotics and artificial intelligence technology. Background Technology

[0002] Point cloud-based 6-DoF (6-Degree of Freedom) grasping algorithms have shown great potential in autonomous intelligent grasping applications for robots. However, in practical applications, the point cloud used for grasping is usually converted from a depth map captured by a single-view depth camera, referred to as a 2.5D point cloud, which only contains visible points on one side of the object's surface. Furthermore, surface reflections are common in everyday life, and commonly used depth cameras such as infrared cameras and LiDAR may lose the target point cloud due to reflections, further contributing to the incompleteness of the point cloud. Incomplete point clouds lead to a significant reduction in grasping accuracy and success rate. Therefore, improving the grasping algorithm to enhance the grasping posture under incomplete point cloud input is crucial for intelligent robot grasping.

[0003] Existing algorithms mainly fall into two categories. The first category is end-to-end neural networks, which directly input incomplete point clouds into the network to generate target grasping poses. The results of this type of method are severely affected by the incomplete point clouds, resulting in inaccurate results and low grasping success rates. The second category is to first generate a complete target point cloud through point cloud completion methods, and then generate the grasping pose from the completed point cloud. This type of method requires the target object to exist in the point cloud completion dataset, resulting in poor generalization of grasping and severely limiting the application scope of grasping. Summary of the Invention

[0004] This invention aims to address the problems of low accuracy and poor generalization in existing robotic autonomous grasping processes by providing a 6-DoF grasping method for robotic arms based on shape features.

[0005] The 6-DoF grasping posture generation method for robotic arms based on shape features described in this invention includes:

[0006] Step 1: Establish the point cloud completion network and the capture network;

[0007] Step 2: Extract the original target object point cloud P from the depth map input by the camera, and input the target object point cloud P into the point cloud completion network to generate the target object completed point cloud P. c ;

[0008] Step 3: Complete the obtained point cloud P c This generates a stitched point cloud P by stitching the point cloud P with the original target object point cloud P. n ;

[0009] Step 4: Patch the point cloud nThe data is input into a grasping network, which predicts the grasping posture and obtains the grasping posture base parameters.

[0010] The gripping posture base parameters include: gripper gripping orientation base vector Z2 and gripper gripping baseline direction base vector Z1, gripping posture confidence score matrix S and gripper gripping width matrix w;

[0011] The elements in the gripper gripping orientation base vector Z2 and gripper gripping baseline direction base vector Z1, the confidence score matrix S of the gripping posture and the gripping width matrix w of the gripper are in one-to-one correspondence.

[0012] Step 5: Extract the maximum value from the confidence score matrix S in the grasping posture base parameters, and obtain the corresponding elements in the gripper gripping orientation base vector Z2, gripper gripping baseline direction base vector Z1, and gripper gripping width matrix w corresponding to the maximum value, to obtain the optimal grasping posture parameters.

[0013] Furthermore, in this invention, in step one, the point cloud completion network is implemented using Point Completion Network, PCN (Point Cloud Completion Network), FoldingNet (Folding Network), Gridding Residual Network GRNet (Grid Residual Network), or PointR (Point Cloud Converter Network).

[0014] Furthermore, in this invention, in step one, the grasping network includes: a feature mapping layer, a point cloud encoding layer, and a grasping prediction layer;

[0015] The feature mapping layer is used to stitch together the point cloud P n Mapping onto the original point cloud data, we obtain the complete shape features of the original point cloud and the stitched point cloud corresponding to the original point cloud;

[0016] The point cloud coding layer is used to aggregate, fuse, and extract features from the original point cloud data with complete shape features to obtain enhanced point cloud features based on the original point cloud and with complete point cloud features as additional features.

[0017] The grasping prediction layer uses a multilayer perceptron to decode the enhanced point cloud features and generate grasping posture parameters.

[0018] Furthermore, in this invention, the feature mapping layer includes three neighborhood point set acquisition modules, three feature extraction network modules, and a splicing module;

[0019] The three neighborhood point set acquisition modules are used to acquire the set of point clouds within three spheres of different radii centered on each point cloud, and transmit the set of point clouds within the three spheres of different radii to three feature extraction network modules respectively. The three feature extraction network modules perform feature extraction respectively, and stitch the extracted features together to obtain the complete shape features of the stitched point cloud, which corresponds to the original point cloud.

[0020] Furthermore, in this invention, the point cloud encoding layer includes two point cloud information aggregation layers, one point cloud information extraction layer, and three point cloud information propagation layers;

[0021] The first point cloud information aggregation layer aggregates the complete shape features of the original point cloud and the spliced ​​point cloud corresponding to the original point cloud, and outputs the point cloud features corresponding to n downsampling points and 2n sampling points.

[0022] The second point cloud information aggregation layer aggregates the point cloud features of the n sampling points and 2n sampling points, and outputs n / 2 downsampling points and the point cloud features corresponding to the n downsampling points;

[0023] The point cloud information extraction layer aggregates the point cloud information of the neighborhood of n / 4 sampling points and the point cloud features of n sampling points to obtain the point cloud feature information of n / 2 sampling points;

[0024] The first point cloud information transmission layer fuses the point cloud feature information of n / 4 sampling points with the point cloud features corresponding to n / 4 downsampling points and n downsampling points, and then learns and extracts them through a neural network to extract the enhanced features of n / 4 points.

[0025] The second point cloud information transmission layer fuses the enhanced features of the n / 4 points with the point cloud features corresponding to the n downsampled points and 2n sampled points output by the first point cloud information transmission layer, and then learns and extracts them through a neural network to extract the enhanced features of the n points.

[0026] The enhanced features of the n points in the second point cloud information transmission layer are fused with the point cloud features corresponding to the n downsampled points and 2n sampled points output by the first point cloud information aggregation layer, and then the enhanced features of the 2n points are extracted through learning by a neural network.

[0027] Furthermore, in this invention, the grasping prediction layer includes four prediction modules, which are respectively used to predict the confidence score matrix S, the gripper gripping orientation base vector Z2, the gripper gripping baseline direction base vector Z1, and the gripping width w of the gripper.

[0028] Furthermore, in this invention, the prediction module is implemented using a multilayer perceptron.

[0029] Furthermore, in this invention, the method for obtaining the optimal grasping posture parameters in step five is as follows:

[0030] The gripping position and attitude are calculated using the base vector Z2 corresponding to the maximum value in the confidence score matrix S and the base vector Z1 corresponding to the gripping baseline direction of the gripper.

[0031] The relative value t of the grasped position g for:

[0032]

[0033] The relative value of the grasping posture is:

[0034]

[0035] in, ||a|| = 1, where a represents the orientation vector of the grippers. ||b||=1, where b represents the gripper's gripping baseline direction vector; c∈R 3 is the spatial coordinate of the contact point between the gripper and the object; W∈R is the gripping width of the gripper; d∈R is the distance from the gripper coordinate system to the gripping baseline;

[0036] in,

[0037]

[0038]

[0039] A gripper coordinate system is established with the connection point between the robot gripper and the robotic arm as the origin Go. Based on the gripper size v, the true value of the gripping posture v is obtained through a gripping network. pred :

[0040] v pred =vR g +t g

[0041] The 6-DoF grasping method based on shape features described in this invention utilizes a neural network to autonomously learn the approximate shape of the target object from the point cloud completion result, and uses the learned shape features to guide grasping training. This not only improves the accuracy of intelligent grasping by robotic arms under incomplete point clouds, but also avoids the limitations of object generalization caused by point cloud completion. Quantitative and qualitative experimental results show that the grasping results generated by this method have higher grasping accuracy and a significantly improved grasping success rate. Attached Figure Description

[0042] Figure 1 This is a flowchart of the present invention;

[0043] Figure 2This is a physical flow chart of the present invention;

[0044] Figure 3 This is the overall network flowchart;

[0045] Figure 4 This is a detailed diagram of the feature mapping layer network;

[0046] Figure 5 To capture the pose representation graph;

[0047] Figure 6 Comparison chart of results generated by different crawling methods;

[0048] Figure 7 This is a screenshot showing the result of the capture generated by the method of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0050] Specific implementation method one: Refer to Figures 1-7 This embodiment specifically describes the 6-DoF grasping posture generation method for a robotic arm based on shape features, which includes:

[0051] Step 1: Establish the point cloud completion network and the capture network;

[0052] Step 2: Extract the original target object point cloud P from the depth map input by the camera, and input the target object point cloud P into the point cloud completion network to generate the target object completed point cloud P. c ;

[0053] Step 3: Complete the obtained point cloud P c This generates a stitched point cloud P by stitching the point cloud P with the original target object point cloud P. n ;

[0054] Step 4: Patch the point cloud n The data is input into a grasping network, which predicts the grasping posture and obtains the grasping posture base parameters.

[0055] The gripping posture base parameters include: gripper gripping orientation base vector Z2 and gripper gripping baseline direction base vector Z1, gripping posture confidence score matrix S and gripper gripping width matrix w;

[0056] The elements in the gripper gripping orientation base vector Z2 and gripper gripping baseline direction base vector Z1, the confidence score matrix S of the gripping posture and the gripping width matrix w of the gripper are in one-to-one correspondence.

[0057] Step 5: Extract the maximum value from the confidence score matrix S in the grasping posture base parameters, and obtain the corresponding elements in the gripper gripping orientation base vector Z2, gripper gripping baseline direction base vector Z1, and gripper gripping width matrix w corresponding to the maximum value, to obtain the optimal grasping posture parameters.

[0058] When humans grasp objects, they can roughly judge the shape of an object by observing only one side, and then reliably grasp it based on the visible part. Inspired by this, this invention uses a neural network algorithm to mimic the human grasping process, enabling the approximate shape judgment of an input defect cloud. This shape judgment generates a precise grasping result, greatly improving the accuracy of autonomous intelligent grasping for robots.

[0059] Furthermore, in this invention, in step one, the point cloud completion network is implemented using Point Completion Network, PCN (Point Cloud Completion Network), FoldingNet (Folding Network), Gridding Residual Network GRNet (Grid Residual Network), or PointR (Point Cloud Converter Network).

[0060] Furthermore, in this invention, in step one, the grasping network includes: a feature mapping layer, a point cloud encoding layer, and a grasping prediction layer;

[0061] The feature mapping layer is used to stitch together the point cloud P n Mapping onto the original point cloud data, we obtain the complete shape features of the original point cloud and the stitched point cloud corresponding to the original point cloud;

[0062] The point cloud coding layer is used to aggregate, fuse, and extract features from the original point cloud data with complete shape features to obtain enhanced point cloud features based on the original point cloud and with complete point cloud features as additional features.

[0063] The grasping prediction layer uses a multilayer perceptron to decode the enhanced point cloud features and generate grasping posture parameters.

[0064] Furthermore, in this invention, the feature mapping layer includes three neighborhood point set acquisition modules, three feature extraction network modules, and a splicing module;

[0065] The three neighborhood point set acquisition modules are used to acquire the set of point clouds within three spheres of different radii centered on each point cloud, and transmit the set of point clouds within the three spheres of different radii to three feature extraction network modules respectively. The three feature extraction network modules perform feature extraction respectively, and stitch the extracted features together to obtain the complete shape features of the stitched point cloud, which corresponds to the original point cloud.

[0066] Furthermore, in this invention, the point cloud encoding layer includes two point cloud information aggregation layers, one point cloud information extraction layer, and three point cloud information propagation layers;

[0067] The first point cloud information aggregation layer aggregates the complete shape features of the original point cloud and the spliced ​​point cloud corresponding to the original point cloud, and outputs the point cloud features corresponding to n downsampling points and 2n sampling points.

[0068] The second point cloud information aggregation layer aggregates the point cloud features of the n sampling points and 2n sampling points, and outputs n / 2 downsampling points and the point cloud features corresponding to the n downsampling points;

[0069] The point cloud information extraction layer aggregates the point cloud information of the neighborhood of n / 4 sampling points and the point cloud features of n sampling points to obtain the point cloud feature information of n / 2 sampling points;

[0070] The first point cloud information transmission layer fuses the point cloud feature information of n / 4 sampling points with the point cloud features corresponding to n / 4 downsampling points and n downsampling points, and then learns and extracts them through a neural network to extract the enhanced features of n / 4 points.

[0071] The second point cloud information transmission layer fuses the enhanced features of the n / 4 points with the point cloud features corresponding to the n downsampled points and 2n sampled points output by the first point cloud information transmission layer, and then learns and extracts them through a neural network to extract the enhanced features of the n points.

[0072] The enhanced features of the n points in the second point cloud information transmission layer are fused with the point cloud features corresponding to the n downsampled points and 2n sampled points output by the first point cloud information aggregation layer, and then the enhanced features of the 2n points are extracted through learning by a neural network.

[0073] Furthermore, in this invention, the grasping prediction layer includes four prediction modules, which are respectively used to predict the confidence score matrix S, the gripper gripping orientation base vector Z2, the gripper gripping baseline direction base vector Z1, and the gripping width w of the gripper.

[0074] Furthermore, in this invention, the prediction module is implemented using a multilayer perceptron.

[0075] Furthermore, in this invention, the method for obtaining the optimal grasping posture parameters in step five is as follows:

[0076] The gripping position and attitude are calculated using the base vector Z2 corresponding to the maximum value in the confidence score matrix S and the base vector Z1 corresponding to the gripping baseline direction of the gripper.

[0077] The relative value t of the grasped position g for:

[0078]

[0079] The relative value of the grasping posture is:

[0080]

[0081] in, ||a|| = 1, where a represents the orientation vector of the grippers. ||b||=1, where b represents the gripper's gripping baseline direction vector; c∈R 3 is the spatial coordinate of the contact point between the gripper and the object; W∈R is the gripping width of the gripper; d∈R is the distance from the gripper coordinate system to the gripping baseline;

[0082] in,

[0083]

[0084]

[0085] A gripper coordinate system is established with the connection point between the robot gripper and the robotic arm as the origin Go. Based on the gripper size v, the true value of the gripping posture v is obtained through a gripping network. pred :

[0086] v pred =vR g +t g .

[0087] In this embodiment, the point cloud coding layer uses the PointNet++ network coding layer to extract features from the original point cloud and shape features, specifically as follows: Figure 3As shown. Point cloud downsampling (first three layers) The purpose of point cloud downsampling is to downsample the point cloud sequentially (farthest point sampling FPS, from 1024 original point cloud inputs to the first layer PN++AbsMsg to 512 output points as inputs to the second layer PN++AbsMsg, and then the second layer outputs 128 points as inputs to the third layer PN++Abs. These point numbers (1024, 512, 128) are used as point inputs for each layer to learn features from point clouds of different densities, i.e., to learn features from different point cloud counts), thereby learning a better representation of point cloud information.

[0088] 1. PN++AbsMsg layer (1)

[0089] The Abstract Message Layer aggregates the information from the points in the neighborhood of the original point cloud and the complete point cloud features of the input. It outputs downsampled points (512 points) and point cloud features corresponding to 1024 points.

[0090] 2. PN++AbsMsg layer (2)

[0091] Further point cloud downsampling information aggregation is performed, which aggregates the points in the neighborhood of the 512 points and the input complete point cloud information features, and outputs the downsampled points (128 points) and the point cloud features corresponding to the 512 downsampled points.

[0092] 3. PN++Abs layer (3)

[0093] The point cloud information extraction layer only includes the aggregation of neighborhood point information from the preceding PN++AbsMsg layer, realizing the aggregation of point cloud information from 128 neighborhood points and the output of the previous layer to generate 128 point cloud feature information.

[0094] Point cloud upsampling (last three layers): The purpose of point cloud upsampling is to pass the features learned from downsampling back to the original point cloud layer by layer (because the grasping is performed on the original point cloud). The additional purpose of downsampling and upsampling is to learn more complete and comprehensive point cloud information through feature learning, thus making the point cloud features used to generate the grasping pose more accurate.

[0095] 4. PN++FP layer (4)

[0096] The Feature Propagation Layer takes 128 points and 128 points, along with their corresponding features and those of 512 points as input (the dotted lines indicate that the parameters are not passed in immediately). It concatenates the features corresponding to the 512 point cloud features (because the 128 points are sampled from the 512 points, it needs to identify which of the 512 points the 128 points belong to and concatenate the corresponding features to the 512 points). Then, it uses a neural network to learn and extract an enhanced feature for the 128 points (because it contains both the dense features of the 512 points and the sparse features after downsampling).

[0097] 5. PN++FP layer (5)

[0098] Repeat the above function, inputting 128 point clouds and 512 point clouds, as well as enhanced features and dense features of the original 1024 points, and repeat the above to output the enhanced features of the 512 point clouds.

[0099] 6. PN++FP layer (6)

[0100] Repeat the above function, inputting a point cloud of 512 points and an original point cloud of 1024 points, as well as enhanced features and the features of the original complete point cloud, and output the enhanced features of the original 1024 point cloud using a neural network, which is the point cloud feature based on the original point cloud as mentioned in the patent, with the complete point cloud features as additional point cloud features. Specific implementation examples:

[0102] (1) A pre-trained point cloud completion network is used to predict the complete point cloud of the target object from the point cloud converted from the depth map input by the camera.

[0103] (2) Freeze the pre-trained point cloud completion network parameters and perform inference for the grasping network. Specifically, the grasping network includes a feature mapping layer, a point cloud encoding layer, and a grasping prediction layer, as follows: Figure 3 As shown. The network architecture and feature mapping layer will be introduced in the following steps. The point cloud encoding layer uses Pointnet++[2], and the capture prediction layer consists of 4 MLP blocks with sizes of (128,128,3)→Z1, (128,128,3)→Z2, (128,128,10)→w, and (128,128,1)→S.

[0104] (3) First, extract the target object point cloud p∈(1024,3) from the depth map, and input it into the point cloud completion network to generate the target object completed point cloud p. c ∈(1024,3), contains the approximate shape information of the target object.

[0105] (4) Subsequently, the obtained completed point cloud P c This generates a stitched point cloud p by stitching the point cloud p of the original target object P together. n ∈(1024,3), and input it into the feature mapping layer to generate the complete point cloud feature Fn. The feature mapping layer is as follows: Figure 4 As shown, the original point cloud P is used as input to perform three progressive samplings of the farthest point with different numbers of points, resulting in a sampling point set p. Q1 ∈(1024,32,3), p Q2 ∈(1024,32,3), p Q3 ∈(1024,64,3). Then, through three convolutional neural network layers, the point set is used to complete the point cloud P. c Feature learning is performed on neighboring points of different radii, and the three feature learning results are concatenated to form a complete point cloud feature F. n The inner radius of the sampling points was set to (0.04, 0.08, 0.16) m, and the feature learning MLP layers were (32, 32, 64), (64, 64, 128), and (64, 96, 128).

[0106] (5) Then, the complete point cloud features F n The original target object point cloud P is input into a point cloud encoding layer based on PointNet++, with sampling points at (512,3) and (128,3), and in-layer sampling radii of (0.04, 0.08, 0.16) m and (0.08, 0.16, 0.32) m, respectively. The in-layer feature learning MLP layers are (64, 64, 128), (128, 128, 256), and (128, 128, 256). Finally, the feature aggregation layer is entered, with an in-layer feature learning MLP layer of (256, 512, 1024). The output is based on the original point cloud, with the complete point cloud features being additional point cloud features of size (128, 1640). Finally, the point cloud features are input into the MLP grasping prediction layer containing 4 heads, and the grasping prediction results are output. The grasping prediction results are used to represent the grasping posture in step 6). Among the confidence scores of the grasping posture, the highest score is selected as the grasping posture to be executed by the robotic arm.

[0107] (6) Using the following 6-DoF grasping representation method, the grasping posture g is directly associated with the grasping contact point c. Thus, the grasping position and posture are obtained.

[0108] Experimental results:

[0109] In the real robotic arm grasping experiment, we selected a Realsense D435 RGBD camera to acquire the point cloud of the target object, Mask RNNN to segment the target object mask, and a Franka robotic arm. Figure 7The experiment involved grasping objects. The control methods were 6-DoFGrasp (6DG)[3], PointGPD (PGPD)[4], Ma et al.[5], and ContactGraspnet (CG)[6].

[0110] Table 1 Comparison of experiments on grasping individual objects

[0111]

[0112] In the table, * indicates that the object is not present in the dataset for the corresponding method. For example... Figure 7 As shown, even with significant point cloud gaps, the proposed method, using previously obtained point cloud completion features, can still generate a reliable grasping posture. Furthermore, the grasping results in Table 1 demonstrate that the proposed method significantly improves the grasping success rate for robotic arms. Figure 7 The results demonstrated show that this method autonomously extracts the point cloud completion results through a neural network, learns the shape features of the complete target object's point cloud, and integrates them as additional information into the original point cloud. This effectively addresses the inaccuracy of point cloud completion results for unknown objects and produces reliable grasping results.

[0113] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A 6-DoF grasping method for a robotic arm based on shape features, characterized in that, include: Step 1: Establish the point cloud completion network and the capture network; Step 2: Extract the original target object point cloud P from the depth map input by the camera, and input the target object point cloud P into the point cloud completion network to generate the target object completed point cloud P. c ; Step 3: Complete the obtained point cloud P c This generates a stitched point cloud P by stitching the point cloud P with the original target object point cloud P. n ; Step 4: Patch the point cloud n The data is input into a grasping network, which predicts the grasping posture and obtains the grasping posture base parameters. The gripping posture base parameters include: gripper gripping orientation base vector Z2 and gripper gripping baseline direction base vector Z1, gripping posture confidence score matrix S and gripper gripping width matrix w; The elements in the gripper gripping orientation base vector Z2 and gripper gripping baseline direction base vector Z1, the confidence score matrix S of the gripping posture and the gripping width matrix w of the gripper are in one-to-one correspondence. Step 5: Extract the maximum value from the confidence score matrix S in the grasping posture base parameters, and obtain the corresponding elements in the gripper gripping orientation base vector Z2, gripper gripping baseline direction base vector Z1, and gripper gripping width matrix w corresponding to the maximum value, to obtain the optimal grasping posture parameters.

2. The 6-DoF grasping method for a robotic arm based on shape features according to claim 1, characterized in that, In step one, the point cloud completion network is implemented using a point cloud completion network, a folding network, a mesh residual network, or a point cloud converter network.

3. The 6-DoF grasping method for a robotic arm based on shape features according to claim 1, characterized in that, In step one, the grasping network includes: a feature mapping layer, a point cloud encoding layer, and a grasping prediction layer; The feature mapping layer is used to stitch together the point cloud P n Mapping onto the original point cloud data, we obtain the complete shape features of the original point cloud and the stitched point cloud corresponding to the original point cloud; The point cloud coding layer is used to aggregate, fuse, and extract features from the original point cloud data with complete shape features to obtain enhanced point cloud features based on the original point cloud and with complete point cloud features as additional features. The grasping prediction layer uses a multilayer perceptron to decode the enhanced point cloud features and generate grasping posture parameters.

4. The 6-DoF grasping method for a robotic arm based on shape features according to claim 1, characterized in that, The feature mapping layer includes: three neighborhood point set acquisition modules, three feature extraction network modules, and a concatenation module; The three neighborhood point set acquisition modules are used to acquire the set of point clouds within three spheres of different radii centered on each point cloud, and transmit the set of point clouds within the three spheres of different radii to three feature extraction network modules respectively. The three feature extraction network modules perform feature extraction respectively, and stitch the extracted features together to obtain the complete shape features of the stitched point cloud, which corresponds to the original point cloud.

5. The 6-DoF grasping method for a robotic arm based on shape features according to claim 1, 2, or 3, characterized in that, The point cloud encoding layer includes two point cloud information aggregation layers, one point cloud information extraction layer, and three point cloud information propagation layers; The first point cloud information aggregation layer aggregates the complete shape features of the original point cloud and the stitched point cloud corresponding to the original point cloud, and outputs the point cloud features corresponding to n downsampling points and 2n sampling points; The second point cloud information aggregation layer aggregates the point cloud features of the n sampling points and 2n sampling points, and outputs n / 2 downsampling points and the point cloud features corresponding to the n downsampling points; The point cloud information extraction layer aggregates the point cloud information of the neighborhood of n / 4 sampling points and the point cloud features of n sampling points to obtain the point cloud feature information of n / 2 sampling points; The first point cloud information transmission layer fuses the point cloud feature information of n / 4 sampling points with the point cloud features corresponding to n / 4 downsampling points and n downsampling points, and then learns and extracts them through a neural network to extract the enhanced features of n / 4 points. The second point cloud information transmission layer fuses the enhanced features of the n / 4 points with the point cloud features corresponding to the n downsampled points and 2n sampled points output by the first point cloud information transmission layer, and then learns and extracts them through a neural network to extract the enhanced features of the n points. The enhanced features of the n points in the second point cloud information transmission layer are fused with the point cloud features corresponding to the n downsampled points and 2n sampled points output by the first point cloud information aggregation layer. Then, the enhanced features of the 2n points are extracted through learning by a neural network, where n is a positive integer.

6. The 6-DoF grasping method for a robotic arm based on shape features according to claim 4, characterized in that, n＝512。 7. The 6-DoF grasping method for a robotic arm based on shape features according to claim 1, 2, or 3, characterized in that, The grasping prediction layer includes four prediction modules, which are used to predict the confidence score matrix S, the gripper gripping orientation base vector Z2, the gripper gripping baseline direction base vector Z1, and the gripping width w of the gripper, respectively.

8. The 6-DoF grasping method for a robotic arm based on shape features according to claim 1, 2, or 3, characterized in that, The prediction module is implemented using a multilayer perceptron.

9. The 6-DoF grasping method for a robotic arm based on shape features according to claim 1, 2, or 3, characterized in that, In step five, the method for obtaining the optimal grasping posture parameters is as follows: The gripping position and attitude are calculated using the base vector Z2 corresponding to the maximum value in the confidence score matrix S and the base vector Z1 corresponding to the gripping baseline direction of the gripper. The relative value t of the grasped position g for: The relative value of the grasping posture is: in, ||a|| = 1, where a represents the orientation vector of the grippers. ||b||=1, where b represents the gripper's gripping baseline direction vector; c∈R 3 is the spatial coordinate of the contact point between the gripper and the object; W∈R is the gripping width of the gripper; d∈R is the distance from the gripper coordinate system to the gripping baseline; in, A gripper coordinate system is established with the connection point between the robot gripper and the robotic arm as the origin Go. Based on the gripper size v, the true value of the gripping posture v is obtained through a gripping network. pred : v pred = vR g +t g .

Images