Small-size object point cloud segmentation and space grabbing method

Abstract

A small-size object point cloud segmentation and space grabbing method belongs to the field of small-size object identification and grabbing, and comprises two parts of point cloud segmentation based on width residual attention, rapid sampling and pose estimation. The method comprises the following steps: firstly, extracting shallow spatial features of point cloud data, carrying out splicing and dimension raising on the shallow spatial features and normal vector information, and capturing local features through a multi-layer set sampling module and a width residual attention module; afterwards, the feature expression capability of the point cloud is enhanced by using a multi-layer feature propagation module, and the features of the point cloud are mapped to a category space to predict the label of each point, so that local point cloud data are obtained; and finally, the local point cloud data are reconstructed and smoothed, and the optimal grabbing posture of the mechanical arm is generated according to the generated grabbing plane and the multi-constraint principle. According to the method, the problems that the fine segmentation accuracy of small-size objects is poor and the grabbing efficiency is low can be solved, the optimal grabbing posture of the mechanical arm is generated, the search time in the grabbing space can be shortened, and the grabbing accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the field of small-sized object recognition and grasping, and relates to a small-sized object point cloud segmentation and spatial grasping method, which is used for fine segmentation of small-sized objects and guiding robot grasping, aiming to solve the problems of poor fine segmentation accuracy and low grasping efficiency of small-sized objects. Background Art

[0002] The assembly or grasping of objects has attracted the attention and research of many scholars, and many achievements have been made. With the increasing requirements for robot intelligence and the development of high-precision visual sensors, people's attention has begun to shift from large objects to small objects. Limited by the size and volume of small objects, the collected point cloud is sparse and thin, easily affected by noise, and difficult to capture detailed features, resulting in loss of detailed information, thus affecting the recognition and segmentation accuracy. Under some working conditions, the position and posture of objects in space are random and irregular, resulting in reduced grasping accuracy and stability, and more stringent grasping posture sampling and constraint conditions, which increases the difficulty of grasping. A method for point cloud segmentation and spatial grasping of small objects is of great significance to the research in the field of recognition and grasping of small objects.

[0003] Most existing methods focus on large or medium-sized object recognition, and their ability to recognize fine features of small objects is still limited. The segmentation quality and accuracy are poor, and there are cases of mis-segmentation or incomplete segmentation. They are also very dependent on the quality and evaluation strategy of the sampling posture, and require a lot of time and resources. First, the PointNet deep learning model is used as a spatial encoder, combined with K nearest neighbors and multi-layer perceptron MLP to obtain the shallow spatial features of the point cloud data, and splice and upgrade the dimension with the normal vector information. Secondly, the local features are captured by multi-layer set sampling SA modules and width residual attention WRA modules. The WRA module groups the point cloud data into multiple branches, increases the width of the data, reduces the difficulty of data processing and improves the processing speed. The attention mechanism is used in each branch to enhance the model's sensitivity to spatial position and improve the model's adaptability and robustness to different scenes. By adding residuals, the gradient disappearance is alleviated. Finally, a multi-layer feature propagation FP module is used to enhance the feature expression ability of the point cloud, and the features of the point cloud are mapped to a category space to predict the label of each point to obtain local point cloud data to meet the needs of point cloud segmentation. By reconstructing and smoothing the local point cloud data, noise points are removed and the surface flatness is optimized. According to the generated grasping plane, the optimal grasping posture of the robot arm is generated according to the principle of multiple constraints, which reduces the search time in the grasping space and improves the accuracy of grasping. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention proposes a method for point cloud segmentation and spatial grasping of small-sized objects. The present invention gives full play to the advantages of deep learning algorithms, attention mechanisms and multiple constraint principles, and is superior to traditional methods in terms of the accuracy of point cloud segmentation tasks and the efficiency of generating grasping postures. PointNet is used as a spatial encoding module for shallow spatial feature extraction. Multi-level feature extraction and representation of point cloud data are performed through multi-layer set abstraction modules, width residual attention modules and feature propagation modules. Finally, the segmented local point cloud data is obtained. Through reconstruction and smoothing, the noise points of local point cloud data are removed and the surface flatness is optimized to improve the surface quality of the point cloud. According to the generated grasping plane, the optimal grasping posture of the robot arm is generated according to the principle of multiple constraints, which reduces the search time in the grasping space and improves the accuracy of grasping.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A method for point cloud segmentation and spatial grasping of small-sized objects comprises the following steps:

[0007] Step 1: Point cloud segmentation based on width residual attention;

[0008] First, a deep learning algorithm is used to extract shallow spatial features of point cloud data, and then concatenate and upgrade the dimensions with normal vector information. Second, a multi-layer set sampling module and a width residual attention module are used to capture more detailed local features. Then, a multi-layer feature propagation module is used to enhance the feature expression ability of the point cloud, and the features of the point cloud are mapped to a category space to predict the label of each point, thereby meeting the needs of point cloud segmentation. Specifically:

[0009] Step 1.1: Use the K-Nearest Neighbors (KNN) algorithm on the input raw point cloud data to calculate the Euclidean distance between each point and other points, find the K points closest to it, and extract the local features of the point based on these neighborhood information.

[0010]

[0011] in, represents the neighborhood centralization set; p i represents the coordinates of the center point; i represents the serial number of each point in the input point cloud; N represents the number of all points contained in the input point cloud; p j represents the coordinates of the neighbor point; j represents the sequence number of the neighbor point; n represents the number of neighbor points in the neighborhood;

[0012] In step 1.2, PointNet is used as a spatial encoding module to perform fine-grained feature extraction on the local features obtained in step 1.1, so that each point obtains a shallow spatial feature containing the feature information of the surrounding neighborhood, and the normal vector information is concatenated to the back of the shallow spatial feature of each point to increase its feature information.

[0013] Step 1.3: The point cloud data in step 1.2 is upgraded to a high dimension through a multilayer perceptron (MLP), and each point is further mapped and feature fused to give each point richer feature information.

[0014]

[0015] Among them, s i Indicates the feature extraction of each point in the input point cloud; f i Represents the normal vector feature information of each point in the input point cloud;

[0016] Step 1.4, input the point cloud data with high dimension and rich features obtained in step 1.3 into the Set Abstract (SA) module, and perform downsampling (Subsample), grouping (Grouping) and MLP feature mapping on the input point cloud data through the SA module to reduce the amount of input point cloud data, and further map and update the features of the point cloud data. Point cloud data with high-level features and a small number are obtained, where the total number of point clouds N′=N / 4.

[0017] Step 1.5: Input the point cloud data obtained in step 1.4 into the Wide Residual Attention (WRA) module, and group the input point cloud data into multiple branches through average pooling and maximum pooling to increase the width of the data. Use the attention mechanism in each branch to capture the contextual relationship between different regions to dynamically adjust the weight of each region, and use the Sigmod function to map the obtained weight to [0,1].

[0018] e i ={s j -s i}j=0,1,...n

[0019] X1=AvgPool(e i )

[0020] X2=MaxPool(e i )

[0021] Weight=Sigmod(relu(mlp(group c(X1))))+relu(mlp(group c (X2)))

[0022] Among them, e i Represents the neighborhood set of each feature extracted point; s j Represents the features of neighboring points in a neighborhood; X1 represents the features after average pooling; X2 represents the features after maximum pooling; Weight represents the attention weight;

[0023] Step 1.6, multiply the point cloud data obtained in step 1.3 by the weights obtained in step 1.5 element by element. Different weights are assigned to the features of each point cloud to highlight important features and obtain weighted point cloud data. Then, the point cloud data obtained in step 1.3 is added to the weighted point cloud data in the form of residuals to reduce oscillation and instability during training and alleviate gradient disappearance. Finally, a reduction operation is used to aggregate complex multidimensional data or information to improve computational efficiency or enhance the expressiveness of the model.

[0024] F att =Weight×e i

[0025] F out =F att +e i

[0026] F att Indicates the weighted result; F out Represents the residual addition result;

[0027] Step 1.7, repeat steps 1.4 to 1.6 three more times, perform feature extraction at different levels of point cloud quantity, enhance feature expression capabilities, and thus improve segmentation accuracy and robustness. Finally, the total number of point clouds N″=N / 256. The input of each operation is the output of the previous operation.

[0028] Step 1.8, input the result of step 1.7 into the feature propagation (FP) module, and perform skip link concatenation and upsampling on the input point cloud data through the FP module to propagate local features to the global level and enhance the feature expression ability of the point cloud. Finally, after multiple layers of FP, the features of each point are updated, and the features of the point cloud are mapped to a category space to predict the label of each point, and points with the same label value (i.e., the same features) are divided together to form a local point cloud as output.

[0029] Step 1.9, the completed original point cloud input in step 1.1 is segmented into multiple local point clouds according to its features after steps 1.1 to 1.8, and the required local point clouds are retained through the corresponding label values.

[0030] Step 2: Fast sampling and pose estimation;

[0031] By reconstructing and smoothing the local point cloud data segmented in step 1, noise points are removed and surface flatness is optimized. Based on the generated grasping plane, the optimal grasping posture of the robot arm is generated according to the principle of multiple constraints. Specifically:

[0032] In step 2.1, the local point cloud data obtained in step 1 is used as input using the scale-space surface reconstruction algorithm (3SR), which is reconstructed in a high-scale space. The reconstruction result is then interpolated back to the original scale to adjust the reconstruction result, remove noise points, and represent the input point cloud data in the form of a three-dimensional grid.

[0033] Step 2.2, use the "Mesh-Shape-Mesh" algorithm (MSM) to perform mesh optimization and smoothing operations on the three-dimensional mesh data processed in step 2.1. First, adjust the surface mesh size and area through mesh smoothing, then improve the surface quality of the object through shape smoothing, and finally perform mesh smoothing again to achieve the effect of point cloud surface mesh optimization and smoothing.

[0034] In step 2.3, mesh shape detection is performed based on the region growing method on the surface smoothed data in step 2.2. The seed triangle plane is selected, the similarity between the adjacent plane and the seed plane is calculated, and the same surface is continuously generated outward to divide the point cloud into different surfaces.

[0035] Step 2.4, based on step 2.3, estimate the normal of the generated plane, obtain the corresponding normal vector, find a pair of planes with the smallest normal vector angle as the grasping plane for detection, and calculate the centroid coordinates of the grasping plane.

[0036] Step 2.5: Use the grasping points output in step 2.4 as sampling points in the spherical space. Sample V grasping directions uniformly, sample A angles for each grasping direction, generate V×A grasping postures, and sample D grasping widths uniformly, generating a total of M samples.

[0037] Step 2.6: In step 2.5, multiple grasping postures are generated, and the collision with the object before grasping is eliminated by the width constraint principle. The distance d between the two contact points p1 and p2 on the grasping plane is calculated using the Euclidean distance. The grasping requirement is met only when the grasping width is greater than the distance d. This reduces the number of grasping postures.

[0038] Step 2.7, based on step 2.6, the rotation posture around the X-axis and Y-axis of the gripper coordinate system is restricted by the angle constraint principle. Among them, the constraint of the Y-axis of the gripper is mainly reflected in the normal line of the gripping plane. The X-axis constraint of the gripper is mainly reflected in the angle constraint with the main axis normal of the object. The principal axis Z of the object is calculated using the principal component analysis (PCA). Combined with the known grasping plane normal, the object principal axis constraint and the grasping plane normal constraint are performed simultaneously.

[0039] Step 2.8: Based on step 2.7, the direction constraint principle is used to avoid collision between the gripper and the desktop. The gripping point is taken as the center of the circle, and the rotation sampling is used as the direction of the gripping insertion. The gripper insertion direction should be outside the surface of the object. The Random Sample Consensus (RANSAC) algorithm is used to segment the gripped desktop and calculate the normal vector. By limiting the desktop normal vector The angle between the Z axis (approach direction) of the gripper can be used to remove inappropriate poses.

[0040] Step 2.9: Based on step 2.8, the grasping depth is limited by the position constraint principle. Find a contact point p close to the boundary point on the projection boundary point and the center line of the grasping plane. t , sampling on the boundary to find the contact points can get a contact point set P t , all contact points in the set meet the grasping requirements.

[0041] Step 2.10, through the multiple constraint principle of steps 2.6 to 2.9, delete the grasping postures that may cause collision and invalid grasping postures, and finally generate the optimal grasping posture of the robot arm.

[0042] The effects and benefits of the present invention are:

[0043] Aiming at the point cloud segmentation and grasping problem of small-sized objects, the present invention proposes a point cloud segmentation and spatial grasping method for small-sized objects, which gives full play to the advantages of deep learning algorithms, attention mechanisms and multiple constraint principles, and is superior to traditional methods in terms of the accuracy of point cloud segmentation tasks and the efficiency of generating grasping postures. PointNet is used as a spatial encoding module for shallow spatial feature extraction. Multi-level feature extraction and representation of point cloud data are performed through multi-layer SA modules, WRA modules and FP modules. Among them, the WRA module groups point cloud data into multiple branches, increases the width of the data, reduces the difficulty of data processing and improves the processing speed. An attention mechanism is added to each branch, and a dynamic attention mechanism is used to enhance the sensitivity of the model to spatial position, and improve the adaptability and robustness of the model to different scenes. By adding residuals, the gradient disappearance is alleviated. Finally, the segmented local point cloud data is obtained. Through reconstruction and smoothing, the noise points of the local point cloud data are removed and the surface flatness is optimized, thereby improving the surface quality of the point cloud. According to the generated grasping plane, the optimal grasping posture of the robot arm is generated according to the principle of multiple constraints, which reduces the search time in the grasping space and improves the accuracy of grasping. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Point cloud segmentation network based on width residual attention;

[0045] Figure 2 It is a fast sampling and pose estimation method flow;

[0046] Figure 3 This is the result of the cylindrical object after fast sampling and pose estimation method. DETAILED DESCRIPTION

[0047] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0048] The present invention is a method for point cloud segmentation and spatial capture of small-sized objects, comprising the following steps:

[0049] Step 1: Point cloud segmentation based on width residual attention;

[0050] First, a deep learning algorithm is used to extract shallow spatial features of point cloud data, and then concatenate and upgrade the dimensions with normal vector information. Second, a multi-layer set sampling module and a width residual attention module are used to capture more detailed local features. Then, a multi-layer feature propagation module is used to enhance the feature expression ability of the point cloud, and the features of the point cloud are mapped to a category space to predict the label of each point, thereby meeting the needs of point cloud segmentation. Specifically:

[0051] Step 1.1: Use the K-Nearest Neighbors (KNN) algorithm on the input raw point cloud data to calculate the Euclidean distance between each point and other points, find the K points closest to it, and extract the local features of the point based on these neighborhood information.

[0052]

[0053] in, represents the neighborhood centralization set; p i represents the coordinates of the center point; i represents the serial number of each point in the input point cloud; N represents the number of all points contained in the input point cloud; p j represents the coordinates of the neighbor point; j represents the sequence number of the neighbor point; n represents the number of neighbor points in the neighborhood;

[0054] In step 1.2, PointNet is used as a spatial encoding module to perform fine-grained feature extraction on the local features obtained in step 1.1, so that each point obtains a shallow spatial feature containing the feature information of the surrounding neighborhood, and the normal vector information is concatenated to the back of the shallow spatial feature of each point to increase its feature information.

[0055] Step 1.3: The point cloud data in step 1.2 is upgraded to a high dimension through a multilayer perceptron (MLP), and each point is further mapped and feature fused to give each point richer feature information.

[0056]

[0057] Among them, s i Indicates the feature extraction of each point in the input point cloud; f i Represents the normal vector feature information of each point in the input point cloud;

[0058] Step 1.4, input the point cloud data with high dimension and rich features obtained in step 1.3 into the Set Abstract (SA) module, and perform downsampling (Subsample), grouping (Grouping) and MLP feature mapping on the input point cloud data through the SA module to reduce the amount of input point cloud data, and further map and update the features of the point cloud data. Point cloud data with high-level features and a small number are obtained, where the total number of point clouds N′=N / 4.

[0059] Step 1.5: Input the point cloud data obtained in step 1.4 into the Wide Residual Attention (WRA) module, and group the input point cloud data into multiple branches through average pooling and maximum pooling to increase the width of the data. Use the attention mechanism in each branch to capture the contextual relationship between different regions to dynamically adjust the weight of each region, and use the Sigmod function to map the obtained weight to [0,1].

[0060] e i ={s j -s i}j=0,1,...n

[0061] X1=AvgPool(e i )

[0062] X2=MaxPool(e i )

[0063] Weight=Sigmod(relu(mlp(group c (X1))))+relu(mlp(group c (X2)))

[0064] Among them, e i Represents the neighborhood set of each feature extracted point; s j Represents the features of neighboring points in a neighborhood; X1 represents the features after average pooling; X2 represents the features after maximum pooling; Weight represents the attention weight;

[0065] Step 1.6, multiply the point cloud data obtained in step 1.3 by the weights obtained in step 1.5 element by element. Different weights are assigned to the features of each point cloud to highlight important features and obtain weighted point cloud data. Then, the point cloud data obtained in step 1.3 is added to the weighted point cloud data in the form of residuals to reduce oscillation and instability during training and alleviate gradient disappearance. Finally, a reduction operation is used to aggregate complex multidimensional data or information to improve computational efficiency or enhance the expressiveness of the model.

[0066] F att =Weight×e i

[0067] F out =F att +e i

[0068] F att Indicates the weighted result; F outRepresents the residual addition result;

[0069] Step 1.7, repeat steps 1.4 to 1.6 three more times, perform feature extraction at different levels of point cloud quantity, enhance feature expression capabilities, and thus improve segmentation accuracy and robustness. Finally, the total number of point clouds N″=N / 256. The input of each operation is the output of the previous operation.

[0070] Step 1.8, input the result of step 1.7 into the feature propagation (FP) module, and perform skip link concatenation and upsampling on the input point cloud data through the FP module to propagate local features to the global level and enhance the feature expression ability of the point cloud. Finally, after multiple layers of FP, the features of each point are updated, and the features of the point cloud are mapped to a category space to predict the label of each point, and points with the same label value (i.e., the same features) are divided together to form a local point cloud as output.

[0071] Step 1.9, the completed original point cloud input in step 1.1 is segmented into multiple local point clouds according to its features after steps 1.1 to 1.8, and the required local point clouds are retained through the corresponding label values.

[0072] Step 2: Fast sampling and pose estimation;

[0073] By reconstructing and smoothing the local point cloud data segmented in step 1, noise points are removed and surface flatness is optimized. Based on the generated grasping plane, the optimal grasping posture of the robot arm is generated according to the principle of multiple constraints. Specifically:

[0074] Step 2.1, use the scale-space surface reconstruction algorithm (3SR) to take the local point cloud data obtained in step 1.8 as input, reconstruct it in a high-scale space, and then interpolate it back to the original scale to adjust the reconstruction result, remove noise points, and represent the input point cloud data in the form of a three-dimensional grid.

[0075] Step 2.2, use the "Mesh-Shape-Mesh" algorithm (MSM) to perform mesh optimization and smoothing operations on the three-dimensional mesh data processed in step 2.1. First, adjust the surface mesh size and area through mesh smoothing, then improve the surface quality of the object through shape smoothing, and finally perform mesh smoothing again to achieve the effect of point cloud surface mesh optimization and smoothing.

[0076] In step 2.3, mesh shape detection is performed based on the region growing method on the surface smoothed data in step 2.2. The seed triangle plane is selected, the similarity between the adjacent plane and the seed plane is calculated, and the same surface is continuously generated outward to divide the point cloud into different surfaces.

[0077] Step 2.4, based on step 2.3, estimate the normal of the generated plane, obtain the corresponding normal vector, find a pair of planes with the smallest normal vector angle as the grasping plane for detection, and calculate the centroid coordinates of the grasping plane.

[0078] Step 2.5: Use the grasping points output in step 2.4 as sampling points in the spherical space. Sample V grasping directions uniformly, sample A angles for each grasping direction, generate V×A grasping postures, and sample D grasping widths uniformly, generating a total of M samples, where M=V×A×D. In this experiment, V=9, A=100, D=1, and M=900.

[0079] Step 2.6: In step 2.5, multiple grasping postures are generated, and the collision with the object before grasping is eliminated by the width constraint principle. The distance d between the two contact points p1 and p2 of the grasping plane is calculated using the Euclidean distance. When the grasping width d g When the distance is greater than d, the grasping requirement is met. This reduces the number of grasping postures. g =[20mm,60mm].

[0080] Step 2.7, based on step 2.6, the rotation posture around the X-axis and Y-axis of the gripper coordinate system is restricted by the angle constraint principle. Among them, the constraint of the Y-axis of the gripper is mainly reflected in the normal line of the gripping plane. The X-axis constraint of the gripper is mainly reflected in the angle constraint with the main axis normal of the object. The principal axis Z of the object is calculated using the principal component analysis (PCA). Combined with the known grasping plane normal, the object principal axis constraint and the grasping plane normal constraint are performed simultaneously. The grasping angle constraint conditions set in this experiment are: ① The Y axis of the gripper and the grasping plane normal are The angle α between z It needs to be greater than 40° and less than 140°. ② The normal line between the X-axis of the gripper and the main axis of the object The angle α between x It needs to be greater than 55° and less than 125°.

[0081] Step 2.8: Based on step 2.7, the direction constraint principle is used to avoid collision between the gripper and the desktop. The gripping point is taken as the center of the circle, and the rotation sampling is used as the direction of the gripping insertion. The gripper insertion direction should be outside the surface of the object. The Random Sample Consensus (RANSAC) algorithm is used to segment the gripped desktop and calculate the normal vector. By limiting the desktop normal vector The angle between the Z axis of the gripper coordinate system and the normal vector of the desktop is 0. The angle α between y It needs to be greater than 130° and less than 230°.

[0082] Step 2.9: Based on step 2.8, the grasping depth is limited by the position constraint principle. Find a contact point p close to the boundary point on the projection boundary point and the center line of the grasping plane. t , sampling on the boundary to find the contact points can get a contact point set P t , all contact points in the set meet the grasping requirements.

[0083] Step 2.10, through the multiple constraint principle of steps 2.6 to 2.9, delete the grasping postures that may cause collision and invalid grasping postures, and finally generate the optimal grasping posture of the robot arm.

[0084] The above-described embodiments merely express the embodiments of the present invention and are not intended to limit the present invention to the precise forms described. It is possible for a person of ordinary skill in the art to make several modifications based on the above teachings without departing from the concept of the present invention. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other persons of ordinary skill in the art can easily understand, implement and utilize the various exemplary embodiments of the present invention and its various selected forms and modified forms. The scope of protection of the present invention is intended to be limited by the appended claims and their equivalent forms.

Claims

1. A method for point cloud segmentation and spatial grasping of small-sized objects, characterized in that: The following steps are involved: Step 1: Point cloud segmentation based on width residual attention; First, the shallow spatial features of the point cloud data are extracted and concatenated with the normal vector information to increase the dimension. Second, the local features are captured through a multi-layer set sampling module and a width residual attention module. Finally, a multi-layer feature propagation module is used to enhance the feature expression ability of the point cloud, and the features of the point cloud are mapped to a category space to predict the label of each point, thereby obtaining local point cloud data and meeting the needs of point cloud segmentation. Step 2: Fast sampling and pose estimation; The local point cloud data segmented in step 1 is reconstructed and smoothed to remove noise points and optimize surface flatness; based on the generated grasping plane, the optimal grasping posture of the robot arm is generated according to the principle of multiple constraints.

2. A method for point cloud segmentation and spatial grasping of small-sized objects according to claim 1, characterized in that: The step 1 is specifically as follows: Step 1.1, use the K nearest neighbor algorithm to calculate the Euclidean distance between each point and other points on the input raw point cloud data, find the K nearest points, and extract the local features of the point based on these neighborhood information; Step 1.2: Use PointNet as a spatial encoding module to perform fine-grained feature extraction on the local features obtained in step 1.1 to obtain point cloud data, so that each point obtains a shallow spatial feature containing the feature information of the surrounding neighborhood, and concatenates the normal vector information to the back of the shallow spatial feature of each point to increase its feature information; Step 1.3: The point cloud data in step 1.2 is upgraded to a high dimension through a multi-layer perceptron (MLP), and each point is mapped and feature fused; Step 1.4, input the point cloud data with high dimension and rich features obtained in step 1.3 into the set sampling SA module, and perform downsampling, grouping and MLP feature mapping on the input point cloud data through the SA module to obtain point cloud data with high-level features and a small number, wherein the total number of point clouds N′=N / 4; Step 1.5: Input the point cloud data obtained in step 1.4 into the width residual attention WRA module, and group the input point cloud data into multiple branches through average pooling and maximum pooling; use the attention mechanism in each branch to capture the contextual relationship between different regions, dynamically adjust the weight of each region, and map the obtained weight to [0,1]; e i ={s j -s i }j=0,1,...n X1=AvgPool(e i ) X2=MaxPool(e i ) Weight=Sigmod(relu(mlp(group) c (X1))))+relu(mlp(group c (X2))) Among them, e i Represents the neighborhood set of each feature extracted point; s j Represents the features of neighboring points in a neighborhood; X1 represents the features after average pooling; X2 represents the features after maximum pooling; Weight represents the attention weight; Step 1.6, multiply the point cloud data obtained in step 1.3 by the weight obtained in step 1.5 element by element; assign different weights to the features of each point cloud to obtain weighted point cloud data; then add the point cloud data obtained in step 1.3 to the weighted point cloud data in the form of residuals; finally, use reduction operations to aggregate complex multidimensional data or information; F att =Weight×e i F out =F att +e i Among them, F att Indicates the weighted result; F out Represents the residual addition result; Step 1.7, repeat steps 1.4 to 1.6 for multiple times, and perform feature extraction at different point cloud quantity levels, wherein the input of each operation is the output of the previous operation; Step 1.8: Input the result of step 1.7 into the feature propagation FP module. The input point cloud data is skip-chained and up-sampled through the FP module to propagate local features to the global level. After multiple layers of FP, the features of each point are updated, and the features of the point cloud are mapped to a category space to predict the label of each point. Points with the same label value are divided together to form a local point cloud as output. Step 1.9, the completed original point cloud input in step 1.1 is segmented into multiple local point clouds according to its features after steps 1.1 to 1.8, and the required local point clouds are retained through the corresponding label values.

3. The method for point cloud segmentation and spatial grasping of small-sized objects according to claim 2, characterized in that: The processing formula of step 1.1 is: in, represents the neighborhood centralization set; p i represents the coordinates of the center point; i represents the serial number of each point in the input point cloud; N represents the number of all points contained in the input point cloud; p j represents the coordinates of the neighbor point; j represents the sequence number of the neighbor point; n represents the number of neighbor points in the neighborhood.

4. The method for point cloud segmentation and spatial grasping of small-sized objects according to claim 2, characterized in that: The processing formula of step 1.3 is: Among them, s i Indicates the feature extraction of each point in the input point cloud; f i Represents the normal vector feature information of each point in the input point cloud.

5. The method for point cloud segmentation and spatial grasping of small-sized objects according to claim 2, characterized in that: In step 1.5, the obtained weight is mapped to [0, 1] using the Sigmod function.

6. The method for point cloud segmentation and spatial grasping of small-sized objects according to claim 2, characterized in that: The step 2 is specifically as follows: Step 2.1, using the scale space surface reconstruction algorithm to take the local point cloud data obtained in step 1 as input, reconstruct it in a high-scale space, and then interpolate it back to the original scale to adjust the reconstruction result, remove noise points, and represent the input point cloud data in the form of a three-dimensional grid; Step 2.2, using the "grid-shape-grid" algorithm to perform mesh optimization and smoothing operations on the three-dimensional mesh data processed in step 2.1; Step 2.3, based on the surface smoothed data in step 2.2, mesh shape detection is performed based on the region growing method. By selecting a seed triangle plane, calculating the similarity between the adjacent plane and the seed plane, the same surface is continuously generated outward, and the point cloud is divided into different surfaces; Step 2.4, based on step 2.3, estimate the normal of the generated plane to obtain the corresponding normal vector, find a pair of planes with the smallest normal vector angle as the grasping plane for detection, calculate the centroid coordinates of the grasping plane, and obtain the grasping point; Step 2.5, using the grasping points output in step 2.4 as sampling points in the spherical space to generate multiple grasping postures and multiple samples; Step 2.6, in step 2.5, multiple grasping postures are generated, and the collision with the object before grasping is eliminated by the width constraint principle; the distance d between the two contact points p1 and p2 of the grasping plane is calculated by using the Euclidean distance, and the grasping requirement is met only when the grasping width is greater than the distance d; Step 2.7, based on step 2.6, the rotation posture around the X-axis and Y-axis of the gripper coordinate system is restricted by the angle constraint principle, and the object principal axis constraint and the grasping plane normal constraint are performed; Step 2.8, based on step 2.7, first avoid the collision between the gripper and the desktop, determine the gripper's insertion direction, and secondly, limit the desktop normal vector The angle of the Z axis of the gripper, delete the inappropriate posture; Step 2.9: Based on step 2.8, the grasping depth is limited by the position constraint principle; a contact point p close to the boundary point is found on the projection boundary point and the center line of the grasping plane. t , sampling on the boundary to find the contact points can get a contact point set P t , all contact points in the set meet the grasping requirements; Step 2.10, through the multiple constraint principle of steps 2.6 to 2.9, delete the grasping postures that may cause collision and invalid grasping postures, and finally generate the optimal grasping posture of the robot arm.

7. The method for point cloud segmentation and spatial grasping of small-sized objects according to claim 6, characterized in that: In the step 2.2, the surface mesh size and area are first adjusted by mesh smoothing, the surface quality of the object is then improved by shape smoothing, and finally mesh smoothing is performed again, thereby achieving the effect of optimizing and smoothing the point cloud surface mesh.

8. The method for point cloud segmentation and spatial grasping of small-sized objects according to claim 6, characterized in that: In the step 2.5, V grasping directions are uniformly sampled, A angles of each grasping direction are sampled to generate V×A grasping postures, and D grasping widths are uniformly sampled to generate M samples in total.

9. The method for point cloud segmentation and spatial grasping of small-sized objects according to claim 6, characterized in that: In step 2.7, the Y-axis constraint of the gripper is mainly reflected in the normal to the grasping plane. Angle constraint; The constraint of the gripper's X axis is mainly reflected in the normal to the main axis of the object. angle constraint; use principal component analysis PCA to calculate the object's principal axis Z; combine the known grasping plane normal to perform object principal axis constraint and grasping plane normal constraint at the same time.

10. The method for point cloud segmentation and spatial grasping of small-sized objects according to claim 6, characterized in that: In the step 2.8, the direction constraint principle is used to avoid collision between the gripper and the desktop. The gripping point is taken as the center of the circle, and the rotation sampling is used as the direction of gripping and insertion. The gripper insertion direction is outside the surface of the object. The desktop normal vector is calculated by segmenting the grasped desktop using the random sampling consistency algorithm. and the Z-axis angle of the gripper to delete inappropriate postures.

Images