Object identification detection method based on improved GSNet network

Abstract

The invention provides an object recognition detection method based on an improved GSNet network, and the method comprises the steps: collecting a depth image of a target scene through a depth camera, and converting the depth image into point cloud data; the method comprises the following steps: replacing a ResUNet 14 backbone network in a GSNet network by adopting a Point Net + + network to obtain an improved GSNet network; training the improved GSNet network by using the obtained point cloud data; and inputting target object data to be detected in the target scene, and using the trained improved GSNet network to identify and detect a target object. According to the method, the Point Net + + network which is good at processing point cloud data is introduced into the GSNet network, so that local detail features are better extracted. By segmenting the background and the target object in the target scene, interference of the environment and other objects on target object recognition and detection is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of object recognition, and in particular to an object recognition detection method based on an improved GSNet network. Background Art

[0002] Grasping is one of the basic directions in the field of robotics. With the continuous increase in work demands, the grasping operations of robotic arms face many challenges, especially in achieving accurate grasping of small target objects in complex environments.

[0003] In the existing technology, researchers mainly obtain the grasping posture by analyzing the shape of the object, or operate through the extracted primary features of the object. Limited by the development of basic research, artificially designed features are difficult to describe objects with complex shapes in detail, and can only describe simple objects, which is not very practical in real scenes. In the field of deep learning, researchers use neural network technology to detect and identify targets in images and estimate their postures. The robotic arm completes the grasping task based on the information provided by the target detection algorithm. Deep learning can learn deeper features to improve recognition accuracy, but this type of method requires a lot of time and effort to create data sets.

[0004] With the development of data acquisition equipment in recent years, more and more affordable 3D cameras have been launched, making it easier to obtain point clouds. The method of obtaining six degrees of freedom (6-Dof) grasping posture based on point clouds has begun to emerge. Compared with planar grasping, 6-Dof grasping does not limit the grasping approach vector of the robot arm and is more suitable for complex grasping work.

[0005] Traditional 6-Dof grasping methods treat all points in the target scene equally, and usually select candidate grasping points by uniform sampling. However, this will ignore the issue of where to grasp, and cannot avoid the interference of the environment and other objects on the target object recognition and detection, which will greatly affect the speed and accuracy of the current grasping posture detection method. Summary of the invention

[0006] Purpose of the invention: The purpose of the present invention is to provide an object recognition detection method with higher accuracy of object and background segmentation based on an improved GSNet network.

[0007] Technical solution: An object recognition and detection method based on GSNet network, comprising the following steps: S1. Use a depth camera to acquire a depth image of the target scene and convert the depth image into point cloud data; S2. Use PointNet++ network to replace the ResUNet14 backbone network in the GSNet network to obtain an improved GSNet network; S3, using the point cloud data obtained in step S1 to train the improved GSNet network; S4. Input the target object data to be tested in the target scene, and use the trained improved GSNet network to identify and detect the target object.

[0008] Specifically, step S1 includes: using a camera to acquire a color image of the target scene, adjusting the size of the depth image to be the same as the color image, using the depth information and the parameters of the depth camera to convert the depth image into point cloud data, and using a workspace mask to separate valid points from invalid points in the color image and point cloud, and extracting valid points in the color image and point cloud.

[0009] Specifically, in step S2, the improved GSNet network includes a first abstract layer, a second abstract layer, a third abstract layer, a fourth abstract layer, a first feature upsampling layer, a second feature upsampling layer, a first fully connected layer, and a second fully connected layer. After the point cloud data is extracted through the first abstract layer, the second abstract layer, the third abstract layer, and the fourth abstract layer in sequence, the first feature upsampling layer and the second feature upsampling layer fuse and upsample the extracted features, and the features of the point cloud data are output through the first fully connected layer and the second fully connected layer.

[0010] Preferably, step S2 further includes: A CBAM attention module is added after improving the first feature upsampling layer and the second feature upsampling layer in the GSNet network.

[0011] The CBAM attention module extracts feature map information through average pooling and maximum pooling operations to obtain average pooling features and maximum pooling features. It then uses a network composed of a multi-layer perceptron to generate an attention mechanism, applies average pooling and maximum pooling operations along the spatial coordinate axis, aggregates the channel information of the two feature maps, generates two two-dimensional feature maps, and performs convolution operations through standard convolutional layers to obtain a spatial attention mechanism.

[0012] Specifically, in step S2, the network generates the attention mechanism formula as follows: , Where: is the channel attention of the CBAM attention module, is the Sigmoid function, , is the weight, is the average pooling feature, is the maximum pooling feature; The formula of spatial attention mechanism is: , Where: is the spatial attention of the CBAM attention module, is the average pooled two-dimensional feature map, is the maximum pooled two-dimensional feature map, The convolution sum size of the convolution operation is 7 7.

[0013] Specifically, in step S2, the abstract layer selects a set number of downsampling points from the point cloud data through uniformly distributed downsampling, and defines a region of interest with each downsampling point as the center and a set distance as the radius, selects several points in the region of interest that are closest to the downsampling point as neighboring points for grouping, extracts features through a multilayer perceptron, and normalizes the coordinates in the point cloud data when extracting features, and directly uses them for feature learning.

[0014] Specifically, in step S2, the first feature upsampling layer fuses and upsamples the features extracted by the third abstract layer and the fourth abstract layer to improve the resolution of the point cloud data. Based on the first feature upsampling layer, the second feature upsampling layer fuses and upsamples the features extracted by the second abstract layer and the third abstract layer to further improve the resolution of the point cloud data.

[0015] Preferably, step S2 further includes replacing the seed point sampling method in the GSNet network with a multi-scale cylindrical sampling method. The multi-scale features obtained by the multi-scale cylindrical sampling method are adaptively weighted through the SE attention mechanism module to obtain enhanced features, and the enhanced features are compressed to a fixed length through a multi-layer perceptron to match the feature dimensions obtained by the seed point feature sampling method, and added to the features obtained by the seed point feature sampling method to obtain local features.

[0016] Beneficial effects: Compared with the prior art, the present invention has the following significant effects: (1) The present invention introduces the PointNet++ network, which is better at processing point cloud data, into the GSNet network, so as to better extract local detail features. By segmenting the background and target object in the target scene, the interference of the environment and other objects on the recognition and detection of the target object is avoided.

[0017] (2) In order to solve the problem that the GSNet network only uses single-scale cylindrical sampling, resulting in insufficient ability to extract multi-scale geometric features, the present invention introduces multi-scale cylindrical sampling to improve the sampling of seed point features of objects of different scales, especially small target objects. At the same time, the SE attention mechanism is used to adaptively weight features of different sizes, enhance the attention to features of different sizes, and improve the attention to seed point features of small objects, thereby effectively improving the accuracy of target object recognition and detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the method of the present invention.

[0019] Figure 2 It is a schematic diagram of the backbone network structure of the improved GSNet network of the present invention. DETAILED DESCRIPTION

[0020] A preferred embodiment of the present invention is further described below in conjunction with the accompanying drawings.

[0021] Example 1

[0022] See also Figure 1 As shown, this embodiment provides an object recognition detection method based on an improved GSNet network, comprising the following steps: S1. Use a depth camera to acquire a depth image of the target scene and convert the depth image into point cloud data.

[0023] In this embodiment, a color image of the target scene is acquired using a camera. First, the size of the depth image is adjusted to be the same as the color image to maintain spatial consistency; the depth information and depth camera parameters are used to convert the two-dimensional depth image into point cloud data in three-dimensional space; the workspace mask (workspace_mask) is used to separate the valid points (objects) from the invalid points (background) in the image; the mask array is used to extract valid points in the point cloud and color image to ensure that subsequent processing only focuses on actual objects. In order to prevent overfitting, random sampling is used to select 15,000 points from the valid points as point cloud data for subsequent training. The sampled point cloud data is converted into a PyTorch tensor for input into the model for processing.

[0024] S2. Use PointNet++ network to replace the ResUNet14 backbone network in the GSNet (GraspNet) network to obtain an improved GSNet network.

[0025] See also Figure 2 As shown, the improved GSNet network includes a first abstract layer SA1, a second abstract layer SA2, a third abstract layer SA3, a fourth abstract layer SA4, a first feature upsampling layer FP1, a second feature upsampling layer FP2, a first fully connected layer fc, and a second fully connected layer fc-512. In this embodiment, a CBAM attention module is added after the first feature upsampling layer FP1 and the second feature upsampling layer FP2 in the improved GSNet network.

[0026] The CBAM attention module extracts feature map information through average pooling and maximum pooling operations to obtain average pooling features and maximum pooling features. It then uses a network composed of a multi-layer perceptron to generate an attention mechanism, applies average pooling and maximum pooling operations along the spatial coordinate axis, aggregates the channel information of the two feature maps, generates two two-dimensional feature maps, and performs convolution operations through standard convolutional layers to obtain a spatial attention mechanism.

[0027] The network generates the attention mechanism formula as follows: , Where: is the channel attention of the CBAM attention module, is the Sigmoid function, , is the weight, is the average pooling feature, is the maximum pooling feature; The formula of spatial attention mechanism is: , Where: is the spatial attention of the CBAM attention module, is the average pooled two-dimensional feature map, is the maximum pooled two-dimensional feature map, The convolution sum size of the convolution operation is 7 7.

[0028] In this embodiment, the abstract layer SA1 selects 2048 down-sampling points from the point cloud data by uniformly distributing down-sampling, and defines the region of interest with each down-sampling point as the center and a distance of 0.04 as the grouping radius, selects the first 64 points closest to the down-sampling point in the region of interest as neighboring points for grouping, extracts features through a multi-layer perceptron (MLP), normalizes the coordinates in the point cloud data when extracting features, and directly uses them for feature learning. Referring to the above steps, similar operations are performed in the remaining abstract layers, and the number of down-sampling points in the abstract layers SA2, SA3, and SA4 are 1024, 512, and 256, respectively, the grouping radius is 0.1, 0.2, and 0.3, respectively, and the number of neighboring points is 32, 16, and 16, respectively.

[0029] The first feature upsampling layer FP1 fuses and upsamples the features extracted by the third abstract layer SA3 and the fourth abstract layer SA4 to improve the resolution of the point cloud data so that the features can match the higher resolution point cloud data. Based on the first feature upsampling layer FP1, the second feature upsampling layer FP2 fuses and upsamples the features extracted by the second abstract layer SA2 and the third abstract layer SA3 to further improve the resolution of the point cloud data.

[0030] A CBAM attention module is added after the two feature upsampling modules to enhance the feature representation, so that the network can learn important features more effectively and increase the attention to local features. Finally, the feature dimensions of the point cloud data are adjusted and output through two fully connected layers.

[0031] Considering that the seed point extraction module of the original GSNet network only uses single-scale cylinder sampling, resulting in insufficient extraction of multi-scale geometric features, a multi-scale cylinder sampling method is introduced to improve the sampling of seed point features of different scales, especially small target objects, thereby improving the recognition and detection accuracy of small targets. After extracting features of three scales, the SE (Squeeze and Excitation) attention mechanism module is combined to adaptively weight the input multi-scale features, thereby enhancing the attention to local features and effectively aggregating the extracted multi-scale features. Finally, the enhanced features are compressed to a fixed length to match the original seed point feature dimension through a multi-layer perceptron, and added to the seed point features to form new local features, which are used to improve the GSNet network for posture parameter prediction.

[0032] The SE module redistributes the information between channels by learning an attention weight vector that represents the relationship between channels, thereby enhancing the network's attention to important feature channels. This is achieved through squeeze and excitation operations. In the squeeze step, the input feature map is compressed into a vector through a global average pooling operation, and then mapped to a 1×1×C vector through a fully connected layer; in the excitation step, one or more fully connected layers are used to compress each element in this vector to between 0 and 1 to generate a weight vector with the same number of input feature channels; finally, the weight vector obtained in the excitation step is multiplied by the original input feature map to obtain a weighted feature map. In this way, the network can pay more attention to important feature channels and ignore unimportant channels.

[0033] S3. Use the point cloud data obtained in step S1 to train the improved GSNet network constructed in step S2.

[0034] S4. Input the target object data to be tested in the target scene, and use the trained improved GSNet network to identify and detect the target object.

Claims

1. An object recognition detection method based on an improved GSNet network, characterized in that: The following steps are involved: S1. Use a depth camera to acquire a depth image of the target scene and convert the depth image into point cloud data; S2. Use PointNet++ network to replace the ResUNet14 backbone network in the GSNet network to obtain an improved GSNet network; S3, using the point cloud data obtained in step S1 to train the improved GSNet network; S4. Input the target object data to be tested in the target scene, and use the trained improved GSNet network to identify and detect the target object.

2. The object recognition detection method according to claim 1, characterized in that: The step S1 includes: using a camera to acquire a color image of the target scene, adjusting the size of the depth image to be the same as the color image, using the depth information and the parameters of the depth camera to convert the depth image into point cloud data, and using a workspace mask to separate valid points from invalid points in the color image and the point cloud, and extracting valid points in the color image and the point cloud.

3. The object recognition detection method according to claim 1, characterized in that: In step S2, the improved GSNet network includes a first abstract layer, a second abstract layer, a third abstract layer, a fourth abstract layer, a first feature upsampling layer, a second feature upsampling layer, a first fully connected layer, and a second fully connected layer. After the point cloud data is sequentially extracted through the first abstract layer, the second abstract layer, the third abstract layer, and the fourth abstract layer, the first feature upsampling layer and the second feature upsampling layer fuse and upsample the extracted features, and the features of the point cloud data are output through the first fully connected layer and the second fully connected layer.

4. The object recognition detection method according to claim 3, characterized in that: The step S2 further comprises: A CBAM attention module is added after improving the first feature upsampling layer and the second feature upsampling layer in the GSNet network.

5. The object recognition detection method according to claim 4, characterized in that: In step S2, the CBAM attention module extracts feature map information through average pooling and maximum pooling operations to obtain average pooling features and maximum pooling features, and then uses a network composed of a multi-layer perceptron to generate an attention mechanism, applies average pooling and maximum pooling operations along the spatial coordinate axis, aggregates the channel information of the two feature maps, generates two two-dimensional feature maps, and performs convolution operations through standard convolutional layers to obtain a spatial attention mechanism.

6. The object recognition detection method according to claim 5, characterized in that: In step S2, the network generates the attention mechanism formula as follows: ， Where: is the channel attention of the CBAM attention module, is the Sigmoid function, , is the weight, is the average pooling feature, is the maximum pooling feature; The formula of spatial attention mechanism is: ， Where: is the spatial attention of the CBAM attention module, is the average pooled two-dimensional feature map, is the maximum pooled two-dimensional feature map, The convolution sum size of the convolution operation is 7 7.

7. The object recognition detection method according to claim 3, characterized in that: In step S2, the abstract layer selects a set number of down-sampling points from the point cloud data by uniformly distributing down-sampling, defines a region of interest with each down-sampling point as the center and a set distance as the radius, selects several points in the region of interest that are closest to the down-sampling point as neighboring points for grouping, extracts features through a multi-layer perceptron, and normalizes the coordinates in the point cloud data when extracting features, and directly uses them for feature learning.

8. The object recognition detection method according to claim 3, characterized in that: In step S2, the first feature upsampling layer fuses and upsamples the features extracted by the third abstraction layer and the fourth abstraction layer to improve the resolution of the point cloud data. Based on the first feature upsampling layer, the second feature upsampling layer fuses and upsamples the features extracted by the second abstraction layer and the third abstraction layer to further improve the resolution of the point cloud data.

9. The object recognition detection method according to claim 1, characterized in that: The step S2 also includes: using a multi-scale cylindrical sampling method to replace the seed point feature sampling method in the GSNet network.

10. The object recognition detection method according to claim 9, characterized in that: The step S2 also includes: adaptively weighting the multi-scale features obtained by the multi-scale cylindrical sampling method through the SE attention mechanism module to obtain enhanced features, compressing the enhanced features to a fixed length through a multi-layer perceptron, matching the enhanced features with the feature dimensions obtained by the seed point feature sampling method, and adding the enhanced features to the features obtained by the seed point feature sampling method to obtain local features.

Images