A point cloud analysis method based on dynamic graph convolutional neural network

By adopting dynamic graph convolutional neural network in point cloud analysis, combining attention and residual network, the problem of neglecting geometric relationships between points in point cloud analysis is solved, the accuracy of point cloud classification and segmentation is improved, and the realization of More efficient point cloud analysis.

CN113313176BInactive Publication Date: 2024-11-26SOUTHEAST UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110616319.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-06-02
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When processing three-dimensional point cloud data, the existing point cloud analysis methods ignore the geometric relationship between points, resulting in the inability to effectively capture local features. The feature extraction method is too simple and loses part of local information.

Method used

Using a dynamic graph convolution neural network method, the features of the point cloud disordered domain are extracted through the attention graph convolution module, combined with the residual network for information transmission, and dynamically updated the module fusion features, using LeakyReLU activation function and Softmax logistic regression model, improve the robustness of feature extraction.

Benefits of technology

It improves the accuracy of point cloud classification and segmentation, enhances the efficiency and robustness of point cloud analysis, and provides a new solution for point cloud analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113313176B_ABST
    Figure CN113313176B_ABST
Patent Text Reader

Abstract

The present invention discloses a point cloud analysis method based on a dynamic graph convolutional neural network, and belongs to the field of image processing. The method includes: an attention graph convolution module is used to extract features of the disordered field of the point cloud, find the K nearest neighboring points of each point, train the weights of each edge feature through the MLP network, and extract the depth features of each point by multiplying each edge feature by the weight sum; a dynamic graph convolution module, for the point cloud in the feature space, still uses the graph convolution module to update the feature data; a residual network module, based on the dynamic graph convolution network, introduces a residual network, and constructs a residual dynamic graph convolution network based on the attention mechanism according to the residual direction and the graph convolution network. Point cloud data is input into the network, and better point cloud classification results and segmentation results can be obtained. The present invention provides an end-to-end method to better obtain local information of point clouds, and improves the accuracy of point cloud classification and segmentation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a point cloud analysis method based on a dynamic graph convolutional neural network. Background Art

[0002] With the rapid development of 3D acquisition technology, 3D sensors are becoming more and more widely used and cheaper, including various types of 3D scanners, LiDAR, RGB-D cameras (such as Kinect, RealSense and Apple depth cameras). The 3D data collected by these sensors can provide rich geometry, shape and scale information. Assisted by 2D images, 3D data provides an opportunity for machines to better understand the surrounding environment. 3D data has many applications in different fields, including autonomous driving, robotics, remote sensing and medical treatment. At present, 3D point cloud analysis is mainly divided into three categories of methods: point-based, voxel-based and multi-view-based. Among them, the point-based method is currently the more mainstream method. The typical 2D image deep learning method cannot be directly applied to 3D point clouds due to its inherent data irregularity. PointNet, as a pioneering work, proposes a network that directly processes point clouds. Due to its simplicity and strong representation ability, many later works such as Mo-Net, PointWet, PointASNL and other networks are developed on the basis of PointNet.

[0003] Advanced deep neural networks are specifically designed to handle the irregularity of point clouds, processing raw point cloud data directly without passing it to an intermediate regular representation. This approach was first proposed by PointNet, which can achieve permutation invariance of points by operating on each point independently and then applying symmetric functions to accumulate features. Various extensions of PointNet consider the neighborhood of points instead of acting on each point independently, which allows the network to exploit local features, thereby improving the performance of the basic model. These techniques process points independently at local scales to a large extent to maintain permutation invariance. However, this independence ignores the geometric relationship between points, thus presenting the fundamental limitation of not being able to capture local features. The recently proposed DGCNN method solves the problem of overly independent features well, but the parameters considered in calculating edge features are relatively few and the maximum pooling method used to extract features is too simple, so some local information is still lost. Summary of the invention

[0004] To solve the above problems, the present invention discloses a point cloud analysis method based on a dynamic graph convolutional neural network, which can improve the accuracy of point cloud classification and segmentation.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A point cloud analysis method based on a dynamic graph convolutional neural network comprises the following steps:

[0007] Step 1: Collect point cloud data of objects and scenes in the radar field of view, collect data sets related to point cloud analysis, and create training and test data sets for model training, which specifically includes:

[0008] Step 1.1: Directly obtain the three-dimensional information of the object through hardware devices (depth sensor, structured light, radar equipment, etc.). If the data set does not include the training set and the test set, the obtained data set is divided into a training set and a test set in a ratio of 4:1.

[0009] Step 1.2: If the dataset already includes a training set and a test set, go to step 2.

[0010] Step 2: Establish a deep learning network model, which consists of three modules, including an attention graph convolution module, a residual connection module, and a dynamic update module; input the point cloud data obtained in step 1 into the network, and use the attention graph convolution module to extract the features of the disordered area of ​​the point cloud. For the point cloud in the feature space, the graph convolution module is still used to update the feature data. The residual network is used for information transmission. The n×1024 feature map obtained by fusing the output of each layer is output after the pooling module and 1×1 convolution, and the classification result is output. The 1×1024 feature vector is repeated n times and combined with the previous feature maps after 1×1 convolution, and the segmentation result is output. Specifically, it includes:

[0011] Step 2.1: Input the point cloud data obtained in step 1 into the first layer network, and the first layer network outputs N×64 high-dimensional features after attention graph convolution. The calculation formulas of attention graph convolution and LeakyReLU are as follows:

[0012]

[0013]

[0014] in: is a learnable parameter, x j and x i The initial point cloud is the 3D coordinates, and the parameters of the edge features are trained through two MLP layers. The attention mechanism is introduced, and the nearest neighbor algorithm is used for each point in the point cloud to obtain the vector edge features of the K points around the point. The Softmax logistic regression model is used to normalize the K features to obtain the features of the current point.

[0015] Step 2.2: Input the N×64 high-dimensional feature space output by step 2.1 into the second layer network, and the second layer network outputs the N×64 high-dimensional features after the attention graph convolution.

[0016] Step 2.3: Input the N×64 high-dimensional feature space output by step 2.2 into the third-layer network, and the third-layer network outputs the N×128 high-dimensional features after the attention graph convolution.

[0017] Step 2.4: Input the N×128 high-dimensional feature space output by step 2.3 into the fourth-layer network, and the fourth-layer network outputs the N×256 high-dimensional features after the attention graph convolution.

[0018] Step 2.5: For the output of each layer of the network, a residual network module is used, skipping the lower layer network and directly connecting its output. For example, the output of the first layer of the network needs to be connected to the output of the second layer of the network as the input of the third layer of the network. If the dimensions of the two are different, a 1×1 convolution is required for dimensionality increase. A residual block can be expressed as: where x l represents the input of the residual block, x l+1 Represents the output of the residual block, which is divided into two parts: the direct mapping part and the residual part. The direct mapping part is h(x l ), the residual is This is reflected in the network as the need to skip the next layer of graph convolution modules after each layer to connect to the input of the next layer.

[0019] Step 2.6: Connect the outputs of each layer of the network, and upgrade the dimension to 1024 through a fully connected layer. Fuse the output of each layer to obtain an n×1024 feature map, and then output the classification result after passing it through a pooling module and a 1×1 convolution. Repeat the 1×1024 feature vector n times and combine it with the previous feature maps, and then output the segmentation result after passing it through a 1×1 convolution.

[0020] Step 3: Input the point cloud data into the network and train the network parameters, which specifically includes:

[0021] Step 3.1: Input the initial training set obtained in step 1 into the network. Learn the parameters in the network, and let the parameters in step 2 be Θ net , the output of the network is P net , use the cross entropy loss function to update the parameters: The network is trained by stochastic gradient descent. After a certain number of times, the model parameters are saved for testing.

[0022] Step 4: Use the test set to test the network trained in step 3, which specifically includes:

[0023] Step 4.1: Input the test set in step 1 into the saved network model with the best training performance, repeat the training and testing multiple times, and select the model with the best performance.

[0024] The beneficial effects of the present invention are:

[0025] The present invention is distinguishable, can be inserted into the existing point cloud system, and improves the efficiency of point cloud analysis. It not only meets the requirements in terms of accuracy and has strong robustness, but also provides a new solution for point cloud analysis.

[0026] The present invention uses the LeakyReLU activation function instead of the ReLU activation function. The LeakyReLU activation function multiplies a very small weight in the negative half interval of the input, so that the negative area is no longer saturated and dead, thereby avoiding the problem that the neurons in the negative interval no longer learn. The present invention integrates the attention network and the residual network module, and uses a dynamic method to update the features. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a network model framework diagram in the present invention;

[0028] Figure 2 It is an overall flow chart of the invention method. DETAILED DESCRIPTION

[0029] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0030] Example 1: See Figure 1 , a point cloud analysis method based on dynamic graph convolutional neural network, comprising the following steps:

[0031] Step 1: Collect point cloud data of objects and scenes in the radar field of view, collect data sets related to point cloud analysis, and create training and test data sets for model training, which specifically includes:

[0032] Step 1.1: Directly obtain the three-dimensional information of the object through hardware devices (depth sensor, structured light, radar equipment, etc.). If the data set does not include the training set and the test set, the obtained data set is divided into a training set and a test set in a ratio of 4 to 1.

[0033] Step 1.2: If the dataset already includes a training set and a test set, go to step 2.

[0034] Step 2: Establish a deep learning network model. The model consists of three modules, including an attention graph convolution module, a residual connection module, and a dynamic update module. Input the point cloud data obtained in step 1 into the network, and use the attention graph convolution module to extract the features of the disordered area of ​​the point cloud. For the point cloud in the feature space, the graph convolution module is still used to update the feature data. The residual network is used for information transmission. The n×1024 feature map obtained by fusing the output of each layer is output after the pooling module and 1×1 convolution. The classification result is output. The 1×1024 feature vector is repeated n times and combined with the previous feature maps after 1×1 convolution. The segmentation result is output, which specifically includes:

[0035] Step 2.1: Input the point cloud data obtained in step 1 into the first layer network, and the first layer network outputs N×64 high-dimensional features after attention graph convolution. The calculation formulas of attention graph convolution and LeakyReLU are as follows:

[0036]

[0037]

[0038] in: is a learnable parameter, x j and x i The initial point cloud is the 3D coordinates, and the parameters of the edge features are trained through two MLP layers. The attention mechanism is introduced, and the nearest neighbor algorithm is used for each point in the point cloud to obtain the vector edge features of the K points around the point. The Softmax logistic regression model is used to normalize the K features to obtain the features of the current point.

[0039] Step 2.2: Input the N×64 high-dimensional feature space output by step 2.1 into the second layer network, and the second layer network outputs the N×64 high-dimensional features after the attention graph convolution.

[0040] Step 2.3: Input the N×64 high-dimensional feature space output by step 2.2 into the third-layer network, and the third-layer network outputs the N×128 high-dimensional features after the attention graph convolution.

[0041] Step 2.4: Input the N×128 high-dimensional feature space output by step 2.3 into the fourth-layer network, and the fourth-layer network outputs the N×256 high-dimensional features after the attention graph convolution.

[0042] Step 2.5: For the output of each layer of the network, a residual network module is used, skipping the lower layer network and directly connecting its output. For example, the output of the first layer of the network needs to be connected to the output of the second layer of the network as the input of the third layer of the network. If the dimensions of the two are different, a 1×1 convolution is required for dimensionality increase. A residual block can be expressed as: The residual block is divided into two parts: the direct mapping part and the residual part, where the direct mapping part is h(x l ), the residual is This is reflected in the network as the need to skip the next layer of graph convolution modules after each layer to connect to the input of the next layer.

[0043] Step 2.6: Connect the outputs of each layer of the network, and upgrade the dimension to 1024 through a fully connected layer. Fuse the output of each layer to obtain an n×1024 feature map, and then output the classification result after passing it through a pooling module and a 1×1 convolution. Repeat the 1×1024 feature vector n times and combine it with the previous feature maps, and then output the segmentation result after passing it through a 1×1 convolution.

[0044] Step 3: Input the point cloud data into the network and train the network parameters, which specifically includes:

[0045] Step 3.1: Input the initial training set obtained in step 1 into the network. Learn the parameters in the network, and let the parameters in step 2 be Θ net , the output of the network is P net , use the cross entropy loss function to update the parameters: The network is trained by stochastic gradient descent. After a certain number of times, the model parameters are saved for testing.

[0046] Step 4: Use the test set to test the network trained in step 3, which specifically includes:

[0047] Step 4.1: Input the test set in step 1 into the saved network model with the best training performance, repeat the training and testing multiple times, and select the model with the best performance.

[0048] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A point cloud analysis method based on dynamic graph convolutional neural network, It is characterized in that The following steps are involved: Step 1: Collect point cloud data of objects and scenes in the radar field of view, collect data sets related to point cloud analysis, and create training and test data sets for model training; Step 2, establish a deep learning network model, which consists of three modules, including an attention graph convolution module, a residual connection module, and a dynamic update module; input the point cloud data obtained in step 1 into the network, and use the attention graph convolution module to extract the features of the disordered area of ​​the point cloud. For the point cloud in the feature space, the graph convolution module is still used to update the feature data. The residual network is used for information transmission. The n×1024 feature map obtained by fusing the output of each layer is output after the pooling module and 1×1 convolution, and the classification result is output. The 1×1024 feature vector is repeated n times and combined with the previous feature maps after 1×1 convolution to output the segmentation result; The deep learning network model structure is as follows: 2.1 Attention Graph Convolution Module Define the edge features as: The calculation formula of LeakyReLU is as follows: in: is a learnable parameter, x j and x i The initial point cloud is the 3D coordinates. The parameters of the edge features are trained through two MLP layers. The attention mechanism is introduced. The nearest neighbor algorithm is used for each point in the point cloud to obtain the vector edge features of the K points around the point. The Softmax logistic regression model is used to normalize the K features to obtain the features of the current point. 2.2 Residual Network Module A residual block is expressed as: The residual block is divided into two parts: the direct mapping part and the residual part, where the direct mapping part is h(x l ), the residual part is This is reflected in the network as the next layer of graph convolutional modules needs to be skipped after each layer to connect the input of the next layer; in different layers of the network, x l and x l+1 The dimensions of the feature maps may be different, in which case 1×1 convolution is needed to increase or decrease the dimension; 2.3 Dynamic Update Module Each layer still uses the graph convolution module to update the feature data of the point cloud in the feature space. The specific method is to use the graph convolution module to update after each time the high-dimensional features of N points are obtained, and the high-dimensional features of N points obtained each time are saved; Step 3: Input the point cloud data into the network and train the network parameters; Step 4: Use the test set to test the network trained in step 3.

2. According to claim 1, a point cloud analysis method based on a dynamic graph convolutional neural network, It is characterized in that Step 1 collects point cloud data of objects and scenes in the radar field of view, collects data sets related to point cloud analysis, and creates training and test data sets for model training, which specifically includes: The three-dimensional information of the object is directly obtained through hardware devices such as depth sensors, structured light, and radar. If the dataset does not include the training set and the test set, the obtained dataset is divided into a training set and a test set in a ratio of 4:1; if the dataset already includes the training set and the test set, go to step 2.

3. According to claim 1, a point cloud analysis method based on a dynamic graph convolutional neural network, It is characterized in that In step 4, the point cloud data is input into the network to train the parameters of the network; Specifically include: Input the initial training set obtained in step 1 into the network; Learn the parameters in the network, and let the parameters in step 2 be Θ net , the output of the network is P net , use the cross entropy loss function to update the parameters: The network is trained by stochastic gradient descent, and after a certain number of times, the model parameters are saved for testing.

4. According to claim 1, a point cloud analysis method based on a dynamic graph convolutional neural network, It is characterized in that Step 4 uses the test set to test the network trained in step 3; Specifically include: Input the test set in step 1 into the saved network model with the best training performance, repeat the training and testing multiple times, and select the model with the best performance.