PointNet + + three-dimensional point cloud target classification method fusing context anchor point attention mechanism

Abstract

The invention discloses a PointNet + + three-dimensional point cloud target classification method fusing a context anchor point attention mechanism, and belongs to the technical field of computer vision, and the method comprises the steps: firstly, preparing a to-be-classified point cloud data set, inputting the data set into a PointNet + + network, and carrying out the sampling and grouping processing of the point cloud; thirdly, extracting features by using a PointNet module, adding a context anchor attention mechanism in a feature extraction layer, and enhancing the features of a central region by using average pooling and 1D bar convolution; and finally, testing the model, inputting a non-label point cloud, outputting an identification result, and judging the accuracy of the result. The method is applied to a Modelnet40 data set, and it is verified through one embodiment that the accuracy of the point cloud classification algorithm fusing the attention mechanism module is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a PointNet++ three-dimensional point cloud target classification method integrating a contextual anchor point attention mechanism. Background Art

[0002] With the development of technologies such as laser scanning, point cloud data processing has been widely used in many fields. Three-dimensional point clouds are rich in spatial information and are less affected by the environment. However, due to the disorder and unevenness of point clouds, there are still certain challenges in network construction and model training. At first, researchers projected three-dimensional point clouds into two-dimensional images. Wu et al. proposed a convolutional deep belief network 3D shapenets model based on voxel grids, which represents three-dimensional point clouds as binary probability distributions in voxel grids and then processes them with convolutional deep networks. However, information is lost during the conversion process, and the classification performance is limited. The MVCNN proposed by Su et al. projects point clouds from multiple perspectives, integrates multi-view feature information into one through convolutional layers and pooling layers, and then obtains classification results through fully connected layers, but the camera's perspective will be blocked, affecting the training results. Later, Qi et al. first proposed the PointNet network, which directly input the unprocessed 3D point cloud into the network for processing, solved the problem of point cloud disorder by encoding global information through multi-layer perceptrons and symmetric functions, and solved the problem of point cloud rotation invariance through 3D space transformation networks. However, PointNet did not consider the local features of the point cloud, which would lead to limited generalization of the model in actual use. On this basis, Qi et al. proposed PointNet++, which divides the point set into overlapping local areas, extracts local features of the point cloud, and then further groups these local features into larger units, extracts features again, and repeats this process until the features of the entire point set are obtained. However, PointNet++ ignores the relationship between local areas, has no context, and loses spatial geometric information. Summary of the invention

[0003] To solve the above problems, the present invention proposes a PointNet++ three-dimensional point cloud target classification method that integrates the contextual anchor attention mechanism. The contextual anchor attention mechanism is added on the basis of PointNet++, and average pooling and 1D strip convolution are used to enhance the features of the central area to improve the accuracy of point cloud classification.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] The PointNet++ 3D point cloud object classification method integrating the contextual anchor attention mechanism includes the following steps:

[0006] S1. Input the training set of point clouds to be classified into the PointNet++ network, and pass through two abstraction layers. Each abstraction layer samples and groups the point clouds in turn, and adds a context anchor attention mechanism to the feature extraction module.

[0007] S2, after two abstract layers, a PointNet is connected to obtain a global feature tensor, the number of channels is changed through a multi-layer perceptron, and the probability of each category is output through a Softmax function;

[0008] S3. Save the training results after each round of training, and then evaluate the performance of the model through the test set. Input an unlabeled point cloud and calculate the classification result through the network. Use matplotlib to draw a three-dimensional graph of the point cloud to determine whether the result is correct.

[0009] Furthermore, the specific method of adding the context anchor attention mechanism to the feature extraction module in step S1 is:

[0010] S101: After average pooling the input features, a local feature map is obtained through 1×1 convolution. The formula is as follows:

[0011] Avg Pool(X) = Conv 1×1 (avg_pool(X)) where: X is the input feature, avg_pool represents average pooling, Conv 1×1 Represents 1×1 convolution;

[0012] S102: Use two depth-wise separable convolutions as an approximation of the standard large kernel depth-wise convolution. The formula is as follows:

[0013]

[0014] in: and Respectively represent the width of 1×k b and height k b ×1 depthwise separable convolution;

[0015] S103: The output of S102 is convolved by 1×1 and then activated using the Sigmoid activation function to obtain the attention factor A. The attention factor is multiplied by the original input X step by step and then added step by step to obtain the final output result F. The formula is as follows:

[0016] A=Sigmoid(Conv 1×1 (H_Conv(X)))

[0017]

[0018] Among them, Sigmoid represents the Sigmoid activation function, represents multiplication at each level, and ⊕ represents addition at each level.

[0019] Furthermore, in step S2, the multilayer perceptron uses the LeakyReLU activation function for nonlinear transformation, and then uses batch normalization to accelerate the training process and improve the model stability.

[0020] Furthermore, the multi-layer perceptron uses the LeakyReLU activation function for nonlinear transformation, and then uses batch normalization to accelerate the training process and improve the stability of the model:

[0021] S201: The formula of LeakyReLU activation function is:

[0022]

[0023] Among them, ω defaults to 100;

[0024] S202: The batch normalization formula is:

[0025]

[0026] in: represents the normalized value, y i represents the value of batch normalization, ω>0 is a small constant, and α, β and χ are three learnable parameters.

[0027] Furthermore, the optimizer used in step S3 is AdamW, and the formula is:

[0028] m t =α1·m t-1 +(1-α1)·g t

[0029]

[0030] Where: g t is the gradient, θ t is a parameter, m t and v t are the first-order and second-order momentum estimates, η is the learning rate, α1 and α2 are the first-order and second-order exponential decay rates, ε is a small constant for numerical stability, λ is the weight decay coefficient, and γ and are two learnable parameters.

[0031] Beneficial effects of the present invention:

[0032] In order to solve the problem that PointNet++ ignores the relationship between local areas of point clouds, has no connection context, and loses spatial geometric information, the present invention integrates the contextual anchor attention mechanism in the feature extraction module. First, the local feature extraction capability of PointNet++ is enhanced. By introducing contextual information, the spatial relationship between points in point cloud data can be better understood, thereby improving the recognition capability of local structures. Secondly, the contextual anchor attention mechanism can help PointNet++ better integrate global and local information, especially when dealing with complex scenes. Finally, the contextual anchor attention mechanism guides the PointNet++ network model to focus on more important areas by calculating attention weights. Combined with multi-scale feature extraction, the robustness and accuracy of the model are enhanced by capturing contextual information of different scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below in conjunction with the accompanying drawings.

[0034] Figure 1 It is a network structure diagram of the three-dimensional point cloud multi-target classification method of the present invention;

[0035] Figure 2 The global point cloud of the steelmaking area of ​​the steel plant collected in Example 1 of the present invention;

[0036] Figure 3 The global point cloud of the steelmaking area of ​​the steel plant after preprocessing described in Example 1 of the present invention;

[0037] Figure 4 The six target point clouds after segmentation and processing described in Example 1 of the present invention;

[0038] Figure 5 1 is a schematic diagram of the accuracy curve of the original PointNet++ network classification described in Example 1 of the present invention;

[0039] Figure 6 It is a schematic diagram of the accuracy curve of the network classification of the fusion context anchor point attention mechanism module described in Example 1 of the present invention;

[0040] Figure 7 This is a schematic diagram comparing the highest accuracy rates during the iteration of the two classification networks described in Example 1 of the present invention;

[0041] Figure 8 It is the category identified by the network of inputting an unlabeled point cloud as described in Example 1 of the present invention;

[0042] Fig. 9 This is the visualization result of inputting an unlabeled point cloud as described in Example 1 of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The present invention provides a PointNet++ 3D point cloud target classification method integrating contextual anchor point attention mechanism, such as Figure 1 As shown, the method comprises the following steps:

[0045] S1: The training set of point clouds to be classified is input into the PointNet++ network. After two abstraction layers, each abstraction layer samples and groups the point clouds in turn. The context anchor attention mechanism is added to the feature extraction module to promote adaptive feature extraction of local and global context information.

[0046] S101: After average pooling the input features, a local feature map is obtained through 1×1 convolution. The formula is as follows:

[0047] Avg_Pool(X)=Conv 1×1 (avg_pool(X))

[0048] Where: X is the input feature, avg_pool represents average pooling, Conv 1×1 Represents 1×1 convolution;

[0049] S102: Use two depth-wise separable convolutions as an approximation of the standard large kernel depth-wise convolution. The formula is as follows:

[0050]

[0051] in: and Respectively represent the width of 1×k b and height k b ×1 depthwise separable convolution;

[0052] S103: The output of S102 is convolved by 1×1 and then activated using the Sigmoid activation function to obtain the attention factor A. The attention factor is multiplied by the original input X step by step and then added step by step to obtain the final output result F. The formula is as follows:

[0053] A=Sigmoid(Conv 1×1 (H_Conv(X)))

[0054]

[0055] Among them, Sigmoid represents the Sigmoid activation function, represents multiplication at each level, and ⊕ represents addition at each level.

[0056] S2: After two abstract layers, a PointNet is connected to obtain a global feature tensor. The number of channels is changed through three multi-layer perceptrons, and the probability of each category is output through the Softmax function. The multi-layer perceptron uses the LeakyReLU activation function for nonlinear transformation, and then batch normalization is used to accelerate the training process and improve the stability of the model. The details are as follows:

[0057] S201: The formula of LeakyReLU activation function is:

[0058]

[0059] Among them, ω defaults to 100;

[0060] S202: The batch normalization formula is:

[0061]

[0062] in: represents the normalized value, y i represents the value of batch normalization, ω>0 is a small constant, and α, β and χ are three learnable parameters.

[0063] S3: After each round of training, the training results are saved, and the performance of the model is evaluated through the test set. An unlabeled point cloud is input and the classification result is obtained through network calculation. Matplotlib is used to draw a three-dimensional graph of the point cloud to determine whether the result is correct.

[0064] The optimizer used in step S3 is AdamW, and the formula is:

[0065] m t =α1·m t-1 +(1-α1)·g t

[0066]

[0067] Where: g t is the gradient, θ t is a parameter, m t and v t are the first-order and second-order momentum estimates, η is the learning rate, α1 and α2 are the first-order and second-order exponential decay rates, ε is a small constant for numerical stability, λ is the weight decay coefficient, and γ and are two learnable parameters. The classification results of the Modelnet40 dataset by the present invention and the original PointNet++ are shown in Table 1. Experiments show that the point cloud classification accuracy of the method proposed in the present invention is higher than that of the original PointNet++.

[0068] Table 1 Classification results of the two networks on the Modelnet40 dataset

[0069]

[0070] Example 1

[0071] The C32 laser radar was connected to a computer, and the LEGO-LOAM algorithm was used to collect point clouds in the steelmaking area of ​​the steel plant. The laser radar and computer were placed on a trolley and moved forward slowly. Point clouds were collected multiple times from multiple angles. The experimental environment for collecting point clouds is shown in Table 2.

[0072] Table 2 Point cloud acquisition experimental environment

[0073]

[0074] The collected global point cloud is as follows Figure 2 , use Cloudcompare software to remove the ground point cloud, and then use statistical filtering to filter and denoise, such as Figure 3 The preprocessed point cloud is segmented using a clustering algorithm and divided into six categories: aisle, vehicle support, excavator, hanger, steelmaking furnace and wall. Each point cloud is downsampled or upsampled to make the number of points uniformly 7000, such as Figure 4 , where the number in the upper left corner is the point cloud label, indicating the category of the point cloud. Due to the small number of samples, the point cloud is enhanced by random translation and rotation. There are a total of 631 point clouds, 80% of which are randomly selected as training sets and 20% as test sets.

[0075] The processed point cloud is input into the original PointNet++ and the PointNet++ network with the contextual anchor attention mechanism module for training. The bach size is set to 12 and the epoch is set to 150. The experimental environment is shown in Table 3.

[0076] Table 3 Point cloud classification experimental environment

[0077]

[0078]

[0079] The classification accuracy curves of the two model networks are shown in Figure 5 and Figure 6 The accuracy fluctuates greatly in the early stage of iteration and gradually converges in the later stage. The maximum values ​​of mAcc and OA during the iteration process are shown in Table 4, and the comparison diagram is shown in Figure 7 It can be seen that the classification accuracy of PointNet++ integrated with the attention mechanism module has increased, with the mAcc value increasing by 3.94% and the 0A value increasing by 3.79%, which is a significant effect.

[0080] Table 4 Classification results of the two networks on the steelmaking area point cloud dataset

[0081]

[0082] Finally, an unlabeled point cloud is input to verify the classification effect of the model. The output category results are shown in Figure 8 , where [5] means Figure 4 The fifth type of point cloud, the visualization results are shown in Fig. 9 , the result is correct.

[0083] The above contents are merely examples and explanations of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the invention or exceed the scope defined by the claims, they shall all fall within the protection scope of the present invention.

Claims

1. The PointNet++ 3D point cloud target classification method integrating the contextual anchor attention mechanism is characterized by: The steps include: S1. Input the training set of point clouds to be classified into the PointNet++ network, and pass through two abstraction layers. Each abstraction layer samples and groups the point clouds in turn, and adds a context anchor attention mechanism to the feature extraction module. S2, after two abstract layers, a PointNet is connected to obtain a global feature tensor, the number of channels is changed through a multi-layer perceptron, and the probability of each category is output through a Softmax function; S3. Save the training results after each round of training, and then evaluate the performance of the model through the test set. Input an unlabeled point cloud and calculate the classification result through the network. Use matplotlib to draw a three-dimensional graph of the point cloud to determine whether the result is correct.

2. The PointNet++ 3D point cloud target classification method integrating the contextual anchor attention mechanism according to claim 1 is characterized in that: The specific method of adding the context anchor attention mechanism to the feature extraction module in step S1 is: S101: After average pooling the input features, a local feature map is obtained through 1×1 convolution. The formula is as follows: Avg_Pool(X)=Conv 1×1 (avg_pool(X)) Where: X is the input feature, avg_pool represents average pooling, Conv 1×1 Represents 1×1 convolution; S102: Use two depth-wise separable convolutions as an approximation of the standard large kernel depth-wise convolution. The formula is as follows: in: and Respectively represent the width of 1×k b and height k b ×1 depthwise separable convolution; S103: The output of S102 is convolved by 1×1 and then activated using the Sigmoid activation function to obtain the attention factor A. The attention factor is multiplied by the original input X step by step and then added step by step to obtain the final output result F. The formula is as follows: Among them, Sigmoid represents the Sigmoid activation function, represents the multiplication at each level. Indicates step-by-step addition.

3. The PointNet++ three-dimensional point cloud target classification method integrating the contextual anchor attention mechanism according to claim 1 is characterized in that: In step S2, the multilayer perceptron uses the LeakyReLU activation function for nonlinear transformation, and then uses batch normalization to accelerate the training process and improve model stability.

4. The PointNet++ three-dimensional point cloud target classification method integrating the contextual anchor attention mechanism according to claim 3 is characterized in that: The multi-layer perceptron uses the LeakyReLU activation function for nonlinear transformation and then uses batch normalization to accelerate the training process and improve the stability of the model: S201: The LeakyReLU activation function formula is: Among them, ω defaults to 100; S202: The batch normalization formula is: in: represents the normalized value, y i represents the value of batch normalization, ω>0 is a small constant, and α, β and χ are three learnable parameters.

5. The PointNet++ 3D point cloud target classification method integrating the contextual anchor attention mechanism according to claim 1 is characterized in that: The optimizer used in step S3 is AdamW, and the formula is: m t =α1·m t-1 +(1-α1)·g t Where: g t is the gradient, θ t is a parameter, m t and v t are the first-order and second-order momentum estimates, η is the learning rate, α1 and α2 are the first-order and second-order exponential decay rates, ε is a small constant for numerical stability, λ is the weight decay coefficient, and γ and are two learnable parameters.

Images