Three-dimensional point cloud change detection method based on attention mechanism to enhance feature comparison

Abstract

The present invention discloses a three-dimensional point cloud change detection method based on attention mechanism to enhance feature contrast, which relates to the field of computer vision and remote sensing, and includes the following steps: extracting point cloud data and preprocessing the point cloud data; inputting the preprocessed point cloud data into kernel point convolution KPConv to extract preliminary features of the point cloud data, and inputting the preliminary features into the residual block to obtain enhanced features of the point cloud data; inputting the enhanced features of the point cloud data into the Transformer module for feature extraction, global maximum pooling and feature difference calculation to obtain a feature difference vector; inputting the feature difference vector into the feature enhancement module to obtain an enhanced feature vector; inputting the enhanced feature vector into the multi-layer perceptron for classification, and outputting the probability distribution of the dual-phase point cloud change category through the Softmax layer. The present invention solves the problem that the existing method loses some information when extracting features from point clouds.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and remote sensing, and specifically relates to a three-dimensional point cloud change detection method based on an attention mechanism to enhance feature comparison. Background Art

[0002] The key to traditional change detection methods based on two-point distance calculation and point cloud displacement difference analysis methods is to establish correspondences between point clouds in Euclidean space, but these correspondences only represent the distances between surfaces, not the actual displacements of points on the surfaces. Therefore, when detecting changes by comparing the distance differences of point cloud data at different time points, especially in the case of parallel movement with the surface, it is difficult to estimate correct and significant changes. The overall idea of ​​the method based on artificial feature extraction is to extract features between point cloud pairs, and then use machine learning models for classification to obtain change detection results.

[0003] In order to overcome the change detection problem based only on distance correspondence in Euclidean space and extract more advanced abstract features without manual work, deep learning methods have been proposed. Most of the current deep learning-based 3D point cloud change detection methods use classification networks to extract the features of two point clouds and then convert them, then pass through an encoder with shared weights, and finally detect changes and change categories through a classification network. However, the density distribution and partial occlusion problems of point clouds at different times have not been completely solved. When extracting features from point clouds, they will be converted and some information will be lost. In addition, high-quality change detection relies heavily on the high availability of training data. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the three-dimensional point cloud change detection method based on attention mechanism to enhance feature contrast provided by the present invention solves the problem that the existing methods lose some information when extracting features from point clouds.

[0005] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a three-dimensional point cloud change detection method based on attention mechanism to enhance feature comparison, comprising the following steps:

[0006] S1: extracting point cloud data and preprocessing the point cloud data;

[0007] S2: Input the preprocessed point cloud data into the kernel point convolution KPConv to extract the preliminary features of the point cloud data, and input the preliminary features of the point cloud data into the residual block to obtain the enhanced features of the point cloud data;

[0008] S3: Inputting the point cloud data enhanced features into the Transformer module for feature extraction, global maximum pooling and feature difference calculation to obtain a feature difference vector;

[0009] S4: inputting the feature difference vector into a feature enhancement module to obtain an enhanced feature vector;

[0010] S5: Input the enhanced feature vector into a multi-layer perceptron for classification, and output the probability distribution of the dual-phase point cloud change category through the Softmax layer to complete the three-dimensional point cloud change detection based on the attention mechanism to enhance feature comparison.

[0011] Furthermore, the S1 includes the following sub-steps:

[0012] S11: Time point and Point cloud data and Define a fixed-size 3D bounding box and extract all points that fall within the bounding box and ;

[0013] S12: and Perform random downsampling to obtain downsampled point cloud data;

[0014] S13: performing centralization and normalization processing on the downsampled point cloud data to obtain standardized point cloud data and , to realize the preprocessing of point cloud data.

[0015] Furthermore, S2 includes the following sub-steps:

[0016] S21: Standardized point cloud data and Input kernel point convolution KPConv to get the time point The convolution output feature vector and time point The convolution output feature vector ;

[0017] S22: and Input multiple residual blocks for processing to obtain point cloud data enhancement features and .

[0018] Furthermore, the kernel point convolution KPConv in S21 is at point Convolution at Defined as:

[0019]

[0020] in, For Points with radius Neighborhood of is the kernel function, Point cloud data and midpoint The characteristic vector of represents a three-dimensional Euclidean space, Each point in represents a coordinate point in three-dimensional space;

[0021]

[0022] in, is the relative displacement between the input point and the convolution kernel point, is the index in the convolution kernel, Index bounds for all possible convolution kernel points, is the linear relationship function between the input point and the kernel point, is the core point position, is the weight matrix.

[0023] Furthermore, the point cloud data enhancement feature in S22 and The calculation formula is:

[0024]

[0025] in, Enhance features for point cloud data or The characteristic value of is the residual convolution operation, is the weight of the convolution kernel.

[0026] Furthermore, S3 includes the following sub-steps:

[0027] S31: Enhance the features of point cloud data and Input the Transformer module for feature extraction to obtain the time point The Transformer output feature vector and time point The Transformer output feature vector ;

[0028] S32: and Perform global maximum pooling to obtain the global feature vector and ;

[0029] S33: Based on the global eigenvector and , and obtain the feature difference vector :

[0030] .

[0031] Furthermore, in S31 and The calculation formula is:

[0032]

[0033] in, for or The characteristic value of It means to connect the concatenated vectors in sequence on the last dimension to form a new vector. is the query matrix, is the key matrix, is the value matrix, It represents the operation of concatenating the outputs of multiple attention heads by dimension. is the weighted feature representation of each attention head, is the linear transformation matrix;

[0034]

[0035]

[0036]

[0037]

[0038] in, is the weighted feature representation obtained by attention calculation for each head, For attention calculation, To normalize the results, the superscript is the transpose of the matrix, is the dimension of the key vector, Enhance features for point cloud data or The characteristic value of , and is the weight matrix that can be learned.

[0039] Furthermore, the global feature vector in S32 and The calculation formula is:

[0040]

[0041] in, is the global eigenvector or The characteristic value of For the The maximum pooling operation is performed on each feature dimension.

[0042] Furthermore, the enhanced feature vector in S4 The calculation formula is:

[0043]

[0044]

[0045]

[0046] in, and is the intermediate parameter, , and is the weight matrix, , and is the bias term, is the feature difference vector.

[0047] Furthermore, the probability distribution of the change categories of the dual-phase point cloud in S5 is The calculation formula is:

[0048]

[0049]

[0050] in, For enhanced feature variables Input the change detection result obtained by multi-layer perceptron MLP classification, Represents Multilayer Perceptron.

[0051] The beneficial effects of the present invention are as follows: the present invention proposes a multi-scale three-dimensional point cloud change detection method based on an attention mechanism that combines KPConv and Transformer, providing a new solution for processing point cloud data change detection and classification tasks. In view of the problems existing in the current point cloud processing methods, such as insufficient feature extraction and lack of global information of local features, the present invention effectively extracts and fuses the local and global features of point cloud data by introducing KPConv convolutional layers and Transformer modules. In addition, the feature enhancement module further improves the ability to distinguish features, so that the model shows higher robustness and accuracy in different scenarios and applications. Experimental results show that the new model framework is superior to existing methods in terms of accuracy and stability when processing complex point cloud data tasks, providing a more effective technical means for the field of change detection and classification of three-dimensional point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Flowchart of the 3D point cloud change detection method based on attention mechanism to enhance feature comparison.

[0053] Figure 2 Schematic diagram of a 3D point cloud change detection method based on attention mechanism to enhance feature comparison.

[0054] Figure 3 A comparison chart of 3D point cloud change detection and classification effects based on attention mechanism to enhance feature comparison. DETAILED DESCRIPTION

[0055] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0056] like Figure 1 As shown, a 3D point cloud change detection method based on attention mechanism to enhance feature contrast includes the following steps:

[0057] S1: extracting point cloud data and preprocessing the point cloud data;

[0058] S2: Input the preprocessed point cloud data into the kernel point convolution KPConv to extract the preliminary features of the point cloud data, and input the preliminary features of the point cloud data into the residual block to obtain the enhanced features of the point cloud data;

[0059] S3: Inputting the point cloud data enhanced features into the Transformer module for feature extraction, global maximum pooling and feature difference calculation to obtain a feature difference vector;

[0060] S4: inputting the feature difference vector into a feature enhancement module to obtain an enhanced feature vector;

[0061] S5: Input the enhanced feature vector into a multi-layer perceptron for classification, and output the probability distribution of the dual-phase point cloud change category through the Softmax layer to complete the three-dimensional point cloud change detection based on the attention mechanism to enhance feature comparison.

[0062] like Figure 2 As shown, the main modules of the technical solution of the present invention include the following: the model includes three parts. The first part is the feature extraction process of kernel point convolution KPConv (Kernel Point Convolution). KPConv is a convolution operation specially designed for point cloud data. It extracts features by defining convolution kernels in local areas of point clouds to enhance feature extraction capabilities. The second part is the global modeling and feature difference calculation process. The Transformer module is used to capture long-distance dependencies between point cloud features through a multi-head attention mechanism to enhance the global representation of features. The module uses a multi-head attention mechanism to adaptively weight input features to improve the distinguishability and robustness of features, and performs context modeling at the global feature level to further enhance feature expression capabilities. The third part is the feature enhancement process, which uses a feature enhancement module (Feature Enhanced Module) to weight and fuse features and expand feature dimensions through an attention mechanism to improve feature expression capabilities. The module weights and expands features to generate richer and more discriminative feature representations, thereby enhancing the model's ability to capture subtle changes in point clouds.

[0063] The S1 includes the following sub-steps:

[0064] S11: Time point and Point cloud data and Define a fixed-size 3D bounding box and extract all points that fall within the bounding box and ;

[0065] S12: and Perform random downsampling to obtain downsampled point cloud data;

[0066] S13: performing centralization and normalization processing on the downsampled point cloud data to obtain standardized point cloud data and , to realize the preprocessing of point cloud data.

[0067] The S2 includes the following sub-steps:

[0068] S21: Standardized point cloud data and Input kernel point convolution KPConv to get the time point The convolution output feature vector and time point The convolution output feature vector ;

[0069] The kernel point convolution KPConv in S21 is at point Convolution at Defined as:

[0070]

[0071] in, For Points with radius Neighborhood of is the kernel function, Point cloud data and midpoint The characteristic vector of represents a three-dimensional Euclidean space, Each point in represents a coordinate point in three-dimensional space;

[0072]

[0073] in, is the relative displacement between the input point and the convolution kernel point, that is, the input point coordinates and convolution kernel points The difference is the index in the convolution kernel, used to identify the points in the convolution kernel. Index bounds for all possible convolution kernel points, is the linear relationship function between the input point and the kernel point, is the core point position, is the weight matrix;

[0074] S22: and Input multiple residual blocks for processing to obtain point cloud data enhancement features and ;

[0075] The point cloud data enhancement features in S22 and The calculation formula is:

[0076]

[0077] in, Enhance features for point cloud data or The characteristic value of is the residual convolution operation, is the weight of the convolution kernel.

[0078] The S3 includes the following sub-steps:

[0079] S31: Enhance the features of point cloud data and Input the Transformer module for feature extraction to obtain the time point The Transformer output feature vector and time point The Transformer output feature vector ;

[0080] Enhanced features and And input Transformer module, using the Multi-Head Attention mechanism to capture the long-range dependencies between point cloud features, so as to perform global feature representation and obtain the time point The Transformer output feature vector and time point The Transformer output feature vector and The calculation method of the Multi-Head Attention mechanism of the Transformer module is as follows:

[0081] In S31 and The calculation formula is:

[0082]

[0083] in, for or The characteristic value of It means to connect the concatenated vectors in sequence on the last dimension to form a new vector. is the query matrix, is the key matrix, is the value matrix, To concatenate the outputs of multiple attention heads by dimension, is the weighted feature representation of each attention head, is a linear transformation matrix; the Multi-Head Attention mechanism calculates multiple independent attentions in parallel and then concatenates the results;

[0084]

[0085]

[0086]

[0087]

[0088] in, is the weighted feature representation obtained by attention calculation for each head, is the attention calculation, indicating that the query matrix , the bond matrix , value matrix Perform dot product calculations, The results are normalized so that the sum of their weights is 1, ensuring that different attention heads have different weight distributions. is the transpose of the matrix, is the dimension of the key vector, Enhance features for point cloud data or The characteristic value of , and is the weight matrix that can be learned;

[0089] S32: and Perform global maximum pooling to obtain the global feature vector and ;

[0090] The global eigenvector in S32 and The calculation formula is:

[0091]

[0092] in, is the global eigenvector or The characteristic value of For the Perform the maximum pooling operation on feature dimensions;

[0093] S33: Based on the global eigenvector and , and obtain the feature difference vector :

[0094] .

[0095] The feature difference vector Input to Feature Enhanced Module for processing, first perform linear transformation and Tanh function activation on the input feature difference vector to obtain , and then through the second linear transformation and Softmax activation function to get , and finally weighted summation and feature expansion to generate an enhanced feature vector , the calculation process is as follows:

[0096] The enhanced feature vector in S4 The calculation formula is:

[0097]

[0098]

[0099]

[0100] in, and is an intermediate parameter, the former is after the activation function tanh The feature vector processed by the softmax activation function is and The normalized output after linear transformation is: , and is the weight matrix, , and is the bias term, is the feature difference vector.

[0101] The enhanced feature vector Input into the multi-layer perceptron (MLP) for classification and obtain the change detection result , and then the Softmax layer outputs the probability distribution of the change category of the final dual-phase point cloud change detection , the process is as follows:

[0102] Probability distribution of the change categories of the dual-phase point cloud in S5 The calculation formula is:

[0103]

[0104]

[0105] in, For enhanced feature variables Input the change detection result obtained by multi-layer perceptron MLP classification, Represents Multilayer Perceptron, which means that the input feature vector is passed to it for feature extraction and transformation processing to generate classification results.

[0106] In one embodiment of the present invention, the specific process is as follows:

[0107] 1. Data preprocessing

[0108] Input 100 pairs of annotated scene LAS point cloud files, a total of 200 pairs, of which 60 pairs are used for training, 20 pairs for verification, and 20 pairs for testing. and Point cloud data and Define a 3D bounding box with a fixed size of 2.4m in length and width and 10m in height, and extract all points that fall within the bounding box and Then and Perform random downsampling, retain 2048 points, and obtain the downsampled point cloud data and Then, the downsampled point cloud data and Perform centralization and normalization to obtain standardized point cloud data and .

[0109] 2. Preliminary feature extraction of KPConv layer

[0110] The standardized point cloud data and Enter the KPConv layer, perform convolution operation, set the initial voxel size to 0.02 m when extracting preliminary features, and then get the time point The convolution output feature vector and time point The convolution output feature vector and , the input dimension of the KPConv layer is 4 and the output dimension is 64.

[0111] 3. Residual Block (ResBlock) feature processing

[0112] The initial features and Input multiple ResBlock layers for further processing to obtain enhanced features and , the ResBlock layer starts with an input dimension of 64, and then outputs multiple layers with dimensions of 128, 256, 512, and 1024 respectively.

[0113] 4. Feature extraction and enhancement processing

[0114] The enhanced features of 1024 dimensions and And input Transformer module, use Multi-HeadAttention to get the mechanism feature vector and Then the global feature vector is obtained through global maximum pooling and , and then perform difference calculation to obtain the feature difference vector Finally, the Feature Enhanced Module is used to improve the feature difference vector Enhance to get the final feature vector .

[0115] 5. Change detection and classification output of point cloud for test set files

[0116] The final feature vector Input three layers of 256, 128 and 5 neurons Classify and get the final change detection result And the classification results .

[0117] 6. Environment Configuration

[0118] Throughout the process, the Pytorch framework was used to implement the model and training was performed on an NVIDIA GTX 1080Ti GPU. The Adam optimizer was used, with an initial learning rate of 0.01, which decayed to 95% of the original value every 4 epochs, and a weight decay coefficient of 1e. -6 , the maximum number of training rounds is 350, the gradient accumulation step is 1, and the batch size is 1. In the experimental results, PCs at T1 and PCs at T2 represent point clouds at two times. The areas with no change are marked in blue, and the areas with other types of changes are marked in red, such as Figure 3 shown.

[0119] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the invention.

Claims

1. A three-dimensional point cloud change detection method based on attention mechanism to enhance feature contrast, characterized in that: The following steps are involved: S1: extracting point cloud data and preprocessing the point cloud data; S2: Input the preprocessed point cloud data into the kernel point convolution KPConv to extract the preliminary features of the point cloud data, and input the preliminary features of the point cloud data into the residual block to obtain the enhanced features of the point cloud data; S3: Inputting the point cloud data enhanced features into the Transformer module for feature extraction, global maximum pooling and feature difference calculation to obtain a feature difference vector; S4: inputting the feature difference vector into a feature enhancement module to obtain an enhanced feature vector; S5: input the enhanced feature vector into a multilayer perceptron for classification, and output the probability distribution of the dual-phase point cloud change category through a Softmax layer, thereby completing the three-dimensional point cloud change detection based on the attention mechanism to enhance feature comparison; The S2 includes the following sub-steps: S21: Standardized point cloud data and Input kernel point convolution KPConv to get the time point The convolution output feature vector and time point The convolution output feature vector ; S22: and Input multiple residual blocks for processing to obtain point cloud data enhancement features and ; The S3 includes the following sub-steps: S31: Enhance the features of point cloud data and Input the Transformer module for feature extraction to obtain the time point The Transformer output feature vector and time point The Transformer output feature vector ; S32: and Perform global maximum pooling to obtain the global feature vector and ; S33: Based on the global eigenvector and , and obtain the feature difference vector : ； The enhanced feature vector in S4 The calculation formula is: in, and is the intermediate parameter, After activation function tanh The processed feature vector, is the feature vector after softmax activation, indicating and The normalized output after linear transformation is , and is the weight matrix, , and is the bias term, is the feature difference vector.

2. The three-dimensional point cloud change detection method based on attention mechanism enhanced feature contrast according to claim 1 is characterized in that: The S1 includes the following sub-steps: S11: Time point and Point cloud data and Define a fixed-size 3D bounding box and extract all points that fall within the bounding box and ; S12: and Perform random downsampling to obtain downsampled point cloud data; S13: performing centralization and normalization processing on the downsampled point cloud data to obtain standardized point cloud data and , to realize the preprocessing of point cloud data.

3. The three-dimensional point cloud change detection method based on attention mechanism enhanced feature contrast according to claim 1 is characterized in that: The kernel point convolution KPConv in S21 is at point Convolution at Defined as: in, For Points with radius Neighborhood of is the kernel function, Point cloud data and midpoint The characteristic vector of represents a three-dimensional Euclidean space, Each point in represents a coordinate point in three-dimensional space; in, is the relative displacement between the input point and the convolution kernel point, is the index in the convolution kernel, Index bounds for all possible convolution kernel points, is the linear relationship function between the input point and the kernel point, is the core point position, is the weight matrix.

4. The three-dimensional point cloud change detection method based on attention mechanism enhanced feature contrast according to claim 3 is characterized in that: The point cloud data enhancement features in S22 and The calculation formula is: in, Enhance features for point cloud data or The characteristic value of is the residual convolution operation, is the weight of the convolution kernel.

5. The three-dimensional point cloud change detection method based on attention mechanism enhanced feature contrast according to claim 4 is characterized in that: In S31 and The calculation formula is: in, for or The characteristic value of It means to connect the concatenated vectors in sequence on the last dimension to form a new vector. is the query matrix, is the key matrix, is the value matrix, It represents the operation of concatenating the outputs of multiple attention heads by dimension. is the weighted feature representation of each attention head, is the linear transformation matrix; in, is the weighted feature representation obtained by attention calculation for each head, For attention calculation, To normalize the results, the superscript is the transpose of the matrix, is the dimension of the key vector, Enhance features for point cloud data or The characteristic value of , and is the weight matrix that can be learned.

6. The three-dimensional point cloud change detection method based on attention mechanism enhanced feature contrast according to claim 5 is characterized in that: The global eigenvector in S32 and The calculation formula is: in, is the global eigenvector or The characteristic value of For the The maximum pooling operation is performed on each feature dimension.

7. The three-dimensional point cloud change detection method based on attention mechanism enhanced feature contrast according to claim 6, characterized in that: Probability distribution of the change categories of the dual-phase point cloud in S5 The calculation formula is: in, For enhanced feature variables Input the change detection result obtained by multi-layer perceptron MLP classification, Represents Multilayer Perceptron.

Images