A method for detecting three-dimensional changes of UAVs over the ground based on attention of neighborhood display features

Abstract

The present invention discloses a method for detecting three-dimensional changes of unmanned aerial vehicles to the ground based on neighborhood display feature attention, which relates to three-dimensional point cloud change detection technology, including: loading and preprocessing point cloud data; extracting point cloud features based on the preprocessed point cloud data using an explicit neighborhood feature extraction module ENFE; performing hierarchical feature fusion based on point cloud features using a Point Transformer module; and performing change recognition on the fused point cloud feature difference data using a multi-layer perceptron to complete the detection of three-dimensional changes of unmanned aerial vehicles to the ground based on neighborhood display feature attention. The present invention solves the problem that existing methods are difficult to adapt to various complex scenes and have limited feature expression capabilities.

Description

Technical Field

[0001] The present invention relates to a three-dimensional point cloud change detection technology, and in particular to a method for detecting three-dimensional changes of a drone to the ground based on neighborhood display feature attention. Background Art

[0002] With the rapid development of remote sensing and drone technology, change detection is becoming increasingly important in fields such as environmental monitoring, urban planning, and autonomous driving. Change detection can identify changes in time series data to help monitor and analyze the environment and man-made structures. This is critical for applications such as road change detection in autonomous driving, building change monitoring in urban planning, and natural environment change detection in environmental monitoring.

[0003] In 3D change detection, common data sources include laser detection and ranging (LiDAR), synthetic aperture radar (SAR), and point cloud data generated by photogrammetry. LiDAR systems can be divided into aerial laser scanning (ALS), mobile laser scanning (MLS), and terrestrial laser scanning (TLS). In autonomous driving, MLS is often used to obtain high-precision street-level point cloud data to ensure that the vehicle can identify and respond to obstacles and pedestrians in the environment. In environmental monitoring and urban planning, ALS and TLS are widely used to monitor large-scale surface changes and building structure changes. With the development of drone technology, point cloud data generated by drones equipped with LiDAR or high-overlapping high-resolution images has also become an important data source for 3D change detection.

[0004] Existing methods still face many challenges in dealing with complex scenes and large-scale data. Three-dimensional point cloud data is usually highly sparse and irregular, making data processing and feature extraction difficult. Different application scenarios may have significant differences in data, and existing methods are difficult to adapt to various complex scenarios, especially in drone-based ground data, where terrain and object changes increase the difficulty of detection. High-precision change detection methods usually require a lot of computing resources, which may be a limitation in practical applications. In addition, existing methods lack full utilization of contextual information in the data, resulting in limited feature representation capabilities. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for detecting three-dimensional changes of UAVs on the ground based on neighborhood display feature attention, which solves the problem that the existing methods are difficult to adapt to various complex scenes and have limited feature expression capabilities.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a method for detecting three-dimensional changes of UAV to the ground based on neighborhood display feature attention, comprising the following steps:

[0007] S1: Load and preprocess point cloud data;

[0008] S2: Based on the preprocessed point cloud data, the explicit neighborhood feature extraction module ENFE is used to extract point cloud features;

[0009] S3: Based on point cloud features, the Point Transformer module is used for hierarchical feature fusion;

[0010] S4: Use a multi-layer perceptron to identify changes in the fused point cloud feature difference data and complete the UAV-to-ground three-dimensional change detection based on neighborhood display feature attention.

[0011] Furthermore, the point cloud data is loaded and preprocessed in S1, specifically: point cloud data pairs collected at different times are loaded into the model, and the point cloud data pairs are filtered and normalized.

[0012] Furthermore, the explicit neighborhood feature extraction module ENFE in S2 includes an explicit feature extraction submodule EFE and a neighborhood feature extraction submodule NFE, and S2 includes the following sub-steps:

[0013] S21: extracting features from the preprocessed point cloud data using an explicit feature extraction submodule EFE to obtain a first display structure feature;

[0014] S22: extracting features from the preprocessed point cloud data using a neighborhood feature extraction submodule NFE to obtain a second display structure feature;

[0015] S23: Fusing the first display structure feature and the second display structure feature, and generating a final point cloud feature through a convolution layer.

[0016] Furthermore, the S21 includes the following sub-steps:

[0017] S211: Perform initial feature extraction on the preprocessed point cloud data using a one-dimensional convolutional layer;

[0018] S212: using the explicit feature extraction submodule EFE to obtain the neighborhood features of each point through the neighborhood index from the point cloud data after the initial feature extraction, and concatenating the neighborhood features to generate a neighborhood feature matrix;

[0019] S213: Calculate the average value of the neighborhood feature matrix in the neighborhood dimension to obtain the centroid feature;

[0020] S214: expanding the centroid feature to the same shape as the neighborhood feature matrix, and connecting the centroid feature with the original neighborhood feature to obtain a new display structure feature;

[0021] S215: Use a multi-layer perceptron to perform dimension reduction and feature aggregation on the new display structure feature to obtain a first display structure feature.

[0022] Furthermore, the S22 includes the following sub-steps:

[0023] S221: Perform initial feature extraction on the preprocessed point cloud data using a one-dimensional convolutional layer;

[0024] S222: using the neighborhood feature extraction submodule NFE to obtain the neighborhood feature of each point through the neighborhood index from the point cloud data after the initial feature extraction, and concatenate the neighborhood features to generate a neighborhood feature matrix;

[0025] S223: Use a multi-layer perceptron to perform dimension reduction and feature aggregation on the neighborhood feature matrix to obtain a second display structure feature.

[0026] Furthermore, S3 includes the following sub-steps:

[0027] S31: using the fully connected layer to perform feature transformation on the point cloud features output by the explicit neighborhood feature extraction module ENFE;

[0028] S32: Based on the feature transformation results, the self-attention mechanism in the Point Transformer module is used to calculate the relationship between the points in the point cloud data, and the relative position between each point in the point cloud data and its neighboring points is calculated;

[0029] S33: Based on the calculation results, the attention weight of each point in the point cloud data is calculated using the query matrix and the key matrix;

[0030] S34: Perform weighted fusion according to the attention weights to obtain weighted feature representation and realize hierarchical feature fusion.

[0031] Furthermore, the relationship between points in the point cloud data is calculated in S32, specifically: a query matrix, a key matrix and a value matrix of the points in the point cloud data are respectively calculated using a linear layer.

[0032] Furthermore, the point cloud feature difference data in S4 The calculation formula is:

[0033]

[0034] in, and are the feature vectors of two points in the two point cloud data respectively.

[0035] The beneficial effects of the present invention are as follows: the present invention achieves efficient and accurate three-dimensional point cloud change detection by cleverly introducing an explicit neighborhood feature extraction module (ENFE) and a twin Point Transformer structure. The ENFE module comprehensively captures the local geometric features and global structural features of point cloud data through the explicit feature extraction submodule (EFE) and the neighborhood feature extraction submodule (NFE); the twin Point Transformer structure enhances the feature representation capability and change detection accuracy through the self-attention mechanism. This allows the present invention to maintain high precision and robustness when processing large-scale and high-dimensional point cloud data, and is suitable for a variety of complex scenarios such as autonomous driving, urban modeling, and environmental monitoring. It fully solves the shortcomings of previous methods when facing complex terrain and data noise, and achieves high efficiency and accuracy in three-dimensional point cloud change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flowchart of the UAV-to-ground three-dimensional change detection method based on neighborhood display feature attention.

[0037] Figure 2 This is the structural diagram of the UAV-to-ground 3D change detection model based on neighborhood display feature attention.

[0038] Figure 3 This is the structure diagram of the explicit neighborhood feature extraction module ENFE.

[0039] Figure 4 This is the point attention module diagram.

[0040] Figure 5 This is a diagram showing the effect of change detection based on neighborhood explicit feature attention comparison. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0042] like Figure 1 As shown, a method for detecting three-dimensional changes of UAV to ground based on neighborhood display feature attention includes the following steps:

[0043] S1: Load and preprocess point cloud data;

[0044] S2: Based on the preprocessed point cloud data, the explicit neighborhood feature extraction module ENFE is used to extract point cloud features;

[0045] S3: Based on point cloud features, the Point Transformer module is used for hierarchical feature fusion;

[0046] S4: Use a multi-layer perceptron to identify changes in the fused point cloud feature difference data and complete the UAV-to-ground three-dimensional change detection based on neighborhood display feature attention.

[0047] like Figure 2 As shown, the main modules of the technical solution of the present invention include three parts. The first part is to normalize and preprocess the input point cloud data, the second part is to encode explicit features based on the neighborhood and use the point attention layer for hierarchical feature fusion, and the third part is to use a multi-layer perceptron to identify changes in the encoded point cloud features. In the first part, the original point cloud is preprocessed by denoising, stripping the ground, etc., and then the point cloud is filtered to a fixed number using a filter. In the second part, two modules, explicit neighborhood features and point attention, are used to extract point cloud spatial features and fuse and enhance multi-scale spatial features, respectively. In the third part, a multi-layer perceptron is used to compare the point cloud features of two different phases and then output the change probability of each point.

[0048] In S1, point cloud data loading and preprocessing are performed, specifically: point cloud data pairs collected at different times are loaded into the model, and the point cloud data pairs are filtered and normalized.

[0049] In this embodiment, point cloud data pairs (such as PC1 and PC2) collected at different times are loaded into the model. These point cloud data can come from the point cloud generated by the LiDAR carried by the drone or high-overlapping high-resolution images. First, the point cloud data is simply filtered to remove noise points, and the point cloud data is normalized to ensure data consistency.

[0050] The explicit neighborhood feature extraction module ENFE in S2 includes an explicit feature extraction submodule EFE and a neighborhood feature extraction submodule NFE, and S2 includes the following sub-steps:

[0051] S21: extracting features from the preprocessed point cloud data using an explicit feature extraction submodule EFE to obtain a first display structure feature;

[0052] S22: extracting features from the preprocessed point cloud data using a neighborhood feature extraction submodule NFE to obtain a second display structure feature;

[0053] S23: Fusing the first display structure feature and the second display structure feature, and generating a final point cloud feature through a convolution layer.

[0054] like Figure 3As shown in the figure, before entering the ENFE module, the initial features of the point cloud data are extracted and processed by a one-dimensional convolution layer. Assuming that the input point cloud data is xyz, after the initial feature extraction, the shape of the data changes from [B, N, 3] to [B, C, N, 1], where B is the batch size, N is the number of points, and C is the number of channels. After passing through the explicit feature extraction (EFE) submodule, the input point cloud data obtains the neighborhood features of each point through the neighborhood index. Assuming that the neighborhood index is [B, N, K], where K represents the number of neighborhood points of each point. The neighborhood feature collection process will expand the point cloud data and obtain the neighborhood feature matrix of each point, with a shape of [B, C, N, K]. This process uses the index operation of the tensor to obtain the neighborhood features of each point through the index and splice these features into a matrix. For each point, calculate the centroid of its neighborhood points. The centroid is the average value of the neighborhood points and represents the center position of this neighborhood. The specific operation is to average the neighborhood feature matrix in the neighborhood dimension to obtain the centroid feature, the shape of which is [B, C, N, 1]. The centroid feature is expanded to the same shape as the neighborhood feature matrix and connected to the original neighborhood feature to form a new explicit structural feature. The explicit structural feature is reduced in dimension and aggregated by a multi-layer perceptron (MLP) to obtain the final explicit feature representation. The operation of MLP can be implemented through a convolutional layer. The specific operation is to input the explicit structural feature into the convolutional layer for feature dimensionality reduction. The shape of the feature after dimensionality reduction is [B, d_out, N, K]. Then the features in the neighborhood dimension are summed to obtain the aggregated explicit feature, which has a shape of [B, d_out, N, 1].

[0055] The S21 includes the following sub-steps:

[0056] S211: Perform initial feature extraction on the preprocessed point cloud data using a one-dimensional convolutional layer;

[0057] S212: using the explicit feature extraction submodule EFE to obtain the neighborhood features of each point through the neighborhood index from the point cloud data after the initial feature extraction, and concatenating the neighborhood features to generate a neighborhood feature matrix;

[0058] S213: Calculate the average value of the neighborhood feature matrix in the neighborhood dimension to obtain the centroid feature;

[0059] S214: expanding the centroid feature to the same shape as the neighborhood feature matrix, and connecting the centroid feature with the original neighborhood feature to obtain a new display structure feature;

[0060] S215: Use a multi-layer perceptron to perform dimension reduction and feature aggregation on the new display structure feature to obtain a first display structure feature.

[0061] After the neighborhood feature extraction (NFE) module, its purpose is to enhance the feature representation by aggregating neighborhood point features. Similar to the EFE module, the neighborhood features of each point are collected through the neighborhood index. The shape of the collected neighborhood features is [B, C, N, K]. The neighborhood features are processed using a multi-layer perceptron (MLP), and the neighborhood features are input into the convolution layer for feature processing. The explicit structural features of the neighborhood points are calculated and processed through the convolution layer. The specific operation is similar to the EFE module, and the shape of the processed explicit structural features is [B, d_out, N, K].

[0062] The ENFE module fuses the output features of EFE and NFE and generates the final feature representation through another convolutional layer. The output features of EFE and NFE are added together and the fused features are processed through a convolutional layer to obtain the final fused feature representation.

[0063] The S22 includes the following sub-steps:

[0064] S221: Perform initial feature extraction on the preprocessed point cloud data using a one-dimensional convolutional layer;

[0065] S222: using the neighborhood feature extraction submodule NFE to obtain the neighborhood feature of each point through the neighborhood index from the point cloud data after the initial feature extraction, and concatenate the neighborhood features to generate a neighborhood feature matrix;

[0066] S223: Use a multi-layer perceptron to perform dimension reduction and feature aggregation on the neighborhood feature matrix to obtain a second display structure feature.

[0067] The S3 includes the following sub-steps:

[0068] S31: using the fully connected layer to perform feature transformation on the point cloud features output by the explicit neighborhood feature extraction module ENFE;

[0069] S32: Based on the feature transformation results, the self-attention mechanism in the Point Transformer module is used to calculate the relationship between the points in the point cloud data, and the relative position between each point in the point cloud data and its neighboring points is calculated;

[0070] S33: Based on the calculation results, the attention weight of each point in the point cloud data is calculated using the query matrix and the key matrix;

[0071] S34: Perform weighted fusion according to the attention weights to obtain weighted feature representation and realize hierarchical feature fusion.

[0072] The relationship between points in the point cloud data is calculated in S32, specifically: a query matrix, a key matrix and a value matrix of the points in the point cloud data are calculated respectively using a linear layer.

[0073] First, the features output by the ENFE module are transformed. The purpose of feature transformation is to transform the input features from d_points to d_model for self-attention calculation. This process mainly uses a fully connected layer to transform the feature dimension. Then, the self-attention mechanism of Point Transformer is used to calculate the relationship between points in the point cloud data, such as Figure 4 As shown in the figure, the specific operation is to use the linear layer to calculate the query, key and value matrices respectively. The calculated matrix shape is [B, N, d_model]. The query matrix, key matrix and value matrix represent the features of each point in the point cloud data respectively. Then relative position encoding is performed to calculate the relative position between each point in the point cloud and its neighboring points, and a linear layer is used for processing. The purpose of relative position encoding is to consider the relative position relationship between points. Then the attention weight calculation is performed, and the attention score is calculated using the query matrix and the key matrix. The specific operation is to calculate the dot product of the query matrix and the key matrix, and normalize them through the softmax function to obtain the attention weight. The attention weight indicates the degree of attention of each point to its neighboring points. Then weighted fusion is performed, and the value matrix is ​​weighted and summed using the attention weight to obtain the weighted feature representation. The specific operation is to multiply the attention weight with the value matrix and sum them to obtain the enhanced feature representation. Finally, the fully connected layer (linear layer) is used to restore the enhanced feature dimension back to d_points, and the restored feature shape is [B, d_points, N]. Add the restored features to the input features to get the final feature representation.

[0074] After the Point Transformer module is processed, the difference in point cloud features between two time points is obtained. Next, these features need to be further processed through the multi-layer perceptron (MLP) layer to determine the final feature changes. The MLP layer receives the point cloud feature difference data output by the Point Transformer module. The MLP layer consists of multiple fully connected layers (linear layers), each of which is followed by an activation function and a batch normalization layer. The first fully connected layer reduces the input feature dimension from d_out to 64. The linear layer is followed by a ReLU activation function to introduce nonlinearity. After three fully connected layers, it is finally reduced to 2 dimensions for classification (changed or unchanged). Finally, the features of each point are softmax processed to obtain the classification probability of each point.

[0075] The point cloud feature difference data in S4 The calculation formula is:

[0076]

[0077] in, and are the feature vectors of two points in the two point cloud data respectively.

[0078] Then will The corresponding change score can be obtained by sending it to the MLP layer. Based on the change score calculated in the feature fusion stage, the MLP layer is finally used for change detection. This layer classifies the change score and outputs the change status of each point. Through threshold judgment, it is determined which points in the point cloud have changed and which points have not changed.

[0079] The entire training process is end-to-end training. The model is trained using the Negative Log-Likelihood Loss (NLLLoss). Specifically, the model first outputs the category probability distribution of each point, and then compares these outputs with the corresponding true labels through the NLLLoss function to calculate the classification loss of each point. Finally, these loss values ​​are summed up to get the overall loss, which is used for backpropagation to update the model parameters and thus optimize the model performance. The model optimizer uses the Adam optimizer, and the batch size of the input data is set to 8. A total of 40 epochs are trained, and the overall learning rate is set to , the learning rate drops 10 times after 50 epochs, and a warm-up learning rate strategy is adopted in the first 10 epochs.

[0080] After the above steps, the model can output the change detection results. The specific experimental results are shown in Figure 5 The figure shows the original point cloud data and the data output by the proposed method. The blue part in the figure indicates the area where the prediction remains unchanged, and the red part indicates the area where the prediction changes. Compared with the standard answer, the proposed method has significant advantages in the accuracy and robustness of change detection.

[0081] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the invention.

Claims

1. A method for detecting three-dimensional changes of UAVs on the ground based on neighborhood display feature attention, characterized in that: The following steps are involved: S1: Load and preprocess point cloud data; S2: Based on the preprocessed point cloud data, the explicit neighborhood feature extraction module ENFE is used to extract point cloud features; S3: Based on point cloud features, the Point Transformer module is used for hierarchical feature fusion; S4: Use a multi-layer perceptron to identify changes in the fused point cloud feature difference data, and complete the UAV-to-ground three-dimensional change detection based on neighborhood display feature attention; The explicit neighborhood feature extraction module ENFE in S2 includes an explicit feature extraction submodule EFE and a neighborhood feature extraction submodule NFE, and S2 includes the following sub-steps: S21: extracting features from the preprocessed point cloud data using an explicit feature extraction submodule EFE to obtain a first display structure feature; S22: extracting features from the preprocessed point cloud data using a neighborhood feature extraction submodule NFE to obtain a second display structure feature; S23: fusing the first display structure feature and the second display structure feature, and generating a final point cloud feature through a convolution layer; The S21 includes the following sub-steps: S211: Perform initial feature extraction on the preprocessed point cloud data using a one-dimensional convolutional layer; S212: using the explicit feature extraction submodule EFE to obtain the neighborhood features of each point through the neighborhood index from the point cloud data after the initial feature extraction, and concatenating the neighborhood features to generate a neighborhood feature matrix; S213: Calculate the average value of the neighborhood feature matrix in the neighborhood dimension to obtain the centroid feature; S214: expanding the centroid feature to the same shape as the neighborhood feature matrix, and connecting the centroid feature with the original neighborhood feature to obtain a new display structure feature; S215: Using a multi-layer perceptron to perform dimension reduction and feature aggregation on the new display structure feature to obtain a first display structure feature; The S22 includes the following sub-steps: S221: Perform initial feature extraction on the preprocessed point cloud data using a one-dimensional convolutional layer; S222: using the neighborhood feature extraction submodule NFE to obtain the neighborhood feature of each point through the neighborhood index from the point cloud data after the initial feature extraction, and concatenate the neighborhood features to generate a neighborhood feature matrix; S223: Using a multi-layer perceptron to perform dimension reduction and feature aggregation on the neighborhood feature matrix to obtain a second display structure feature; The S3 includes the following sub-steps: S31: using the fully connected layer to perform feature transformation on the point cloud features output by the explicit neighborhood feature extraction module ENFE; S32: Based on the feature transformation results, the self-attention mechanism in the Point Transformer module is used to calculate the relationship between the points in the point cloud data, and the relative position between each point in the point cloud data and its neighboring points is calculated; S33: Based on the calculation results, the attention weight of each point in the point cloud data is calculated using the query matrix and the key matrix; S34: Perform weighted fusion according to the attention weights to obtain weighted feature representation and realize hierarchical feature fusion; The point cloud feature difference data in S4 The calculation formula is: in, and They are the feature vectors of two points in two point cloud data respectively; In S4, a multi-layer perceptron is used to identify changes in the fused point cloud feature difference data, including the following steps: S41: Dimensionality reduction is performed on point cloud feature difference data through three fully connected layers in a multi-layer perceptron; S42: Perform softmax processing on the features of each point after the dimensionality reduction processing to obtain the classification probability of each point, and complete the change recognition of the fused point cloud feature difference data.

2. The method for detecting three-dimensional changes of UAV to ground based on neighborhood display feature attention according to claim 1 is characterized in that: In S1, point cloud data loading and preprocessing are performed, specifically: point cloud data pairs collected at different times are loaded into the model, and the point cloud data pairs are filtered and normalized.

3. The method for detecting three-dimensional changes of UAV to ground based on neighborhood display feature attention according to claim 1 is characterized in that: The relationship between points in the point cloud data is calculated in S32, specifically: a query matrix, a key matrix and a value matrix of the points in the point cloud data are calculated respectively using a linear layer.

Images