Lightweight feature smoothing group attention network for change detection in remote sensing images

Abstract

The present invention discloses a lightweight feature smoothing group attention network for remote sensing image change detection, which relates to the field of remote sensing information extraction technology. First, a group convolution channel self-attention is designed to divide the input features into small windows along the channel dimension. Second, a dual-feature fusion module is designed to filter and reconstruct multi-scale features, thereby smoothing the differences between convolution kernels and enhancing the generalization ability of the model. The present invention solves the interaction capability limitations and computational cost problems of CNN and Transformer in global-local feature extraction. In application, a remote sensing target detection technology with practical value is obtained, in order to effectively promote the application and development of target object monitoring in remote sensing imagery.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing information extraction, and in particular to a lightweight feature smoothing group attention network for remote sensing image change detection. Background Art

[0002] Change detection (CD), as an important technical tool in Earth observation, has been widely used in various fields such as post-disaster assessment, urban management, and environmental monitoring. It can accurately identify changes in land cover and environmental conditions by analyzing remote sensing imagery from different time periods. With the rapid development of optical sensors, automated CD technology has attracted increasing attention. This method can save considerable time and costs. Binary change detection (BCD) is a common and important basic method in CD. It can be divided into between-class CD and within-class CD. The former is mainly used to detect changes between all land cover features, while the latter is specifically used to detect changes related to specific tasks, such as CD construction.

[0003] In recent years, deep learning-based CD methods for VHR remote sensing imagery have gained widespread application. The emergence of convolutional neural networks (CNNs) has spurred rapid development in CD technology, leading to the proposal of a series of classic models. Due to their shared weights and local receptive field, CNNs demonstrate exceptional ability to capture rich, abstract local features. Deeper network architectures and larger convolution kernels often allow models to achieve a wider receptive field. However, this also leads to parameter redundancy, making model training more challenging. Furthermore, the increase in receptive field achieved through this approach is limited. Since the advent of the Transformer, its ability to establish global connections between features through a self-attention module has been widely applied in natural language processing and image processing. Some studies have integrated the Vision Transformer (ViT) into CD, offering new possibilities for feature extraction and addressing the inability of CNNs to establish global feature correlations. However, since the computational complexity of the self-attention mechanism grows quadratically with image size, model training places higher hardware requirements.

[0004] How to effectively extract global-local features for remote sensing image change detection tasks and improve the efficiency of changed area detection is a problem that needs to be solved urgently. In order to address these challenges, this paper proposes a lightweight feature smoothing group attention network (LAGANet). In this model, the group convolution channel self-attention (CGSA) of CNN is introduced to capture global-local features and reduce attention calculation redundancy. At the same time, the forward propagation (FFN) of CNN is used to process local information and nonlinear transformations. In addition, a dual feature fusion module (DFFM) is designed to filter and reconstruct multi-scale features. This strategy does not require many learnable parameters and can effectively smooth the differences between convolution kernels, thereby enhancing the generalization of the model. To this end, we propose a lightweight feature smoothing group attention network for remote sensing image change detection. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of the background technology, improve the accuracy of remote sensing image target detection, and achieve high-precision remote sensing target detection. Under the guidance of this idea, a lightweight feature smoothing group attention network for remote sensing image change detection is designed. The lightweight feature smoothing group attention network for remote sensing image change detection is characterized by comprising the following steps:

[0006] Step 1: Collect remote sensing image datasets;

[0007] Step 2: Model construction;

[0008] Step 3: Model implementation and training;

[0009] Step 4: Model accuracy evaluation.

[0010] Preferably, in step 1, the CDD, SECOND, LEVIR-CD, LEVIR-CD+, and WHU-CD datasets are downloaded from the Internet, and the image size is uniformly cropped to 256×256 pixels. The images are enhanced by random flipping, rotation, and scaling operations, and all patches are randomly divided into training, validation, and test sets in a 7:1:2 ratio.

[0011] Preferably, the LAGANet in step 2 is composed of a dual encoder and a dual decoder structure, the encoding part is composed of a CGSA module and a downsampling module, and a depth-separable convolution is used instead of a standard convolution block. The downsampling module is composed of multiple groups of depth-separable convolution layers, batch normalization layers, and ReLU activation function layers, which are used to extract features of different scales while minimizing computational complexity and information loss.

[0012] In the decoding stage, the decoding part is formed by integrating two sets of depth-wise separable convolutional layers, batch normalization layers, and ReLU activation function layers. In the feature fusion stage of the decoder, the features from the two time periods are first fused. Then, through the filtering and reconstruction process, the differences between the convolution kernels are smoothed and then merged with the features of the previous stage.

[0013] Preferably, the model in step 2 includes a CGSA module, a dual feature fusion module and a loss function module.

[0014] Preferably, in step 2, a multi-head attention mechanism is applied to couple lightweight and inter-channel features. The CGSA module evenly divides the input features into four attention heads in the channel dimension, ensuring that each attention head can learn different features. The specific formula is as follows:

[0015] Q, K, V = Conv Q，K，V (w(x i ))

[0016]

[0017]

[0018] Where x i represents the full input feature of the i-th block, w(x i ) represents the window segmentation process; where Q, K, and V are obtained by three 1×1 convolutions; T represents the transpose operation, and d represents the dimensionality reduction; It is a channel, namely MHSA, which divides the input window features into n groups according to the number of positive faces. Then, the self-attention calculation is performed on these n channel features respectively. Finally, the forward propagation τ is used f Perceiving and analyzing local features of cumulative attention feature maps.

[0019] Preferably, in the step 2, Characteristic of the pre-time stage, is the post-time stage characteristic, and the specific formula is as follows:

[0020] P1=|F1-F2|

[0021] P2 = Conv(SE(Cat(F1, F2)))

[0022] FF=Cat(σ(P2)×P1)+P2

[0023] Where P1 represents the feature enhancement branch, P2 represents the feature retention branch, Cat represents the feature concatenation of F1 and F2 along the channel dimension, the SE module is used to enhance the channel features of the input feature map, and 1×1 convolution is used to align the channels. In order to enhance the interaction between P1 and P2, a sigmoid activation function is applied to normalize P2. Then, the normalized P2 is multiplied by the enhanced P1 feature. Finally, the features of the two paths are fused.

[0024]

[0025] In the group normalization (GN) operation, the dynamically updated parameter γ is used to record the weight of each feature channel. By comparing the preset empirical coefficient τ, different channels are filtered out. Then, these channels are compared with the original input feature F i and weighted attention features Multiply them together to get the final high-weight and low-weight features, and finally, perform inter-kernel smoothing through channel cascade.

[0026] Preferably, in step 2, LossNet is designed to provide comprehensive supervision across different feature levels. The specific formula is as follows:

[0027]

[0028] Where, Represents binary cross entropy loss, which is a commonly used loss function in change detection tasks. At the same time, use Further optimize the model training. Specifically, freeze a VGG16 model pre-trained on ImageNet and use it as the backbone network for this loss function. By comparing the prediction results with the ground truth at multiple scales, the model's extraction of real features is improved. and They represent the features extracted by VGG16 from the predicted true value and the ground truth of each layer, respectively. The root mean square error loss function is used to calculate the difference between the predicted value and the true value of each layer, and they are accumulated as the final depth loss.

[0029] Preferably, in step 3, the model is implemented using the PyTorch framework, and the batch size, number of epochs, initial learning rate, and weight decay are set to 8, 200, 0.02, and 0.0003, respectively. The optimization algorithm used is the Nesterov momentum stochastic gradient descent method.

[0030] Preferably, in step 4, the performance of the model is evaluated using IoU, Precision, Recall, F1-score, and OverallAccuuracy indicators. The specific formula is as follows:

[0031]

[0032] Where TP, FP, FN, and TN represent the number of true positive predictions, false positive predictions, false negative predictions, and true negative predictions, respectively.

[0033] Beneficial effects

[0034] The method of the present invention solves the interaction capability limitations and computational cost problems of CNN and Transformer in global-local feature extraction. In application, it obtains a remote sensing target detection technology with practical value, in order to effectively promote the application and development of target object monitoring in remote sensing imagery. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a model architecture diagram of the present invention;

[0037] Figure 2 This is the model change detection result diagram based on the CDD dataset (Note: the white part in the figure represents TP, the black part represents TN, the dark gray part represents FP, and the light gray part represents FN). DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0039] See also Figure 1-2 The present invention provides a technical solution: a lightweight feature smoothing group attention network for remote sensing image change detection, comprising the following steps:

[0040] Step 1: Collect remote sensing image datasets

[0041] The CDD, SECOND, LEVIR-CD, LEVIR-CD+, and WHU-CD datasets were downloaded from the Internet, and the images were uniformly cropped to 256×256 pixels. In addition, random flipping, rotation, and scaling operations were used to enhance the images. All patches were randomly divided into training, validation, and test sets in a 7:1:2 ratio.

[0042] Step 2: Model Construction

[0043] LAGANet is a dual-encoder, dual-decoder architecture. The encoding part consists of a CGSA module and a downsampling module. Depthwise separable convolutions are used instead of standard convolution blocks. The downsampling module consists of multiple sets of depthwise separable convolution layers, batch normalization layers, and ReLU activation function layers to extract features at different scales while minimizing computational complexity and information loss. The decoding part is formed by integrating two sets of depthwise separable convolution layers, batch normalization layers, and ReLU activation function layers.

[0044] In the feature fusion stage of the decoder, the features from the two time periods are first fused, and then the differences between the convolution kernels are smoothed through the filtering and reconstruction process, and then merged with the features of the previous stage.

[0045] Specifically, this model includes a CGSA module, a dual feature fusion module, and a loss function module.

[0046] CGSA module: To solve the computational redundancy problem in attention calculation, the multi-head attention mechanism (MHSA) is applied to couple lightweight, inter-channel features. The CGSA module evenly divides the input features into four attention heads in the channel dimension, ensuring that each attention head can learn different features. The specific formula is as follows:

[0047] Q, K, V = Conv Q，K，V (w(x i ))

[0048]

[0049] Where x i represents the full input feature of the i-th block, w(x i ) represents the window segmentation process. Among them, Q, K, and V are obtained by three 1×1 convolutions, T represents the transposition operation, and d represents the dimension reduction. It is a channel, namely MHSA, which divides the input window features into n groups according to the number of positive faces. Then, the self-attention calculation is performed on these n channel features respectively. Finally, the forward propagation τ is used f Perceiving and analyzing local features of cumulative attention feature maps.

[0050] Dual feature fusion module: Characteristic of the pre-time stage, is the post-time stage characteristic, and the specific formula is as follows:

[0051] P1=|F1-F2|

[0052] P2 = Conv(SE(Cat(F1, F2)))

[0053] FF=Cat(σ(P2)×P1)+P2

[0054] Where P1 represents the feature enhancement branch, P2 represents the feature retention branch, Cat represents the feature concatenation of F1 and F2 along the channel dimension, the SE module is used to enhance the channel features of the input feature map, and 1×1 convolution is used to align the channels. In order to enhance the interaction between P1 and P2, a sigmoid activation function is applied to normalize P2. Then, the normalized P2 is multiplied by the enhanced P1 feature. Finally, the features of the two paths are fused.

[0055]

[0056] In the group normalization (GN) operation, the dynamically updated parameter γ is used to record the weight of each feature channel. By comparing the preset empirical coefficient τ, different channels are filtered out. Then, these channels are compared with the original input feature F i and weighted attention features Multiply them together to get the final high-weight and low-weight features, and finally, perform inter-kernel smoothing through channel cascade.

[0057] Loss function module: LossNet is designed to provide comprehensive supervision across different feature levels to better perceive the intrinsic characteristics of the changing area. The specific formula is as follows:

[0058]

[0059] Where, Represents binary cross entropy (BCE) loss, which is a commonly used loss function in change detection tasks. At the same time, use Further optimize the model training, specifically, freeze a VGG16 model pre-trained on ImageNet and use it as the backbone network for this loss function. Improve the model's extraction of real features by comparing the prediction results with the ground truth at multiple scales. and They represent the features extracted by VGG16 from the predicted true value and the ground truth of each layer, respectively. The root mean square error loss function is used to calculate the difference between the predicted value and the true value of each layer, and they are accumulated as the final depth loss.

[0060] Step 3: Model implementation and training

[0061] The model is implemented using the PyTorch framework, with the batch size, number of epochs, initial learning rate, and weight decay set to 8, 200, 0.02, and 0.0003, respectively. The optimization algorithm used is the Nesterov momentum stochastic gradient descent (SGD).

[0062] Step 4: Model Accuracy Evaluation

[0063] The performance of the model is evaluated using indicators such as IoU, Precision, Recall, F1-score, and Overall Accuuracy. The specific formula is as follows:

[0064]

[0065] Where TP, FP, FN, and TN represent the number of true positive predictions, false positive predictions, false negative predictions, and true negative predictions, respectively;

[0066] Specifically, the CDD dataset is used according to the above steps to provide an implementation case. The experiment was conducted on a Windows 10 operating system using an Intel Core i7-13700F CPU and an NVIDIA GeForce GTX 4060Ti GPU (16GB memory). At the same time, FC-EF, FC-EF-Cat, FC-EF-diff, SNUNet, AFCF3DNet, Changeformer, BiT, and ICIFNet models were selected for comparative analysis to verify the effectiveness and superiority of this method. As shown in Table 1 and Figure 2 As shown in the figure, LAGANet's F1-score and IoU are 3.25% and 5.97% higher than the latest AFCF3DNet model, respectively. This shows that LAGANet performs very well in extracting the boundaries of changed regions, detecting changes in changed regions, and avoiding false detections in unchanged regions, and is able to skillfully extract low-frequency features from images.

[0067] Table 1 Comparison of results of different model methods

[0068]

[0069] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0070] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A lightweight feature smoothing group attention network for remote sensing image change detection, characterized by: The following steps are involved: Step 1: Collect remote sensing image datasets; Step 2: Model construction; Step 3: Model implementation and training; Step 4: Model accuracy evaluation; In step 1, the CDD, SECOND, LEVIR-CD, LEVIR-CD+, and WHU-CD datasets are downloaded from the Internet, and the images are uniformly cropped to 256×256 pixels. Random flipping, rotation, and scaling operations are used to enhance the images. All patches are randomly divided into training, validation, and test sets in a 7:1:2 ratio. The LAGANet in step 2 consists of a dual encoder and dual decoder structure. The encoding part consists of a CGSA module and a downsampling module. Depthwise separable convolution is used instead of a standard convolution block. The downsampling module consists of multiple groups of depthwise separable convolution layers, batch normalization layers, and ReLU activation function layers to extract features of different scales while minimizing computational complexity and information loss. In the decoding stage, the decoding part is formed by integrating two sets of depth-separable convolutional layers, batch normalization layers and ReLU activation function layers. In the feature fusion stage of the decoder, the features from the two time periods are first fused. Then, through the filtering and reconstruction process, the differences between the convolution kernels are smoothed and then merged with the features of the previous stage. The model in step 2 includes a CGSA module, a dual feature fusion module and a loss function module. In step 2, a multi-head attention mechanism is applied to couple lightweight and inter-channel features. The CGSA module evenly divides the input features into four attention heads in the channel dimension to ensure that each attention head can learn different features. The specific formula is as follows: Q, K, V = Conv Q，K，V (w(x i )) Where x i represents the full input feature of the i-th block, w(x i ) represents the window segmentation process; where Q, K, and V are obtained by three 1×1 convolutions; T represents the transpose operation, and d represents the dimensionality reduction; It is a channel, that is, the multi-head attention mechanism MHSA is applied to divide the input window features into n groups according to the number of positive faces, and then the self-attention calculation is performed on these n channel features respectively. Finally, the forward propagation τ is used f Perceiving and analyzing local features of cumulative attention feature maps.

2. The lightweight feature smoothing group attention network for remote sensing image change detection according to claim 1 is characterized in that: In the step 2, Characteristic of the pre-time stage, is the post-time stage characteristic, and the specific formula is as follows: P1=|F1-F2| P2=Conv(SE(Cat(F1,F2))) FF=Cat(σ(P2)×P1)+P2 Where P1 represents the feature enhancement branch, P2 represents the feature retention branch, Cat represents the feature concatenation of F1 and F2 along the channel dimension, the SE module is used to enhance the channel features of the input feature map, and 1×1 convolution is used to align the channels. In order to enhance the interaction between P1 and P2, a sigmoid activation function is applied to normalize P2. Then, the normalized P2 is multiplied by the enhanced P1 feature. Finally, the features of the two paths are fused. In the group normalization (GN) operation, the dynamically updated parameter γ is used to record the weight of each feature channel. By comparing the preset empirical coefficient τ, different channels are filtered out. Then, these channels are compared with the original input feature F i and weighted attention features Multiply them together to get the final high-weight and low-weight features, and finally, perform inter-kernel smoothing through channel cascade.

3. The lightweight feature smoothing group attention network for remote sensing image change detection according to claim 2 is characterized in that In step 2, LossNet is designed to provide comprehensive supervision across different feature levels. The specific formula is as follows: Where, Represents binary cross entropy loss, which is a commonly used loss function in change detection tasks. At the same time, use Further optimize the model training. Specifically, freeze a VGG16 model pre-trained on ImageNet and use it as the backbone network for this loss function. By comparing the prediction results with the ground truth at multiple scales, the model's extraction of real features is improved. and They represent the features extracted by VGG16 from the predicted true value and the ground truth of each layer, respectively. The root mean square error loss function is used to calculate the difference between the predicted value and the true value of each layer, and they are accumulated as the final depth loss.

4. The lightweight feature smoothing group attention network for remote sensing image change detection according to claim 3 is characterized in that: In step 3, the model is implemented using the PyTorch framework, and the batch size, number of epochs, initial learning rate, and weight decay are set to 8, 200, 0.02, and 0.0003, respectively. The optimization algorithm used is the Nesterov momentum stochastic gradient descent method.

5. The lightweight feature smoothing group attention network for remote sensing image change detection according to claim 4 is characterized in that: In step 4, the performance of the model is evaluated using the IoU, Precision, Recall, F1-score, and OverallAccuuracy indicators. The specific formulas are as follows: Where TP, FP, FN, and TN represent the number of true positive predictions, false positive predictions, false negative predictions, and true negative predictions, respectively.

Images