A remote sensing change detection method fusing lightweight backbone network and convolutional decoding

Abstract

The present application relates to a kind of fusion lightweight backbone network and remote sensing change detection method of convolution decoding, including using backbone network feature extraction module, for extracting multi-scale features with strong representation ability from input dual-phase remote sensing image;CNN-Decode feature alignment module, using convolutional coding structure is spatially aligned to dual-phase feature, avoid the feature confusion problem of traditional attention mechanism;And change detection module, based on the feature after alignment realizes pixel-level change identification.The present application is optimized by double-path cooperation, while enhancing the feature expression ability of model, it enhances spatial alignment accuracy, effectively improves the integrity and boundary accuracy of change detection under complex scene, under the premise of keeping low computational complexity, significantly improves detection performance, can be widely applied in urban planning, disaster assessment and environmental monitoring and other fields.

Description

Technical Field

[0001] This invention relates to the field of computer vision remote sensing change detection technology, and in particular to a remote sensing change detection method that integrates a lightweight backbone network and convolutional decoding. Background Technology

[0002] Remote sensing change detection refers to the technology of automatically identifying and quantifying land cover and land use changes by analyzing and processing remote sensing images of the same area acquired at different times. With the rapid development of Earth observation technology, remote sensing change detection has played an important role in areas such as urban expansion monitoring, natural disaster assessment, crop growth analysis, forest resource surveys, and environmental change monitoring. Although traditional change detection methods are computationally simple, they heavily rely on manually designed features, require high image registration accuracy and radiometric correction quality, and struggle to effectively distinguish between real changes and spurious changes caused by factors such as illumination, season, and sensor angle.

[0003] In recent years, the rise of deep learning technology, especially convolutional neural networks (CNNs), has brought revolutionary breakthroughs to remote sensing change detection. CNN-based methods can automatically learn discriminative deep features, significantly improving robustness against false changes and the accuracy of change detection. However, these methods still have significant limitations: the receptive field of CNNs is limited, making it difficult to effectively model multi-scale targets and long-range spatial dependencies commonly found in remote sensing imagery.

[0004] To overcome these challenges, researchers have begun to introduce the Transformer architecture, which possesses global modeling capabilities, into the field of change detection. Vision Transformer and its variants effectively capture global contextual information through self-attention mechanisms, demonstrating significant potential for improving detection accuracy. However, the application of Transformers in remote sensing change detection faces two key issues: first, the computational complexity of the self-attention mechanism is quadratic with the input image size, leading to enormous computational overhead; second, directly fusing attentional features from two temporal phases can easily cause feature confusion between different land cover categories, affecting the accurate localization of change boundaries. Therefore, how to maintain detection accuracy while achieving efficient model operation and enhancing robustness to spatiotemporal inconsistencies has become a critical technical challenge that urgently needs to be addressed in the field of remote sensing change detection.

[0005] Against this backdrop, this invention proposes a remote sensing change detection method that integrates a lightweight backbone network and convolutional decoding. By constructing a lightweight grouped attention network and an efficient convolutional attention decoding module, the method significantly reduces computational complexity while ensuring global and local feature extraction capabilities, and effectively alleviates feature confusion problems, providing an innovative solution for high-precision remote sensing change detection in resource-constrained environments. Summary of the Invention

[0006] The purpose of this invention is to provide a remote sensing change detection method that integrates a lightweight backbone network and convolutional decoding to solve the problems existing in the prior art, improve the model's ability to extract global and local features, and significantly reduce computational complexity.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A remote sensing change detection method integrating a lightweight backbone network and convolutional decoding includes:

[0009] A lightweight module for remote sensing image feature extraction, used to extract multi-scale features from input bi-temporal images;

[0010] The convolutional attention decoding module takes the aforementioned multi-scale feature pyramid as input, fuses dual-temporal features through a convolution-enhanced attention mechanism, and outputs a set of change-robust category representations.

[0011] The change map prediction module uses the robust category representation to activate and compare the dual-temporal features extracted by the backbone network, and refines the details of the prediction map to obtain a more accurate prediction map, and finally outputs a pixel-level change detection map.

[0012] Preferably, the lightweight grouped attention backbone network includes:

[0013] The feature extraction structure consists of three stages, each containing multiple lightweight grouped attention modules.

[0014] Each LWGA module contains sub-modules for gated attention, regular local attention, sparse mid-range attention, and sparse global attention.

[0015] By using grouped feature processing and multi-scale attention mechanisms, efficient feature extraction of multi-scale targets in remote sensing images is achieved.

[0016] The feature outputs of each module are then combined to form a feature list, which serves as the input to the convolutional attention decoding module.

[0017] Preferably, the convolutional attention decoding module is constructed by:

[0018] The multi-scale bi-temporal feature maps extracted from the backbone network are added to the corresponding location codes, then flattened in the spatial dimension, and the two flattened features are spliced ​​together to form a multi-scale memory sequence.

[0019] A set of learnable change query embeddings is used as the initial input to the decoder, allowing the model to automatically summarize 'standard templates' for several core land features.

[0020] It consists of six identical convolutional decoding layers stacked together. Each decoding layer uses one-dimensional convolutional operations to process query, key, and value features, and uses feature maps of different scales in a loop between different layers to fuse multi-scale contextual information and enhance feature representation.

[0021] Preferably, the attention feature matrix is ​​constructed, including:

[0022] Qurey convolutional layer: contains a one-dimensional convolutional layer used to perform convolutional enhancement on the input learnable embedding and extract local dependencies;

[0023] Key-value convolutional layer: contains a one-dimensional convolutional layer, which is used to convolve and enhance the input memory sequence, and divides its output feature channels into two parts, which serve as the key and the value respectively.

[0024] Attention Calculation: This is used to calculate the dot product attention weight between the convolutionally enhanced value and the key, and then sum this weight with the convolutionally enhanced value to obtain the attention context vector.

[0025] Feature fusion layer: contains a one-dimensional convolutional layer, used to concatenate and fuse the output of the query convolutional layer with the context vector output by the attention calculation unit.

[0026] Feedforward Network (FFN) employs a depthwise separable convolutional structure;

[0027] Layer normalization and dropout layers are used to stabilize the training process.

[0028] Preferably, constructing the global feature attention weights includes:

[0029] S5.1: First, the learnable embedding and positional encoding of the input are added together and then convolved through the corresponding query convolutional layer to obtain the convolutionally enhanced query vector Q. The processed bi-temporal features of the input are then convolved through a one-dimensional key-value convolutional kernel, and the output features are divided into two equal parts in the channel dimension, which are used as the convolutionally enhanced key features and value features, respectively. The formula is as follows:

[0030]

[0031]

[0032]

[0033] in, It is a learnable embedding. It is a positional encoding. These are the characteristics of the input.

[0034] S5.2: Transpose the Q-features after convolution enhancement and multiply them with the K-features after convolution enhancement to obtain an initial attention score matrix; divide the attention score matrix by the square root of the key feature channel dimension and perform a scaling operation to stabilize the training process. The formula is as follows:

[0035]

[0036] in, Let K be the size of the feature dimension.

[0037] S5.3: Apply the Softmax function along the last dimension (i.e., the dimension of the memory feature sequence) of the scaled attention score matrix to obtain the normalized attention weight matrix. Each element in this matrix represents the correlation strength between a query and a key.

[0038] S5.6: Perform matrix multiplication between the normalized attention weight matrix and the transpose of the value after convolution enhancement, and calculate the weighted summation output, which is the context vector that incorporates global memory features.

[0039] Preferably, constructing the feature fusion layer includes:

[0040] S6.1: Align the dimensions and concatenate the channels of the query feature matrix enhanced by convolution with the context feature matrix output by global attention; specifically, the context feature matrix... Perform a dimension transpose to obtain

[0041]

[0042] S6.2: The concatenated feature matrix The input is fed into a one-dimensional convolutional layer for feature fusion and dimensionality reduction. The kernel size of this convolution is [size missing]. Fill with Number of output channels This generates a fused feature matrix with enhanced features. :

[0043]

[0044] in, Indicates the kernel size as Fill as One-dimensional convolution operation.

[0045] S6.3: The fused feature matrix Perform a dimension transpose to restore the sequence representation. This is to facilitate subsequent residual join and normalization processing.

[0046]

[0047] The beneficial effects of this invention are as follows:

[0048] By introducing a lightweight grouped attention mechanism and a convolution-enhanced query feature generation method, this invention effectively addresses the shortcomings of traditional change detection methods in recognizing multi-scale targets and subtle changes in complex scenarios. Simultaneously, the convolutional attention decoding module models the local context of the query features through one-dimensional convolutional layers, establishing spatial dependencies between query tokens in the sequence dimension. This ensures that the generated query vector not only contains semantic information but also incorporates structured spatial features, significantly enhancing the discriminative power of the feature representation. This method improves model performance while reducing the number of parameters. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of the remote sensing change detection method integrating a lightweight backbone network and convolutional decoding according to an embodiment of the present invention.

[0051] Figure 2 This invention relates to a remote sensing change detection model that integrates a lightweight backbone network and convolutional decoding, specifically the convolutional attention decoding module. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] like Figure 1 , 2 As shown, this invention provides a remote sensing change detection method that integrates a lightweight backbone network and convolutional decoding, specifically including:

[0055] First, LWGANet is used for feature extraction, extracting three-scale features. , , (t=0,1 represents two time points) A feature pyramid is formed. Input features are uniformly converted to the same output dimension through successive 1×1 convolutions, then bilinear interpolation is used for upsampling, and finally 3×3 convolutions and batch normalization are used to eliminate the aliasing effect caused by upsampling. Specifically, the deepest layer features are first convolved with 1×1 convolutions. The convolution process yields p4, which is then upsampled via bilinear interpolation and fused with mid-level features to obtain p3. p3 is then upsampled again and fused with shallow features to obtain p2. Finally, three fused features at different resolutions—p4, p3, and p2—are output. This design propagates deep semantic information step-by-step to shallow layers, enhancing feature representation capabilities and making it particularly suitable for computer vision tasks requiring multi-scale perception.

[0056] The fused features are used as input to the convolutional attention decoding module, and the learnable embeddings are also used as input. The learnable embeddings are a set of trainable parameter matrices with dimensions of [missing information]. Where N is the number of learnable embeddings and C represents the number of feature channels. In the feature and processing stage of this module, the dual-temporal multi-scale feature maps extracted by the lightweight grouped attention backbone network are flattened along the spatial dimension into feature sequences: for the left-temporal feature tensor of each scale... and right phase characteristic tensor Transform it into a dimension of by flattening. The sequence is then used to construct the feature sequence from the two time phases; subsequently, the feature sequence dimensions of the two time phases are concatenated to form a memory feature matrix that fuses the information from both time phases. To preserve the spatial location information of the features, the location encoding module generates a location encoding matrix that matches the dimensions of the memorized features. Positional information is injected into the feature matrix through element-wise addition, ultimately yielding position-enhanced memory features. This processing method fully preserves the temporal characteristics of the dual-phase data through feature concatenation, while maintaining spatial structure information through positional encoding. This provides a feature representation that combines temporal contrast and spatial structure for the subsequent attention mechanism. Similarly, positional encoding is also added to the input learnable vector.

[0057] Enhanced bi-temporal memory features The input is fed into a key-value convolutional layer, which generates joint features through a one-dimensional convolutional layer with 2C output channels, and then equally segments the features into key features along the channel dimension. Sum value characteristics ,in To remember the sequence length, the learnable input is embedded into the input query convolutional layer to generate an enhanced query vector with spatial structure features.

[0058]

[0059]

[0060]

[0061] The enhanced query features are transposed and then multiplied with the key feature K to generate the original attention score matrix. This is then divided by... The standardized attention score is obtained by scaling. The formula is as follows:

[0062]

[0063] in, Let K be the size of the feature dimension.

[0064] The attention weight matrix is ​​generated by applying the Softmax function to the scaled attention scores. We then perform a weighted summation with the value feature V to obtain the context feature that incorporates global information. The contextual feature C and the enhanced query feature are concatenated through channels. Feature compression and information integration are performed through a fusion convolutional layer. The output result is then connected to the initial query vector via residuals and normalized to complete the first feature update.

[0065]

[0066] in, Indicates the kernel size as Fill as One-dimensional convolution operation.

[0067] The updated features are input into a feedforward network composed of depthwise separable convolutions. A nonlinear transformation is performed using a cascaded combination of 3×3 depthwise convolutions and 1×1 pointwise convolutions. This is followed by a second residual connection and layer normalization to complete single-layer decoding. Steps 1-5 are repeated, cyclically using memory features of different scales during the L-layer decoding process to achieve progressive fusion of multi-scale contextual information. Finally, a refined change-aware feature vector is output.

[0068] The feature vector output from the decoder is input into the change map prediction module. A preliminary change map is generated by class activation mapping and bi-temporal comparison. Finally, the detail enhancement module performs boundary optimization using the original resolution image and outputs the final change detection result.

[0069] This invention designs a convolutional attention decoding module that achieves efficient feature decoding through a one-dimensional convolution-enhanced attention mechanism. This module concatenates bi-temporal features and injects positional information, utilizes convolution-enhanced query-key-value interaction to capture feature dependencies, and combines a multi-scale fusion strategy to improve feature representation quality.

[0070] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A remote sensing change detection method integrating a lightweight backbone network and convolutional decoding, characterized in that, include: A lightweight feature extraction module for remote sensing images, used to extract multi-scale features from input bi-temporal images; The convolutional attention decoding module takes the aforementioned multi-scale feature pyramid as input, fuses dual-temporal features through a convolution-enhanced attention mechanism, and outputs a set of change-robust category representations. The change map prediction module uses the robust category representation to activate and compare the dual-temporal features extracted by the backbone network, and refines the details of the prediction map to obtain a more accurate prediction map, and finally outputs a pixel-level change detection map.

2. A remote sensing change detection method integrating a lightweight backbone network and convolutional decoding, characterized in that... The lightweight grouped attention backbone network includes: The feature extraction structure consists of three stages, each containing multiple lightweight grouped attention modules. Each LWGA module contains sub-modules for gated attention, regular local attention, sparse mid-range attention, and sparse global attention. By using grouped feature processing and multi-scale attention mechanisms, efficient feature extraction of multi-scale targets in remote sensing images is achieved. The feature outputs of each module are then combined to form a feature list, which serves as the input to the convolutional attention decoding module.

3. The remote sensing change detection method integrating lightweight backbone network and convolutional decoding according to claim 1, characterized in that, The convolutional attention decoding module includes: The multi-scale bi-temporal feature maps extracted from the backbone network are added to the corresponding location codes, then flattened in the spatial dimension, and the two flattened features are spliced ​​together to form a multi-scale memory sequence. A set of learnable change query embeddings is used as the initial input to the decoder, allowing the model to automatically summarize 'standard templates' for several core land features. It consists of six identical convolutional decoding layers stacked together. Each decoding layer uses one-dimensional convolutional operations to process query, key, and value features, and uses feature maps of different scales in a loop between different layers to fuse multi-scale contextual information and enhance feature representation.

4. The convolutional attention decoding module of the remote sensing change detection method integrating lightweight backbone network and convolutional decoding according to claim 3, characterized in that, The convolutional decoding layer includes: Qurey convolutional layer: contains a one-dimensional convolutional layer used to perform convolutional enhancement on the input learnable embedding and extract local dependencies; Key-value convolutional layer: contains a one-dimensional convolutional layer, which is used to convolve and enhance the input memory sequence, and divides its output feature channels into two parts, which serve as the key and the value respectively. Attention Calculation: This is used to calculate the dot product attention weight between the convolutionally enhanced value and the key, and then sum this weight with the convolutionally enhanced value to obtain the attention context vector. Feature fusion layer: contains a one-dimensional convolutional layer, used to concatenate and fuse the output of the query convolutional layer with the context vector output by the attention calculation unit. Feedforward Network (FFN) employs a depthwise separable convolutional structure; Layer normalization and dropout layers are used to stabilize the training process.

5. The convolutional decoding layer of the remote sensing change detection method integrating a lightweight backbone network and convolutional decoding according to claim 4, characterized in that, The specific method for constructing global feature attention weights is as follows: S5.1: First, the learnable embedding and positional encoding of the input are added together and then convolved through the corresponding query convolutional layer to obtain the convolutionally enhanced query vector Q. The processed bi-temporal features of the input are then convolved through a one-dimensional key-value convolutional kernel, and the output features are divided into two equal parts in the channel dimension, which are used as the convolutionally enhanced key features and value features, respectively. The formula is as follows: in, It is a learnable embedding. It is a positional encoding. These are the characteristics of the input. S5.2: Transpose the Q-features after convolution enhancement and multiply them with the K-features after convolution enhancement to obtain an initial attention score matrix; divide the attention score matrix by the square root of the key feature channel dimension and perform a scaling operation to stabilize the training process. The formula is as follows: in, Let K be the size of the feature dimension. S5.3: Apply the Softmax function along the last dimension (i.e., the dimension of the memory feature sequence) of the scaled attention score matrix to obtain the normalized attention weight matrix. Each element in this matrix represents the correlation strength between a query and a key. S5.6: Perform matrix multiplication between the normalized attention weight matrix and the transpose of the value after convolution enhancement, and calculate the weighted summation output, which is the context vector that incorporates global memory features.

6. The feature fusion layer of the remote sensing change detection method integrating a lightweight backbone network and convolutional decoding as described in claim 4, characterized in that, The specific method for constructing the video feature matrix after feature enhancement is as follows: S6.1: Align the dimensions and concatenate the channels of the query feature matrix enhanced by convolution with the context feature matrix output by global attention; specifically, the context feature matrix... Perform a dimension transpose to obtain S6.2: The concatenated feature matrix The input is fed into a one-dimensional convolutional layer for feature fusion and dimensionality reduction. The kernel size of this convolution is [size missing]. Fill with Number of output channels This generates a fused feature matrix with enhanced features. : in, Indicates the kernel size as Fill as One-dimensional convolution operation. S6.3: The fused feature matrix Perform a dimension transpose to restore the sequence representation. This is to facilitate subsequent residual join and normalization processing.

7. The change map prediction module of the remote sensing change detection method integrating lightweight backbone network and convolutional decoding according to claim 1, comprising: The feature activation unit is used to perform matrix multiplication between the class representation output by the spatiotemporal robust representation generation module and the bi-temporal high-level features extracted by the backbone network to generate a class activation map of the bi-temporal image. The feature fusion unit is used to concatenate the dual-temporal category activation maps through channels and perform feature fusion through a convolutional layer. The change discrimination unit is used to generate the final change result based on the fused features through the ArgMax operation.

Images