A multi-scale feature fusion change detection method based on feature enhancement

Abstract

The application provides a multi-scale feature fusion change detection method based on feature enhancement, mainly improves and designs a multi-scale feature enhancement network MSE-Net for remote sensing image change detection, integrates a DAFA module, an MS-CFEM module and a BFA module, realizes deep fusion of high-level semantic information and low-level fine-grained features through multi-level feature extraction and bidirectional feature interaction mechanism in an encoding stage. The framework can enhance the global discrimination ability while maintaining the local detail expression ability, obtain more robust multi-scale feature representation, and effectively improve the recognition ability of different scale change regions in the change detection task.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image change detection technology, and in particular relates to a multi-scale feature fusion change detection method based on feature enhancement. Background Technology

[0002] With the rise of big data and artificial intelligence, most current research on remote sensing change detection based on deep learning is based on convolutional neural networks or Transformers.

[0003] In convolutional neural network (CNN)-based methods, many classic neural network architectures have been utilized. For example, Qin et al. innovatively improved the CNN architecture, significantly reducing the number of network parameters and computational cost while enhancing classification accuracy. Shi et al. proposed the DSAMNet metric network with deep supervised attention, which enhances the discriminative power of change features by integrating an attention mechanism with a deep supervision module, effectively suppressing noise and spurious change interference. Esfandiari et al. combined the Mask R-CNN algorithm with high-resolution satellite imagery to automatically detect the demolition and reconstruction of urban buildings, conducting experiments on the New Brunswick dataset in Canada, providing an effective solution for change detection in multi-source remote sensing data. Gong et al. proposed an end-to-end change detection method for SAR images based on deep learning, directly generating change maps by fusing unsupervised feature learning and supervised fine-tuning of a deep network, effectively avoiding interference from traditional differential images. Ronneberger et al. proposed a data-augmented, high-efficiency U-Net architecture, achieving end-to-end training through a shrink-expand symmetric path, significantly improving the accuracy of microscopic image segmentation and cell tracking with limited labeled data, setting a new performance record in the ISBI Challenge with a processing speed of less than one second. Following this, Zhang Cuijun et al. combined an improved U-Net model with asymmetric convolutional blocks and attention mechanisms. By enhancing feature robustness to suppress overfitting and focusing on changing regions to improve small target detection capabilities, they significantly improved the F1 score in remote sensing building change detection tasks, effectively solving the problem of missed detections in complex backgrounds. Chang Zhenliang et al.'s improved remote sensing image change detection method based on DeepLabv3+ optimized edge segmentation accuracy through multi-receptive field pooling pyramids, multi-scale fusion of decoders, and lightweight network design, significantly improving detail feature preservation and computational efficiency.

[0004] Besides convolutional neural networks, Transformers are also a mainstream method for change detection in remote sensing images. Kaur et al. proposed the DAHiTrA network based on hierarchical Transformers, which can classify building damage in satellite images after hurricanes by fusing multi-resolution spatial features and temporal differences. Zhang et al. proposed a twin U-shaped pure Transformer network based on Swin Transformers, which uses Swin blocks to construct the encoder, fusion module, and decoder, overcoming the limitation of traditional CNNs in capturing global spatiotemporal information, and enabling efficient fusion of global context modeling and multi-scale features, thus improving detection accuracy. Bandara et al. proposed a Transformer-based twin network—Changeformer, which combines a hierarchical Transformer encoder with an MLP decoder to capture multi-scale long-range details to improve the accuracy of remote sensing image change detection. Feng Weiming et al. also fused twin structures, skip connections, and Transformers, and obtained global contextual information of remote sensing images by hybridizing CNN-Transformers encoders, achieving significant accuracy improvements on multiple datasets.

[0005] Currently, high spatial and temporal resolution remote sensing images provide a wealth of data sources for monitoring land surface changes, and in recent years, deep convolutional neural networks have made significant progress in remote sensing image change detection tasks due to their powerful feature representation capabilities. However, due to the complexity of remote sensing image scenes and significant differences in target scales, existing methods still have shortcomings in multi-scale difference feature representation and dual-temporal feature fusion, making it difficult to simultaneously take into account global semantic representation and local detail characterization.

[0006] Therefore, there is an urgent need to design a new method for detecting changes in remote sensing images, so as to improve the model's ability to distinguish changed areas in remote sensing images and achieve more accurate and complete change detection. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] To address the aforementioned problems, this invention proposes a multi-scale feature fusion change detection method based on feature enhancement. By constructing a multi-scale feature enhancement and cross-scale fusion mechanism, the method enhances the differential expression of dual-temporal features and the multi-scale feature fusion capability, thereby improving the model's ability to discriminate changed regions and achieving more accurate and complete change detection.

[0009] (II) Technical Solution

[0010] This invention discloses a multi-scale feature fusion change detection method based on feature enhancement, characterized by the following steps:

[0011] Step 1: Obtain a dual-temporal remote sensing image dataset, wherein the dual-temporal remote sensing images include a first temporal image T1 and a second temporal image T2 acquired at different time points in the same area;

[0012] Step 2: Construct the multi-scale feature enhancement change detection network MSE-Net, which includes: VGG16-BN backbone network, differentially guided attention and flow alignment module DAFA, multi-scale contextual feature enhancement module MS-CFEM, bidirectional feature fusion module BFA, and classifier.

[0013] Step 3: Input the dual temporal images T1 and T2 into the VGG16-BN backbone network. The VGG16-BN backbone network contains five sequentially connected blocks. Each block contains 2 to 3 convolutional layers and 1 max pooling layer. The five blocks output features at five different levels.

[0014] Step 4: Input the five-layer dual-temporal features from the first to the fifth layer of VGG16-BN into five independent DAFA modules in sequence for difference-guided feature alignment and enhancement to obtain the aligned and enhanced multi-scale features. The multi-scale features output by the five DAFA modules are denoted as feature nodes E, D, C, B, and A, respectively.

[0015] Step 5: Input the aligned and enhanced high-level feature nodes A, B, and C into three independent MS-CFEM modules for multi-scale contextual feature enhancement to obtain the enhanced high-level features, which are denoted as feature nodes A', B', and C' respectively.

[0016] Step 6: Input the enhanced high-level feature nodes A', B', and C' into the BFA module, perform top-down fusion, and generate global weight W;

[0017] Step 7: Multiply the global weight W element-wise with the low-level feature nodes C', D, and E respectively; input the weighted low-level feature nodes into three independent MS-CFEM modules for feature enhancement to obtain the enhanced low-level features, denoted as feature nodes C'', D', and E'.

[0018] Step 8: Input the enhanced shallow feature nodes C'', D', and E' into the BFA module and perform bottom-up fusion. Then, input the fused output features into the convolution module and the classifier in sequence to adjust the output features to the same spatial resolution as the input image and obtain the predicted output.

[0019] Preferably, when the BFA module performs top-down fusion in step 6, it specifically includes:

[0020] Step 6.1: After upsampling, the spatial size of node A' is doubled. The features of node A' after convolution are added element by element and then activated by ReLU to obtain the first fusion result.

[0021] Step 6.2: Further upsample the fusion result from Step 6.1, and add it element-wise to the three identical-sized features of node A' and node C' after convolution transformation following the two upsampling steps in Step 6.1. After ReLU activation, the second fusion result is obtained.

[0022] Step 6.3: The second fusion result is subjected to downsampling and convolution operations in sequence to generate global weights W.

[0023] Preferably, when the BFA module performs bottom-up fusion in step 8, it specifically includes:

[0024] Step 8.1: After downsampling, the spatial size of node E' is reduced by half. The features of node D' after convolution are added element by element, and after ReLU activation, the first fusion result is obtained.

[0025] Step 8.2: Further downsample the first fusion result from Step 8.1, and add it element-wise to the three identical features of node E' and node C'' after convolution transformation in Step 8.1 after two downsampling steps. After ReLU activation, the second fusion result is obtained.

[0026] Step 8.3: The second fusion result is subjected to upsampling and convolution operations in sequence, and finally used as the fused output feature.

[0027] Preferably, the difference-guided attention and flow alignment module consists of two parts: a difference-guided attention enhancement part and a flow bidirectional alignment fusion module. The difference-guided attention enhancement part includes a channel attention module and a spatial attention module connected in sequence. Input features X1 and X2 pass through the channel attention module and then enter the spatial attention module, finally outputting features. and ;

[0028] The channel attention module performs the following operations: first, it performs global max pooling and global average pooling on the input features in parallel to extract global statistical information of the feature map along the channel dimension; then, it inputs the two pooling results into the two layers respectively. Convolution is used to generate channel descriptions, and in two layers A ReLU activation function is introduced in the middle of the convolution to enhance non-linear expressive power, thereby strengthening the ability to model differences between different channels. Finally, the two layers... The parallel output of the convolution is summed element by element and then fed into the Sigmoid activation function. The Sigmoid activation function generates normalized channel weights, which are then applied to the original features channel by channel to highlight the response to changes in key channels.

[0029] Preferably, the spatial attention module performs the following operations: the input features are first subjected to max pooling and average pooling in parallel along the channel dimension; then the two results are fused along the channel dimension, and then... Convolutional layers model spatial information, and a batch normalization layer is introduced after convolution for normalization. Finally, the outputs of the two normalization layers are concatenated and input into the sigmoid activation function. The sigmoid activation function generates spatial attention weights, which are applied pixel-by-pixel to the feature map, guiding the network to pay more attention to potentially changing regions.

[0030] Preferably, after the aforementioned attention enhancement is completed, the two-phase features output by the spatial attention module are sent to the flow bidirectional alignment and fusion module, where the following operations are performed:

[0031] Features after the aforementioned difference enhancement and The components are spliced ​​along the channel dimension, as shown in equation (1).

[0032]

[0033] in Indicates a stream generator The predicted flow field is generated by a flow generator that sequentially includes depthwise separable convolution (Dw Conv), normalized IN, GELU activation function, and pointwise convolution (PW Conv).

[0034] Next, the input features are remapped using a Warp operation to obtain the corrected aligned features, as shown in Equation (2).

[0035]

[0036] in, and They represent from and The offset of each displacement field, the remapping Warp operation interpolates and samples the features in a continuous space;

[0037] Subsequently, the feature after the Warp operation is subtracted from the original feature of another time phase to obtain explicit change information;

[0038]

[0039] The difference features are concatenated by equation (3.3), and the resulting x will be fed into the subsequent network after fusion.

[0040] Preferably, the multi-scale context feature enhancement module includes three parallel convolutional branches, each composed of a CBR structure of different sizes. The CBR structure consists of a convolutional Conv, a batch normalized BN, and a ReLU function connected in sequence. One branch uses three depthwise separable convolutions with kernels of (1,1) / (3,3) / (1,1), and the other two branches use four asymmetric convolutions with kernels of (1,3) / (3,1) / (1,5) / (5,1). The three branches are concatenated in the channel dimension to form multi-scale local features.

[0041] Simultaneously, a global context branch is introduced. This branch generates a global description through average pooling, flattening, and decompression operations. The input features, after passing through the global context branch, are concatenated with the multi-scale local features. The concatenated and fused features are then input into a ReLU activation function, and subsequently... The residual connections of convolution are combined with input features to achieve the integration of local details and global structure.

[0042] In another aspect, the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the detection method described in any of the preceding claims.

[0043] (III) Beneficial Effects

[0044] (1) This invention constructs a multi-scale feature enhancement network MSE-Net, which integrates the DAFA module, MS-CFEM module, and BFA module. Through multi-level feature extraction and bidirectional feature interaction mechanism in the encoding stage, it achieves deep fusion of high-level semantic information and low-level fine-grained features. This framework can enhance global discrimination ability while maintaining the ability to express local details, and obtain more robust multi-scale feature representations, thereby effectively improving the ability to identify regions with changes at different scales in change detection tasks.

[0045] (2) The difference-guided attention and flow alignment module of this invention enhances the response of change-related regions at the feature level and suppresses irrelevant background interference through the synergistic effect of channel attention and spatial attention. On this basis, a flow bidirectional alignment fusion mechanism is introduced to effectively alleviate the feature space offset problem caused by factors such as illumination and viewing angle in dual-temporal images. This module achieves a good balance between preserving change information and suppressing false detections, significantly improving the accuracy and robustness of change detection. The multi-scale context feature enhancement module adopts a strategy that combines parallel multi-scale local convolution branches with global context branches, which can fully explore multi-scale spatial features under limited annotation conditions and improve the quality of feature representation. Through the effective fusion of local details and global structure, this module enhances the model's ability to distinguish change regions in complex terrain structures and scale-changing scenarios, and improves the problem of insufficient feature representation. The bidirectional feature fusion module constructs a multi-stage cross-scale feature interaction mechanism through the synergistic fusion of two sub-modules, one from top to bottom and one from bottom to top. Top-down fusion utilizes high-level semantic information to generate global weights and adaptively enhances shallow features, thus effectively obtaining global weights. Bottom-up fusion aggregates the enhanced shallow details level by level, forming fused features that combine semantic consistency and spatial accuracy. This module effectively solves the problems of inconsistency in multi-scale features and loss of detail information in traditional methods, providing strong support for the accurate identification of changing regions. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the embodiments will be briefly described below:

[0047] Figure 1 This is a schematic diagram of the MSE-Net network structure of the present invention.

[0048] Figure 2 This is a structural diagram of the Differential Guided Attention and Flow Alignment Module (i.e., DAFA module structural diagram) in this invention.

[0049] Figure 3 This is a structural diagram of the multi-scale context feature enhancement module (i.e., the MS-CFEM module structural diagram) in this invention.

[0050] Figure 4 This is a structural diagram of the multi-scale feature fusion and bidirectional feature fusion enhancement module (i.e., BFA module structural diagram) in this invention.

[0051] Figure 5 The results of the visualization detection experiments of the MSE-Net of this invention on different datasets are presented. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0053] In remote sensing image change detection, high-resolution images exhibit diverse types of changes with significant scale differences, while the model must achieve high-precision feature extraction and representation. Therefore, this invention proposes a multi-scale feature fusion change detection method based on feature enhancement. An improved multi-scale feature enhancement network (MSE-Net) is designed, incorporating feature enhancement and fusion mechanisms within the network framework to strengthen the model's ability to represent features and effectively integrate multi-scale contextual information, thereby improving overall detection performance.

[0054] The overall framework of MSE-Net is as follows: Figure 1 As shown, during the encoding stage, the input dual-temporal remote sensing images are processed by the VGG16-BN backbone network to extract multi-scale features at five levels, obtaining feature representations at different semantic levels. The extracted dual-temporal features are then aligned using five difference-guided attention and flow alignment (DAFA) modules to enhance potential change areas and highlight multi-scale features of change. Based on this, high-level features (layers A, B, and C) are enhanced with contextual information through a multi-scale contextual feature enhancement module (MS-CFEM) to enrich their feature representation, before entering the multi-scale feature fusion process. Specifically, high-level features are fused step-by-step through a top-down aggregation module, integrating deep semantic information while generating cross-scale global weight information. This global weight is then weighted and applied to low-level features (layers C', D, and E), achieving the fusion of high-level semantic information and low-level fine-grained features. The low-level features, after being weighted globally, are further enhanced by the MS-CFEM module (C'', D', E' layers), and then fused step-by-step through a bottom-up aggregation module to integrate detailed and global information. Through this bidirectional feature interaction process, the network can enhance its global discriminative ability while preserving local detail representation capabilities, resulting in more robust multi-scale feature representations. Finally, the fused features are adjusted to the same spatial resolution as the input image using a convolutional module and a classifier to obtain the predicted output.

[0055] based on Figure 1As can be seen from the overall network structure of the MSE-Net multi-scale feature enhancement network, specifically, this invention discloses a multi-scale feature fusion change detection method based on feature enhancement, which specifically includes the following steps:

[0056] Step 1: Obtain a dual-temporal remote sensing image dataset, which includes a first temporal image T1 and a second temporal image T2 acquired at different time points in the same area.

[0057] Step 2: Construct the multi-scale feature enhancement change detection network MSE-Net, which includes: VGG16-BN backbone network, difference-guided attention and flow alignment module DAFA, multi-scale context feature enhancement module MS-CFEM, bidirectional feature fusion module BFA, and classifier.

[0058] Step 3: Input the dual temporal images T1 and T2 into the VGG16-BN backbone network. The VGG16-BN backbone network contains five sequentially connected blocks. Each block contains 2 to 3 convolutional layers and 1 max pooling layer. The five blocks output features at five different levels.

[0059] See Figure 1 As shown in the diagram, VGG16-BN is divided into 5 blocks, each followed by a max-pooling layer: Block 1: 2 convolutional layers (64 channels) + max-pooling layer pool1; Block 2: 2 convolutional layers (128 channels) + max-pooling layer pool2; Block 3: 3 convolutional layers (256 channels) + max-pooling layer pool3; Block 4: 3 convolutional layers (512 channels) + max-pooling layer pool4; Block 5: 3 convolutional layers (512 channels) + max-pooling layer pool5. Therefore, the outputs of blocks 1-5 correspond to layers 2, 4, 7, 10, and 13 of the 16-layer VGG16-BN, respectively.

[0060] Step 4: Input the five-layer dual-temporal features from the first to the fifth layer of VGG16-BN into five independent DAFA modules in sequence for difference-guided feature alignment and enhancement to obtain the aligned and enhanced multi-scale features. The multi-scale features output by the five DAFA modules are denoted as feature nodes E, D, C, B, and A, respectively.

[0061] See Figure 1It can be seen that feature node E corresponds to the shallow features output by the first layer of VGG16-BN, with the highest spatial resolution and the lowest semantic level; feature node D corresponds to the shallow features output by the second layer of VGG16-BN; feature node C corresponds to the mid-level features output by the third layer of VGG16-BN; feature node B corresponds to the high-level features output by the fourth layer of VGG16-BN; and feature node A corresponds to the high-level features output by the fifth layer of VGG16-BN, with the lowest spatial resolution and the highest semantic level.

[0062] Step 5: Input the aligned and enhanced high-level feature nodes A, B, and C into three independent MS-CFEM modules for multi-scale contextual feature enhancement to obtain the enhanced high-level features, which are denoted as feature nodes A', B', and C', respectively.

[0063] Step 6: Input the enhanced high-level feature nodes A', B', and C' into the BFA module, perform top-down fusion, and generate global weight W.

[0064] Specifically, when the BFA module performs top-down fusion in step 6, it includes:

[0065] Step 6.1: After upsampling, the spatial size of node A' is doubled. The features of node A' after convolution are added element by element and then activated by ReLU to obtain the first fusion result.

[0066] Step 6.2: Further upsample the fusion result from Step 6.1, and add it element-wise to the three identical-sized features of node A' and node C' after convolution transformation following the two upsampling steps in Step 6.1. After ReLU activation, the second fusion result is obtained.

[0067] Step 6.3: The second fusion result is subjected to downsampling and convolution operations in sequence to generate global weights W.

[0068] Step 7: Multiply the global weight W element-wise with the low-level feature nodes C', D, and E respectively; input the weighted low-level feature nodes into three independent MS-CFEM modules for feature enhancement to obtain the enhanced low-level features, denoted as feature nodes C'', D', and E'.

[0069] Step 8: Input the enhanced shallow feature nodes C'', D', and E' into the BFA module and perform bottom-up fusion. Then, input the fused output features into the convolution module and the classifier in sequence to adjust the output features to the same spatial resolution as the input image and obtain the predicted output.

[0070] Specifically, when the BFA module performs bottom-up fusion in step 8, it includes the following:

[0071] Step 8.1: After downsampling, the spatial size of node E' is reduced by half. The features of node D' after convolution are added element by element, and after ReLU activation, the first fusion result is obtained.

[0072] Step 8.2: Further downsample the first fusion result from Step 8.1, and add it element-wise to the three identical features of node E' and node C'' after convolution transformation in Step 8.1 after two downsampling steps. After ReLU activation, the second fusion result is obtained.

[0073] Step 8.3: The second fusion result is subjected to upsampling and convolution operations in sequence, and finally used as the fused output feature.

[0074] Furthermore, the present invention in Figure 1 Based on, combined Figure 2-4 Each sub-module of the Differential Guided Attention and Flow Alignment (DAFA), Multi-Scale Contextual Feature Enhancement (MS-CFEM), and Bidirectional Feature Fusion (BFA) modules is described in detail.

[0075] A. Differential-Guided Attention and Flow Alignment Module (DAFA Module)

[0076] In change detection tasks, due to factors such as illumination variations, viewpoint differences, and scene dynamics, dual-temporal remote sensing images exhibit spatial offsets and semantic differences at the feature level. Directly calculating these differences on unaligned features can easily misclassify unchanged factors as changed ones, leading to false positives and false negatives. Therefore, a difference-guided attention and flow alignment module (DAFA) is designed to achieve feature alignment at the feature level. The structure of this DAFA module is as follows: Figure 2 As shown, it mainly consists of two parts: a difference-guided attention enhancement part and a flow bidirectional alignment fusion module.

[0077] Before feature alignment, the bi-temporal features are first enhanced using an attention mechanism to highlight significant information related to change. Input features X1 and X2 pass through a channel attention module, then into a spatial attention module, ultimately outputting the feature... and Attention mechanisms include channel attention and spatial attention. These mechanisms adaptively adjust the importance of different channels or spatial locations along the feature dimension, thereby enhancing the response of changing regions and suppressing irrelevant background interference. Compared to direct feature alignment, pre-enhancement through attention mechanisms can provide more explicit difference guidance information for subsequent alignment processes.

[0078] like Figure 2As shown, the channel attention mechanism primarily improves feature representation by learning the weight distribution between channels, strengthening important channels, and suppressing unimportant channels. In channel attention, global max pooling (Max_Pool) and global average pooling (Avg_Pool) operations are first performed in parallel on the input features to extract global statistical information of the feature map along the channel dimension. Then, the results of the two pooling operations are input into two layers respectively. Convolution is used to generate channel descriptions, and in two layers A ReLU activation function is introduced in the middle of the convolution to enhance non-linear expressiveness, thereby strengthening the ability to model differences between different channels. Finally, the two layers... The parallel output of the convolution is summed element by element and then fed into the Sigmoid activation function. The Sigmoid activation function generates normalized channel weights, which are then applied to the original features channel by channel to highlight the response to changes in key channels.

[0079] Unlike channel attention, spatial attention mechanisms are primarily used to model the importance distribution of feature maps in a spatial dimension. Their input features are the output features of the channel attention module. The input features are first subjected to max pooling and average pooling in parallel along the channel dimension, and then the two results are fused along the channel dimension. Convolutional layers are used to model spatial information. Simultaneously, to stabilize the feature distribution and maintain consistency across different scales, a batch normalization (BN) layer is introduced after convolution for normalization. Finally, the outputs of the two normalization layers are concatenated and input into a sigmoid activation function. This sigmoid activation function generates spatial attention weights, which are then applied pixel-by-pixel to the feature map, guiding the network to focus more on potentially changing regions.

[0080] The synergy of channel attention and spatial attention can enhance the difference-related features of remote sensing images at both the channel and spatial levels, providing a more reliable basis for difference guidance in the subsequent feature alignment process.

[0081] After the aforementioned attention enhancement, the two-phase features are fed into the flow bidirectional alignment fusion module (FDAF module). Feature alignment can further alleviate the feature misalignment problem between the two-phase images. By learning a deformable flow field, FDAF can achieve bidirectional feature alignment, reducing errors caused by spatial offset while preserving change information. Specifically, the features after the aforementioned difference enhancement... and Concat along the channel dimension, as shown in Equation (1).

[0082]

[0083] in Indicates a stream generator The predicted flow field is generated by extracting the differential response through multi-layer convolution, normalization, and activation functions. The flow generator sequentially includes depthwise separable convolution (Dw Conv), normalization (IN), GELU activation function, and pointwise convolution (PW Conv).

[0084] Next, a warp operation is performed on the input features to obtain the corrected aligned features, as shown in Equation (2).

[0085]

[0086] in, and They represent from and The offset of each displacement field. The remapping warp operation interpolates and samples features in a continuous space, achieving fine-grained spatial alignment.

[0087] Subsequently, the feature after the Warp operation is subtracted from the original feature of another time phase to obtain explicit change information.

[0088]

[0089] The differential features are concatenated using Equation (3.3), and the resulting x is then fed into the subsequent network after fusion.

[0090] B. Multi-scale contextual feature enhancement (MS-CFEM module)

[0091] In change detection, complex land cover structures and different scales pose challenges to feature representation, easily leading to insufficient representation or inadequate discriminative information. Existing methods discard information at some scales during feature extraction, resulting in the inability to depict some complex details. Therefore, to enhance the expressive power of multi-scale spatial features and alleviate the problem of single feature extraction, a Multi-Scale Contextual Feature Enhancement Module (MS-CFEM) was designed. MS-CFEM can more fully mine effective supervision information with limited annotations through parallel multi-scale local convolutional collaboration, improving the quality of feature representation and the ability to discriminate changed regions. The module structure is shown below. Figure 3 .

[0092] MS-CFEM contains three parallel convolutional branches. The CBR structure consists of sequentially connected convolutional Conv, batch normalization (BN), and ReLU functions. Each branch is composed of CBR structures of different sizes, responsible for extracting local features at different scales and orientations. One of the branches uses three depthwise separable convolutions with kernels of (1,1) / (3,3) / (1,1). Figure 3 The three-layer yellow CBR module extracts local features at low cost. The other two layers use four asymmetric convolutions with kernels of (1,3) / (3,1) / (1,5) / (5,1). Figure 3 The four-layer green CBR module enhances spatial features and boundary information at different scales, and concats the three branches in the channel dimension to form multi-scale local features.

[0093] Simultaneously, to further supplement global context information, a global context branch is introduced. This branch generates a global description through average pooling, flattening, and decompression operations, fusing multi-scale local features into a unified feature space. The input features, after passing through the global context branch, are concatenated with the multi-scale local features. The concatenated and fused features are then input into the ReLU activation function, and subsequently... The residual connections of convolution are combined with input features to achieve the integration of local details and global structure.

[0094] C. Enhanced Multi-Scale Feature Fusion and Bidirectional Feature Fusion (BFA Module)

[0095] Change detection tasks often suffer from inconsistencies in multi-scale features and loss of detail. Low-level features preserve rich texture and edge details but lack global contextual information; high-level features contain strong global information but often sacrifice spatial accuracy. To effectively compensate for this difference, a multi-stage, multi-scale feature fusion strategy is constructed. The Bidirectional Feature Aggregation (BFA) module, as the core structure of multi-scale feature fusion, consists of two sub-modules: a top-down fusion module and a bottom-up fusion module. Its structure is as follows: Figure 4 As shown.

[0096] Unlike the simple step-by-step alignment fusion in traditional multi-scale feature fusion structures, MSE-Net's complete fusion process not only includes bidirectional feature fusion, but also combines key steps such as global semantic weight generation and shallow feature modulation to achieve more comprehensive cross-scale feature interaction.

[0097] Multi-scale feature fusion first involves a top-down module that progressively fuses high-level features (layers {A′, B′, C′}). These high-level features are aligned to a uniform spatial resolution and then fused through element-wise addition and convolutional thinning to gradually integrate deep semantic information and generate global weights containing global information. The specific fusion process is as follows: Figure 4 As shown in Figure (a).

[0098] Input A': dimensions are (32, 16, 16), input B': dimensions are (32, 32, 32), input C': dimensions are (32, 64, 64).

[0099] The first stage: Feature map A' is first upsampled once, doubling its spatial size to (32, 32, 32). Feature map B' is transformed by a convolutional layer (Conv). Then, the upsampled A' and the convolutional B' are added element-wise. The result is activated by ReLU, and the output size is kept at (32, 32, 32).

[0100] Second stage: The features of A' in the first stage (size 32, 32, 32) are upsampled again to (32, 64, 64). The features output by ReLU in the first stage (size 32, 32, 32) are upsampled again to (32, 64, 64). Feature map C' passes through a convolutional layer (Conv) and its size remains (32, 64, 64). The three feature maps with the same size (32, 64, 64) are then added element-wise.

[0101] Then, the ReLU activation function is applied, followed by downsampling to reduce the spatial resolution by half, and the feature map size is restored to (32, 32, 32). Finally, a convolutional layer (Conv) is applied to obtain a global weight W with a size of (1, 32, 32).

[0102] The global weights W are then used to weight the shallow features (layers {C′, D, E}). The global weights W are upsampled to the same scale as the shallow features, then multiplied element-wise with the shallow features, and finally fused with the original features using residuals. This achieves adaptive enhancement of the low-level features, allowing the shallow features to obtain stronger global constraints while preserving detailed information.

[0103] After global weighting, the shallow features (layers {C′, D, E}) are augmented using the MS-CFEM module to obtain C′′, D′, and E′, thus enhancing the expressive power of local features. The augmented features are then fused using a bottom-up module. The specific fusion process is as follows: Figure 4 As shown in Figure (b).

[0104] Enter C'': dimensions are (32, 64, 64), enter D': dimensions are (32, 128, 128), enter E': dimensions are (32, 256, 256).

[0105] The first stage: Feature map E' first undergoes downsampling, reducing its spatial size by half to (32, 128, 128). Feature map D' then undergoes a convolutional layer (Conv) for feature transformation, maintaining its size at (32, 128, 128). The convolutional D' and the downsampled E' are then added element-wise, and the result is activated by the ReLU function, maintaining the output size at (32, 128, 128).

[0106] Second stage: The dimensionality-reduced features of E' in the first stage (32, 128, 128) are downsampled again, and the size becomes (32, 64, 64). The features output by ReLU in the first stage (32, 128, 128) are downsampled again, and the size becomes (32, 64, 64). The input feature map C'' is directly passed through a convolutional layer (Conv), and the size remains (32, 64, 64). The three feature maps with the size of (32, 64, 64) are then added element-wise.

[0107] Finally, the feature map after the sum of the three sides is first passed through the ReLU activation function, then upsampled once to double the spatial resolution, restoring the feature map size to a medium resolution (32, 128, 128). Finally, it goes through a convolutional layer (Conv) to expand the number of channels from 32 to 96, resulting in an output feature map of size (96, 128, 128).

[0108] By fusing information through stepwise downsampling and element-wise addition, information aggregation from local details to global semantics is achieved. The resulting fused features maintain fine-grained spatial structure information while also possessing stronger semantic consistency, providing strong support for the accurate identification of changing regions.

[0109] Furthermore, top-down fusion and bottom-up fusion share a certain structural symmetry and are complementary in function. The former emphasizes the global constraint of high-level semantics on low-level features, while the latter highlights the supplementation of low-level details with high-level detailed features. Together, they complete bidirectional feature interaction, enabling the model to maintain strong feature representation and detail description capabilities in semi-supervised scenarios, thus improving the stability and accuracy of change detection.

[0110] Figure 5 The visualization results of MSE-Net on different remote sensing change detection datasets are shown below. Green indicates missed detections, and red indicates false detections. (a), (b), and (c) represent three different scene types: changes to large-target buildings, changes involving large targets in combination, and changes involving a mixture of large and small targets, respectively. The overall results show that MSE-Net achieves excellent detection performance in all three scenes across different datasets.

[0111] In summary, the MSE-Net overall framework constructed in this invention forms a closely collaborative technical system with the DAFA module, MS-CFEM module, and BFA module. Each module performs its specific function at different stages, jointly improving the accuracy and robustness of change detection.

[0112] The synergistic effect of feature alignment and multi-scale fusion: The DAFA module performs difference-guided alignment and enhancement of bi-temporal features during the encoding stage, providing semantically consistent and highly variable feature inputs for subsequent multi-scale feature fusion. The effectiveness of this module directly affects the expression quality of high-level features, thus laying a reliable foundation for the global weight generation of the BFA module, ensuring that the feature fusion process focuses on the truly changing regions and reduces the interference of non-changing factors.

[0113] Complementary enhancement of global semantics and local details: The MS-CFEM module introduces multi-scale contextual enhancement at both the high-level and shallow feature stages to strengthen feature representation. During the top-down fusion process of the BFA module, high-level semantic information is transformed into global weights to guide the modulation of shallow features. The modulated shallow features are then further enhanced by MS-CFEM, and finally, through bottom-up fusion, fine spatial structure information is transmitted to the final prediction. This alternating combination achieves a two-way enhancement that combines global semantic guidance with preservation of local details.

[0114] Closed-loop optimization of bidirectional interaction and multi-level feature representation: The MSE-Net framework organically integrates the alignment features provided by the DAFA module, the contextual information enhanced by the MS-CFEM module, and the cross-scale interaction mechanism constructed by the BFA module through two fusion paths: top-down and bottom-up. This forms a complete processing chain from feature alignment to multi-scale enhancement and then to bidirectional fusion. This closed-loop structure enables the network to simultaneously optimize feature representation at different semantic levels, effectively avoiding the local optimum problem caused by independent optimization of a single module, and significantly improving the model's change detection capability in complex remote sensing scenarios.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-scale feature fusion change detection method based on feature enhancement, characterized in that, Includes the following steps: Step 1: Obtain a dual-temporal remote sensing image dataset, wherein the dual-temporal remote sensing images include a first temporal image T1 and a second temporal image T2 acquired at different time points in the same area; Step 2: Construct the multi-scale feature enhancement change detection network MSE-Net, which includes: VGG16-BN backbone network, differentially guided attention and flow alignment module DAFA, multi-scale contextual feature enhancement module MS-CFEM, bidirectional feature fusion module BFA, and classifier. Step 3: Input the dual temporal images T1 and T2 into the VGG16-BN backbone network. The VGG16-BN backbone network contains five sequentially connected blocks. Each block contains 2 to 3 convolutional layers and 1 max pooling layer. The five blocks output features at five different levels. Step 4: Input the five-layer dual-temporal features from the first to the fifth layer of VGG16-BN into five independent DAFA modules in sequence for difference-guided feature alignment and enhancement to obtain the aligned and enhanced multi-scale features. The multi-scale features output by the five DAFA modules are denoted as feature nodes E, D, C, B, and A, respectively. Step 5: Input the aligned and enhanced high-level feature nodes A, B, and C into three independent MS-CFEM modules for multi-scale contextual feature enhancement to obtain the enhanced high-level features, which are denoted as feature nodes A', B', and C' respectively. Step 6: Input the enhanced high-level feature nodes A', B', and C' into the BFA module, perform top-down fusion, and generate global weight W; Step 7: Multiply the global weight W element-wise with the low-level feature nodes C', D, and E respectively; input the weighted low-level feature nodes into three independent MS-CFEM modules for feature enhancement to obtain the enhanced low-level features, denoted as feature nodes C'', D', and E'. Step 8: Input the enhanced shallow feature nodes C'', D', and E' into the BFA module and perform bottom-up fusion. Then, input the fused output features into the convolution module and the classifier in sequence to adjust the output features to the same spatial resolution as the input image and obtain the predicted output.

2. The detection method according to claim 1, characterized in that, When the BFA module performs top-down fusion in step 6, it specifically includes: Step 6.1: After upsampling, the spatial size of node A' is doubled. The features of node A' after convolution are added element by element and then activated by ReLU to obtain the first fusion result. Step 6.2: Further upsample the fusion result from Step 6.1, and add it element-wise to the three identical-sized features of node A' and node C' after convolution transformation following the two upsampling steps in Step 6.

1. After ReLU activation, the second fusion result is obtained. Step 6.3: The second fusion result is subjected to downsampling and convolution operations in sequence to generate global weights W.

3. The detection method according to claim 1, characterized in that, When the BFA module performs bottom-up fusion in step 8, it specifically includes: Step 8.1: After downsampling, the spatial size of node E' is reduced by half. The features of node D' after convolution are added element by element, and after ReLU activation, the first fusion result is obtained. Step 8.2: Further downsample the first fusion result from Step 8.1, and add it element-wise to the three identical features of node E' and node C'' after convolution transformation in Step 8.1 after two downsampling steps. After ReLU activation, the second fusion result is obtained. Step 8.3: The second fusion result is subjected to upsampling and convolution operations in sequence, and finally used as the fused output feature.

4. The detection method according to claim 1, characterized in that, The difference-guided attention and flow alignment module consists of two parts: a difference-guided attention enhancement part and a flow bidirectional alignment fusion module. The difference-guided attention enhancement part includes a channel attention module and a spatial attention module connected in sequence. Input features X1 and X2 pass through the channel attention module and then enter the spatial attention module, finally outputting features. and ; The channel attention module performs the following operations: first, it performs global max pooling and global average pooling on the input features in parallel to extract global statistical information of the feature map along the channel dimension; then, it inputs the two pooling results into the two layers respectively. Convolution is used to generate channel descriptions, and in two layers A ReLU activation function is introduced in the middle of the convolution to enhance non-linear expressive power, thereby strengthening the ability to model differences between different channels. Finally, the two layers... The parallel output of the convolution is summed element by element and then fed into the Sigmoid activation function. The Sigmoid activation function generates normalized channel weights, which are then applied to the original features channel by channel to highlight the response to changes in key channels.

5. The detection method according to claim 4, characterized in that, The spatial attention module performs the following operations: the input features are first subjected to max pooling and average pooling in parallel along the channel dimension; then the two results are fused along the channel dimension, and then... Convolution is used to model spatial information. After convolution, a batch normalization layer is introduced for normalization. Finally, the outputs of the two normalization layers are concatenated and input into the Sigmoid activation function. The Sigmoid activation function generates spatial attention weights, which are then applied pixel-by-pixel to the feature map, guiding the network to pay more attention to potential change regions.

6. The detection method according to claim 5, characterized in that, After the aforementioned attention enhancement is completed, the two-phase features output by the spatial attention module are sent to the bidirectional alignment and fusion module, where the following operations are performed: Features after the aforementioned difference enhancement and The concatenation is performed along the channel dimension, as shown in equation (1): in Indicates a stream generator The predicted flow field is generated by a flow generator that sequentially includes depthwise separable convolution (Dw Conv), normalized IN, GELU activation function, and pointwise convolution (PW Conv). Next, a remapping warp operation is performed on the input features to obtain the corrected aligned features, as shown in equation (2): in, and They represent from and The offset of each displacement field, the remapping Warp operation interpolates and samples the features in a continuous space; Subsequently, the feature after the Warp operation is subtracted from the original feature of another time phase to obtain explicit change information; The difference features are spliced ​​together by equation (3), and the result x will be fed into the subsequent network after fusion.

7. The detection method according to claim 1, characterized in that, The multi-scale context feature enhancement module includes three parallel convolutional branches, each composed of CBR structures of different sizes. The CBR structure consists of a convolutional Conv, a batch normalized BN, and a ReLU function connected in sequence. One branch uses three depthwise separable convolutions with kernels of (1,1) / (3,3) / (1,1), while the other two branches use four asymmetric convolutions with kernels of (1,3) / (3,1) / (1,5) / (5,1). The three branches are concatenated along the channel dimension to form multi-scale local features. Simultaneously, a global context branch is introduced. This branch generates a global description through average pooling, flattening, and decompression operations. The input features, after passing through the global context branch, are concatenated with the multi-scale local features. The concatenated and fused features are then input into a ReLU activation function, and subsequently... The residual connections of convolution are combined with input features to achieve the integration of local details and global structure.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the detection method according to any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When the program is executed by the processor, it implements the detection method according to any one of claims 1 to 7.

Images