Remote sensing image change detection method and device

Abstract

The invention discloses a remote sensing image change detection method and device which are applied to the technical field of change detection, and the method comprises the steps: obtaining a dual-temporal remote sensing image, carrying out the change detection of the dual-temporal remote sensing image through a change detection model, and obtaining a binary change detection image, the change detection model is a neural network model obtained by adding an attention module and a change guide module behind an up-sampling module of each decoder of a ChangeMama model; the attention module is used for performing layered pixel convolution operation and channel interaction processing on the input of the attention module to obtain a first attention feature map; and the change guiding module is used for carrying out output guiding processing on the first attention characteristic pattern by using the coding characteristics of each encoder in the ChangeMa mba model to obtain the output of the decoder. According to the invention, the change detection precision of the dual-time-phase remote sensing image can be improved.

Description

Technical Field

[0001] This application relates to the field of change detection technology, and in particular to a method and apparatus for detecting changes in remote sensing images. Background Technology

[0002] Since the birth of remote sensing technology, change detection (CD) has been a popular research direction in the field of remote sensing. Its goal is to detect changes in surface objects from multi-temporal remote sensing images obtained at different times. It can be widely used in land cover change analysis, urban expansion research, disaster response, geographic information system updates, and ecological monitoring.

[0003] High-resolution remote sensing images have become one of the most widely used and researched data sources in the field of remote sensing. They provide detailed texture and geometric information of surface features, enabling more sophisticated change detection tasks. However, with the increase in spatial resolution, noise interference in remote sensing images has become more severe, making it difficult for traditional pixel-based change detection methods to achieve satisfactory results.

[0004] The emergence of deep learning has brought new models and paradigms to change detection, significantly improving its efficiency and accuracy. Since Daudt et al. introduced fully convolutional networks into the field of change detection, methods based on convolutional neural networks (CNNs) have been dominant. Although these methods have achieved good results, the limited receptive field of the CNN architecture makes it unable to capture long-range dependencies between different pixels, which still makes it insufficient when processing remote sensing images with different spatiotemporal resolutions.

[0005] The Transformer architecture offers a novel approach to addressing the shortcomings of the aforementioned CNN architectures. Through a stacked self-attention mechanism, it learns the relationships between pixels across the entire image. Currently, an increasing number of change detection architectures employ Transformers as encoders to extract representative and robust features of ground cover changes, while simultaneously using them in the decoder to capture the spatiotemporal relationships between multi-temporal features. However, the computational complexity of the self-attention mechanism in the Transformer architecture is proportional to the square of the image size, resulting in high computational costs. This is highly detrimental to high-density detection tasks on large-scale remote sensing image datasets. To address this issue, some related techniques have proposed solutions, such as adjusting the size or stride of the computation window. While these methods improve attention efficiency, they all come at the cost of limiting the receptive field.

[0006] State-space models (SSMs), as a viable alternative to the Transformer architecture, especially the Structured State-space Sequence Model (S4), have demonstrated outstanding performance in analyzing continuous long sequence data, exhibiting the favorable characteristic that computational complexity scales linearly with sequence length. The Mamba architecture further improves upon the S4 model by introducing a selection mechanism, allowing the model to select relevant information in an input-dependent manner. Combined with hardware-aware algorithms, the Mamba architecture outperforms the Transformer architecture in some downstream tasks. Recently, the Mamba architecture has been extended to image data and has achieved promising results in several vision tasks.

[0007] While the Mamba architecture has achieved remarkable success in vision tasks, and the ChangeMamba model, when applied to remote sensing image change detection, has also demonstrated certain advantages, the ChangeMamba model relies heavily on the 2D cross-scanning mechanism of the Vmamba model to handle interactions between ground feature changes. This makes it difficult to cope with complex scenarios such as changes in small targets and the coexistence of multiple scale changes. Furthermore, the ChangeMamba model uses a simple stitching or direct cross-scanning mechanism at the decoder input to fuse feature information from both temporal phases, resulting in insufficient sensitivity to subtle changes or occluded areas. All these factors contribute to the poor performance of the ChangeMamba model in remote sensing image change detection tasks. Summary of the Invention

[0008] This application provides a method and apparatus for detecting changes in remote sensing images, which can improve the accuracy of change detection in dual-temporal remote sensing images.

[0009] On the one hand, embodiments of this application provide a method for detecting changes in remote sensing images, including the following steps:

[0010] Acquire dual-temporal remote sensing images;

[0011] The dual-temporal remote sensing image is subjected to change detection using a change detection model to obtain a binary change detection map.

[0012] The change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model.

[0013] The attention module is used to perform hierarchical pixel convolution operations and channel interaction processing on the input of the attention module to obtain a first attention feature map;

[0014] The change guidance module is used to perform output guidance processing on the first attention feature map using the encoding features of each encoder in the ChangeMamba model, so as to obtain the output of the decoder.

[0015] On the other hand, embodiments of this application provide a remote sensing image change detection device, including:

[0016] The acquisition module is used to acquire dual-temporal remote sensing images;

[0017] The remote sensing change detection module is used to perform change detection on the dual-temporal remote sensing image using a change detection model to obtain a binary change detection map.

[0018] The change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model.

[0019] The attention module is used to perform hierarchical pixel convolution operations and channel interaction processing on the input of the attention module to obtain a first attention feature map;

[0020] The change guidance module is used to perform output guidance processing on the first attention feature map using the encoding features of each encoder in the ChangeMamba model, so as to obtain the output of the decoder.

[0021] According to an embodiment of this application, a method and apparatus for detecting changes in remote sensing images are provided. This involves acquiring dual-temporal remote sensing images, performing change detection on the dual-temporal remote sensing images using a change detection model to obtain a binary change detection map. The change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in a ChangeMamba model. The attention module performs hierarchical pixel convolution operations and channel interaction processing on the input of the attention module to obtain a first attention feature map. The change guidance module uses the encoding features of each encoder in the ChangeMamba model to guide the output of the first attention feature map, obtaining the output of the decoder. According to the technical solution of this application, while comprehensively capturing subtle, multi-scale ground feature changes and complex dependencies between channels in dual-temporal remote sensing images, it also models the global spatial context and spatiotemporal relationships of the dual-temporal remote sensing images. This achieves accurate fusion of multi-scale ground feature changes and effective modeling of refined channel relationships, thereby improving the change detection accuracy of dual-temporal remote sensing images, especially in complex backgrounds such as small target changes and the coexistence of multi-scale changes.

[0022] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0023] Figure 1A This is a diagram of the overall structure of the existing ChangeMamba model;

[0024] Figure 1B This is a structural diagram of the encoder in the ChangeMamba model;

[0025] Figure 1C This is a structural diagram of the decoder in the ChangeMamba model;

[0026] Figure 2 This is a structural diagram of the MCMamba model provided in this application;

[0027] Figure 3 This is a structural diagram of the MSPA module provided in this application;

[0028] Figure 4 This is a structural diagram of the HPC module provided in this application;

[0029] Figure 5 This is a structural diagram of the SPR module provided in this application;

[0030] Figure 6 This is a structural diagram of the CGM module provided in this application;

[0031] Figure 7 This is a structural diagram of the PCM component provided in this application;

[0032] Figure 8 This is a structural diagram of the CGA component provided in this application;

[0033] Figure 9 These are comparison images of the effects provided in this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0036] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0038] The ChangeMamba model is the first remote sensing image change detection framework to apply the Mamba architecture to change detection tasks. Its core design aims to overcome the limitations of the receptive field in CNN architectures and the high computational complexity of Transformer architectures. Specifically, by combining the linear sequence modeling capabilities of the Mamba architecture with the characteristics of visual tasks, this model achieves efficient global context learning and spatiotemporal relationship modeling for multi-temporal remote sensing images.

[0039] Reference Figure 1A T1 and T2 are paired dual-temporal remote sensing images. Encoder is the encoder structure, with Stage 1 to Stage 4 of the encoder referring to the first to fourth encoders respectively. Weight Sharing represents the weight sharing among the encoders. Decoder is the decoder structure, with Stage 1 to Stage 4 of the decoder referring to the first to fourth decoders respectively. Binary Change map represents the binary change detection map. The ChangeMamba model adopts an encoder-decoder structure, and its key modules are the weight-sharing twin encoder structure based on the Variational State Space Model (VSSM) architecture and the three-temporal interactive transform decoder structure based on the Mamba architecture.

[0040] Reference Figure 1BH represents the height of the input image, W represents the width of the input image, C0 represents the number of channels of the input image, and C1 to C4 refer to the number of channels of the feature maps output by the first to fourth encoders, respectively. The weight-shared Siamese encoder structure based on the VSSM architecture uses VMamba as the backbone network and has four encoders. Each encoder first downsamples the input data, then uses multiple VSS blocks to fully model the spatial context information, and then outputs the features of this stage. Since the input data are paired bi-temporal remote sensing images, the features output by each encoder are also paired, called encoded feature pairs, which contain two feature maps, each corresponding to one of the two temporal phases.

[0041] Reference Figure 1C H represents the height of the input image, W represents the width of the input image, C0 represents the number of channels of the input image, C1 to C4 refer to the number of channels of the feature maps output by the fourth decoder to the first decoder, respectively, Stage1 output to Stage4 output refer to the feature maps output by the first decoder to the fourth decoder, respectively, and Binary Change map represents the binary change detection map. The three spatiotemporal interactive transform decoder structure based on the Mamba architecture consists of four decoders. Each decoder models the spatiotemporal relationship of dual-phase features through a three spatiotemporal state space module (STSS Block), restores the image resolution through an upsampling module (2×Upsampling), and introduces a fusion module (FusionBlock) in the last three decoders to fuse low-level semantic features and high-level semantic low-resolution features. In the STSS module, three spatio-temporal token generators are first used to model the features of the dual-temporal images, further extracting robust and representative features. These features are then input into three VSS blocks to obtain features that fully exploit the global spatio-temporal correlation of the dual-temporal features. Finally, the three VSS blocks are concatenated and passed through a convolutional layer (Conv) to obtain the output of the STSS module. In the fusion module, the low-level semantic features output from the SSTS block of the current decoder are concatenated with the high-level semantic low-resolution features output from the previous decoder. Features are refined and noise is suppressed through 1×1 convolution (1×1 Conv) and 3×3 depthwise convolution (3×3 Conv).

[0042] While the ChangeMamba model can capture global contextual and spatiotemporal information, it has limitations in fusing and refining the modeling of multi-scale land cover change features. This is because it primarily relies on the 2D cross-scanning mechanism of the ChangeMamba model to assess the interactions between land cover change features. It cannot model the semantic relationships between land cover change features at different scales, which affects its change detection accuracy in complex scenarios (such as changes in small targets or the coexistence of multiple scale changes). Furthermore, the ChangeMamba model uses a simple concatenation or direct cross-scanning mechanism at the decoder input to fuse features from both temporal phases. It lacks flexible spatiotemporal feature selection capabilities, which can introduce redundant and useless information or suppress key land cover change features, resulting in insufficient sensitivity to subtle changes or occluded areas.

[0043] Specifically, on the one hand, in complex contexts such as changes in small targets and the coexistence of changes at multiple scales, the ChangeMamba model is insufficient in capturing subtle, multi-scale changes in ground features. Because remote sensing images contain a rich variety of ground features, and the changes in different ground features vary significantly in scale, the ChangeMamba model struggles to extract these features comprehensively and accurately, leading to the missed detection of some ground feature changes and thus affecting the accuracy of its remote sensing image change detection.

[0044] On the other hand, interference factors such as radiation differences, illumination changes, and complex topography in remote sensing images at different time phases will cause significant changes in the ground features of remote sensing images. This makes the ChangeMamba model susceptible to being misled when modeling the global spatial context and spatiotemporal relationships of dual-temporal remote sensing images, thereby reducing its change detection performance.

[0045] On the other hand, the ChangeMamba model has shortcomings in the fusion of multi-scale land cover change features and the modeling of refined channel relationships. Land cover change features at different scales often cannot be fully and effectively integrated, which limits the ChangeMamba model's ability to understand and analyze complex scenarios.

[0046] Therefore, to address the shortcomings of the ChangeMamba model in remote sensing image change detection tasks, this application provides a remote sensing image change detection method and apparatus. The ChangeMamba model is improved for remote sensing image change detection tasks to obtain a change detection model, called the MCMamba model. This model can effectively model the global context and spatiotemporal relationships in dual-phase images, thereby achieving efficient and robust remote sensing image change detection.

[0047] The following section will describe in detail, with reference to the accompanying drawings, a method for detecting changes in remote sensing images provided in this application.

[0048] This application provides a remote sensing image change detection method, which can be applied to a terminal, a server, or software running on either a terminal or a server. The terminal can be a tablet, laptop, desktop computer, etc., but is not limited to these. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Furthermore, the server can be a node server in a blockchain network, but is not limited to these. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0049] Reference Figure 2 The remote sensing image change detection method may include the following steps S101-S102:

[0050] S101, acquire dual-temporal remote sensing images;

[0051] S102, change detection is performed on the dual-temporal remote sensing image using a change detection model to obtain a binary change detection map; wherein, the change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model; the attention module is used to perform hierarchical pixel convolution operation and channel interaction processing on the input of the attention module to obtain a first attention feature map; the change guidance module is used to use the encoding features of each encoder in the ChangeMamba model to perform output guidance processing on the first attention feature map to obtain the output of the decoder.

[0052] In this embodiment of the application, firstly, pairs of dual-temporal remote sensing images are acquired, i.e., as shown below. Figure 2 Images T1 and T2 are shown in the diagram. The paired bi-temporal remote sensing images are then input into a change detection model for change detection. The change detection model is a neural network model improved from the existing ChangeMamba model. It is trained using multiple pre-set bi-temporal remote sensing image samples and the corresponding label information for each sample. The label information refers to the binary change detection map corresponding to the bi-temporal remote sensing image. The change detection model can then detect the desired binary change detection map from the paired bi-temporal remote sensing images.

[0053] Specifically, the embodiments of this application mainly improve the decoder of the ChangeMamba model, such as... Figure 1C As shown, in the existing ChangeMamba model, its decoder structure consists of four decoders. The first decoder contains an STSS module and an upsampling module (containing two upsampling layers), while the subsequent three decoders each contain an STSS module, a fusion module, and an upsampling module. Accordingly, as... Figure 2 As shown, in this embodiment of the application, a multi-scale spatial pyramid attention (MSPA) module and a change guide module (CGM) are added after the upsampling module (2×Upsampling) of each decoder. The MSPA module is designed to perform hierarchical pixel convolution operations and channel interaction processing, while the CGM module is used to perform output guidance operations.

[0054] In the change detection model, the encoder structure of the existing ChangeMamba model is retained. The data flow is as follows: First, the encoder structure is used to perform feature extraction on the input dual-temporal remote sensing image, thereby obtaining four sets of feature maps at different scales corresponding to the two time periods, i.e., eight feature maps in pairs, which are referred to as coded feature pairs in this embodiment. In other words, a coded feature pair contains two feature maps at the same scale corresponding to the two time periods. Then, the four coded feature pairs are input into the corresponding decoders.

[0055] In the first decoder Scd1, the encoded feature pairs output by the first encoder are first processed by the STSS module, and then processed by the upsampling module (2×Upsampling) before being input into the MSPA module. Both the STSS and upsampling modules follow the existing ChangeMamba model description. Then, the MSPA module performs hierarchical pixel convolution and channel interaction processing on the output of the upsampling module. Finally, the CGM module performs an output guidance operation on the output of the MSPA module to obtain the output of the first decoder. In the subsequent three decoders Scd2 to Scd4, the encoded feature pairs output by the corresponding encoders are first processed by the STSS module. Then, the output of the previous decoder is fused with the output of the STSS module by the Fusion Block, and then processed by the upsampling module (2×Upsampling) before being input into the MSPA module. The fusion module also follows the existing ChangeMamba model description. Then, the MSPA module performs hierarchical pixel convolution and channel interaction processing on the output of the upsampling module. Finally, the CGM module performs an output guidance operation on the output of the MSPA module to obtain the decoder output. The output of the fourth decoder is the result of the remote sensing image change detection.

[0056] Therefore, this application improves upon the existing ChangeMamba model to obtain a change detection model. Specifically, the improvement involves adding an MSPA module and a CGM module after the upsampling module of each decoder in the ChangeMamba model. The MSPA module is designed to perform hierarchical pixel convolution operations and channel interaction processing. Hierarchical pixel convolution operations utilize hierarchical residual connections to extract multi-scale spatial information from bi-temporal remote sensing images in a finer-grained manner, accurately locating subtle land cover changes in the bi-temporal remote sensing images. Channel interaction processing models the inter-channel relationship information in the bi-temporal remote sensing images through an adaptive combination mechanism and captures long-distance channel dependencies between different land cover change features. Thus, the MSPA module effectively integrates multi-scale spatial information and cross-channel attention, enabling the change detection model to fully explore the structural information and complex inter-channel dependencies in the bi-temporal remote sensing images, thereby capturing more and more refined land cover change features and improving its ability to detect subtle, multi-scale land cover changes. The CGM module is used to perform output guidance operations. By combining deep feature semantic information and self-attention calculation, it can accurately capture changes in ground features between two temporal remote sensing images, thereby solving the problems of inaccurate edge detection and missed detection of internal holes.

[0057] In summary, the embodiments of this application comprehensively capture subtle, multi-scale features of ground cover changes in dual-temporal remote sensing images, as well as the complex dependencies between channels in dual-temporal remote sensing images. At the same time, they model the global spatial context and spatiotemporal relationships of dual-temporal remote sensing images, achieving accurate fusion of multi-scale features of ground cover changes and effective modeling of refined channel relationships. This improves the change detection accuracy of dual-temporal remote sensing images, especially in complex backgrounds such as changes in small targets and the coexistence of multi-scale changes.

[0058] The MSPA module will be further explained below.

[0059] In some implementations, refer to Figure 3 The above-mentioned hierarchical pixel convolution operation and channel interaction processing on the input of the attention module to obtain the first attention feature map may include:

[0060] Hierarchical pixel convolution operations are performed on the input of the attention module to obtain multi-scale feature maps;

[0061] Channel interaction processing is performed on multi-scale feature maps to obtain channel attention weights;

[0062] The channel attention weights and multi-scale feature maps are fused to obtain the first attention feature map.

[0063] In this embodiment, attention mechanisms are gradually becoming a necessary means to enhance the representational capabilities of convolutional neural networks. Despite progress in attention mechanism research, some problems remain to be solved. In the existing ChangeMamba model, the modeling of multi-scale feature representations and structural information is neglected, which are crucial for generating more discriminative attention maps. Therefore, this embodiment proposes a novel, low-overhead, high-performance attention mechanism with strong generalization capabilities, integrated into the aforementioned MSPA module, which can be used to address the limitations of other attention methods.

[0064] Reference Figure 3 For the key components of the MSPA module mentioned above, the Hierarchical-pixel convolution (HPC) module is referenced. This module can extract multi-scale spatial information in a finer-grained manner by utilizing hierarchical residual connections. The Spatial Pyramid Recalibration Module (SPR) module can model inter-channel relationship information in an adaptive combination mechanism, and use the Softmax operation to establish long-distance channel dependencies.

[0065] First, the input feature map F∈R is processed by the HPC module.H×W×C Processing is performed at multiple scales to obtain enhanced multi-scale feature maps. It contains the first convolutional feature map Second convolution feature map and the third convolution feature map

[0066] Secondly, multi-scale feature maps The input is fed into the SPR module to learn channel attention, thereby obtaining the channel attention weights V = [V1, V2, V3] ∈ R. 1×1×C It includes a first attention weight V1, a second attention weight V2, and a third attention weight V3, where the first attention weight V1 corresponds to the first convolutional feature map. The second attention weight V2 corresponds to the second convolutional feature map. The third attention weight V3 corresponds to the third convolutional feature map.

[0067] Subsequently, the cross-channel soft attention mechanism allows for the adaptive selection of different spatial dimensions while facilitating interaction between local and global channel attention. Therefore, the Softmax function is applied to the channel attention weights V, generating recalibrated channel attention weights A∈R. C×1×1 This is called calibrating attention weights. In this process, the i-th convolutional feature map... The corresponding calibration channel attention weight A i It can be expressed as the following formula (1):

[0068]

[0069] In equation (1), V i For the i-th convolutional feature map The corresponding channel attention weights; s is the number of convolutional feature maps.

[0070] Through the above operations, long-distance channel dependencies between the enhanced multi-scale feature maps were established, and these dependencies were then analyzed using each convolutional feature map. Calibration channel attention weight A i By concatenating (Concat) i = 1, 2, 3, we can obtain the overall recalibrated channel attention weights A = Concat([A1, A2, A3]).

[0071] Finally, for each convolutional feature map Convolutional feature maps Its corresponding calibration channel attention weight A i Perform element-wise multiplication (i.e.) This yields a refined set of feature maps, i.e., convolutional feature maps. The corresponding attention feature map Then all convolutional feature maps The corresponding attention feature map Concatenate the features to obtain the first attention feature map (Refined Feature).

[0072] In some implementations, refer to Figure 4 The above-mentioned hierarchical pixel convolution operation on the input of the attention module yields a multi-scale feature map, which may include:

[0073] The input to the attention module is segmented to obtain the first feature map, the second feature map, and the third feature map;

[0074] Perform a convolution operation on the first feature map to obtain the first convolutional feature map;

[0075] The second feature map and the first convolutional feature map are fused and convolved to obtain the second convolutional feature map.

[0076] The third feature map and the second convolutional feature map are fused and convolved to obtain the third convolutional feature map.

[0077] The first, second, and third convolutional feature maps are fused together to obtain a multi-scale feature map.

[0078] In this embodiment, the HPC module consists of three operations: channel splitting (Split), convolution (Conv), and fusion (Concat). First, channel splitting is performed. For a given input feature map F∈R... H×W×C The feature map is uniformly divided into three feature map subsets F along the channel dimension using a segmentation operator. i ∈R H×W×ω Let i = 1, 2, 3, and each feature map subset have the same spatial shape and the same channel dimension. For ease of understanding, these three feature map subsets are represented as the first feature map F1, the second feature map F2, and the third feature map F3.

[0079] Then, convolution is performed. For each feature map subset F i For each of the numbers i = 1, 2, and 3, a corresponding convolution operator is configured. This operator consists of a 3×3 standard convolution, batch normalization, and an activation function. The subset of feature maps processed by the convolution operator is denoted as [i, i, i]. For ease of understanding, these three subsets of feature maps processed by the convolution operator are represented as the first convolution feature map. Second convolution feature map and the third convolution feature map In hierarchical convolution operations, different convolution operators are connected in a hierarchical residual manner to increase the number of scales that the output features can represent. Specifically, the first feature map F1 can be used to obtain the first convolutional feature map through the first set of convolution operators. The first convolutional feature map Add the second feature map F2 (i.e.) Then, the second set of convolution operators is used to obtain the second convolution feature map. The second convolutional feature map The third convolutional feature map is obtained by adding it to the third feature map F3 and then passing it through the third set of convolution operators. This process can be expressed as the following formula (2):

[0080]

[0081] In equation (2), Conv i (·) denotes the i-th group of convolution operators (convolution operation).

[0082] Finally, the first convolutional feature map is concatenated along the channel dimension. Second convolution feature map and the third convolution feature map Obtain the entire enhanced multi-scale feature map

[0083] Therefore, it is evident that in the HPC module, each convolutional operation extracts crucial information on ground feature changes from feature maps at different scales, and each 3×3 convolution results in a larger receptive field for the output. Consequently, the feature maps output by the HPC module contain different combinations and numbers of receptive field scales. Outputs with smaller receptive fields can capture more details of ground feature changes in remote sensing image scenes, which is crucial for detecting changes in small targets. Outputs with larger receptive fields can focus on larger ground targets, which helps improve the global modeling capability of the change detection model. In short, the HPC module processes the input feature maps with a more efficient multi-scale approach, thereby promoting the extraction of local and global information from dual-temporal remote sensing images by the change detection model. It also uses feature splitting and channel number control to limit the number of parameters and computational cost of the change detection model. A larger ω corresponds to a stronger multi-scale feature representation capability, meaning that the change detection model can obtain feature maps with richer information on ground feature changes.

[0084] In some implementations, refer to Figure 5 The above-mentioned channel interaction processing of multi-scale feature maps to obtain channel attention weights can include:

[0085] Global and local aggregation processing is performed on the multi-scale feature maps to obtain aggregated feature maps;

[0086] Channel interaction processing is performed on the aggregated feature map to obtain channel attention weights.

[0087] In this embodiment, since the HPC module does not involve modeling the inter-channel relationships in the bi-temporal remote sensing images, an SPR module is introduced after the HPC module to establish the inter-channel dependencies in the bi-temporal remote sensing images. This improves the sensitivity of the change detection model to the inter-channel dependencies in the bi-temporal remote sensing images, thereby extracting feature information that is more beneficial for remote sensing image change detection tasks. Therefore, extracting inter-channel dependency information from the multi-scale feature map output by the HPC module is particularly important.

[0088] Accordingly, the SPR module is mainly divided into two parts: the first is the Spatial Pyramid Aggregation Block (SPA block), which aggregates global and local contextual features in dual-temporal remote sensing images through adaptive fusion; the second is the Channel-wise Interaction Block (CI block), which consists of two pointwise convolutions (PW conv) and a sigmoid activation function to extract the interrelationships between channels in dual-temporal remote sensing images.

[0089] Here, the SPA block performs global and local aggregation processing on the multi-scale feature map to obtain the aggregated feature map, which can effectively aggregate global and local contextual information in dual-temporal remote sensing images, thereby realizing the exploration of structural information while performing structural regularization. Since the channel features generated by the SPA block cannot be directly used to learn the relationship between channels in dual-temporal remote sensing images, the CI block is introduced to perform channel interaction processing on the aggregated feature map to obtain channel attention weights, which are then further processed to generate the corresponding channel attention weights.

[0090] In some implementations, refer to Figure 5 The above-mentioned global and local aggregation processing of multi-scale feature maps to obtain aggregated feature maps may include:

[0091] Parallel pooling operations are performed on the multi-scale feature maps to obtain global and local feature maps;

[0092] The global feature map and the local feature map are fused to obtain an aggregated feature map.

[0093] In this implementation, traditional feature extraction modules often employ only a single-scale pooling operation. While global average pooling can capture the global semantics of bi-temporal remote sensing images, it loses a significant amount of local detail information, making it difficult for change detection models to locate subtle changes in ground features within the bi-temporal images. Conversely, relying solely on local pooling can lead to a lack of understanding of the overall scene, making the change detection model susceptible to local noise. For example, in some early change detection algorithms, a single pooling operation caused the model to either fail to accurately identify changed areas or misjudge normal local fluctuations as changes when faced with complex scenes. Furthermore, traditional methods rarely integrate pooling results from different scales, failing to fully leverage the synergistic effect of global and local information. The SPA block, however, utilizes two sizes of average pooling to aggregate global and local contextual information, thereby achieving structure regularization while exploring structural information, effectively solving the aforementioned problems.

[0094] For each convolutional feature map The specific data flow is as follows:

[0095] First, the input feature map The data is mapped to two average pooling channels of different sizes for processing. The first channel uses traditional 1×1 global average pooling to obtain the global feature map F′1∈R. 1 ×1×ω The first channel is used to obtain global channel information; the second channel is treated with 2×2 local average pooling to obtain the local feature map F2′∈R. 2×2×ω This allows for the acquisition of richer local feature representations and structural information.

[0096] Here, based on the need for balanced extraction of global and local features, while considering both computational efficiency and model complexity, the SPA block uses 1×1 and 2×2 pooling kernels. The 1×1 pooling kernel can extract global contextual information from dual-temporal remote sensing images, while the 2×2 pooling kernel can extract local structural information. By weighted summation and fusion of the two pooling results, the change detection model can simultaneously perceive global semantic dependencies and local structural details in dual-temporal remote sensing images, significantly improving the expressive power of remote sensing change features. Of course, in other embodiments, pooling kernels of other sizes can be selected, but the advantages and disadvantages need to be weighed. Larger pooling kernels are suitable for tasks that model long-range dependent spaces, such as large target detection, but may introduce too much irrelevant background information, leading to blurred local details, and the computational cost will increase accordingly.

[0097] In this way, by compressing the entire feature map through 1×1 global average pooling, global channel information in dual-temporal remote sensing images can be obtained, and the overall semantics and structural trends of dual-temporal remote sensing images can be grasped. For example, in remote sensing image change detection, the overall category of the scene (such as city or farmland) can be quickly determined. On the other hand, by using 2×2 local average pooling, we can focus on capturing local detail features in dual-temporal remote sensing images, and retain information such as texture and edges in dual-temporal remote sensing images. For example, we can identify details such as the outline of buildings and the direction of roads, providing support for subsequent localization of subtle changes.

[0098] Then, the spatial shape of the global feature map F1′ is upsampled to the size of the local feature map F2′ to obtain the upsampled global feature map F1″. The upsampled global feature map F1″ and the local feature map F2′ are then fused using a weighted summation to obtain the output feature map F. 1+2 ∈R 2×2×ω .

[0099] Here, the feature map obtained from 1×1 pooling is upsampled to the same size as the feature map obtained from 2×2 pooling and then weighted and summed. Through weighted summation, the change detection model can dynamically adjust the ratio between global and local information in the dual-temporal remote sensing images according to task requirements. This ensures that the fused feature map reflects both the overall semantics of the dual-temporal remote sensing images and contains rich details of local land cover changes, enhancing the adaptability of the change detection model to complex scenes. This fusion method achieves complementarity between global and local information, avoiding the information loss problem caused by single-scale pooling.

[0100] Finally, the output feature map F 1+2 ∈R 2×2×ω By spatially scaling (resize) to a one-dimensional vector with a length of 4ω, this one-dimensional vector becomes the aggregated feature map Z. i .

[0101] Here, the fused feature map is scaled down to a one-dimensional vector. This step not only achieves structural regularization but also further integrates feature information, enabling the output one-dimensional vector to serve as an effective input for subsequent CI block processing and laying the foundation for modeling inter-channel relationships.

[0102] In some implementations, refer to Figure 5 The channel interaction processing of the aggregated feature map described above, to obtain channel attention weights, may include:

[0103] Multiple pointwise convolution operations are performed on the aggregated feature map to obtain the channel attention weights.

[0104] In this implementation, the CI block is generated by further processing the SPA block through two pointwise convolutions and a sigmoid activation function to produce corresponding channel attention weights. For each aggregated feature map Z... i For i = 1, 2, 3, the specific data flow is as follows: First, process the input aggregated feature map Z... i The first pointwise convolution operation is performed, consisting of a 1×1 pointwise convolution and a ReLU activation function. Then, a second pointwise convolution operation is performed on the result of the first operation, consisting of a 1×1 pointwise convolution and a Sigmoid activation function. This yields the channel attention weights V. i As shown in formula (3) below:

[0105]

[0106] In equation (3), This represents a 1×1 pointwise convolution with parameter matrix (4ω, ω / r); σ represents a 1×1 pointwise convolution with parameter matrix (ω / r, ω); σ represents the Sigmoid activation function.

[0107] Here, the one-dimensional vector output from the SPA block is processed by two 1×1 pointwise convolutions. The first 1×1 pointwise convolution reduces the number of channels in the feature map, lowering the computational complexity of the change detection model, and simultaneously extracting preliminary relationships between channels in the dual-temporal remote sensing images. The second 1×1 pointwise convolution restores the number of channels to its original level, further refining the dependencies between channels in the dual-temporal remote sensing images, allowing each channel to fully absorb information from other channels and enhancing the interaction between channels in the dual-temporal remote sensing images. After pointwise convolution, the output is mapped to 0 and 1 using the Sigmoid activation function, generating channel attention weights. These weights reflect the importance of each channel for the change detection task. The change detection model can adaptively adjust the features of different channels based on these weights, enhancing the feature representation of key channels and suppressing interference from irrelevant channels, thereby extracting feature information more conducive to change detection.

[0108] In this way, the CI block achieves deep modeling of the inter-channel relationships while ensuring computational efficiency through simple and effective pointwise convolution and the Sigmoid activation function. Compared with traditional methods, it can capture channel information related to change detection tasks more accurately and improve the detection performance of the model.

[0109] In summary, the MSPA module described above has the following technical effects:

[0110] First, the input feature map is processed by the HPC module, which extracts ground feature change features from multiple scales using hierarchical residual connections. In dual-temporal remote sensing image change detection scenarios, small-scale ground feature change features can capture detailed information such as building outlines and road textures, while large-scale ground feature change features can focus on macroscopic information such as urban layout and topography. Through this hierarchical extraction method, the obtained multi-scale feature map covers rich spatial information, providing a more comprehensive feature foundation for subsequent processing. Compared with single-scale feature extraction, it can more accurately locate subtle ground feature changes in dual-temporal remote sensing images.

[0111] Secondly, the feature maps output by the HPC module are input into the SPR module. In the SPR module, the SPA block uses parallel 1×1 global average pooling and 2×2 local average pooling to acquire global channel information and local structural information from the dual-temporal remote sensing images, respectively, and then performs weighted summation to fuse the two. The CI block uses two pointwise convolutions and a sigmoid activation function to deeply mine the interrelationships between channels and generate channel attention weights. This allows the change detection model to clearly define the importance of each channel in the change detection task. For example, when identifying building changes, it enhances the weights of channels related to building features, suppresses interference from irrelevant background channels, and improves the targeting and effectiveness of ground feature change features.

[0112] Furthermore, a Softmax function is applied to the channel attention weights generated by the SPR module to implement a cross-channel soft attention mechanism. This mechanism allows the change detection model to adaptively select different spatial sizes, promotes interaction between local and global channel attention, and establishes long-distance channel dependencies between enhanced multi-scale feature maps. Taking land use change detection over a large area as an example, the change detection model can use this mechanism to integrate land cover change features from different regions and scales, avoiding misjudgments of actual changes due to local land cover change features, thereby improving the accuracy of change detection in complex scenes.

[0113] Finally, the enhanced multi-scale feature map is multiplied by the calculated channel attention weights to obtain a refined subset of feature maps and the final output. This operation adjusts the weights of features based on channel importance, highlighting key land cover change features and suppressing redundant and useless feature information. This makes the land cover change features extracted by the change detection model more discriminative, further improving the performance of change detection in dual-temporal remote sensing images.

[0114] Existing attention mechanisms have shortcomings in multi-scale feature processing and channel relationship modeling. Some methods focus only on single-scale features, failing to consider both macroscopic and microscopic information, making it difficult to accurately identify changes in complex scenes. Regarding channel relationship modeling, simple channel addition, averaging, or weight generation based on statistical information cannot fully explore the complex dependencies between channels, making it difficult to dynamically adjust channel importance. Based on the above description, by using the MSPA module to extract multi-scale features from the fused feature layer and fully explore the structural information and long-distance channel dependencies of dual-temporal remote sensing images, multi-scale spatial information and cross-channel attention in dual-temporal remote sensing images can be effectively integrated. This is beneficial for capturing more detailed ground feature change features and improving the change detection model's learning ability for multi-scale ground feature change features, thereby enhancing the performance of the change detection model in dual-temporal remote sensing image change detection tasks.

[0115] The CGM module will be explained further below.

[0116] In some implementations, refer to Figure 6 Encoder represents the encoder's output, Weight Sharing represents weight sharing, and Decoder Stages 1 to 4 refer to the outputs of the MSPA modules in the first to fourth decoders, respectively. The above-mentioned output guidance processing of the first attention feature map using the encoding features of each encoder in the ChangeMamba model can be obtained by the following:

[0117] Based on the coding features of each encoder, a guide map generation process is performed to obtain a remote sensing change guide map.

[0118] The first attention feature map is guided by a remote sensing change guidance map to obtain the decoder output.

[0119] In this embodiment, in the existing ChangeMamba model, the multi-scale feature maps obtained by the Siamese Mamba encoder need to be further processed by the decoder to obtain change detection maps. The decoder simply stitches the two sets of feature maps from the two temporal phases together along the channel dimension. This direct fusion method cannot effectively detect changes between the two temporal images, especially due to inaccurate edge detection and missed detection of internal holes. To address this, this application introduces a CGM module, which, by combining deep feature semantic information and self-attention computation, can effectively solve the problems of edge and internal hole detection.

[0120] The CGM module consists of two components: a Produce Change Map (PCM) component and a Change Guide Attention (CGA) component. The PCM component generates a guide map based on the encoded feature pairs from each encoder, resulting in a remote sensing change guide map. This way, the encoded feature pairs from each encoder represent deep ground feature features with rich semantic information. The change guide map is then used as prior information by the CGA component, effectively improving the expressive power of ground feature features. The CGA component uses the remote sensing change guide map as prior information to guide the processing of the first attention feature map, resulting in the decoder's output. This allows the change detection model to focus more on ground feature areas that have changed between the two temporal remote sensing images and ignore areas that have not changed, thereby improving its change detection accuracy.

[0121] In some implementations, refer to Figure 7 The above-mentioned guidance map generation process based on the coding features of each encoder, to obtain a remote sensing change guidance map, may include:

[0122] The coding feature pairs of each encoder are fused to obtain the coding feature map of each encoder;

[0123] The dimensionality reduction of the encoding feature map of each encoder is performed to obtain the dimensionality reduction feature map of each encoder;

[0124] Upsampling is performed on the dimensionality-reduced feature maps of the last three encoders to obtain the upsampled feature maps of the last three encoders.

[0125] The upsampled feature maps of the last three encoders and the dimensionality-reduced feature map of the first encoder are fused together to generate a remote sensing change guidance map.

[0126] In this embodiment, within the PCM component, firstly, for each encoder, the encoder's encoded feature pairs are concatenated along the channel dimension to obtain the encoder's encoded feature map. For ease of understanding, the encoded feature maps of the first to fourth encoders can be represented as layer_1, layer_2, layer_3, and layer_4. Then, the encoder's encoded feature maps are dimensionality-reduced to half their original dimensions, thus obtaining the dimensionality-reduced feature maps of each encoder. This dimensionality reduction operation can be implemented using a 1×1 convolution (Conv 1×1), but is not limited to this. For ease of understanding, the dimensionality-reduced feature maps of the first to fourth encoders can be represented as layer_1_2, layer_2_2, layer_3_2, and layer_4_2.

[0127] Then, the first encoder is defined as the shallow encoder, while the second through fourth encoders are defined as deep encoders. For each deep encoder, the dimensionality-reduced feature map of the deep encoder is upsampled to make its spatial shape consistent with the dimensionality-reduced feature map layer_1_2 (i.e., the shallow feature map) of the shallow encoder, resulting in the upsampled feature map of the deep encoder (i.e., the deep feature map). Specifically, the upsampling operation for the fourth encoder is stacked eight times (i.e., Upsample×8), the upsampling operation for the third encoder is stacked four times (i.e., Upsample×4), and the upsampling operation for the second encoder is stacked twice (i.e., Upsample×2), but this is not a limitation. For ease of understanding, the upsampled feature maps of the fourth through second encoders can be represented as layer_4_3, layer_3_3, and layer_2_3, respectively.

[0128] Finally, the upsampled feature maps layer_4_3, layer_3_3, and layer_2_3 from the last three encoders are concatenated with the dimensionality-reduced feature map layer_1_2 from the first encoder, and a remote sensing change map is generated through a GUI. The GUI can refer to a 1×1 convolution (Conv 1×1), but is not limited to this.

[0129] Here, the remote sensing change guidance map is generated from deep features with rich semantic information and used as prior information for subsequent CGA components, thereby improving feature representation capabilities. Remote sensing image scenes often tend to be complex, and the output of a single encoder is easily affected by environmental interference, exhibiting problems such as feature homogeneity, poor adaptability, and low information utilization. It cannot simultaneously capture the detailed and semantic information in dual-temporal remote sensing images, thus struggling to cope with complex scenes and limiting model performance. Therefore, this implementation considers the outputs of four encoders to generate the remote sensing change guidance map. From a macroscopic perspective, the feature maps output by encoders at different stages have different semantic levels. Integrating the outputs of the four encoders can effectively fuse multi-level semantic information in dual-temporal remote sensing images, understand the changes in ground features in dual-temporal remote sensing images from multiple perspectives, accurately determine the types of changes, enhance the global understanding capability of the change detection model, and ensure the reliability of the change guidance map under environmental changes, stably detecting actual change areas in dual-temporal remote sensing images. From a detailed perspective, the output of the shallow encoder preserves the details of the ground features, while the output of the deep encoder contains high-level semantics of the ground features. By fusing the two, subtle changes can be accurately located, while the scene can be understood from a macro perspective, avoiding problems such as misjudgment, blurred edges, and missed detection caused by using only the output of the shallow or deep encoder.

[0130] In some implementations, refer to Figure 8The above-mentioned use of remote sensing change guidance maps to guide the processing of the first attention feature map yields the decoder output, including:

[0131] The remote sensing change guidance map and the first attention feature map are fused together to obtain a fused feature map;

[0132] Multi-head attention operations are performed on the fused feature map to obtain the second attention feature map;

[0133] The first attention feature map and the second attention feature map are fused together to obtain the decoder output.

[0134] In this implementation, within the CGM module, a change guidance map is generated from deep ground feature features rich in semantic information using the PCM component. Then, the CGA component is used to broaden the receptive field between different pixels in the dual-temporal remote sensing images. The change guidance map serves as prior knowledge, guiding the fusion process of multi-scale ground feature change features. The goal of the attention mechanism is to ignore most unimportant information and select a small amount of important information from a large amount of data. In highly imbalanced binary classification problems, the attention mechanism for change detection locates important change features with a small number of pixels in the image, ignoring background information with a large number of pixels. When calculating attention, higher weights result in greater weights for the corresponding values. Therefore, the CGA component enables the model to focus more on ground feature areas that have changed between the dual-temporal remote sensing images, ignoring ground feature areas that have not changed.

[0135] In the CGA component, firstly, the Sigmoid activation function is used to guide the remote sensing change map f. CGM The (Change map) is processed to obtain a weighted map W with values ​​of (0, 1). CGM The weighted graph W CGM This represents the feature maps extracted at different stages of the encoder, and is also the constantly changing guide map in this component. The probability of pixel changes increases with the increase of this weight map. Simultaneously, the first attention feature map... The Decoder Output is convolved to obtain the first attention feature map after convolution. The convolution operation can be a 1×1 convolution (Conv 1×1) to achieve dimensionality reduction, but it is not limited to this. After obtaining the weight graph W... CGM and the first attention feature map after convolution Then, the two are multiplied element by element (i.e., This yields the transformed feature map, i.e., the fused feature map f. CG。 Then, the fused feature map f CGThe query Q, value V, and key K are obtained through linear mapping. Then, a second attention feature map H is calculated using a multi-head attention mechanism. This multi-head attention mechanism can extract rich and potential features related to ground cover changes. The calculation of the multi-head attention mechanism is existing technology and will not be elaborated further. Finally, the second attention feature map H is convolved, and the first attention feature map is then processed. The residual is connected to the layer containing the second attention feature map H, and then the first attention feature map is... Add the second attention feature map H′ after the convolution operation (i.e.) This yields the decoder's output. The convolution operation can be a 1×1 convolution for dimensionality reduction or a 3×3 convolution for feature refinement, but it is not limited to these two methods.

[0136] Here, the CGA component first processes the change guidance map using the Sigmoid function to generate a weight map to locate change areas. Then, the feature map is convolved and multiplied with the weight map, followed by linear mapping to enhance ground feature change characteristics. Next, a multi-head attention mechanism is used to extract refined ground feature change features from multiple dimensions. Finally, the original and optimized ground feature change features are fused through residual connections. Compared to traditional self-attention mechanisms, the CGA component reduces the computational cost of the change detection model, avoids ineffective computation in background areas, and addresses the problem of features being easily overlooked in ground feature change areas. Compared to simple attention methods, the CGA component can deeply mine ground feature change features, reduce false positives and false negatives due to illumination misjudgments, and take into account multi-scale ground feature changes, thus significantly improving the accuracy of numbering and detection of dual-temporal remote sensing images.

[0137] The performance of the change-guided model (i.e., the MCMamba model) provided in the embodiments of this application will be verified below.

[0138] (I) Dataset:

[0139] In this embodiment, the publicly available datasets selected are the LEVIR-CD dataset, the SYSU-CD dataset, and the WHU-CD dataset. The specific information of these three datasets is shown in Table 1 below.

[0140] Table 1: Details of the dataset

[0141]

[0142] (II) Experimental Setup:

[0143] This embodiment's experimental environment is based on the Windows 10 operating system, and a deep learning framework is built using Python 3.10 and PyTorch. Comparative experiments were conducted on three standard change detection datasets: LEVIR-CD, SYSU-CD, and WHU-CD (cropped to a uniform size of 256×256 pixels). An ablation experiment was added for the LEVIR-CD dataset to comprehensively verify the algorithm's effectiveness. Dual-temporal remote sensing image pairs and their associated labels were cropped to 256×256 pixels and input into the network. Then, the network was used for forward inference and model evaluation using the test set. During training, the AdamW optimizer was used to optimize the network with a learning rate of 1e-4 and a weight decay of 5e-3. The batch size was set to 16. The number of training iterations for all datasets was set to 50,000. Random rotation, horizontal flipping, and vertical flipping were used as training data augmentation methods.

[0144] The loss functions used in this embodiment include the cross-entropy loss function and the Dice loss function, as shown in the following formula (4).

[0145] L total =λ1L ce +λ2L dice ,

[0146]

[0147]

[0148] In equation (4), L total It is the overall loss function; L ce This refers to the cross-entropy loss function; L dice This refers to the Dice loss function; N represents the total number of pixels; y j It is the true value of the j-th pixel; λ1 and λ2 represent the predicted probability value of the j-th pixel; λ1 and λ2 represent the preset weight coefficients.

[0149] (III) Performance Indicators:

[0150] In this embodiment, recall, precision, overall accuracy (OA), F1 score, and intersection over union (IoU) are selected as metrics to evaluate model performance.

[0151] (iv) Comparative Experiment:

[0152] To verify the effectiveness of the MCMamba model proposed in this application, 10 comparative experiments were conducted, including the selection of eight popular deep learning change detection algorithms, the baseline model ChangeMamba model of this application, and the improved model MCMamba model proposed in this application. The eight popular deep learning change detection algorithms are: FC-EF model, FC-Siam-conc model, FC-Siam-diff model, and SNUnet model based on CNN architecture; Changeformer model, PA-Former model, and ACABFNet model based on Transformer architecture; and RSMamba model based on Mamba architecture. All of the above were trained and tested on three publicly available datasets. For fair comparison, all experiments were trained under the same conditions. The results of all models are shown in Tables 2 to 4 below.

[0153] Table 2: Performance of all models on the LEVIR-CD dataset

[0154]

[0155] As shown in Table 2, the MCMamba model has the highest values ​​for all evaluation metrics, further validating the superiority of the MCMamba model. Specifically: (1) The Pre value of the MCMamba model is the best among Mamba-based models, 0.88% higher than the baseline model ChangeMamba (91.59%), indicating that the MCMamba model performs best in the ability to correctly classify pixels; (2) The Rec value of the MCMamba model is 1.06% higher than the baseline model ChangeMamba, proving that the MCMamba model performs well in the ability to identify all actual change regions; (3) The F1 score of the MCMamba model is 91.13%, 0.97% higher than the baseline model ChangeMamba, showing that the MCMamba model has a good balance between accuracy and recall; (4) The IoU value of the MCMamba model is 83.72%, 1.63% higher than the baseline model ChangeMamba, indicating that the detection results of the MCMamba model have a higher degree of overlap between change regions and actual change regions; (5) The OA value of the MCMamba model reaches 99.17%, 0.16% higher than the baseline model ChangeMamba, indicating that the MCMamba model performs best overall on the semantic segmentation task of the LEVIR-CD dataset. Therefore, the MCMamba model performs exceptionally well on this dataset, especially in terms of IoU, where it outperforms the baseline model ChangeMamba by the most and achieves the greatest improvement.

[0156] Table 3: Performance of all models on the SYSU-CD dataset

[0157]

[0158] Table 3 shows that the MCMamba model has the best performance in four evaluation metrics: Rec = 81.35%, F1 = 82.36%, IoU = 70.31%, and OA = 92.04%. Specifically: (1) The Pre value of the MCMamba model decreased by 1.32% compared to the baseline model ChangeMamba, ranking 3rd among all models; (2) The Rec value of the MCMamba model increased by 1.76% compared to the baseline model ChangeMamba; (3) The F1 score of the MCMamba model increased by 1.07% compared to the baseline model ChangeMamba; (4) The IoU value of the MCMamba model increased by 1.83% compared to the baseline model ChangeMamba; and (5) The OA value of the MCMamba model increased by 0.68% compared to the baseline model ChangeMamba.

[0159] Table 4: Performance of all models on the WHU-CD dataset

[0160]

[0161] As shown in Table 4, among all models, the MCMamba model has the highest Recall (93.81%), F1 score (94.50%), and IoU (89.99%). Its Pre (95.22%) ranks second among all models, and its OA (98.89%) ranks in the middle. Specifically: (1) The Pre value of the MCMamba model is slightly lower than that of the RSMamba model (95.50%) among Mamba-based models, but it is 1% higher than the baseline model ChangeMamba and higher than all other models. This indicates that the accuracy of the MCMamba model is very high, and the proportion of samples predicted as positive is indeed positive. (2) The Rec value of the MCMamba model is the highest, which is 2.87% higher than the baseline model ChangeMamba. This reflects that the MCMamba model can detect most of the samples that are actually positive. (3) The optimal F1 score of the MCMamba model indicates that it has achieved the best balance between precision and recall. Compared with the baseline model ChangeMamba, it is higher than the baseline model ChangeMamba. The Mamba model improved by 1.95%; (4) The MCMamba model had the best IoU performance, higher than the RSMamba model (86.55%) and the ChangeMamba model (86.13%), and improved by 3.86% compared with the baseline model ChangeMamba model. This indicates that the MCMamba model has the largest overlap between the predicted and real regions; (5) The OA value of the MCMamba model was slightly lower than that of the RSMamba model (99.44%) and the ChangeMamba model (99.42%), ranking in the middle among all models and performing slightly weaker. However, considering its F1 score and IoU performance, it shows that the MCMamba model focuses more on the detection of changing regions rather than overall classification. Overall, the MCMamba model outperformed all Transformer-based models, all CNN-based models and the baseline model ChangeMamba model, which demonstrates the great potential of the MCMamba model in the change detection task.

[0162] Based on the validation results of the three public datasets mentioned above, the MCMamba model has the best overall performance. Thanks to its ability to handle complex dependencies and capture long-range features, the MCMamba model has demonstrated excellent results in remote sensing image change detection tasks.

[0163] (V) Ablation Experiment:

[0164] To verify the effectiveness of each key module in the MCMamba model, this application designed six ablation experiments on the LEVIR-CD dataset. The original ChangeMamba framework was configured to match the baseline model of MCMamba. It was verified that regardless of whether the key modules were added individually or in combination, the overall results of the MCMamba model were superior to the baseline ChangeMamba model. The results are shown in Table 5 below.

[0165] Table 5: Ablation Experiment Results

[0166]

[0167] In this ablation experiment, the MSPA and CGM modules were added individually, and the MSPA, HP C, SPR, and CGM modules were added progressively to evaluate their contributions to model performance. The baseline ChangeMamba model had a Pre value of 91.59%, a Rec value of 88.78%, an F1 score of 90.16%, an IoU of 82.09%, and an OA value of 99.01%. The improvements of each module to the baseline ChangeMamba model are as follows:

[0168] 1) Introducing the MSPA module alone (excluding the HPC and SPR modules) slightly decreased the Rec value of the ChangeMamba model to 88.71%, but improved its Pre value, F1 score, IoU, and OA value, reaching 88.71%, 90.71%, 83.01%, and 99.11%, respectively. This indicates that the MSPA module has a positive effect on improving the ChangeMamba model.

[0169] 2) The introduction of the CGM module alone improved the ChangeMamba model in terms of Pre value, F1 score, IoU and overall Pre value. This shows that the CGM module has a positive impact on the ChangeMamba model and can help the ChangeMamba model to better extract subtle land cover change areas, thus proving the effectiveness of the CGM module.

[0170] 3) The introduction of the HP C module on the basis of the ChangeMamba model + MSPA module (excluding HPC and SPR modules) brought an overall improvement to the performance of the ChangeMamba model. The Pre value increased by 0.87%, the Rec value increased by 0.58%, the F1 score increased by 0.71%, and the IoU and OA values ​​increased by 1.2% and 0.12% respectively. Among them, the IoU increased the most, which proved that the introduction of this module reduced the false detection and false negative detection of change detection.

[0171] 4) Introducing the SPR module into the ChangeMamba model + MSPA module (excluding HPC and SPR modules) + HPC module also improved the overall performance of the ChangeMamba model, with IoU and F1 score increasing by 1.52% and 0.9% respectively, which verifies the effectiveness of the SPR module.

[0172] 5) Based on the above, the CGM module is introduced. Compared with the previous experiment (ChangeMamba model + MSPA module + HPC module + SPR module), its Pre value decreases slightly, but its Rec value, F1 score, IoU and overall Pre value are significantly improved. This verifies that the CGM module effectively assists the ChangeMamba model in completing the detection of edge and small target regions.

[0173] (vi) Results Visualization:

[0174] To further verify the effectiveness of the embodiments of this application, the remote sensing image change detection results of the MCMamba model are visualized, and a qualitative analysis of the LEVIR-CD test set is performed to more intuitively demonstrate the performance of the MCMamba model. Visualized experimental results of the CNN architecture model FC-EF, the Transformer architecture model PA-Former, and the baseline model ChangeMamba are also presented, such as... Figure 9 As shown, Figure 9 In the image, T1 and T2 represent paired two-temporal remote sensing images, and GT represents the standard image. Figure 9 As can be seen, compared to the baseline ChangeMamba model, the MC Mamba model shows improvements in both false negatives and false positives. As indicated by the red circle in Figure (b), the MC Mamba model exhibits no false positives. For scenarios involving changes in large buildings, the MC Mamba model also demonstrates superior edge detection performance. As shown by the green and red circles in Figure (a), the MC Mamba model outperforms the baseline ChangeMamba model in edge detection. This proves the effectiveness and reliability of the MC Mamba model in handling change detection tasks in high-resolution remote sensing images. Furthermore, the change detection results in the figure show that the MC Mamba model can identify not only large change areas but also small, subtle change areas, indicating that the model has high spatial resolution and change sensitivity.

[0175] In summary, this application proposes an improved ChangeMamba model, called the MCMamba model, for change detection tasks in dual-temporal remote sensing images. It aims to address the shortcomings of existing methods, such as insufficient feature capture capability in complex backgrounds, poor robustness, and limitations in multi-scale feature fusion. Specifically, an HPC module is introduced to process the input feature map more effectively using a multi-scale approach, enhancing the change detection model's extraction of local and global information from dual-temporal remote sensing images. An SPR module is added to establish inter-channel dependencies in dual-temporal remote sensing images, improving the change detection model's sensitivity to these dependencies. Based on this, an MSPA module is designed to integrate multi-scale spatial information and cross-channel attention from dual-temporal remote sensing images, improving the change detection model's learning ability for multi-scale land cover change features in dual-temporal remote sensing images. Furthermore, a CGM module, including PCM and CGA components, is constructed to address edge and interior detection problems, enabling the change detection model to focus more on actually changed land cover areas in dual-temporal remote sensing images and ignore unchanged areas.

[0176] Experimental results on the publicly available datasets LEVIR-CD, SYSU-CD, and WHU-CD demonstrate the superior performance of the MCMamba model. In comparative experiments, compared to eight popular deep learning change detection algorithms and baseline models, the MCMamba model outperforms in multiple metrics. On the LEVIR-CD dataset, the MCMamba model achieves a Pre score 0.88% higher, a Rec score 1.06% higher, an F1 score 0.97% higher, an IoU score 1.63% higher, and an OA score 0.16% higher than the baseline model ChangeMamba. On the SYSU-CD dataset, the MCMamba model achieves the best Rec, F1, IoU, and OA scores among all models. On the WHU-CD dataset, the MCMamba model has the highest Rec, F1, and IoU scores. These experimental results strongly demonstrate the superiority of the MCMamba model in remote sensing image change detection tasks.

[0177] Ablation experiments further validated the effectiveness of each module. Stepwise additions of the MSPA module (excluding HPC and SPR modules), HPC module, SPR module, and CGM module improved model performance to varying degrees. For example, the HPC module improved the model's Pre value by 0.87%, Rec value by 0.58%, and IoU value by 1.2%; the SPR module improved the model's IoU value and F1 score by 1.52% and 0.9%, respectively. Visual analysis of the experimental results showed that the MCMamba model significantly improved in terms of false negatives and false positives compared to the baseline ChangeMamba model, exhibiting excellent edge detection performance and the ability to identify change regions at different scales, demonstrating high spatial resolution and sensitivity to change.

[0178] Furthermore, embodiments of this application also provide a remote sensing image change detection device, which includes:

[0179] The acquisition module acquires dual-temporal remote sensing images;

[0180] The remote sensing change detection module uses a change detection model to detect changes in dual-temporal remote sensing images and obtains a binary change detection map.

[0181] The change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model. The attention module is used to perform hierarchical pixel convolution operation and channel interaction processing on the input of the attention module to obtain the first attention feature map. The change guidance module is used to use the encoding features of each encoder in the ChangeMamba model to perform output guidance processing on the first attention feature map to obtain the output of the decoder.

[0182] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0183] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0184] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for detecting changes in remote sensing images, characterized in that, Includes the following steps: Acquire dual-temporal remote sensing images; The dual-temporal remote sensing image is subjected to change detection using a change detection model to obtain a binary change detection map. The change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model. The attention module is used to perform hierarchical pixel convolution operations and channel interaction processing on the input of the attention module to obtain a first attention feature map; The change guidance module is used to perform output guidance processing on the first attention feature map using the encoding features of each encoder in the ChangeMamba model, so as to obtain the output of the decoder.

2. The method according to claim 1, characterized in that, The process of performing hierarchical pixel convolution and channel interaction processing on the input of the attention module to obtain a first attention feature map includes: Hierarchical pixel convolution operation is performed on the input of the attention module to obtain a multi-scale feature map; Channel interaction processing is performed on the multi-scale feature map to obtain channel attention weights; The channel attention weights and the multi-scale feature map are fused to obtain the first attention feature map.

3. The method according to claim 2, characterized in that, The step of performing hierarchical pixel convolution operations on the input of the attention module to obtain a multi-scale feature map includes: The input to the attention module is segmented to obtain a first feature map, a second feature map, and a third feature map; Perform a convolution operation on the first feature map to obtain a first convolutional feature map; The second feature map and the first convolutional feature map are fused and convolved to obtain the second convolutional feature map. The third feature map and the second convolutional feature map are fused and convolved to obtain the third convolutional feature map; The first convolutional feature map, the second convolutional feature map, and the third convolutional feature map are fused together to obtain the multi-scale feature map.

4. The method according to claim 2, characterized in that, The step of performing channel interaction processing on the multi-scale feature map to obtain channel attention weights includes: The multi-scale feature map is subjected to global and local aggregation processing to obtain an aggregated feature map; The aggregated feature map is subjected to channel interaction processing to obtain the channel attention weights.

5. The method according to claim 4, characterized in that, The process of performing global and local aggregation on the multi-scale feature map to obtain an aggregated feature map includes: Parallel pooling operations are performed on the multi-scale feature maps to obtain global feature maps and local feature maps; The global feature map and the local feature map are fused to obtain the aggregated feature map.

6. The method according to claim 4, characterized in that, The step of performing channel interaction processing on the aggregated feature map to obtain the channel attention weights includes: The aggregated feature map is subjected to multiple pointwise convolution operations to obtain the channel attention weights.

7. The method according to claim 1, characterized in that, The step of using the encoding features of each encoder in the ChangeMamba model to perform output guidance processing on the first attention feature map to obtain the output of the decoder includes: Based on the coding features of each encoder, a guide map generation process is performed to obtain a remote sensing change guide map; The remote sensing change guidance map is used to guide the processing of the first attention feature map to obtain the output of the decoder.

8. The method according to claim 7, characterized in that, The process of generating a remote sensing change guidance map based on the coding features of each encoder includes: The coding feature pairs of each encoder are fused to obtain the coding feature map of each encoder; The dimensionality reduction process is performed on the encoding feature maps of each encoder to obtain the dimensionality reduction feature maps of each encoder; Upsampling is performed on the dimensionality-reduced feature maps of the latter three encoders to obtain the upsampled feature maps of the latter three encoders; The upsampled feature maps of the latter three encoders and the dimensionality-reduced feature map of the first encoder are fused together to generate the remote sensing change guidance map.

9. The method according to claim 7, characterized in that, The step of guiding the first attention feature map using the remote sensing change guidance map to obtain the decoder output includes: The remote sensing change guidance map and the first attention feature map are fused together to obtain a fused feature map. Perform multi-head attention operation on the fused feature map to obtain a second attention feature map; The first attention feature map and the second attention feature map are fused together to obtain the output of the decoder.

10. A remote sensing image change detection device, characterized in that, include: The acquisition module is used to acquire dual-temporal remote sensing images; The remote sensing change detection module is used to perform change detection on the dual-temporal remote sensing image using a change detection model to obtain a binary change detection map. The change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model. The attention module is used to perform hierarchical pixel convolution operations and channel interaction processing on the input of the attention module to obtain a first attention feature map; The change guidance module is used to perform output guidance processing on the first attention feature map using the encoding features of each encoder in the ChangeMamba model, so as to obtain the output of the decoder.

Images