Remote sensing image change detection method and device

Abstract

The application discloses a remote sensing image change detection method and device, which is applied to the technical field of change detection and comprises the following steps: acquiring double-time-phase remote sensing images, performing change detection on the double-time-phase remote sensing images by using a change detection model to obtain a binary change detection graph, wherein the change detection model is a neural network model obtained by adding an attention module and a change guide module behind the up-sampling module of each decoder of a ChangeMamba model; the attention module is used for performing layered pixel convolution operation and channel interaction processing on the input of the attention module to obtain a first attention feature map; and the change guide module is used for performing output guide processing on the first attention feature map by using the encoding features of each encoder in the ChangeMamba model to obtain the output of the decoder. The application can improve the change detection precision of double-time-phase remote sensing images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of change detection, and particularly relates to a remote sensing image change detection method and device. BACKGROUND

[0002] Since the birth of remote sensing technology, change detection (CD) has been a popular research direction in the field of remote sensing. The goal of change detection is to detect changes in ground objects from multi-temporal remote sensing images obtained at different times, which can be widely applied to land cover change analysis, urban expansion research, disaster response, geographic information system updating and ecological monitoring, etc.

[0003] High-resolution remote sensing images have become one of the most popular data sources in the field of remote sensing application and research, which provides detailed texture and geometric structure information of ground features, making it possible to perform more fine-grained change detection tasks. However, with the increase of spatial resolution, the noise interference in remote sensing images becomes more serious, and the traditional pixel-based change detection method is difficult to achieve satisfactory results.

[0004] The emergence of deep learning brings new models and paradigms to change detection, significantly improving the efficiency and accuracy of change detection. Since Daudt et al. introduced the fully convolutional network into the field of change detection, methods based on convolutional neural networks (CNN) have been dominant. Although these methods have achieved good results, due to the limited receptive field of the CNN architecture, it is difficult to capture long-distance dependencies between different pixels, which makes it still insufficient when dealing with remote sensing images with different temporal and spatial resolutions.

[0005] The Transformer architecture provides a new way to solve the above-mentioned deficiencies of the CNN architecture. Through the stacked self-attention mechanism, it can learn the relationship between pixels in the entire image. At present, more and more change detection architectures use Transformer as an encoder to extract representative and robust features of ground objects, and at the same time use it in the decoder to capture the spatio-temporal relationship between multi-temporal features. However, the computational complexity of the self-attention mechanism in the Transformer architecture is proportional to the square of the image size, which makes the computational overhead of the Transformer architecture expensive, which is very disadvantageous for high-density detection tasks in large-scale remote sensing image datasets. In view of this problem, some related technologies have proposed some solutions, such as adjusting the size or step of the calculation window. Although these methods improve the attention efficiency, they all come at the cost of limiting the receptive field.

[0006] State space model (SSM), especially structured state space sequence model (S4), shows outstanding performance in continuous long sequence data analysis and has the good characteristic that the computational complexity expands linearly with the sequence length. Mamba architecture further improves the S4 model by introducing a selective mechanism that allows the model to select relevant information in a way that depends on the input. Combined with a hardware-aware algorithm, the Mamba architecture outperforms the Transformer architecture in some downstream tasks. Recently, the Mamba architecture has been extended to image data and has achieved satisfactory results in some visual tasks.

[0007] Although the Mamba architecture has made remarkable achievements in the field of visual tasks, the ChangeMamba model, which applies the Mamba architecture to the remote sensing image change detection task, also shows certain advantages. However, the ChangeMamba model mainly relies on the 2D cross-scan mechanism of the Vmamba model to interact between the features of the ground object changes, which makes it difficult to deal with complex scenarios such as small target changes and coexistence of multi-scale changes. Moreover, the ChangeMamba model uses a simple concatenation or direct cross-scan mechanism to fuse the feature information of the double-time-phase at the input of the decoder, which leads to insufficient sensitivity to subtle changes or occluded areas. These factors will lead to low performance of the ChangeMamba model in the remote sensing image change detection task. SUMMARY

[0008] The embodiments of the present application provide a remote sensing image change detection method and device, which are used to improve the change detection accuracy of double-time-phase remote sensing images.

[0009] In one aspect, the embodiments of the present application provide a remote sensing image change detection method, which comprises the following steps:

[0010] obtaining double-time-phase remote sensing images;

[0011] performing change detection on the double-time-phase remote sensing images by using a change detection model to obtain a binary change detection map;

[0012] The change detection model is a neural network model obtained by adding an attention module and a change guide module behind the up-sampling module of each decoder of the ChangeMamba model.

[0013] The attention module is used to perform hierarchical pixel convolution operation and channel interaction processing on the input of the attention module to obtain a first attention feature map.

[0014] The change guiding module is configured to perform output guiding processing on the first attention feature map by using the encoded features of each encoder in the ChangeMamba model, to obtain the output of the decoder.

[0015] In another aspect, the embodiments of the present application provide a remote sensing image change detection device, comprising:

[0016] The acquisition module is configured to acquire double-time-phase remote sensing images.

[0017] The remote sensing change detection module is configured to perform change detection on the double-time-phase remote sensing images by using a change detection model, to obtain a binary change detection map.

[0018] The change detection model is a neural network model obtained by adding an attention module and a change guiding module behind the up-sampling module of each decoder of the ChangeMamba model.

[0019] The attention module is configured to perform layered pixel convolution operation and channel interaction processing on the input of the attention module, to obtain a first attention feature map.

[0020] The change guiding module is configured to perform output guiding processing on the first attention feature map by using the encoded features of each encoder in the ChangeMamba model, to obtain the output of the decoder.

[0021] According to the remote sensing image change detection method and device provided by the embodiments of the present application, double-time-phase remote sensing images are acquired, and change detection is performed on the double-time-phase remote sensing images by using a change detection model, to obtain a binary change detection map. The change detection model is a neural network model obtained by adding an attention module and a change guiding module behind the up-sampling module of each decoder of the ChangeMamba model. The attention module is configured to perform layered pixel convolution operation and channel interaction processing on the input of the attention module, to obtain a first attention feature map. The change guiding module is configured to perform output guiding processing on the first attention feature map by using the encoded features of each encoder in the ChangeMamba model, to obtain the output of the decoder. According to the technical solution of the embodiments of the present application, the global spatial context and the spatio-temporal relationship of the double-time-phase remote sensing images are modeled while the subtle, multi-scale feature of ground object change in the double-time-phase remote sensing images and the complex dependence relationship between channels in the double-time-phase remote sensing images are comprehensively captured. The precise fusion of multi-scale ground object change features and the effective modeling of the fine channel relationship are realized, so as to improve the change detection precision of the double-time-phase remote sensing images, especially in complex backgrounds such as small target change and multi-scale change coexistence.

[0022] Other features and advantages of the present application will be set forth in the following specification, and in part will be apparent from the description, or can be learned by practice of the application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1A is the overall structure diagram of the existing ChangeMamba model;

[0024] Figure 1B is the structure diagram of the encoder in the ChangeMamba model;

[0025] Figure 1C is the structure diagram of the decoder in the ChangeMamba model;

[0026] Figure 2 is the structure diagram of the MCMamba model provided by the present application;

[0027] Figure 3 is the structure diagram of the MSPA module provided by the present application;

[0028] Figure 4 is the structure diagram of the HPC module provided by the present application;

[0029] Figure 5 is the structure diagram of the SPR module provided by the present application;

[0030] Figure 6 is the structure diagram of the CGM module provided by the present application;

[0031] Figure 7 is the structure diagram of the PCM component provided by the present application;

[0032] Figure 8 is the structure diagram of the CGA component provided by the present application;

[0033] Figure 9 is the effect comparison diagram provided by the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0035] The present application will be further described below in combination with the drawings and specific embodiments of the present application. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those skilled in the art without making creative efforts fall within the scope of protection of the present application.

[0036] In the following description, reference is made to "some embodiments," which describe only a subset of all possible embodiments. It is understood that "some embodiments" can be a same subset or different subset of all possible embodiments and can be combined with each other unless otherwise indicated.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to be limiting of this application.

[0038] ChangeMamba model is the first remote sensing image change detection framework that applies Mamba architecture to change detection tasks. The core design aims to overcome the problems of limited receptive field of CNN architecture and high computational complexity of Transformer architecture. Specifically, the model combines the linear sequence modeling capability of Mamba architecture with the characteristics of visual tasks to achieve efficient global context learning and spatio-temporal relationship modeling of multi-temporal remote sensing images.

[0039] With reference to Figure 1A , T1 and T2 are pairs of dual temporal remote sensing images, Encoder is an encoder structure, Stage1 to tage4 of Encoder respectively refer to the first encoder to the fourth encoder, Weight Sharing represents that the weights of each encoder are shared; Decoder is a decoder structure, Stage1 to Stage4 of Decoder respectively refer to the first decoder to the fourth decoder, Binary Change map represents a binary change detection map, ChangeMamba model adopts an "encoder-decoder" structure, and the key module is a weight sharing twin encoder structure based on a variational state space model (VSSM) architecture and a three spatio-temporal interaction transformation decoder structure based on a Mamba architecture.

[0040] With reference to Figure 1B, H represents the height of the input image, W represents the width of the input image, C0 represents the number of channels of the input image, C1 to C4 respectively refer to the number of channels of the feature maps output by the first encoder to the fourth encoder, the weight sharing twin encoder structure based on the VSSM architecture takes VMamba as the backbone network, which has four encoders, each of which first downsamples the input data, then fully models the spatial context information using multiple VSS blocks, and then outputs the features of this stage. Since the input data is a pair of dual-phase remote sensing images, the features output by each encoder are also paired and referred to as an encoding feature pair, which contains two feature maps corresponding to two time phases respectively.

[0041] With reference to Figure 1C , H represents the height of the input image, W represents the width of the input image, C0 represents the number of channels of the input image, C1 to C4 respectively refer to the number of channels of the feature maps output by the fourth decoder to the first decoder, Stage1 output to Stage4 output respectively refer to the feature maps output by the first decoder to the fourth decoder, Binary Change map represents the binary change detection map, the three spatio-temporal interaction transformation decoder structure based on the Mamba architecture is composed of four decoders, each of which models the spatio-temporal relationship of dual-phase features through a three spatio-temporal state space module (STSS Block), restores the image resolution through an upsampling module (2x Upsampling), and introduces a fusion module (Fusion Block) in the last three decoders to fuse low-level semantic features and high-level semantic low-resolution features. Among them, in the STSS module, first, the dual-phase image features are modeled using three spatio-temporal relationship modeling mechanisms (Spatio-temporal token generator), further extracting features with robustness and representativeness, then inputting the features into three VSS blocks to obtain features that fully exploit the global spatio-temporal correlation of dual-phase features, and finally concatenating (Concat) the three VSS blocks and obtaining the output of the STSS module through a convolution layer (Conv). In the fusion module, the low-level semantic features output by the SSTS block of the current decoder are concatenated with the high-level semantic low-resolution features output by the previous decoder, and 1x1 convolution (1x1Conv), 3x3 deep convolution (3x3Conv) and other processing are used to refine the features and suppress noise.

[0042] Although the ChangeMamba model can capture global context information and spatio-temporal information, it has deficiencies in the fusion and fine modeling of multi-scale feature changes. This is because it mainly relies on the 2D cross-scan mechanism of the Vmamba model for the interaction between feature change characteristics, which cannot model the semantic relevance between feature change characteristics at different scales, which will affect the change detection accuracy in complex scenarios (such as small target changes and multi-scale changes coexisting). In addition, the ChangeMamba model uses a simple splicing or direct cross-scan mechanism to fuse the feature information of the two time phases at the input of the decoder, which lacks flexible spatio-temporal feature selection capability, which will introduce redundant information or suppress key feature changes, resulting in insufficient sensitivity to subtle changes or occlusion areas.

[0043] Specifically, on the one hand, in complex scenarios such as small target changes and multi-scale changes coexisting, the ChangeMamba model has insufficient ability to capture subtle and multi-scale feature changes. Because of the rich and diverse types of features in remote sensing images, and the large differences in scale of different feature changes, the ChangeMamba model has difficulty in comprehensively and accurately extracting these feature characteristics, resulting in some feature changes being missed, thereby affecting the accuracy of remote sensing image change detection.

[0044] On the other hand, the radiation difference, light change and complex terrain and topography in different time phases of remote sensing images will cause the feature characteristics of remote sensing images to change greatly, which makes the ChangeMamba model prone to be misled when modeling the global spatial context and spatio-temporal relationship of the two time phases of remote sensing images, thereby reducing its change detection performance.

[0045] On the other hand, the ChangeMamba model has deficiencies in the fusion and fine modeling of multi-scale feature changes. Feature changes at different scales often cannot be fully and effectively integrated, which limits the ChangeMamba model's ability to understand and analyze complex scenarios.

[0046] Therefore, in view of the deficiencies of the ChangeMamba model in the remote sensing image change detection task, the embodiments of the present application provide a remote sensing image change detection method and device, which improves the ChangeMamba model for the remote sensing image change detection task to obtain a change detection model, called MCMamba model. The model can effectively model the global context and spatio-temporal relationship in the two time phases of images, thereby achieving efficient and high-robustness remote sensing image change detection.

[0047] A remote sensing image change detection method provided by the present application will be described in detail below with reference to the accompanying drawings.

[0048] This application provides a remote sensing image change detection method, which can be applied to a terminal, a server, or software running on either a terminal or a server. The terminal can be a tablet, laptop, desktop computer, etc., but is not limited to these. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Furthermore, the server can be a node server in a blockchain network, but is not limited to these. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0049] Reference Figure 2 The remote sensing image change detection method may include the following steps S101-S102:

[0050] S101, acquire dual-temporal remote sensing images;

[0051] S102, change detection is performed on the dual-temporal remote sensing image using a change detection model to obtain a binary change detection map; wherein, the change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model; the attention module is used to perform hierarchical pixel convolution operation and channel interaction processing on the input of the attention module to obtain a first attention feature map; the change guidance module is used to use the encoding features of each encoder in the ChangeMamba model to perform output guidance processing on the first attention feature map to obtain the output of the decoder.

[0052] In this embodiment of the application, firstly, pairs of dual-temporal remote sensing images are acquired, i.e., as shown below. Figure 2 Images T1 and T2 are shown in the diagram. The paired bi-temporal remote sensing images are then input into a change detection model for change detection. The change detection model is a neural network model improved from the existing ChangeMamba model. It is trained using multiple pre-set bi-temporal remote sensing image samples and the corresponding label information for each sample. The label information refers to the binary change detection map corresponding to the bi-temporal remote sensing image. The change detection model can then detect the desired binary change detection map from the paired bi-temporal remote sensing images.

[0053] Specifically, the embodiments of the present application mainly improve the decoder of the ChangeMamba model, as shown in the following. Figure 1C In the existing ChangeMamba model, the decoder structure is provided with four decoders, the first decoder includes an STSS module and an upsampling module (including two upsampling layers), and the last three decoders each include an STSS module, a fusion module and an upsampling module. Accordingly, as shown in the following, Figure 2 The embodiments of the present application add a multi-scale spatial pyramid attention (MSPA) module and a change guide module (CGM) behind the upsampling module (2xUpsampling) of each decoder, the MSPA module is used to perform hierarchical pixel convolution operation and channel interaction processing, and the CGM module is used to perform output guide operation.

[0054] In the change detection model, the encoder structure of the existing ChangeMamba model is retained, and the data flow is as follows: first, the encoder structure is used to perform feature extraction operation on the input dual-time-phase remote sensing image, thereby obtaining four groups of feature maps of different scales corresponding to two time phases, i.e. eight feature maps corresponding to each other, which are called encoding feature pairs in the embodiments of the present application. In other words, the encoding feature pair includes two feature maps of the same scale corresponding to two time phases. Then, the four encoding feature pairs are input into the corresponding decoder.

[0055] In the first decoder Scd1, the encoded feature pairs output by the first encoder are first processed by the STSS module and input into the MSPA module after processing by the up-sampling module (2xUpsampling), both of which are described in the existing ChangeMamba model. Then, the output of the up-sampling module is subjected to hierarchical pixel convolution operation and channel interaction processing by the MSPA module. Finally, the output of the MSPA module is subjected to output guiding operation by the CGM module to obtain the output of the first decoder. In the last three decoders Scd2 to Scd4, the encoded feature pairs output by the corresponding encoder are first processed by the STSS module, and then the output of the last decoder and the output of the STSS module are fused by the fusion module (Fusion Block) and input into the MSPA module after processing by the up-sampling module (2xUpsampling), the fusion module is also described in the existing ChangeMamba model. Then, the output of the up-sampling module is subjected to hierarchical pixel convolution operation and channel interaction processing by the MSPA module. Finally, the output of the MSPA module is subjected to output guiding operation by the CGM module to obtain the output of the decoder. The output of the fourth decoder is the change detection result of the remote sensing image.

[0056] As can be seen, the embodiments of the present application improve the existing ChangeMamba model to obtain a change detection model. Specifically, the MSPA module and the CGM module are added behind the up-sampling module of each decoder of the ChangeMamba model. The MSPA module is used to perform hierarchical pixel convolution operation and channel interaction processing. The hierarchical pixel convolution operation can extract multi-scale spatial information in the dual-time remote sensing image in a more fine-grained manner using hierarchical residual connection, accurately locating subtle changes in ground features in the dual-time remote sensing image. The channel interaction processing can model the inter-channel relationship information in the dual-time remote sensing image in an adaptive combination mechanism and capture long-distance channel dependency relationships between different ground feature change features. In this way, the MSPA module can effectively integrate multi-scale spatial information and cross-channel attention, prompting the change detection model to fully exploit the structural information in the dual-time remote sensing image and the complex inter-channel dependency relationship, thereby capturing more and more refined ground feature change features and improving the detection capability of subtle and multi-scale ground feature change features. The CGM module is used to perform output guiding operation, which can accurately capture ground feature changes between dual-time remote sensing images by combining deep feature semantic information and self-attention calculation, thereby solving the problems of inaccurate detection at the edge and internal void detection.

[0057] In summary, the embodiments of the present application can capture the subtle and multi-scale features of ground objects in the two-phase remote sensing images, and the complex inter-channel relationship in the two-phase remote sensing images. At the same time, the global spatial context and spatio-temporal relationship of the two-phase remote sensing images are modeled, the multi-scale ground object change features are precisely fused, and the effective modeling of the fine channel relationship is realized, thereby improving the change detection accuracy of the two-phase remote sensing images, especially in complex backgrounds such as small target changes and multi-scale changes.

[0058] The MSPA module will be further described below.

[0059] In some embodiments, referring to Figure 3 The input of the attention module is subjected to hierarchical pixel convolution operation and channel interaction processing to obtain the first attention feature map, which can include:

[0060] The input of the attention module is subjected to hierarchical pixel convolution operation to obtain a multi-scale feature map;

[0061] The multi-scale feature map is subjected to channel interaction processing to obtain a channel attention weight;

[0062] The channel attention weight and the multi-scale feature map are fused to obtain the first attention feature map.

[0063] In the present embodiment, the attention mechanism gradually becomes a necessary means to enhance the representation ability of the convolutional neural network. Although progress has been made in the study of attention mechanisms, there are still some problems to be solved. In the existing ChangeMamba model, the modeling of multi-scale feature representation and structural information is ignored, which is essential for generating more discriminative attention maps. Therefore, the present embodiment proposes a novel, low-overhead, high-performance attention mechanism with strong generalization ability, which is integrated in the above-mentioned MSPA module and can be used to solve the limitations of other attention methods.

[0064] Referring to Figure 3 For the key components of the above-mentioned MSPA module, a hierarchical pixel convolution (HPC) module is cited, which can extract multi-scale spatial information in a more fine-grained manner using hierarchical residual connections; a spatial pyramid recalibration (SPR) module, which can model inter-channel relationship information in an adaptive combination mechanism, while using a Softmax operation to establish long-distance channel dependency.

[0065] First, the HPC module is used to extract multi-scale spatial information from the input feature map (Input Feature) F ∈ RH×W×C Processing is performed at multiple scales to obtain enhanced multi-scale feature maps. It contains the first convolutional feature map Second convolution feature map and the third convolution feature map

[0066] Secondly, multi-scale feature maps The input is fed into the SPR module to learn channel attention, thereby obtaining the channel attention weights V = [V1, V2, V3] ∈ R. 1×1×C It includes a first attention weight V1, a second attention weight V2, and a third attention weight V3, where the first attention weight V1 corresponds to the first convolutional feature map. The second attention weight V2 corresponds to the second convolutional feature map. The third attention weight V3 corresponds to the third convolutional feature map.

[0067] Subsequently, the cross-channel soft attention mechanism allows for the adaptive selection of different spatial dimensions while facilitating interaction between local and global channel attention. Therefore, the Softmax function is applied to the channel attention weights V, generating recalibrated channel attention weights A∈R. C×1×1 This is called calibrating attention weights. In this process, the i-th convolutional feature map... The corresponding calibration channel attention weight A i It can be expressed as the following formula (1):

[0068]

[0069] In equation (1), V i For the i-th convolutional feature map The corresponding channel attention weights; s is the number of convolutional feature maps.

[0070] Through the above operations, long-distance channel dependencies between the enhanced multi-scale feature maps were established, and these dependencies were then analyzed using each convolutional feature map. Calibration channel attention weight A i By concatenating (Concat) i = 1, 2, 3, we can obtain the overall recalibrated channel attention weights A = Concat([A1, A2, A3]).

[0071] Finally, for each convolutional feature map Convolutional feature maps Its corresponding calibration channel attention weight A i Perform element-wise multiplication (i.e.) This yields a refined set of feature maps, i.e., convolutional feature maps. corresponding attention feature map subsequently all convolution feature maps corresponding attention feature map concatenation (Concat) to obtain the first attention feature map (Refined Feature)

[0072] In some embodiments, referring to Figure 4 The above-mentioned input of the attention module is subjected to hierarchical pixel convolution operation to obtain a multi-scale feature map, which can include:

[0073] The input of the attention module is subjected to segmentation processing to obtain a first feature map, a second feature map and a third feature map;

[0074] The first feature map is subjected to convolution operation to obtain a first convolution feature map;

[0075] The second feature map and the first convolution feature map are subjected to fusion and convolution operation to obtain a second convolution feature map;

[0076] The third feature map and the second convolution feature map are subjected to fusion and convolution operation to obtain a third convolution feature map;

[0077] The first convolution feature map, the second convolution feature map and the third convolution feature map are subjected to fusion processing to obtain a multi-scale feature map.

[0078] In this embodiment, the HPC module is composed of three operations, namely, channel splitting (Split), convolution (Conv) and fusion (Concat). First, channel splitting is performed. For a given input feature map F∈R H×W×C , the feature map is uniformly split into 3 feature map subsets F i ∈R H×W×ω , i = 1, 2, 3, each feature map subset has the same spatial shape and the same channel dimension. For ease of understanding, the three feature map subsets are denoted as a first feature map F1, a second feature map F2 and a third feature map F3.

[0079] Then, convolution is performed. For each feature map subset F i , i = 1, 2, 3, it is configured with a corresponding convolution operator (convolution operation) composed of a 3x3 standard convolution, batch normalization and an activation function, and the feature map subset processed by the convolution operator is denoted as For ease of understanding, the three feature map subsets processed by the convolution operator are denoted as a first convolution feature map a second convolution feature map and a third convolution feature map In hierarchical convolution operations, different convolution operators are connected in a hierarchical residual manner to increase the number of scales that the output features can represent. Specifically, the first feature map F1 can be used to obtain the first convolutional feature map through the first set of convolution operators. The first convolutional feature map Add the second feature map F2 (i.e.) Then, the second set of convolution operators is used to obtain the second convolution feature map. The second convolutional feature map The third convolutional feature map is obtained by adding it to the third feature map F3 and then passing it through the third set of convolution operators. This process can be expressed as the following formula (2):

[0080]

[0081] In equation (2), Conv i (·) denotes the i-th group of convolution operators (convolution operation).

[0082] Finally, the first convolutional feature map is concatenated along the channel dimension. Second convolution feature map and the third convolution feature map Obtain the entire enhanced multi-scale feature map

[0083] Therefore, it is evident that in the HPC module, each convolutional operation extracts crucial information on ground feature changes from feature maps at different scales, and each 3×3 convolution results in a larger receptive field for the output. Consequently, the feature maps output by the HPC module contain different combinations and numbers of receptive field scales. Outputs with smaller receptive fields can capture more details of ground feature changes in remote sensing image scenes, which is crucial for detecting changes in small targets. Outputs with larger receptive fields can focus on larger ground targets, which helps improve the global modeling capability of the change detection model. In short, the HPC module processes the input feature maps with a more efficient multi-scale approach, thereby promoting the extraction of local and global information from dual-temporal remote sensing images by the change detection model. It also uses feature splitting and channel number control to limit the number of parameters and computational cost of the change detection model. A larger ω corresponds to a stronger multi-scale feature representation capability, meaning that the change detection model can obtain feature maps with richer information on ground feature changes.

[0084] In some implementations, refer to Figure 5 The above-mentioned channel interaction processing of multi-scale feature maps to obtain channel attention weights can include:

[0085] Global and local aggregation processing is performed on the multi-scale feature maps to obtain aggregated feature maps;

[0086] The channel attention weight is obtained by performing channel interaction processing on the aggregated feature map.

[0087] In the embodiment, since the HPC module does not involve modeling of the inter-channel in the dual-temporal remote sensing image, the SPR module is introduced behind the HPC module to establish the interdependence between the channels in the dual-temporal remote sensing image, which is conducive to improving the sensitivity of the change detection model to the interdependence between the channels in the dual-temporal remote sensing image, thereby extracting more feature information conducive to the remote sensing image change detection task. Therefore, it is particularly important to extract the inter-channel dependence information from the multi-scale feature map output by the HPC module.

[0088] Accordingly, the SPR module mainly includes two parts, one is a spatial pyramid aggregation block (SPA block) that aggregates global and local context features in the dual-temporal remote sensing image by adaptive fusion, and the other is a channel-wise interaction block (CI block) that is composed of two point-wise convolutions (PW convs) and a sigmoid activation function to extract the inter-channel relationship in the dual-temporal remote sensing image.

[0089] Here, the global and local aggregation processing of the multi-scale feature map is performed by the SPA block to obtain an aggregated feature map, which can effectively aggregate the global and local context information in the dual-temporal remote sensing image, thereby realizing structure regularization while exploring structure information. Since the channel features generated by the SPA block cannot be directly used to learn the relationship between the channels in the dual-temporal remote sensing image, the CI block is introduced to perform channel interaction processing on the aggregated feature map to obtain a channel attention weight, thereby further processing to generate a corresponding channel attention weight.

[0090] In some embodiments, with reference to Figure 5 The global and local aggregation processing of the multi-scale feature map to obtain the aggregated feature map can include:

[0091] Parallel pooling operations are performed on the multi-scale feature map to obtain a global feature map and a local feature map;

[0092] The global feature map and the local feature map are fused to obtain the aggregated feature map.

[0093] In this embodiment, the traditional feature extraction module often only uses a single scale of pooling operation. If only global average pooling is used, although the global semantics of the dual temporal remote sensing images can be obtained, a large amount of local detail information in the dual temporal remote sensing images will be lost, which makes it difficult for the change detection model to locate the subtle ground feature changes in the dual temporal images; and simply relying on local pooling will make the change detection model lack understanding of the overall scene and be easily disturbed by local noise. For example, in some early change detection algorithms, single pooling operation makes the model either unable to accurately identify the change area when facing complex scenes or misjudges the normal local fluctuations as changes. In addition, the traditional method rarely reasonably fuses the pooling results of different scales, and cannot fully play the synergistic effect of global and local information, while the SPA block uses two sizes of average pooling to aggregate global and local context information, so as to realize structure regularization while exploring structure information, effectively solving the above problems.

[0094] For each convolutional feature map The specific data flow is as follows:

[0095] First, the input feature map is mapped to two size average pooling channels for processing. The first channel is the traditional 1x1 global average pooling (Global Average Pooling), which obtains a global feature map F'1eR 1 ×1×ω to obtain global channel information; the second channel is 2x2 local average pooling (Local Average Pooling), which obtains a local feature map F2' eR 2×2×ω to obtain more rich local feature representation and structure information.

[0096] Here, based on the balanced extraction requirement of global and local features, while considering the calculation efficiency and model complexity, the SPA block selects 1x1 pooling kernel and 2x2 pooling kernel. The 1x1 pooling kernel can extract the global context information in the dual temporal remote sensing images, and the 2x2 pooling kernel can extract the local structure information in the dual temporal remote sensing images. By weighted sum fusion of the two pooling results, the change detection model can simultaneously perceive the global semantic dependence and local structure details in the dual temporal remote sensing images, which significantly improves the expression ability of remote sensing change features. Of course, in other embodiments, other size pooling kernels can be selected, but the pros and cons need to be weighed. Larger pooling kernels are suitable for modeling long-distance dependence space tasks, such as large target detection objects, but may introduce too much irrelevant background information, which will blur the local details, and the computational complexity will increase accordingly.

[0097] In this way, the whole feature map is compressed by 1x1 global average pooling, the global channel information in the dual-time remote sensing image can be obtained, and the overall semantic and structural trend of the dual-time remote sensing image can be grasped. For example, in remote sensing image change detection, the overall category of the scene (such as city, farmland) can be quickly determined; and through 2x2 local average pooling, the local detail features in the dual-time remote sensing image can be focused on capturing, and the texture, edge and other information in the dual-time remote sensing image can be reserved, for example, the outline of the building, the direction of the road and other details can be identified, which provides support for subsequent positioning of subtle changes.

[0098] Then, the spatial shape of the global feature map F1' is up-sampled to the size of the local feature map F2', to obtain an up-sampled global feature map F1'', and the up-sampled global feature map F1'' and the local feature map F2' are fused in a weighted sum manner to obtain an output feature map F 1+2 ∈R 2×2×ω .

[0099] Here, the feature map obtained by 1x1 pooling is up-sampled to the same size as the 2x2 pooling feature map and then weighted summed. Through the weighted sum, the change detection model can dynamically adjust the proportion of global information and local information in the dual-time remote sensing image according to the task requirements, so that the fused feature map can not only reflect the overall semantics of the dual-time remote sensing image, but also contain rich local feature change details, thereby enhancing the adaptability of the change detection model to complex scenes. This fusion method realizes the complementation of global and local information, and avoids the information loss problem caused by single-scale pooling.

[0100] Finally, the output feature map F 1+2 ∈R 2×2×ω through spatial scaling (Resize) to a one-dimensional vector with a vector length of 4ω. The one-dimensional vector is the aggregated feature map Z i .

[0101] Here, the fused feature map is scaled to a one-dimensional vector. This step not only realizes structural regularization, but also further integrates feature information, so that the output one-dimensional vector can be used as an effective input for subsequent CI block processing, thereby laying a foundation for modeling inter-channel relationships.

[0102] In some embodiments, with reference to Figure 5 The above processing of the aggregated feature map to obtain the channel attention weight can include:

[0103] The aggregated feature map is subjected to multiple point-by-point convolution operations to obtain the channel attention weight.

[0104] In this embodiment, the CI block is further processed on the basis of the SPA block through two pointwise convolutions and a Sigmoid activation function to generate corresponding channel attention weights, and for each aggregated feature map Z i , i = 1, 2, 3, and the specific data flow is as follows: first, a first pointwise convolution operation is performed on the input aggregated feature map Z i , the first pointwise convolution operation including a 1x1 pointwise convolution and a ReLU activation function, and then a second pointwise convolution operation is performed on the result of the first pointwise convolution operation, the second pointwise convolution operation including a 1x1 pointwise convolution and a Sigmoid activation function, so that the channel attention weight V i is obtained, as shown in the following formula (3):

[0105]

[0106] In formula (3), represents a 1x1 pointwise convolution with a parameter matrix of (4ω, ω / r); represents a 1x1 pointwise convolution with a parameter matrix of (ω / r, ω); and σ represents a Sigmoid activation function.

[0107] Here, the one-dimensional vector output by the SPA block is processed through two 1x1 pointwise convolutions, the first 1x1 pointwise convolution serving to reduce the dimension, reducing the number of channels of the feature map, reducing the computational complexity of the change detection model, and extracting the preliminary relationship between channels in the dual-time-phase remote sensing image; the second 1x1 pointwise convolution restores the number of channels to the original level, further refines the dependency relationship between channels in the dual-time-phase remote sensing image, so that each channel can fully absorb the information of other channels, and the interaction between channels in the dual-time-phase remote sensing image is enhanced. After the pointwise convolution, the output is mapped between 0 and 1 through the Sigmoid activation function to generate the channel attention weight. These weights reflect the importance of each channel for the change detection task, and the change detection model can adaptively adjust the features of different channels according to the weights, enhance the feature expression of key channels, and suppress the interference of irrelevant channels, so as to extract more favorable feature information for change detection.

[0108] In this way, the CI block realizes deep modeling of the inter-channel relationship through simple and effective pointwise convolution and Sigmoid activation function while ensuring computational efficiency, and can more accurately capture channel information related to the change detection task compared with traditional methods, thereby improving the detection performance of the model.

[0109] As can be seen from the above, the MSPA module has the following technical effects:

[0110] Firstly, the input feature maps are processed by the HPC module, which extracts the features of ground objects from multiple scales by hierarchical residual connection. In the context of dual-temporal remote sensing image change detection, small-scale ground object change features can capture detailed information such as building outlines and road textures, while large-scale ground object change features can focus on macro information such as urban layout and topography. Through this hierarchical extraction method, the multi-scale feature maps obtained cover a wealth of spatial information, providing a more comprehensive feature basis for subsequent processing. Compared to single-scale feature extraction, it can more accurately locate the subtle ground object changes in dual-temporal remote sensing images.

[0111] Secondly, the feature maps output by the HPC module are input into the SPR module. In the SPR module, the SPA block processes the global average pooling and the local average pooling in parallel through 1x1 global average pooling and 2x2 local average pooling, respectively, to obtain the global channel information and the local structure information in the dual-temporal remote sensing images, and then realizes the fusion of the two through weighted summation; the CI block uses two point-by-point convolutions and a Sigmoid activation function to deeply explore the interrelation between channels and generate channel attention weights. This allows the change detection model to clearly determine the importance of each channel in the change detection task, such as when identifying building changes, enhancing the weight of channels related to building features and suppressing irrelevant background channel interference, improving the relevance and effectiveness of ground object change features.

[0112] Thirdly, the channel attention weights generated by the SPR module are applied with the Softmax function to realize the cross-channel soft attention mechanism. This mechanism allows the change detection model to adaptively select different spatial sizes, promotes the interaction between local and global channel attention, and establishes the long-distance channel dependency between enhanced multi-scale feature maps. Taking large-area land use change detection as an example, the change detection model can integrate ground object change features of different regions and different scales through this mechanism, avoiding misjudgment of the actual change due to local ground object change features, and thus improving the accuracy of change detection in complex scenes.

[0113] Finally, the enhanced multi-scale feature maps are multiplied by the calculated channel attention weights to obtain the refined feature map subset and the final output. This operation adjusts the features based on channel importance, highlights key ground object change features, and suppresses redundant and useless feature information, making the ground object change features extracted by the change detection model more distinctive and further improving the performance of dual-temporal remote sensing image change detection.

[0114] The existing attention mechanism has deficiencies in multi-scale feature processing and channel relationship modeling. Some methods only focus on single-scale features, which cannot balance macro and micro information, leading to difficulty in accurately identifying changes in complex scenes. In terms of channel relationship modeling, simple channel addition, averaging operation or weight generation based on statistical information cannot fully explore the complex dependence relationship between channels and dynamically adjust the importance of channels. Based on the above description, the MSPA module is used to extract multi-scale features from the fused feature layer, fully explore the structural information and long-distance channel dependence relationship of the dual-time remote sensing image, which can effectively integrate the multi-scale spatial information and cross-channel attention in the dual-time remote sensing image. This is conducive to capturing more refined land cover change features and improving the learning ability of the change detection model for multi-scale land cover change features, thereby improving the performance of the change detection model in the dual-time remote sensing image change detection task.

[0115] The CGM module will be further described below.

[0116] In some embodiments, referring to Figure 6 , Encoder represents the output of the encoder, Weight Sharing represents weight sharing, Decoder Stage1 to Decoder Stage4 respectively refer to the output of the first decoder MSPA module to the output of the fourth decoder MSPA module; the above output guiding processing of the first attention feature map using the encoding features of each encoder in the ChangeMamba model to obtain the output of the decoder can include:

[0117] Based on the encoding feature pair of each encoder, a guide map generation process is performed to obtain a remote sensing change guide map;

[0118] The remote sensing change guide map is used to guide the processing of the first attention feature map to obtain the output of the decoder.

[0119] In the existing ChangeMamba model, the multi-scale feature map obtained by the twin Mamba encoder needs to be further processed by the decoder to obtain the change detection map. The decoder simply splices the two groups of feature maps in the channel dimension, and this direct fusion method cannot well detect the changes between the dual-time images, especially the problems of inaccurate detection at the edges and internal voids. To this end, the CGM module is introduced in the present application, which can well solve the detection problems of edges and interiors by combining deep feature semantic information and self-attention calculation.

[0120] The CGM module is composed of two components, namely a change map production (PCM) component and a change guide attention (CGA) component. The PCM component is used to generate a change guide map based on the encoded feature pairs of each encoder, thereby obtaining a remote sensing change guide map. In this way, the encoded feature pairs of each encoder are deep features of ground objects with rich semantic information, based on which the change guide map is generated and used as prior information for the subsequent CGA component, thereby effectively improving the expression ability of the ground object features. The CGA component guides the first attention feature map based on the remote sensing change guide map as prior information to obtain the output of the decoder, so that the change detection model can pay more attention to the ground object area that has changed between the two time-phase remote sensing images, and ignore the ground object area that has not changed, thereby improving the change detection accuracy.

[0121] In some embodiments, with reference to Figure 7 The above-mentioned generation of a change guide map based on the encoded feature pairs of each encoder can include:

[0122] fusing the encoded feature pairs of each encoder to obtain the encoded feature map of each encoder;

[0123] dimensionally reducing the encoded feature map of each encoder to obtain the dimensionally reduced feature map of each encoder;

[0124] up-sampling the dimensionally reduced feature map of the last three encoders to obtain the up-sampled feature map of the last three encoders;

[0125] fusing the up-sampled feature map of the last three encoders and the dimensionally reduced feature map of the first encoder to generate a remote sensing change guide map.

[0126] In the PCM component, first, for each encoder, the encoded feature pairs of the encoder are concatenated (Concat) in the channel dimension to obtain the encoded feature map of the encoder. For ease of understanding, the encoded feature maps of the first to fourth encoders can be represented as layer_1, layer_2, layer_3, and layer_4. Then, the encoded feature map of the encoder is dimensionally reduced (Reduce) to reduce the dimension to one-half of the original, thereby obtaining the dimensionally reduced feature map of each encoder. The dimension reduction operation can be implemented by a 1x1 convolution (Conv 1x1), but is not limited thereto. For ease of understanding, the dimensionally reduced feature maps of the first to fourth encoders can be represented as layer_1_2, layer_2_2, layer_3_2, and layer_4_2.

[0127] Then, the first encoder is defined as a shallow encoder, and the second to fourth encoders are defined as deep encoders. For each deep encoder, the dimension-reduced feature map of the deep encoder is upsampled to sample its spatial shape to be consistent with the dimension-reduced feature map layer_1_2 (i.e., the shallow feature map) of the shallow encoder, to obtain an upsampled feature map (i.e., a deep feature map) of the deep encoder. The upsampled operation of the fourth encoder is stacked eight times (i.e., Upsamplex8), the upsampled operation of the third encoder is stacked four times (i.e., Upsamplex4), and the upsampled operation of the second encoder is stacked twice (i.e., Upsamplex2), but not limited thereto. For ease of understanding, the upsampled feature maps of the fourth to second encoders can be denoted as layer_4_3, layer_3_3, and layer_2_3.

[0128] Finally, the upsampled feature maps layer_4_3, layer_3_3, and layer_2_3 of the last three encoders and the dimension-reduced feature map layer_1_2 of the first encoder are concatenated and then passed through a GUI to generate a remote sensing change guide map. The GUI can refer to a 1x1 convolution (Conv 1x1), but is not limited thereto.

[0129] Here, the remote sensing change guide map is generated from deep features with rich semantic information, and is used as prior information for subsequent CGA components, thereby improving the feature expression capability. Remote sensing image scenes tend to be complex, and the output of a single encoder is easily disturbed by the environment, which has problems such as single feature, poor adaptability, low information utilization rate, and cannot balance the detail information and semantic information in the dual-temporal remote sensing image, so it is difficult to deal with complex scenes, thereby limiting the model performance. Therefore, the present embodiment considers the outputs of the four encoders to generate the remote sensing change guide map. From a macro perspective, the semantic levels of the feature maps output by the encoders at different stages are different, and integrating the outputs of the four encoders can effectively fuse the multi-level semantic information in the dual-temporal remote sensing image, understand the ground feature change content of the dual-temporal remote sensing image from multiple perspectives, and accurately judge the change type, enhance the global understanding ability of the change detection model, and thus ensure the reliability of the change guide map when the environment changes, and stably detect the actual change area in the dual-temporal remote sensing image. From a detailed perspective, the shallow encoder output retains the ground feature details, and the deep encoder output contains the high-level semantics of the ground features, and the fusion of the two can accurately locate the subtle changes, while understanding the scene from a macro perspective, avoiding misjudgment, edge blur, and missed detection caused by using only shallow or deep outputs, and other problems.

[0130] In some embodiments, reference is made to Figure 8The above-mentioned use of remote sensing change guidance maps to guide the processing of the first attention feature map yields the decoder output, including:

[0131] The remote sensing change guidance map and the first attention feature map are fused together to obtain a fused feature map;

[0132] Multi-head attention operations are performed on the fused feature map to obtain the second attention feature map;

[0133] The first attention feature map and the second attention feature map are fused together to obtain the decoder output.

[0134] In this implementation, within the CGM module, a change guidance map is generated from deep ground feature features rich in semantic information using the PCM component. Then, the CGA component is used to broaden the receptive field between different pixels in the dual-temporal remote sensing images. The change guidance map serves as prior knowledge, guiding the fusion process of multi-scale ground feature change features. The goal of the attention mechanism is to ignore most unimportant information and select a small amount of important information from a large amount of data. In highly imbalanced binary classification problems, the attention mechanism for change detection locates important change features with a small number of pixels in the image, ignoring background information with a large number of pixels. When calculating attention, higher weights result in greater weights for the corresponding values. Therefore, the CGA component enables the model to focus more on ground feature areas that have changed between the dual-temporal remote sensing images, ignoring ground feature areas that have not changed.

[0135] In the CGA component, firstly, the Sigmoid activation function is used to guide the remote sensing change map f. CGM The (Change map) is processed to obtain a weighted map W with values ​​of (0, 1). CGM The weighted graph W CGM This represents the feature maps extracted at different stages of the encoder, and is also the constantly changing guide map in this component. The probability of pixel changes increases with the increase of this weight map. Simultaneously, the first attention feature map... The Decoder Output is convolved to obtain the first attention feature map after convolution. The convolution operation can be a 1×1 convolution (Conv 1×1) to achieve dimensionality reduction, but it is not limited to this. After obtaining the weight graph W... CGM and the first attention feature map after convolution Then, the two are multiplied element by element (i.e., This yields the transformed feature map, i.e., the fused feature map f. CG。 Then, the fused feature map f CGThe query Q, the value V and the key K are obtained through linear mapping, and the second attention feature map H is calculated through the multi-head attention mechanism. Thus, the multi-head attention mechanism can extract rich and potential features of the feature change. The calculation of the multi-head attention mechanism is prior art, and will not be described again. Finally, the second attention feature map H is subjected to convolution operation, and the first attention feature map is connected to the layer where the second attention feature map H is located, and then the first attention feature map and the second attention feature map H' after convolution operation are added (i.e. ), to obtain the output (Output) of the decoder. The convolution operation can be 1x1 convolution to realize dimension reduction, or 3x3 convolution to realize feature refinement, but is not limited thereto.

[0136] Here, the CGA component first processes the change guide map through the Sigmoid function to generate a weight map to locate the change area; then the feature map is convolved and multiplied by the weight map and linearly mapped to strengthen the feature of the feature change; then the multi-head attention mechanism is used to extract the refined feature of the feature change in multiple dimensions; finally, the residual connection is used to fuse the original feature of the feature change and the optimized feature of the feature change. Compared with the traditional self-attention mechanism, the CGA component reduces the calculation amount of the change detection model, avoids invalid calculation of the change detection model in the background area, solves the problem that the feature in the feature change area is easy to be ignored, and compared with the simple attention method, the CGA component can deeply mine the feature of the feature change, reduce the situation of missed detection and false detection caused by light misjudgment, and take into account the multi-scale feature change, thereby significantly improving the detection accuracy of the dual-time remote sensing image.

[0137] The performance of the change guide model (i.e., the MCMamba model) provided in the embodiments of the present application will be verified below.

[0138] (I) Dataset:

[0139] The public datasets LEVIR-CD dataset, SYSU-CD dataset and WHU-CD dataset are selected in this embodiment, and the specific information of the three datasets is shown in Table 1.

[0140] Table 1: Details of the dataset

[0141]

[0142] (II) Experimental setup:

[0143] The experimental environment of the embodiment is based on a Windows 10 operating system, and a deep learning framework is built by using Python 3.10 and PyTorch. Comparative experiments are carried out on three standard change detection data sets (LEVIR-CD, SYSU-CD and WHU-CD) which are cropped to a uniform size of 256x256 pixels, and ablation experiments are supplemented for the LEVIR-CD data set to comprehensively verify the effectiveness of the algorithm. The dual-phase remote sensing image pairs and related labels are cropped to 256x256 pixels to input into the network, and then the test set is used for forward inference of the network and model evaluation. In the training process, the AdamW optimizer is used to optimize the network, the learning rate is 1e-4, the weight decay is 5e-3. The batch size is set to 16. The number of training iterations of all data sets is set to 50000. Random rotation, left-right flipping and up-down flipping are used as training data augmentation methods.

[0144] The loss function used in the embodiment includes a cross-entropy loss function and a Dice loss function, as shown in the following formula (4).

[0145] L total =λ1L ce +λ2L dice ,

[0146]

[0147]

[0148] In formula (4), L total is the total loss function; L ce is the cross-entropy loss function; L dice is the Dice loss function; N represents the total number of pixels; y j is the true value of the jth pixel; represents the predicted probability value of the jth pixel; λ1 and λ2 represent preset weight coefficients.

[0149] (Three) Performance indicators:

[0150] The embodiment selects recall, precision, overall accuracy (OA), F1 score and intersection over union (IoU) as indicators for evaluating the performance of the model.

[0151] (Four) Comparative experiments:

[0152] To verify the effectiveness of the MCMamba model proposed in the embodiments of the present application, 10 comparative experiments were performed, including selecting 8 popular deep learning change detection algorithms, the baseline model ChangeMamba model of the embodiments of the present application, and the improved model MCMamba model proposed in the embodiments of the present application. Among them, the 8 popular deep learning change detection algorithms are: FC-EF model, FC-Siam-conc model, FC-Siam-diff model and SNUnet model based on CNN architecture; Changeformer model, PA-Former model and ACABFNet model based on Transformer architecture; RSMamba model based on Mamba architecture. All the above are trained and tested on three public data sets. In order to compare fairly, all experiments are trained under the same conditions. The results of all models are shown in Tables 2 to 4.

[0153] Table 2: Performance of all models on LEVIR-CD dataset

[0154]

[0155] From Table 2, it can be seen that the values of all evaluation indicators of the MCMamba model are the highest, further verifying the superiority of the MCMamba model. Specifically: (1) The Pre value of the MCMamba model is best among the Mamba-based models, which is 0.88% higher than that of the baseline model ChangeMamba model (91.59%), which shows that the MCMamba model performs best in the ability to correctly classify pixels; (2) The Rec value of the MCMamba model is 1.06% higher than that of the baseline model ChangeMamba model, which proves that the MCMamba model performs well in the ability to identify all actual change regions; (3) The F1 score of the MCMamba model is 91.13%, which is 0.97% higher than that of the baseline model ChangeMamba model, showing that the MCMamba model has a good balance between accuracy and recall; (4) The IoU value of the MCMamba model is 83.72%, which is 1.63% higher than that of the baseline model ChangeMamba model, indicating that the detection result of the MCMamba model has a higher degree of overlap between the change region and the actual change region; (5) The OA value of the MCMamba model reaches 99.17%, which is 0.16% higher than that of the baseline model ChangeMamba model, which shows that the MCMamba model performs best on the overall semantic segmentation task of the LEVIR-CD dataset. Therefore, the MCMamba model performs excellently on this dataset, especially in the IoU value, which is superior to the baseline model ChangeMamba model by the most, with the maximum improvement.

[0156] Table 3: Performance of all models on SYSU-CD dataset

[0157]

[0158] From Table 3, it can be seen that the Rec value of the MCMamba model is 81.35%, the F1 value is 82.36%, the IoU value is 70.31%, and the OA value is 92.04%, which are the best among all models. Specifically: (1) The Pre value of the MCMamba model decreases by 1.32% compared with the baseline model ChangeMamba model, ranking third among all models; (2) The Rec value of the MCMamba model increases by 1.76% compared with the baseline model ChangeMamba model; (3) The F1 score of the MCMamba model increases by 1.07% compared with the baseline model ChangeMamba model; (4) The IoU value of the MCMamba model increases by 1.83% compared with the baseline model ChangeMamba model; (5) The OA value of the MCMamba model increases by 0.68% compared with the baseline model ChangeMamba model.

[0159] Table 4: Performance of all models on WHU-CD dataset

[0160]

[0161] From Table 4, it can be seen that: in all models, the Recall value of the MCMamba model is 93.81%, the F1 score is 94.50%, and the IoU is 89.99%, which is the highest value among all models, the Pre value is 95.22%, ranking second among all models, and the OA value is 98.89%, ranking in the middle among all models. Specifically: (1) The Pre value of the MCMamba model is slightly lower than that of the RSMamba model (95.50%) among Mamba-based models, which is 1% higher than that of the baseline model ChangeMamba model, and higher than that of all other models, which indicates that the precision of the MCMamba model is very high, and the proportion of samples that are actually positive in the predicted positive samples is relatively high; (2) The Rec value of the MCMamba model is the highest, which is 2.87% higher than that of the baseline model ChangeMamba model, which reflects that the MCMamba model can detect most samples that are actually positive; (3) The F1 score of the MCMamba model is the best, which is 1.95% higher than that of the baseline model ChangeMamba model, indicating that it has achieved the best balance between precision and recall; (4) The IoU of the MCMamba model is the best, which is higher than that of the RSMamba model (86.55%) and the ChangeMamba model (86.13%), which is 3.86% higher than that of the baseline model ChangeMamba model, which indicates that the overlap ratio of the predicted region and the true region of the MCMamba model is the largest; (5) The OA value of the MCMamba model is slightly lower than that of the RSMamba model (99.44%) and the ChangeMamba model (99.42%), ranking in the middle among all models, and the performance is slightly weaker, but considering the performance of F1 score and IoU, it shows that the MCMamba model focuses more on the detection of change regions rather than the overall classification. Overall, the performance of the MCMamba model is better than that of all Transformer-based models, all CNN-based models and the baseline model ChangeMamba model, which shows the great potential of the MCMamba model in the change detection task.

[0162] Based on the above three public data set verification results, it can be seen that the comprehensive performance of the MCMamba model is the best, which is due to its ability to handle complex dependency relationships and long-distance feature capture, and the MCMamba model performs outstandingly in the remote sensing image change detection task.

[0163] (Five) Ablation experiment:

[0164] To verify the effectiveness of each key module in the MCMamba model, six ablation experiments were designed on the LEVIR-CD dataset. Among them, the original ChangeMamba framework was configured as the baseline model of MCMamba. It was verified that whether the key modules were added alone or in combination, the comprehensive results of the MCMamba model were better than the baseline model ChangeMamba model. The results are shown in Table 5 below.

[0165] Table 5: Ablation experiment results

[0166]

[0167] In this ablation experiment, the MSPA module, the CGM module were added alone, and the MSPA module, the HPC module, the SPR module and the CGM module were added step by step to evaluate their contribution to the performance of the model. The Pre value of the baseline model ChangeMamba model is 91.59%, the Rec value is 88.78%, the F1 score is 90.16%, the IoU is 82.09%, and the OA value is 99.01%. The improvement of each module to the baseline model ChangeMamba model is as follows:

[0168] 1) The introduction of the MSPA module alone (without the HPC module and the SPR module) caused the Rec value of the ChangeMamba model to decrease slightly to 88.71%, but its Pre value, F1 score, IoU and OA value were all improved, reaching 88.71%, 90.71%, 83.01% and 99.11% respectively. This shows that the MSPA module has a positive effect on the improvement of the ChangeMamba model.

[0169] 2) The introduction of the CGM module alone improved the Pre value, F1 score, IoU and overall Pre value of the ChangeMamba model, which shows that the influence of the CGM module on the ChangeMamba model is positive, which can help the ChangeMamba model better extract subtle changes in the ground object area, thereby proving the effectiveness of the CGM module.

[0170] 3) The introduction of the HPC module based on the ChangeMamba model + MSPA module (without the HPC module and the SPR module) brought overall improvement to the performance of the ChangeMamba model, with the Pre value improved by 0.87%, the Rec value improved by 0.58%, the F1 score improved by 0.71%, the IoU and OA value improved by 1.2% and 0.12% respectively, among which the IoU improved the most, proving that the introduction of this module reduces the false detection and missed detection phenomenon of change detection.

[0171] 4) Introducing the SPR module into the ChangeMamba model + MSPA module (excluding HPC and SPR modules) + HPC module also improved the overall performance of the ChangeMamba model, with IoU and F1 score increasing by 1.52% and 0.9% respectively, which verifies the effectiveness of the SPR module.

[0172] 5) Based on the above, the CGM module is introduced. Compared with the previous experiment (ChangeMamba model + MSPA module + HPC module + SPR module), its Pre value decreases slightly, but its Rec value, F1 score, IoU and overall Pre value are significantly improved. This verifies that the CGM module effectively assists the ChangeMamba model in completing the detection of edge and small target regions.

[0173] (vi) Results Visualization:

[0174] To further verify the effectiveness of the embodiments of this application, the remote sensing image change detection results of the MCMamba model are visualized, and a qualitative analysis of the LEVIR-CD test set is performed to more intuitively demonstrate the performance of the MCMamba model. Visualized experimental results of the CNN architecture model FC-EF, the Transformer architecture model PA-Former, and the baseline model ChangeMamba are also presented, such as... Figure 9 As shown, Figure 9 In the image, T1 and T2 represent paired two-temporal remote sensing images, and GT represents the standard image. Figure 9 As can be seen, compared to the baseline ChangeMamba model, the MC Mamba model shows improvements in both false negatives and false positives. As indicated by the red circle in Figure (b), the MC Mamba model exhibits no false positives. For scenarios involving changes in large buildings, the MC Mamba model also demonstrates superior edge detection performance. As shown by the green and red circles in Figure (a), the MC Mamba model outperforms the baseline ChangeMamba model in edge detection. This proves the effectiveness and reliability of the MC Mamba model in handling change detection tasks in high-resolution remote sensing images. Furthermore, the change detection results in the figure show that the MC Mamba model can identify not only large change areas but also small, subtle change areas, indicating that the model has high spatial resolution and change sensitivity.

[0175] In summary, the embodiments of the present application propose an improved ChangeMamba model, called MCMamba model, for the change detection task of dual temporal remote sensing images, aiming to solve the problems of insufficient feature capture ability, poor robustness and multi-scale feature fusion short board of existing methods in complex background. Specifically, the HPC module is introduced to process the input feature map more effectively with a multi-scale method, promoting the extraction of local and global information in dual temporal remote sensing images by the change detection model; the SPR module is added to establish the interdependence between channels in dual temporal remote sensing images, improve the sensitivity of the change detection model to the interdependence between channels in dual temporal remote sensing images, and accordingly design the MSPA module to integrate the multi-scale spatial information and cross-channel attention in dual temporal remote sensing images, and improve the learning ability of the change detection model to the multi-scale feature of ground objects in dual temporal remote sensing images. In addition, the CGM module is constructed, including the PCM component and the CGA component, to solve the edge and internal detection problems, so that the change detection model pays more attention to the actual changed ground object area in dual temporal remote sensing images, and ignores the unchanged ground object area.

[0176] The experimental results on the public data sets LEVIR-CD, SYSU-CD and WHU-CD show that the MCMamba model has excellent performance. In the comparative experiment, compared with 8 popular deep learning change detection algorithms and the baseline model, the MCMamba model performs well in multiple indicators. On the LEVIR-CD data set, the Pre value of the MCMamba model is 0.88% higher than that of the baseline model ChangeMamba model, the Rec value is 1.06% higher, the F1 score is 0.97% higher, the IoU value is 1.63% higher, and the OA value is 0.16% higher; on the SYSU-CD data set, the Rec value, F1 score, IoU value and OA value are the best among all models; on the WHU-CD data set, the Rec value, F1 score and IoU value of the MCMamba model are the highest. These experimental results fully prove the superiority of the MCMamba model in the change detection task of remote sensing images.

[0177] The ablation experiment further verifies the effectiveness of each module. After gradually adding the MSPA module (without the HPC module and the SPR module), the HPC module, the SPR module and the CGM module, the performance of the model is improved to different degrees. For example, the HPC module improves the Pre value of the model by 0.87%, the Rec value by 0.58%, and the IoU value by 1.2%; the SPR module improves the IoU value and the F1 score of the model by 1.52% and 0.9% respectively. The visualization analysis of the experimental results shows that the MCMamba model significantly improves the missed detection and false detection compared with the baseline model ChangeMamba model, has excellent edge detection effect, and can identify different scales of change areas, showing high spatial resolution and change sensitivity.

[0178] In addition, the embodiment of the present application further provides a remote sensing image change detection device, which comprises:

[0179] an acquisition module, configured to acquire double-time-phase remote sensing images;

[0180] a remote sensing change detection module, configured to perform change detection on the double-time-phase remote sensing images by using a change detection model to obtain a binary change detection map.

[0181] The change detection model is a neural network model obtained by adding an attention module and a change guide module behind an up-sampling module of each decoder of the ChangeMamba model; the attention module is used for performing hierarchical pixel convolution operation and channel interaction processing on the input of the attention module to obtain a first attention feature map; and the change guide module is used for performing output guide processing on the first attention feature map by using the encoding features of each encoder in the ChangeMamba model to obtain the output of the decoder.

[0182] The contents in the method embodiments are all applicable to the device embodiments, the device embodiments specifically implement the functions of the method embodiments, and the device embodiments achieve the same beneficial effects as the method embodiments.

[0183] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and purposes of the present application, the scope of the present application is defined by the claims and their equivalents.

[0184] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A method for detecting changes in remote sensing images, characterized in that, Includes the following steps: Acquire dual-temporal remote sensing images; The dual-temporal remote sensing image is subjected to change detection using a change detection model to obtain a binary change detection map. The change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model. The attention module is used to perform hierarchical pixel convolution operations and channel interaction processing on the input of the attention module to obtain a first attention feature map; The change guidance module is used to perform output guidance processing on the first attention feature map using the encoding features of each encoder in the ChangeMamba model, so as to obtain the output of the decoder; The step of using the encoding features of each encoder in the ChangeMamba model to perform output guidance processing on the first attention feature map to obtain the output of the decoder includes: Based on the coding features of each encoder, a guide map generation process is performed to obtain a remote sensing change guide map; The first attention feature map is guided by the remote sensing change guidance map to obtain the output of the decoder; The step of generating a remote sensing change guidance map based on the coding features of each encoder includes: The coding feature pairs of each encoder are fused to obtain the coding feature map of each encoder; The dimensionality reduction process is performed on the encoding feature maps of each encoder to obtain the dimensionality reduction feature maps of each encoder; Upsampling is performed on the dimensionality-reduced feature maps of the latter three encoders to obtain the upsampled feature maps of the latter three encoders; The upsampled feature maps of the latter three encoders and the dimensionality-reduced feature map of the first encoder are fused together to generate the remote sensing change guidance map. The step of using the remote sensing change guidance map to guide the processing of the first attention feature map to obtain the output of the decoder includes: The remote sensing change guidance map and the first attention feature map are fused together to obtain a fused feature map. Perform multi-head attention operation on the fused feature map to obtain a second attention feature map; The first attention feature map and the second attention feature map are fused together to obtain the output of the decoder.

2. The method according to claim 1, characterized in that, The process of performing hierarchical pixel convolution and channel interaction processing on the input of the attention module to obtain a first attention feature map includes: Hierarchical pixel convolution operation is performed on the input of the attention module to obtain a multi-scale feature map; Channel interaction processing is performed on the multi-scale feature map to obtain channel attention weights; The channel attention weights and the multi-scale feature map are fused to obtain the first attention feature map.

3. The method according to claim 2, characterized in that, The step of performing hierarchical pixel convolution operations on the input of the attention module to obtain a multi-scale feature map includes: The input to the attention module is segmented to obtain a first feature map, a second feature map, and a third feature map; Perform a convolution operation on the first feature map to obtain a first convolutional feature map; The second feature map and the first convolutional feature map are fused and convolved to obtain the second convolutional feature map. The third feature map and the second convolutional feature map are fused and convolved to obtain the third convolutional feature map; The first convolutional feature map, the second convolutional feature map, and the third convolutional feature map are fused together to obtain the multi-scale feature map.

4. The method according to claim 2, characterized in that, The step of performing channel interaction processing on the multi-scale feature map to obtain channel attention weights includes: The multi-scale feature map is subjected to global and local aggregation processing to obtain an aggregated feature map; The aggregated feature map is subjected to channel interaction processing to obtain the channel attention weights.

5. The method according to claim 4, characterized in that, The step of performing global and local aggregation processing on the multi-scale feature map to obtain an aggregated feature map includes: Parallel pooling operations are performed on the multi-scale feature maps to obtain global feature maps and local feature maps; The global feature map and the local feature map are fused to obtain the aggregated feature map.

6. The method according to claim 4, characterized in that, The step of performing channel interaction processing on the aggregated feature map to obtain the channel attention weights includes: The aggregated feature map is subjected to multiple pointwise convolution operations to obtain the channel attention weights.

7. A remote sensing image change detection device, characterized in that, include: The acquisition module is used to acquire dual-temporal remote sensing images; The remote sensing change detection module is used to perform change detection on the dual-temporal remote sensing image using a change detection model to obtain a binary change detection map. The change detection model is a neural network model obtained by adding an attention module and a change guidance module after the upsampling module of each decoder in the ChangeMamba model. The attention module is used to perform hierarchical pixel convolution operations and channel interaction processing on the input of the attention module to obtain a first attention feature map; The change guidance module is used to perform output guidance processing on the first attention feature map using the encoding features of each encoder in the ChangeMamba model, so as to obtain the output of the decoder; The step of using the encoding features of each encoder in the ChangeMamba model to perform output guidance processing on the first attention feature map to obtain the output of the decoder includes: Based on the coding features of each encoder, a guide map generation process is performed to obtain a remote sensing change guide map; The first attention feature map is guided by the remote sensing change guidance map to obtain the output of the decoder; The step of generating a remote sensing change guidance map based on the coding features of each encoder includes: The coding feature pairs of each encoder are fused to obtain the coding feature map of each encoder; The dimensionality reduction process is performed on the encoding feature maps of each encoder to obtain the dimensionality reduction feature maps of each encoder; Upsampling is performed on the dimensionality-reduced feature maps of the latter three encoders to obtain the upsampled feature maps of the latter three encoders; The upsampled feature maps of the latter three encoders and the dimensionality-reduced feature map of the first encoder are fused together to generate the remote sensing change guidance map. The step of using the remote sensing change guidance map to guide the processing of the first attention feature map to obtain the output of the decoder includes: The remote sensing change guidance map and the first attention feature map are fused together to obtain a fused feature map. Perform multi-head attention operation on the fused feature map to obtain a second attention feature map; The first attention feature map and the second attention feature map are fused together to obtain the output of the decoder.

Images