Remote sensing change detection method and device for region-space-semantic modeling based on Mamba

Abstract

The invention relates to a remote sensing image change detection method and device based on Mama region-space-semantic channel modeling, and the method comprises the steps: inputting an obtained remote sensing image data set into a constructed Mama-based region perception, dual-domain space and semantic channel modeling network (S3Mama) for training, and obtaining and storing the trained S3Mama; and inputting a remote sensing image to be processed, and performing prediction through the trained S3Mama to obtain a dual-temporal change prediction map. According to the method, the precision and the performance of remote sensing image change detection are remarkably improved through the design of coupling of triple collaborative modeling mechanisms.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and more specifically, relates to a remote sensing change detection method and apparatus based on Mamba region-spatial-semantic modeling. Background Technology

[0002] Change detection in remote sensing images is of great significance for urban planning, disaster assessment, and environmental protection. Current mainstream methods can be broadly categorized into three types: methods based on convolutional neural networks can extract local features, but are limited by the local receptive field of convolution operations, making it difficult to model long-range spatial dependencies; methods based on Transformers can establish global contextual relationships, but their self-attention mechanism suffers from secondary computational complexity, making them unsuitable for high-resolution remote sensing images; and the recently proposed Mamba architecture, while improving efficiency through a state-space model, still suffers from three major drawbacks:

[0003] (1) Insufficient handling of ambiguous areas of visual perception: changes in lighting, pseudo-changes caused by vegetation growth, and real changes covered by occlusion and shadows are not explicitly distinguished and modeled.

[0004] (2) Weakened local detail features: The global modeling process loses boundary details and high-frequency information, resulting in limited ability to identify targets with small scale changes;

[0005] (3) Lack of cross-channel semantic interaction: The interaction of channel-level features is insufficient, making it difficult to capture subtle semantic changes. Summary of the Invention

[0006] To overcome the above-mentioned shortcomings of the prior art, this invention proposes a remote sensing change detection method and device based on Mamba region-spatial-semantic modeling.

[0007] This invention integrates innovatively designed region-aware modeling branches, dual-domain spatial modeling branches, and semantic channel modeling branches into the dual-domain spatial and semantic channel modeling network S3MambaNet. This eliminates false change interference, identifies real change regions of multiple scales and shapes, strengthens fine-grained boundaries, and successfully applies S3MambaNet to the field of remote sensing image analysis.

[0008] To achieve the above objectives, according to one aspect of the present invention, a remote sensing image change detection method based on Mamba region-spatial-semantic modeling is provided, comprising:

[0009] S1. Obtain a remote sensing image dataset, in which the changed or unchanged regions in each pair of dual-time remote sensing images have the same size, shape, and position in the corresponding change mask; in each change mask, the pixel value of the changed region is 1 or 255, and the pixel value of the unchanged region is 0.

[0010] S2. The remote sensing image dataset is input into the Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network (S3MambaNet) for training. The trained S3MambaNet is then saved. The model structure of S3MambaNet is as follows: the encoder consists of a conjoined backbone with shared weights, and the decoder consists of S3Mamba modules and a fusion module. The decoder includes four stages. Except for stage 1, which includes only one S3Mamba module, the other stages include one S3Mamba module and one fusion module. S3MambaNet performs the following information processing: First, the dual-time remote sensing image is input into the encoder to obtain dual-time features at four scales. Then, the dual-time features at the four scales are respectively processed by the S3Mamba modules in the four stages of the decoder. The fusion module is responsible for integrating the output features of the S3Mamba module in the current stage with the output features of the previous stage after upsampling. Finally, the output of the final stage is upsampled by 4 times to generate a prediction map.

[0011] The S3Mamba module includes a region-aware modeling branch, a dual-domain spatial modeling branch, and a semantic channel modeling branch. The outputs of the dual-temporal features obtained after passing through the three branches are concatenated along the channel dimension, and then passed through a 1×1 convolutional layer to obtain the output of the S3Mamba module.

[0012] S3. Input the remote sensing image to be processed, and use the trained S3MambaNet to make predictions to obtain a two-time-varying prediction map.

[0013] In some alternative implementations, the remote sensing image dataset described in step S1 includes:

[0014] The remote sensing image dataset was constructed using pairs of bi-temporal images and their corresponding change masks. To ensure the consistency and comparability of the dataset, each pair of bi-temporal images and its corresponding change mask must correspond precisely and follow a unified file naming convention. This correspondence ensures that in each pair of images, both changed and stable regions exhibit consistent geometric features in the corresponding change mask, including size, shape, and spatial location. Furthermore, the change mask employs a binary representation, where pixel values ​​in changed regions are encoded as 1 or 255, while pixel values ​​in unchanged regions are set to 0.

[0015] In some optional implementations, training is performed on the Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network (S3MambaNet) described in step S2, including:

[0016] Construct S3MambaNet, input the obtained dataset into S3MambaNet, and set the batch size and training rounds for each iteration to train S3MambaNet.

[0017] In some alternative implementations, the construction of S3MambaNet includes: a weight-sharing conjoined backbone, a decoder consisting of S3Mamba modules and a fusion module.

[0018] In some alternative implementations, the weight-sharing conjoined backbone includes: a feature extractor constructed using a parameter-sharing pre-trained MambaBCD-tiny architecture, which takes a pair of bitemporal remote sensing images as input to the structure to generate bitemporal features. , Here, i and j represent the i-th bitemporal feature pair and the j-th stage of the feature extractor, respectively, and T1 and T2 represent the two time points. The bitemporal features are then input into a decoder consisting of the S3Mamba module and the fusion module.

[0019] In some optional implementations, the region-aware modeling branch includes three parallel sub-branches (salient region sub-branch, similar region sub-branch, and fuzzy region sub-branch), simultaneously processing bitemporal features. Each sub-branch consists of a region-specific label generator (a salient region label generator, a similar region label generator, and a fuzzy region label generator, respectively) and a visual state space module [Visual State Space Advances in Neural Information Processing Systems 37 (NeurIPS 2024)]. These three types of label generators are modeled independently through their respective visual state space modules, and their outputs are connected along the channel dimension to form a unified region-aware representation. Specifically, the generation strategies for the three region-specific label generators are as follows:

[0020] The salient region label generator described above obtains the salient region labels by expanding the bitemporal features after absolute value subtraction, as shown in the following formula (1):

[0021]

[0022] The similar region label generator first calculates the cosine similarity matrix of the two-time features. In this matrix, the values ​​of significantly changing regions are significantly lower. Therefore, multiplying this matrix by the two-time features can suppress significantly changing regions while enhancing high similarity regions, thus obtaining similar region labels. Subsequently, the similar region labels are cross-sorted using the following formulas (2) and (4):

[0023]

[0024]

[0025]

[0026] In formulas (2) and (3) Sim and Sim represent the cosine similarity function and similarity score, respectively.

[0027] The fuzzy region marker generator obtains the fuzzy region marker by subtracting the absolute value of similar region markers using the following formula (5):

[0028]

[0029] The aforementioned dual-domain spatial modeling branch includes two parallel branches (global context learner and wavelet transform local compensator), which simultaneously process dual-temporal features. Specifically, the global context learner first generates global spatial labels by concatenating the dual-temporal features along the channel dimension through a spatial label generator, and then the global spatial labels are input into the visual state space module for global context learning.

[0030] The wavelet transform local compensator concatenates the bi-temporal features along the channel dimension and then performs channel dimensionality reduction through a 1×1 convolutional layer to generate feature maps. This feature map After discrete wavelet transform Then, the frequency domain representation is obtained. Subsequently, local information is extracted through convolution operations, followed by inverse wavelet transform. The original spatial resolution is restored, and the final output feature map is obtained. We obtain this from the following formulas (6) and (7):

[0031]

[0032]

[0033] In formulas (6) and (7) It is a low-pass filter. It is a high-pass filter.

[0034] Finally, features obtained from the global context learner and the wavelet transform local compensator were fused by element-wise addition.

[0035] The semantic channel modeling branch includes: first, concatenating bitemporal features along the channel dimension and then performing channel dimensionality reduction through a 1×1 convolutional layer to obtain merged bitemporal features. Then, sequential and reverse expansions were performed separately to merge the features and obtain sequential fine-grained features. and reverse fine-grained features Two features are grouped into channels separately to obtain bidirectional grouping labels. We obtain the following formulas (8)-(11):

[0036]

[0037]

[0038]

[0039]

[0040] In formulas (8) and (7) , and These represent 1×1 convolution operation, sequential unrolling operation, and reverse unrolling operation, respectively. This indicates that the channel has been divided into group G. Following this, bidirectional grouping is indicated. Modeling is performed using the Mamba module [Mamba: Linear-TimeSequence Modeling with Selective State Spaces Gu A, Dao T. Mamba: Linear-timesequence modeling with selective state spaces[J]. arXiv preprint arXiv:2312.00752, 2023].

[0041] In some alternative implementations, the fusion module first adds the high-level features to the upsampled low-level features. This is followed by the application of two 3×3 convolutional layers and batch normalization.

[0042] In some alternative implementations, the decoder comprises four stages. The first stage directly processes the features output by the encoder via an S3Mamba module; the subsequent three stages each contain an S3Mamba module and a fusion module, whereby the fusion module is responsible for integrating the current features with the upsampled output features from the previous stage. The final stage output generates a prediction map through a 4x upsampling.

[0043] In some alternative implementations, obtaining and saving the trained S3MambaNet includes: iteratively calculating the loss and updating the network parameters, completing the network training, and saving the network parameter file.

[0044] In some optional implementations, the method of obtaining a two-time change prediction map by predicting using a trained S3MambaNet includes: processing the two-time remote sensing image pairs that need to be changed to a resolution size that the model can accept, and then inputting them into the trained S3MambaNet for prediction to obtain the two-time change prediction map.

[0045] A second aspect of the present invention relates to a remote sensing change detection device based on Mamba region-spatial-semantic modeling, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a remote sensing change detection method based on Mamba region-spatial-semantic modeling of the present invention.

[0046] A third aspect of the invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements a remote sensing change detection method based on Mamba region-spatial-semantic modeling.

[0047] The working principle of this invention is:

[0048] The core working principle of this invention is to construct a Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network, S3MambaNet. S3MambaNet achieves remote sensing image change detection through three-dimensional coupling of region, space, and channel. S3MambaNet consists of a weight-shared conjoined backbone, an S3Mamba module, and a decoder composed of a fusion module. First, the dual-time remote sensing images are input into the conjoined backbone, features are extracted, and then input into the decoder composed of the S3Mamba module and the fusion module. The three modeling mechanisms of the S3Mamba module simultaneously focus on key regions, global space, local details, and fine-grained channel relationships, enhancing the multi-dimensional information of the features. Next, the fusion module fuses the decoder features from different stages to increase multi-scale information. Finally, the decoding stage generates a dual-time change prediction map, completing the remote sensing image change detection.

[0049] The advantages of this invention are:

[0050] This invention proposes a Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network (S3MambaNet), which significantly improves the accuracy and performance of remote sensing image change detection through a triple collaborative modeling mechanism. The region-aware modeling branch focuses on explicitly separating pseudo-changes from true changes in visually ambiguous regions. Furthermore, the dual-domain spatial modeling branch utilizes frequency-space joint optimization through global context learning and wavelet local compensation to overcome the bottleneck of boundary ambiguity. Finally, our S3MambaNet, through a semantic channel modeling branch, achieves pixel-level cross-channel interaction using a bidirectional channel grouping mechanism, inferring changes of varying shapes using rich semantics as clues. In summary, this invention not only theoretically demonstrates the significant advantages of deep learning technology in remote sensing image analysis but also showcases its excellent performance and wide applicability in practical applications. Attached Figure Description

[0051] Figure 1 This is a flowchart of the method of the present invention.

[0052] Figure 2 The diagram shows the S3MambaNet model, a Mamba-based network for region awareness, dual-domain spatial and semantic channel modeling, as presented in this invention. Figure 2 Part (a) is a schematic diagram of the overall architecture of S3MambaNet. Figure 2 Part (b) is a schematic diagram of the S3Mamba module and fusion module of the present invention; Figure 2 Part (c) is the region-aware modeling branch of this invention; Figure 2 Part (d) is the dual-domain space modeling branch of this invention; Figure 2 Part (e) is the semantic channel modeling branch of this invention.

[0053] Figure 3 This image shows the change detection performance of the Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network (S3MambaNet) of this invention. Odd-numbered rows (s1, s3, s5, s7) display the predicted change map; even-numbered rows (s2, s4, s6, s8) display the corresponding error map. In the prediction map, different colors represent prediction accuracy: true positive (white), false positive (green), true negative (black), and false negative (red). In the error map, yellow highlights errors, while blue indicates correct predictions. The Image1 and Image2 columns represent real dual-time remote sensing images, the BIT, DMINet, SEIFNet, ChangeMamba, and S3MambaNet columns represent the prediction results of different networks, and GT is the real change mask.

[0054] Figure 4Figure 4 shows the generalization performance of S3MambaNet, a Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network of this invention, on the SYSU dataset for change detection. Different colors represent different percentages of test data. The columns for BIT, ICIFNet, DMINet, SEIFNet, ChangeMamba-B, and S3MambaNet represent the generalization performance of different networks for change detection. The vertical axis in Figure 4 represents the F1 score (%), a key statistical indicator used to evaluate the performance of binary classification models.

[0055] Figure 5 This is a schematic diagram of the structure of the device of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit the scope of protection of this application.

[0057] like Figure 1 This embodiment provides a remote sensing image change detection method based on Mamba region-spatial-semantic modeling, including:

[0058] Step S1: Obtain the remote sensing image dataset; resize the dataset to the size of the input image. .

[0059] Step S2: Construct S3MambaNet, a Mamba-based network for region awareness, dual-domain spatial and semantic channel modeling, as follows: Figure 2 S3MambaNet consists of a weight-sharing conjoined backbone, S3Mamba modules, and a decoder composed of fusion modules.

[0060] The weight-sharing conjoined backbone is a feature extractor constructed using a pre-trained MambaBCD-tiny with shared parameters. A pair of bitemporal remote sensing images is input into this structure to generate bitemporal features. , Here, i and j represent the i-th bitemporal feature pair and the j-th stage of the feature extractor, respectively, and T1 and T2 represent the two time points. The bitemporal features are then input into a decoder consisting of the S3Mamba module and the fusion module.

[0061] The S3Mamba module includes a region-aware modeling branch, a dual-domain spatial modeling branch, and a semantic channel modeling branch. The outputs of the dual-temporal features are concatenated along the channel dimension after being modeled by the three types of models, and then passed through a 1×1 convolutional layer to obtain the output of the S3Mamba module.

[0062] The region-aware modeling branch comprises three parallel sub-branches (salient region branch, similar region branch, and fuzzy region branch), simultaneously processing bitemporal features. Each sub-branch consists of a region-specific label generator (a salient region label generator, a similar region label generator, and a fuzzy region label generator, respectively) and a visual state space module [VMamba: Visual State Space Advances in Neural Information Processing Systems 37 (NeurIPS 2024)]. These three types of labels are modeled independently through their respective visual state space modules, and their outputs are concatenated along the channel dimension to form a unified region-aware representation. Specifically, the generation strategies for the three region-specific label generators are as follows:

[0063] The salient region label generator includes: expanding the bitemporal features after absolute value subtraction to obtain salient region labels, which are obtained by the following formula (1):

[0064]

[0065] The similar region label generator first calculates the cosine similarity matrix of the two-time features. In this matrix, the values ​​of significantly changing regions are significantly lower. Therefore, multiplying this matrix by the two-time features can suppress significantly changing regions while enhancing high-similarity regions, thus obtaining similar region labels. Subsequently, the similar region labels are cross-sorted using the following formulas (2)-(4):

[0066]

[0067]

[0068]

[0069] In formulas (2) and (3) Sim and Sim represent the cosine similarity function and similarity score, respectively.

[0070] The fuzzy region label generator includes: obtaining fuzzy region labels by subtracting the absolute values ​​of similar region labels, which is obtained by the following formula (5):

[0071]

[0072] The dual-domain spatial modeling branch includes two parallel branches (global context learner and wavelet transform local compensator) that simultaneously process bitemporal features. Specifically, the global context learner first generates global spatial labels by concatenating the bitemporal features along the channel dimension using a spatial label generator. These global spatial labels are then input into the visual state space module for global context learning.

[0073] The wavelet transform local compensator concatenates the bi-temporal features along the channel dimension and then performs channel dimensionality reduction through a 1×1 convolutional layer to generate feature maps. This feature map After discrete wavelet transform Then, the frequency domain representation is obtained. Subsequently, local information is extracted through convolution operations, followed by inverse wavelet transform. The original spatial resolution is restored, and the final output feature map is obtained. We obtain this from the following formulas (6) and (7):

[0074]

[0075]

[0076] In formulas (6) and (7) It is a low-pass filter. It is a high-pass filter.

[0077] Finally, features obtained from the global context learner and the wavelet transform local compensator were fused by element-wise addition.

[0078] The semantic channel modeling branch includes: first, concatenating bitemporal features along the channel dimension and then performing channel dimensionality reduction through a 1×1 convolutional layer to obtain merged bitemporal features. Then, sequential and reverse expansions were performed separately to merge the features and obtain sequential fine-grained features. and reverse fine-grained features Two features are grouped into channels separately to obtain bidirectional grouping labels. We obtain the following from formula (8) - formula (11):

[0079]

[0080]

[0081]

[0082]

[0083] In formulas (8)-(7) , and These represent 1×1 convolution operation, sequential unrolling operation, and reverse unrolling operation, respectively. This indicates that the channel has been divided into group G. Following this, bidirectional grouping is indicated. Modeling is performed using the Mamba module [Mamba: Linear-TimeSequence Modeling with Selective State Spaces Gu A, Dao T. Mamba: Linear-timesequence modeling with selective state spaces[J]. arXiv preprint arXiv:2312.00752, 2023].

[0084] The fusion module includes: first, adding the high-level features to the upsampled low-level features; then applying two 3×3 convolutional layers and batch normalization.

[0085] The decoder consists of four stages. The first stage directly processes the features output by the encoder through an S3Mamba module. The subsequent three stages each contain an S3Mamba module and a fusion module, where the fusion module is responsible for integrating the current features with the upsampled output features from the previous stage. The final stage output is upsampled by 4 times to generate a prediction map.

[0086] Step S3: Input the obtained remote sensing image dataset into the constructed S3MambaNet for training. Loss calculation and network parameter updates are performed for each batch until all batches of training data are completed. Finally, the trained weights are obtained, and all updated parameters are saved in the weight file. The trained S3MambaNet is then obtained and saved.

[0087] Step S4: Input the remote sensing image to be processed, and use the trained S3MambaNet to make predictions to obtain a two-time-varying prediction map. Table 1 below compares the performance metrics (F1 / IOU / OA) of S3MambaNet with other current state-of-the-art models. The bold text indicates the metrics of the best models.

[0088] Table 1 Quantitative comparison on the SYSU and GZ datasets

[0089]

[0090] The SYSU dataset, provided by Sun Yat-sen University, is a dedicated dataset for remote sensing change detection, designed to support research on change recognition in complex scenes. This dataset contains 20,000 pairs of orthogonal aerial images, each pair with a fixed spatial size of 256×256 pixels, achieving a spatial resolution of up to 0.5 meters. The data was collected from Hong Kong, China. A key feature of SYSU is the diversity of its covered targets: in addition to common buildings, it includes various land cover types such as ships, roads, and vegetation. This presents a greater challenge to change detection tasks, as the model needs to distinguish between real changes and pseudo-changes (such as seasonality or differences in illumination). In terms of data partitioning, SYSU is divided into 12,000 training pairs, 4,000 validation pairs, and 4,000 test pairs, ensuring the comprehensiveness and reliability of model evaluation. The dataset is publicly available at: https: / / github.com / liumency / SYSU-CD.

[0091] The GZ dataset is an ultra-high resolution satellite imagery change detection dataset focusing on spatiotemporal change analysis in the suburbs of Guangzhou, China. The dataset has a spatial resolution of 0.55 meters and spans from 2006 to 2019. It utilizes 19 pairs of seasonally varied images obtained from Google Earth, with original image sizes ranging from 1006×1168 to 4936×5224 pixels. The data is partitioned into 2,504 training pairs, 313 validation pairs, and 313 test pairs. The GZ dataset's advantages lie in its high resolution and long time span, effectively capturing subtle differences such as urban expansion and land use changes. However, the seasonal variations introduce spurious changes, placing higher demands on the model's ability to eliminate these spurious changes. The dataset is publicly available at: https: / / github.com / daifeng2016 / Change-Detection-Dataset-for-High-Resolution-Satellite-Imagery.

[0092] Example 2

[0093] This embodiment relates to a remote sensing image change detection device based on Mamba region-spatial-semantic modeling, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the remote sensing image change detection method based on Mamba region-spatial-semantic modeling of Embodiment 1.

[0094] like Figure 5At the hardware level, the device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above. Figure 1 The method described herein. Of course, in addition to software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0095] Improvements in a technology can be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology can now be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement in methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0096] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0097] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0098] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this invention, the functions of each unit can be implemented in one or more software and / or hardware components.

[0099] Example 3

[0100] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements the remote sensing image change detection method based on Mamba region-spatial-semantic modeling of Embodiment 1.

[0101] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0106] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0107] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0108] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0111] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0112] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for detecting changes in remote sensing images based on Mamba region-spatial-semantic modeling, characterized in that, include: S1. Obtain a remote sensing image dataset, in which the changed or unchanged regions in each pair of dual-time remote sensing images have the same size, shape, and position in the corresponding change mask; in each change mask, the pixel value of the changed region is 1 or 255, and the pixel value of the unchanged region is 0. S2. Input the remote sensing image dataset into the constructed Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network S3MambaNet for training. The trained S3MambaNet model is then saved. The S3MambaNet model structure consists of an encoder with a weight-shared conjoined backbone, and a decoder composed of S3Mamba modules and a fusion module. The decoder comprises four stages; except for stage 1, which includes only one S3Mamba module, the other stages include one S3Mamba module and one fusion module. S3MambaNet performs the following information processing: First, the dual-time remote sensing image is input into the encoder to obtain dual-time features at four scales. Then, the four-scale dual-time features are processed by the S3Mamba modules in the four stages of the decoder. The fusion module integrates the output features of the current stage's S3Mamba module with the output features of the previous stage after upsampling. Finally, the output of the final stage is upsampled by 4 times to generate a prediction map. The S3Mamba module includes a region-aware modeling branch, a dual-domain spatial modeling branch, and a semantic channel modeling branch. The outputs of the dual-temporal features obtained after passing through the three branches are concatenated along the channel dimension, and then passed through a 1×1 convolutional layer to obtain the output of the S3Mamba module. S3. Input the remote sensing image to be processed, and use the trained S3Mamba to make predictions to obtain a two-phase change prediction map.

2. The method according to claim 1, characterized in that, Step S2 involves training the input into the constructed Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network S3Mamba, including: To build an S3Mamba instance, input the obtained dataset into S3Mamba, and set the batch size and training rounds for each iteration.

3. The method according to claim 2, characterized in that, The S3Mamba network, which is based on Mamba for region awareness, dual-domain spatial and semantic channel modeling, includes a weight-shared conjoined backbone, S3Mamba modules, and a decoder consisting of a fusion module.

4. The method according to claim 3, characterized in that, The weight-sharing conjoined backbone is a feature extractor constructed using a pre-trained MambaBCD-tiny with shared parameters. A pair of bi-temporal remote sensing images are input into this structure to generate bi-temporal features. , , where i and j represent the i-th bitemporal feature pair and the j-th stage of the feature extractor, respectively, and T1 and T2 represent two time points; then the bitemporal features are input into the decoder composed of the S3Mamba module and the fusion module.

5. The method according to claim 3, characterized in that, The region-aware modeling branch includes parallel explicit region sub-branches, similar region sub-branches, and ambiguous region sub-branches, simultaneously processing dual-time features. Each sub-branch consists of a label generator for a specific region and a visual state space module. Each sub-branch is independently modeled using its respective visual state space module, and their outputs are connected along the channel dimension to form a unified region-aware representation. The generation strategy is as follows: A salient region label generator is used to expand the bitemporal features after absolute value subtraction to obtain salient region labels. The similar region label generator first calculates the cosine similarity matrix of the two time features; then multiplies the cosine similarity matrix by the two time features to obtain the similar region labels; subsequently, the similar region labels are cross-sorted. The fuzzy region marker generator obtains fuzzy region markers by subtracting the absolute values ​​of similar region markers.

6. The method according to claim 3, characterized in that, The fusion module first adds the high-level features to the upsampled low-level features; then it applies two 3×3 convolutional layers and batch normalization.

7. The method according to claim 3, characterized in that, The decoder consists of four stages; the first stage directly processes the features output by the encoder through the S3Mamba module; each of the subsequent three stages contains an S3Mamba module and a fusion module, wherein the fusion module is responsible for integrating the current features with the upsampled output features of the previous stage. The final output is generated by upsampling by 4 times to create a prediction map.

8. The method according to claim 1, characterized in that, Step S2, which involves obtaining and saving the trained S3Mamba, includes: calculating the loss iteratively and updating the network parameters, completing the network training, and saving the network parameter file. The method involves using a trained S3Mamba to predict and obtain a two-temporal change prediction map. This includes processing the two-temporal remote sensing images that need to be detected for change to a resolution that the model can accept, and then inputting them into the trained S3Mamba for prediction to obtain the two-temporal change prediction map.

9. A remote sensing change detection device based on Mamba region-spatial-semantic modeling, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the remote sensing change detection method based on Mamba region-spatial-semantic modeling as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements a remote sensing change detection method based on Mamba region-spatial-semantic modeling, as described in any one of claims 1-8.

Images