Mamba-based region-space-semantic modeling method and device for remote sensing change detection

Abstract

The application discloses a remote sensing image change detection method and device based on a Mamba-based region-space-semantic channel modeling, and the method comprises the following steps: inputting an obtained remote sensing image dataset into a constructed Mamba-based region perception, dual-domain space and semantic channel modeling network (S3Mamba) for training, obtaining a trained S3Mamba and saving the same; inputting a remote sensing image to be processed; and performing prediction on the remote sensing image to be processed by using the trained S3Mamba, so as to obtain a dual-time-phase change prediction map. The application improves the precision and performance of remote sensing image change detection by means of a three-way collaborative modeling mechanism coupling design.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of remote sensing image processing, and more particularly relates to a remote sensing change detection method and device based on region-space-semantic modeling of Mamba. BACKGROUND

[0002] Remote sensing image change detection is of great significance for urban construction, disaster assessment and environmental protection. Current mainstream methods mainly fall into three categories: methods based on convolutional neural networks can extract local features, but are limited by the local receptive field of convolutional operations, making it difficult to model long-distance spatial dependencies; methods based on Transformers can establish global context associations, but their self-attention mechanisms have a quadratic computational complexity problem, making them difficult to adapt to high-resolution remote sensing images; the recently proposed Mamba architecture improves efficiency through a state space model, but still has three major key defects:

[0003] (1) Insufficient processing of visually ambiguous regions: false changes caused by changes in light and vegetation growth, as well as real changes hidden by shadows, are not explicitly modeled;

[0004] (2) Weakening of local detail features: the global modeling process loses boundary details and high-frequency information, limiting the ability to identify small-scale change targets;

[0005] (3) Lack of cross-channel semantic interaction: channel-level feature interaction is insufficient, making it difficult to capture subtle semantic changes. SUMMARY

[0006] To overcome the above defects of the prior art, the present application proposes a remote sensing image change detection method and device based on region-space-semantic modeling of Mamba.

[0007] The present application integrates the innovatively designed region perception modeling branch, dual-domain spatial modeling branch and semantic channel modeling branch into the dual-domain spatial and semantic channel modeling network S3MambaNet, excludes false change interference, identifies real change regions of multiple scales and shapes, and strengthens fine-grained boundaries, successfully applying S3MambaNet to the field of remote sensing image analysis.

[0008] To achieve the above purpose, according to one aspect of the present application, a remote sensing image change detection method based on region-space-semantic modeling of Mamba is provided, comprising:

[0009] S1. Obtain a remote sensing image dataset, wherein the change regions or unchanged regions occurring in each pair of dual-time remote sensing images have the same size, shape and position in the corresponding change mask; the pixel value of the change region in each change mask is 1 or 255, and the pixel value of the unchanged region is 0;

[0010] S2. inputting a remote sensing image dataset into a Mamba-based region-aware, dual-domain space and semantic channel modeling network (S3MambaNet) for training, obtaining a trained S3MambaNet and saving, wherein a model structure of the S3MambaNet is: an encoder composed of a weight-shared conjoined backbone, a decoder composed of an S3Mamba module and a fusion module, the decoder including four stages, and each stage except the first stage including only an S3Mamba module includes an S3Mamba module and a fusion module; the S3MambaNet performs the following information processing: first, a dual-time remote sensing image is input into the encoder to obtain four scales of dual-time features; then, the four scales of dual-time features pass through the S3Mamba module in each of the four stages of the decoder; the fusion module is responsible for integrating the output features of the S3Mamba module in the current stage and the up-sampled output features of the previous stage; and the output of the final stage is generated by 4 times up-sampling to generate a prediction map;

[0011] The S3Mamba module includes a region-aware modeling branch, a dual-domain space modeling branch and a semantic channel modeling branch, and the outputs obtained after the dual-time features pass through the three branches are concatenated along the channel dimension, and then pass through a 1x1 convolution layer to obtain the output of the S3Mamba module.

[0012] S3. inputting a remote sensing image to be processed, performing prediction by the trained S3MambaNet to obtain a dual-time change prediction map.

[0013] In some optional embodiments, the remote sensing image dataset in step S1 includes:

[0014] The remote sensing image dataset is constructed by using paired dual-time images and their corresponding change masks. In order to ensure the consistency and comparability of the dataset, each pair of dual-time images and their corresponding change masks must be accurately corresponding and follow a unified file naming specification. This correspondence ensures that in each pair of images, whether it is a change region or a stable region, they all exhibit consistent geometric features in the corresponding change mask, including size, shape and spatial position. In addition, the change mask uses a binary representation method, in which the pixel values of the change region are coded as 1 or 255, and the pixel values of the unchanged region are set to 0.

[0015] In some optional embodiments, the training in the Mamba-based region-aware, dual-domain space and semantic channel modeling network (S3MambaNet) in step S2 includes:

[0016] constructing S3MambaNet, inputting the obtained dataset into the S3MambaNet, setting a batch size of each iteration, and training the S3MambaNet.

[0017] In some optional embodiments, the constructing S3MambaNet comprises a decoder composed of a weight-shared conjoined backbone, an S3Mamba module, and a fusion module.

[0018] In some optional embodiments, the weight-shared conjoined backbone comprises a feature extractor constructed by a parameter-shared pre-trained MambaBCD-tiny, and a pair of dual-time remote sensing images is input into the structure to generate dual-time features. 、 where i and j represent the i-th pair of dual-time features and the j-th stage of the feature extractor, respectively, and T1 and T2 represent two time points. Then the dual-time features are input into a decoder composed of an S3Mamba module and a fusion module.

[0019] In some optional embodiments, the region-aware modeling branch comprises three parallel sub-branches (a salient region sub-branch, a similar region sub-branch, and a fuzzy region sub-branch) that process the dual-time features simultaneously. Each sub-branch is composed of a specific region label generator (a salient region label generator, a similar label generator, and a fuzzy region label generator, respectively) and a visual state space module [Visual State Space Advances in Neural Information Processing Systems 37 (NeurIPS 2024)]. The three types of label generators are independently modeled by their respective visual state space modules, and their outputs are concatenated along the channel dimension to form a unified region-aware representation. Specifically, the generation strategies of the three specific region label generators are as follows:

[0020] The salient region label generator obtains the salient region label by performing absolute value subtraction on the dual-time features and then unfolding the results, as shown in equation (1):

[0021]

[0022] The similar region label generator first calculates a cosine similarity matrix of the dual-time features. In this matrix, the values of the regions with significant changes are significantly lower. Therefore, multiplying this matrix with the dual-time features can suppress the regions with significant changes while enhancing the regions with high similarity, thereby obtaining the similar region label. Subsequently, the similar region label is cross-ordered, as shown in equations (2) and (4):

[0023]

[0024] The formula (2) and formula (3) in And Sim respectively represent the cosine similarity function and the similarity score.

[0025] The fuzzy region label generator, the absolute value subtraction is carried out to the similar region label to obtain the fuzzy region label is obtained by the following formula (5):

[0026]

[0027] The double-domain space modeling branch includes two parallel branches (global context learner, wavelet transform local compensator), which simultaneously process double-time features. Specifically, the global context learner firstly generates global spatial labels by concatenating double-time features along the channel dimension through a spatial label generator, and then inputs the global spatial labels into a visual state space module for global context learning.

[0028] The wavelet transform local compensator concatenates double-time features along the channel dimension, and then reduces the channel dimension through a 1x1 convolution layer to generate a feature map . After the feature map is subjected to a discrete wavelet transform , a frequency domain representation is obtained. Then, local information is extracted through convolution operation, and the original spatial resolution is restored through inverse wavelet transform , and finally the feature map is output.

[0029]

[0030] The formula (6) and (7) in is a low-pass filter, is a high-pass filter.

[0031] Finally, the features obtained from the global context learner and the wavelet transform local compensator are fused through element-wise addition.

[0032] The semantic channel modeling branch includes: firstly, the double-time features concatenated along the channel dimension are subjected to channel dimension reduction through a 1x1 convolution layer to obtain double-time merged features . Then, the double-time merged features are sequentially unfolded and inversely unfolded to obtain sequential fine-grained features and inverse sequential fine-grained features . The two kinds of features are respectively grouped in channels to obtain bidirectional grouping labels , which are obtained by the following formulas (8)-(11):

[0033]

[0034] formula (8) and formula (7) 、 and respectively represent a 1x1 convolution operation, a sequential expansion operation, and a reverse sequential expansion operation, indicates that the channels are divided into G groups. Then the bidirectional group mark The input Mamba module [selective state space linear time sequence modeling Mamba: Linear-Time Sequence Modeling with Selective State Spaces Gu A, Dao T. Mamba: Linear-time sequence modeling with selective state spaces[J]. arXiv preprint arXiv:2312.00752, 2023] is modeled.

[0035] In some optional embodiments, the fusion module first adds high-level features to up-sampled low-level features. Then two 3x3 convolution layers and batch normalization processing are applied.

[0036] In some optional embodiments, the decoder includes four stages. The first stage directly processes the features output by the encoder through the S3Mamba module; the subsequent three stages each include an S3Mamba module and a fusion module, where the fusion module is responsible for integrating the current features with the up-sampled output features of the previous stage. The final stage outputs a prediction map generated by 4 times up-sampling.

[0037] In some optional embodiments, the trained S3MambaNet is obtained and saved, including: calculating the loss and updating the network parameters step by step, completing the training of the network and saving the parameter file of the network.

[0038] In some optional embodiments, the trained S3MambaNet is used for prediction to obtain a dual-time change prediction map, including: after processing the dual-time remote sensing image pair to be used for change detection into a resolution size that the model can receive, inputting it to the trained S3MambaNet for prediction to obtain a dual-time change prediction map, completing the method.

[0039] The second aspect of the application relates to a Mamba-based regional-spatial-semantic modeling remote sensing change detection device, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a Mamba-based regional-spatial-semantic modeling remote sensing change detection method of the application.

[0040] The third aspect of the application relates to a computer-readable storage medium having a program stored thereon, wherein the program is executed by a processor to implement a Mamba-based regional-spatial-semantic modeling remote sensing change detection method of the application.

[0041] The working principle of the application is as follows:

[0042] The core working principle of the application is to construct a Mamba-based regional perception, dual-domain space and semantic channel modeling network S3MambaNet, and the S3MambaNet realizes remote sensing image change detection through regional-spatial-channel three-dimensional coupling. The S3MambaNet is composed of a weight-shared conjoined backbone, an S3Mamba module and a fusion module to form a decoder. First, the dual-time remote sensing image is input into the conjoined backbone to extract features and input into the decoder composed of the S3Mamba module and the fusion module. The S3Mamba module simultaneously focuses on key regions, global space and local details, and fine-grained channel relationships through three modeling mechanisms, thereby enhancing the multi-dimensional information of the features. Then, the fusion module fuses the decoder features at different stages to increase the multi-scale information. Finally, the dual-time change prediction map is generated through the decoding stage to complete the change detection of the remote sensing image.

[0043] The advantages of the application are as follows:

[0044] The application proposes a Mamba-based regional perception, dual-domain space and semantic channel modeling network (S3MambaNet), which significantly improves the accuracy and performance of remote sensing image change detection through a triple-collaborative modeling mechanism coupling design. By designing a regional perception modeling branch, it focuses on explicitly separating false changes from real changes in visually ambiguous regions. In addition, combined with the dual-domain space modeling branch, the frequency-space joint optimization of global context learning and wavelet local compensation is used to break through the boundary ambiguity bottleneck. Finally, our S3MambaNet uses a bidirectional channel grouping mechanism to realize pixel-level cross-channel interaction through a semantic channel modeling branch, and uses rich semantics as clues to infer changes in different shapes. In summary, the application not only theoretically proves the significant advantages of deep learning technology in the field of remote sensing image analysis, but also demonstrates its excellent performance and wide applicability in practical applications. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of the method of the present application.

[0046] Figure 2 is a model diagram of the Mamba-based region-aware, dual-domain spatial and semantic channel modeling network S3MambaNet of the present application. Figure 2 is a schematic diagram of the S3MambaNet overall architecture of (a) part, Figure 2 is a schematic diagram of the S3Mamba module and fusion module of the present application of (b) part; Figure 2 is a region-aware modeling branch of the present application of (c) part; Figure 2 is a dual-domain spatial modeling branch of the present application of (d) part; Figure 2 is a semantic channel modeling branch of the present application of (e) part.

[0047] Figure 3 is a change detection effect diagram of the Mamba-based region-aware, dual-domain spatial and semantic channel modeling network (S3MambaNet) of the present application. Among them, the odd rows (s1, s3, s5, s7) show the prediction change diagram; the even rows (s2, s4, s6, s8) show the corresponding error diagram. In the prediction diagram, different colors represent the prediction correctness: true positive (white), false positive (green), true negative (black) and false negative (red). In the error diagram, the yellow highlights the errors, and the blue represents the correct prediction. The Image1, Image2 column is the real dual-time remote sensing image, the BIT, DMINet, SEIFNet, ChangeMamba, S3MambaNet column is the prediction result of different networks, and the GT is the real change mask.

[0048] Figure 4 is a change detection generalization performance diagram of the Mamba-based region-aware, dual-domain spatial and semantic channel modeling network S3MambaNet of the present application on the SYSU dataset. Among them, different colors represent the use of different percentages of test data, and the BIT, ICIFNet, DMINet, SEIFNet, ChangeMamba-B, S3MambaNet column is the change detection generalization performance of different networks. The F1-Score (%) of the ordinate in FIG. 4 represents the F1 score, which is a key indicator for evaluating the performance of a binary classification model in statistics.

[0049] Figure 5 is a structural schematic diagram of the device of the present application. DETAILED DESCRIPTION

[0050] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the protection scope of the present application.

[0051] As Figure 1 , the embodiment provides a remote sensing image change detection method based on Mamba regional-space-semantic modeling, comprising:

[0052] Step S1, acquiring a remote sensing image dataset; adjusting the dataset to an input picture size .

[0053] Step S2, constructing a Mamba-based regional perception, dual-domain space and semantic channel modeling network S3MambaNet, as Figure 2 . The S3MambaNet includes a weight-shared conjoined backbone, an S3Mamba module and a decoder composed of a fusion module.

[0054] The weight-shared conjoined backbone is a feature extractor constructed by a parameter-shared pre-trained MambaBCD-tiny, which inputs a pair of dual-time remote sensing images into the structure to generate dual-time features 、 , wherein i, j represent the i-th pair of dual-time features and the j-th stage of the feature extractor, respectively, and T1, T2 represent two time points. Then the dual-time features are input into the decoder composed of the S3Mamba module and the fusion module.

[0055] The S3Mamba module includes a regional perception modeling branch, a dual-domain space modeling branch and a semantic channel modeling branch. The outputs obtained after the dual-time features pass through the three modeling branches are concatenated along the channel dimension, and then pass through a 1x1 convolutional layer to obtain the output of the S3Mamba module.

[0056] The regional perception modeling branch includes three parallel sub-branches (obvious region branch, similar region branch and fuzzy region branch) that process the dual-time features at the same time. Each sub-branch is composed of a specific region label generator (obvious region label generator, similar label generator and fuzzy region label generator) and a visual state space module [Visual State Space Advances in Neural Information Processing Systems 37 (NeurIPS 2024)]. The three types of labels are independently modeled through the respective visual state space modules, and their outputs are connected along the channel dimension to form a unified regional perception representation. Specifically, the generation strategies of the three specific region label generators are as follows:

[0057] The salient region label generator comprises: the absolute value subtraction of the double-time feature is unfolded to obtain the salient region label, which is obtained by the following formula (1):

[0058]

[0059] The similar region label generator first calculates the cosine similarity matrix of the double-time feature. In the matrix, the value of the salient change region is obviously low. Therefore, multiplying the matrix with the double-time feature respectively can suppress the salient change region while enhancing the high similarity region to obtain the similar region label. Subsequently, the similar region label is cross-ordered to obtain the following formula (2)-formula (4):

[0060]

[0061] In formula (2) and formula (3), the and Sim represent the cosine similarity function and the similarity score respectively.

[0062] The fuzzy region label generator comprises: the absolute value subtraction of the similar region label to obtain the fuzzy region label, which is obtained by the following formula (5):

[0063]

[0064] The dual-domain space modeling branch comprises two parallel branches (global context learner, wavelet transform local compensator) and simultaneously processes the double-time feature. Specifically, the global context learner firstly generates the global spatial label by concatenating the double-time feature along the channel dimension through the spatial label generator, and then the global spatial label is input into the visual state space module for global context learning.

[0065] The wavelet transform local compensator concatenates the double-time feature along the channel dimension and then performs channel dimension reduction through a 1×1 convolution layer to generate a feature map . The feature map is subjected to discrete wavelet transform to obtain a frequency domain representation . Subsequently, local information is extracted through convolution operation, and then the original spatial resolution is restored through inverse wavelet transform to finally output a feature map , which is obtained by the following formula (6) and formula (7):

[0066]

[0067] In formula (6) and formula (7), the is a low-pass filter, is a high-pass filter.

[0068] Finally, the features obtained from the global context learner and the wavelet transform local compensator are fused by element-wise addition.

[0069] The semantic channel modeling branch includes: first, the double-time features spliced along the channel dimension are reduced in channel dimension by a 1x1 convolution layer to obtain double-time merged features . Then, the double-time merged features are sequentially expanded and inversely expanded to obtain sequential fine-grained features and inverse order fine-grained features . The two features are respectively grouped in channel to obtain bidirectional grouping labels which are obtained by the following formulas (8)-(11):

[0070]

[0071] In formulas (8)-(7), 1x1 convolution operation, sequential expansion operation and inverse order expansion operation are represented by , and respectively, indicates that the channels are divided into G groups. Then the bidirectional grouping labels are input into the Mamba module [Linear-Time Sequence Modeling with Selective State Spaces Mamba: Linear-Time Sequence Modeling with Selective State Spaces Gu A, Dao T. Mamba: Linear- time sequence modeling with selective state spaces[J]. arXiv preprint arXiv: 2312.00752, 2023 ].

[0072] The fusion module includes: first, the high-level features are added to the up-sampled low-level features. Then two 3x3 convolution layers and batch normalization processing are applied.

[0073] The decoder includes: four stages. The first stage directly processes the features output by the encoder through the S3Mamba module; the subsequent three stages each contain an S3Mamba module and a fusion module, where the fusion module is responsible for integrating the current features with the up-sampled output features of the previous stage. The final stage outputs a prediction map through 4 times up-sampling.

[0074] Step S3, input the obtained remote sensing image dataset into the constructed S3MambaNet for training, perform loss calculation and update network parameters for each batch, until all batches of training data are trained, finally obtain the trained weights, and all updated parameters are saved in the weight file. Get the trained S3MambaNet and save it.

[0075] Step S4, input the remote sensing image to be processed, and predict by the trained S3MambaNet to obtain a double-time change prediction map. The performance indicators (F1 / IOU / OA) of S3MambaNet and the current other optimal model are shown in Table 1 below, and the bold indicates the optimal model indicator:

[0076] Table 1 Quantitative comparison on SYSU and GZ datasets

[0077]

[0078] SYSU dataset is a remote sensing change detection dataset provided by Sun Yat-sen University, aiming to support change identification research in complex scenarios. The dataset contains 20,000 pairs of ortho aerial images, with a fixed spatial size of 256x256 pixels for each pair of images, and a spatial resolution as high as 0.5 meters. The outstanding feature of SYSU is its diversity of covered targets: in addition to common buildings, it also contains various ground object types such as ships, roads, and vegetation, which brings greater challenges to the change detection task, as the model needs to distinguish between real changes and pseudo changes (such as seasonal or lighting differences). In data division, SYSU is divided into 12,000 pairs of training samples, 4,000 pairs of validation samples, and 4,000 pairs of test samples, ensuring the comprehensiveness and reliability of model evaluation.

[0079] GZ dataset is a super-high resolution satellite image change detection dataset, focusing on spatio-temporal change analysis in the suburbs of Guangzhou, China. The spatial resolution of this dataset is 0.55 meters, with a time span from 2006 to 2019. Through GoogleEarth service, 19 pairs of images with seasonal changes were obtained, with original image sizes ranging from 1006x1168 to 4936x5224 pixels. In data division, all images were uniformly divided into 2,504 pairs of training samples, 313 pairs of validation samples, and 313 pairs of test samples. The advantage of GZ dataset is its high resolution and long time span, which can effectively capture subtle differences such as urban expansion and land use change, but at the same time, seasonal changes introduce pseudo-change interference, which puts higher requirements on the model's ability to exclude pseudo changes.

[0080] Embodiment 2

[0081] This embodiment relates to a remote sensing image change detection device based on Mamba regional-spatial-semantic modeling, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the remote sensing image change detection method based on Mamba regional-spatial-semantic modeling of embodiment 1.

[0082] As Figure 5At the hardware level, the apparatus includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the above Figure 1 The method. Of course, in addition to the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0083] For the sake of brevity, conventional techniques and technologies related to semiconductor circuits can not be described in detail herein. Work implementing embodiments described herein can employ any of a number of semiconductor circuit technologies, including but not limited to: bipolar, field effect, and / or other types of integrated circuitry. It is to be understood that the implementing system, apparatus, and / or device can be one integrated circuit. However, for ease of illustration and understanding, portions of the implementing system, apparatus, and / or device can instead be illustrated and described as multiple discrete components, which can actually be integrated together within a common integrated circuit package. Alternatively, the implementing system, apparatus, and / or device can be implemented by discrete components that are distributed across multiple integrated circuit packages.

[0084] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0085] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0086] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this invention, the functions of each unit can be implemented in one or more software and / or hardware components.

[0087] Example 3

[0088] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements the remote sensing image change detection method based on Mamba region-spatial-semantic modeling of Embodiment 1.

[0089] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0090] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 Figure 1

[0091] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams. Figure 1 Figure 1

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams. Figure 1 Figure 1

[0093] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0094] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media. ​​​​​​

[0095] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0096] It should also be noted that the terms "comprising", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0097] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0098] The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The present application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

[0099] The various embodiments in the present application are described in a progressive manner, and the same or similar parts among the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0100] The above merely describes the embodiments of the present application, and is not intended to limit the present application. The present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for detecting changes in remote sensing images based on Mamba region-spatial-semantic modeling, characterized in that, include: S1. Obtain a remote sensing image dataset, in which the changed or unchanged regions in each pair of dual-time remote sensing images have the same size, shape, and position in the corresponding change mask; in each change mask, the pixel value of the changed region is 1 or 255, and the pixel value of the unchanged region is 0. S2. Input the remote sensing image dataset into the constructed Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network S3MambaNet for training. The trained S3MambaNet model is then saved. The S3MambaNet model structure consists of an encoder with a weight-shared conjoined backbone, and a decoder composed of S3Mamba modules and a fusion module. The decoder comprises four stages; except for stage 1, which includes only one S3Mamba module, the other stages include one S3Mamba module and one fusion module. S3MambaNet performs the following information processing: First, the dual-time remote sensing image is input into the encoder to obtain dual-time features at four scales. Then, the four-scale dual-time features are processed by the S3Mamba modules in the four stages of the decoder. The fusion module integrates the output features of the current stage's S3Mamba module with the output features of the previous stage after upsampling. Finally, the output of the final stage is upsampled by 4 times to generate a prediction map. The S3Mamba module includes a region-aware modeling branch, a dual-domain spatial modeling branch, and a semantic channel modeling branch. The outputs of the dual-temporal features obtained after passing through the three branches are concatenated along the channel dimension, and then passed through a 1×1 convolutional layer to obtain the output of the S3Mamba module. The region-aware modeling branch includes parallel explicit region sub-branches, similar region sub-branches, and ambiguous region sub-branches, simultaneously processing dual-time features. Each sub-branch consists of a label generator for a specific region and a visual state space module. Each sub-branch is independently modeled using its respective visual state space module, and their outputs are connected along the channel dimension to form a unified region-aware representation. The generation strategy is as follows: A salient region label generator is used to expand the bitemporal features after absolute value subtraction to obtain salient region labels. The similar region label generator first calculates the cosine similarity matrix of the two time features; then multiplies the cosine similarity matrix by the two time features to obtain similar region labels; subsequently, the similar region labels are cross-sorted. A fuzzy region marker generator generates fuzzy region markers by subtracting the absolute values ​​of similar region markers. S3. Input the remote sensing image to be processed, and use the trained S3Mamba to make predictions to obtain a two-phase change prediction map.

2. The method according to claim 1, characterized in that, Step S2 involves training the input into the constructed Mamba-based region-aware, dual-domain spatial, and semantic channel modeling network S3Mamba, including: To build an S3Mamba instance, input the obtained dataset into S3Mamba, and set the batch size and training rounds for each iteration.

3. The method according to claim 1, characterized in that, The weight-sharing conjoined backbone is a feature extractor constructed using a pre-trained MambaBCD-tiny with shared parameters. A pair of bi-temporal remote sensing images are input into this structure to generate bi-temporal features. , , where i and j represent the i-th bitemporal feature pair and the j-th stage of the feature extractor, respectively, and T1 and T2 represent two time points; then the bitemporal features are input into the decoder composed of the S3Mamba module and the fusion module.

4. The method according to claim 1, characterized in that, The fusion module first adds the high-level features to the upsampled low-level features; then it applies two 3×3 convolutional layers and batch normalization.

5. The method according to claim 1, characterized in that, Step S2, which involves obtaining and saving the trained S3Mamba, includes: calculating the loss iteratively and updating the network parameters, completing the network training, and saving the network parameter file. The method involves using a trained S3Mamba to predict and obtain a two-temporal change prediction map. This includes processing the two-temporal remote sensing image pairs that need to be detected for change to a resolution that the model can accept, and then inputting them into the trained S3Mamba for prediction to obtain the two-temporal change prediction map.

6. A remote sensing change detection device based on Mamba region-spatial-semantic modeling, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the remote sensing change detection method based on Mamba region-spatial-semantic modeling as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements a remote sensing change detection method based on Mamba region-spatial-semantic modeling, as described in any one of claims 1-5.

Images