Remote sensing image semantic change detection method based on time-series remote sensing mamba

Abstract

This application discloses a method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba, belonging to the field of image processing technology. The method includes: firstly, lightweight local feature extraction is performed on the time-series remote sensing image, and then surface dynamic changes are captured to obtain multi-scale features; the multi-scale features are first stitched together, then features are extracted, and then stitched together hierarchically to obtain fused features; the fused features are classified to obtain a binary change mask; specific features are first extracted from the multi-scale features, and then stitched together hierarchically to obtain a semantic segmentation mask; the binary change mask and the semantic segmentation mask are multiplied to obtain a semantic change map. This application combines wavelet multi-scale analysis with a state-space model to achieve dual optimization of high-frequency noise suppression and long-term dynamic capture; after fusing dual-path features, it adaptively balances spatial details and temporal continuity, significantly improving detection accuracy in complex scenes.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba. Background Technology

[0002] High-resolution remote sensing imagery (HRI), with its sub-meter spatial resolution, can clearly characterize the geometric structure and texture details of surface targets, making it a core data source for land cover change detection. Semantic change detection (SCD) not only needs to locate the change area but also needs to analyze the change category (such as "farmland → building" or "woodland → bare land"), which places higher demands on the model's spatiotemporal feature modeling capabilities. Traditional methods, based on statistical models such as change vector analysis (CVA) and Markov random fields (MRF), rely on manually designed features and are difficult to adapt to complex scenarios. While recent deep learning methods (such as Siamese convolutional neural networks (CNN) and Transformers) have improved feature representation capabilities, they still face three major bottlenecks:

[0003] (1) Multi-level feature fragmentation: The lack of collaborative utilization between shallow details (edges, textures) and deep semantics (land cover categories) leads to missed detection of small-scale changes;

[0004] (2) Inefficient spatiotemporal modeling: The existing Transformer architecture has high computational complexity, making it difficult to adapt to long-term remote sensing data, and it is not sensitive enough to gradual changes (such as seasonal evolution of vegetation).

[0005] (3) Lack of dynamic interaction: Multi-temporal feature fusion often adopts simple splicing or weighted averaging, without modeling nonlinear correlations across time phases.

[0006] Existing technologies such as Scanning Network (SCanNet) and Semantic Scene Change Detection (SSCD) have introduced codec structure optimization for feature reconstruction, but they have not solved the aforementioned core problems, which has restricted their practical deployment in complex urban-rural mixed areas. Summary of the Invention

[0007] This application provides a method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba, in order to solve the problems of existing technologies in multi-level feature utilization, spatiotemporal dependency modeling and dynamic interaction.

[0008] This application provides a method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba, including:

[0009] The temporal remote sensing images from different time phases are input into the corresponding encoder. The encoder includes four cascaded visual state space blocks. The visual state space blocks first use depth-separable convolution to perform lightweight local feature extraction on the temporal remote sensing images. Then, the extracted features are used to capture dynamic changes on the land surface using a two-dimensional selective scanning module to obtain multi-scale features.

[0010] Multi-scale features are input into the fusion module, which stitches together the multi-scale features of the same level in the channel dimension, length dimension, and width dimension. Then, the visual state space blocks are used to extract features from the stitched features. The extracted features are stitched together across levels in order from deep to shallow to obtain the fused features.

[0011] A binary classifier is used to classify the fused features to obtain a binary transformation mask;

[0012] Multi-scale features are input into the decoder. The decoder uses a spatiotemporal multimodal module to extract specific features for each multi-scale feature. The specific features of each time phase are sequentially spliced ​​across layers in order from deep to shallow to obtain the semantic segmentation mask for each time phase.

[0013] Multiply the binary change mask by the semantic segmentation mask of the first phase to obtain the semantic change map of the first phase; multiply the semantic segmentation mask of the next phase by the binary change mask to obtain the semantic change map of the next phase.

[0014] In one possible implementation, before inputting the time-series remote sensing image into the encoder, the time-series remote sensing image is preprocessed as follows: the pixel values ​​of the time-series remote sensing image are normalized to [0,1]; dynamic scene enhancement is performed on the time-series remote sensing image after pixel value normalization; and the enhanced time-series remote sensing image is cropped into a tile of a set size.

[0015] In one possible implementation, preliminary feature extraction is performed using a convolutional layer with a kernel size of 3×3 before processing the first-level visual state space block in the four cascaded visual state space blocks.

[0016] In one possible implementation, in the channel dimension, the fusion module stitches multi-scale features along the channel axis to expand the number of feature channels; in the length dimension, the fusion module stitches multi-scale features along the spatial height direction to enhance vertical temporal correlation; in the width dimension, the fusion module stitches multi-scale features along the spatial width direction and swaps the first and last multi-scale features in the temporal phase to capture horizontal dynamic changes.

[0017] In one possible implementation, before the fusion module uses visual state space blocks to extract features from the stitched features, it uses a convolutional layer with a kernel size of 1×1 to perform preliminary feature extraction on the stitched features in order to adjust the number of channels and suppress redundant information.

[0018] In one possible implementation, the fusion module concatenates the extracted features across layers in descending order of depth, and then upsamples the concatenated features to gradually restore the spatial resolution of the features.

[0019] In one possible implementation, the method for extracting specific features from the spatiotemporal multimodal module includes: superimposing multi-scale features with dynamic positional encoding; decomposing the multi-scale features superimposed with dynamic positional encoding into low-frequency sub-bands and high-frequency sub-bands using discrete wavelet transform; extracting features from the high-frequency sub-band using depthwise separable convolution; generating cross-scale attention weights for the low-frequency sub-band and the features extracted from the high-frequency sub-band using wavelet transform; concatenating the low-frequency sub-band and the features extracted from the high-frequency sub-band using multi-head self-attention according to the cross-scale attention weights; and performing inverse wavelet transform on the features concatenated using multi-head self-attention to obtain specific features.

[0020] In one possible implementation, when the decoder performs cross-level splicing of the specific features of each time phase, it uses the spatiotemporal multimodal module to extract specific features again for the specific features of the deeper level to obtain intermediate specific features. The intermediate specific features are then spliced ​​with the specific features of the deeper level, and the spliced ​​features are upsampled to obtain the specific features of the shallower level, until the most shallow level specific features are obtained.

[0021] In one possible implementation, a multi-class classifier is used to classify the most superficial level of specific features to obtain a semantic segmentation mask.

[0022] In one possible implementation, both the binary classifier and the multi-class classifier are trained using a joint loss function, which includes cross-entropy loss, binary cross-entropy loss, cosine distance loss, and mean squared error loss.

[0023] The semantic change detection method for remote sensing images based on time-series remote sensing Mamba in this application has the following advantages:

[0024] (1) Multi-frequency-spatiotemporal joint modeling: combining wavelet multi-scale analysis with state space model to achieve dual optimization of high-frequency noise suppression and long-term dynamic capture.

[0025] (2) Lightweight temporal inference: Based on the linear computational complexity design of Mamba, it reduces the memory usage by 70% compared with the traditional Transformer, and is suitable for multi-temporal big data scenarios.

[0026] (3) Dynamic feature collaboration mechanism: By fusing dual-path features with learnable weights, the spatial details and temporal continuity are adaptively balanced, and the detection accuracy is significantly improved in complex scenarios such as vegetation seasonal changes and gradual urban expansion.

[0027] Experiments on the Hi-UCD mini (high-resolution urban change detection mini dataset) and WUSU datasets show that the proposed method achieves a semantic change detection score (F... scd The accuracy rates reached 89.83% and 85.42% respectively, which is 7%-12% higher than the mainstream methods, providing efficient and reliable technical support for the automated dynamic monitoring of high-resolution remote sensing images. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is an architecture diagram of the remote sensing image semantic change detection method based on time-series remote sensing Mamba provided in the embodiments of this application.

[0030] Figure 2 This is an architecture diagram of the encoder provided in an embodiment of this application.

[0031] Figure 3 This is an architecture diagram of the visual state space block provided in an embodiment of this application.

[0032] Figure 4 This is an architecture diagram of the fusion module provided in an embodiment of this application.

[0033] Figure 5 This is an architecture diagram of the decoder provided in an embodiment of this application.

[0034] Figure 6 This is an architecture diagram of the spatiotemporal multimodal module provided in an embodiment of this application.

[0035] Figure 7 The image shows the change detection results of the Hi-UCD dataset provided in this embodiment of the application.

[0036] Figure 8 The image shows the change detection results of the WUSU dataset provided in this embodiment of the application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Figure 1-6 This document presents an overall architecture diagram and its constituent parts of the remote sensing image semantic change detection method based on time-series remote sensing Mamba, as provided in the embodiments of this application. The embodiments of this application provide a remote sensing image semantic change detection method based on time-series remote sensing Mamba, comprising:

[0039] S100 inputs time-series remote sensing images from different time phases into corresponding encoders (Encoder1, Encoder2, and Encoder3). Each encoder comprises four cascaded visual state space blocks (VSS Blocks). The VSS Blocks first utilize depthwise separable convolution (DW Conv) to extract lightweight local features from the time-series remote sensing images. Then, the extracted features are used in a two-dimensional selective scan module (SS2D) to capture dynamic changes in the land surface, resulting in multi-scale features. F 1. F 2 and F 3).

[0040] For example, the embodiments of this application use time-series remote sensing images of three time phases, namely... I 1. I 2 and I 3.

[0041] Before inputting the time-series remote sensing images into the encoder, the following preprocessing is performed on the time-series remote sensing images:

[0042] The pixel values ​​of the time-series remote sensing images are normalized to [0,1].

[0043] Dynamic scene enhancement is performed on time-series remote sensing images after pixel value normalization to improve generalizability;

[0044] The enhanced time-series remote sensing imagery is cropped into tiles of a set size of 512×512 to fit the input of subsequent modules.

[0045] Furthermore, before processing the first-level visual state space block in the four cascaded visual state space blocks, preliminary feature extraction is performed using a 3×3 convolutional layer. The preliminary extracted features are represented as follows: F 0. Through this preliminary feature extraction operation, low-level texture and edge information of the image can be captured, providing basic features for subsequent temporal modeling.

[0046] Depthwise separable convolution is a lightweight local feature extraction operation for temporal remote sensing images. It can preserve spatial details while reducing the number of parameters. The number of output feature channels increases step by step, while the spatial resolution is halved step by step.

[0047] The two-dimensional selective scanning module is based on the state-space equation and models the temporal dependencies of multi-temporal images with linear complexity. It scans the feature map from four directions: top, bottom, left, and right. Combined with multi-dimensional feature stitching (channel, length, width), it forms 12 scanning directions (3 stitching methods × 4 scanning directions) to comprehensively capture dynamic changes on the land surface.

[0048] Among the features output by the four cascaded visual state space blocks, those related to the image... I The features corresponding to 1 can be represented as follows: , , and With images I The features corresponding to 2 can be represented as follows: , , and With images I The features corresponding to 3 can be represented as follows: , , and Among these features, shallow features , and It preserves high-resolution details, such as small areas of variation, while retaining deep features. , and This encodes global semantic information, such as the distribution of land cover categories.

[0049] S110, input the multi-scale features into the fusion module (Fusion). The fusion module stitches together the multi-scale features of the same level in the channel dimension, length dimension and width dimension. Then, it uses visual state space blocks to extract features from the stitched features. The extracted features are stitched together across levels in order from deep to shallow to obtain the fused features.

[0050] For example, in the channel dimension (dim=1), the fusion module stitches multi-scale features along the channel axis to expand the number of feature channels; in the length dimension (dim=2), the fusion module stitches multi-scale features along the spatial height direction to enhance vertical temporal correlation, such as vertical building expansion; in the width dimension (dim=3), the fusion module stitches multi-scale features along the spatial width direction and swaps the most recent and latest temporal features, for example, swapping... and The position of the feature is determined to capture lateral dynamic changes, such as the horizontal extension of a road. The purpose of swapping the positions of the two features is to avoid redundancy in the width and length dimensions.

[0051] Furthermore, before the fusion module extracts features from the stitched features using visual state space blocks, it uses a convolutional layer with a kernel size of 1×1 to perform preliminary feature extraction on the stitched features in order to adjust the number of channels and suppress redundant information.

[0052] During the feature extraction process using visual state space blocks, the fusion module utilizes the four-directional scanning mechanism of the two-dimensional selective scanning module to extract temporal dependencies from 12 directions (such as the continuous evolution of construction sites in multi-temporal images). Through the latent state transfer of the state space model (Mamba), long-range temporal correlations (such as cross-year vegetation degradation) are modeled.

[0053] The fusion module concatenates the extracted features across layers in descending order of depth. Then, it upsamples these concatenated features to gradually restore their spatial resolution. This upsampling operation can employ deconvolution or interpolation. The final fused feature representation is as follows: .

[0054] S120 uses a binary classifier to classify the fused features, resulting in multi-temporal binary change maps (MBCM).

[0055] S130: The multi-scale features are input into the decoders (Decoder1, Decoder2, and Decoder3). The decoders extract specific features for each multi-scale feature using a spatiotemporal multimodal module (STM Block). The specific features of each time phase are sequentially concatenated across layers in descending order of depth to obtain the semantic segmentation mask for each time phase. X 1. X 2 and X 3, also known as SSM 1. SSM 2 and SSM 3.

[0056] For example, a method for extracting specific features from a spatiotemporal multimodal module includes:

[0057] Encoding multi-scale features with dynamic location f ( x Superimposed to enhance spatial sensitivity;

[0058] The multi-scale features superimposed with dynamic position coding are decomposed into low-frequency subbands using Discrete Wavelet Transform (DWT Forward). F L and high-frequency subband F LH , F HL and F HH ;

[0059] Features in high-frequency subbands are extracted using depthwise separable convolution (DW Conv) to preserve local details (such as building edges and vegetation textures).

[0060] Cross-scale attention weights are generated by using discrete wavelet transform to generate low-frequency subbands and features extracted from high-frequency subbands. The low-frequency subbands and features extracted from high-frequency subbands are then concatenated using multi-head self-attention (MHSA) according to the cross-scale attention weights to model long-distance dependencies.

[0061] Specific features are obtained by performing inverse wavelet transform (DWI) on the features spliced ​​using multi-head self-attention.

[0062] Furthermore, when the decoder performs cross-level splicing of the specific features of each time phase, it also splices the specific features of the deeper level (e.g., The spatiotemporal multimodal module is used to extract specific features again to obtain intermediate specific features. These intermediate specific features are then concatenated with the more advanced specific features. After upsampling the concatenated features, a shallower level of specific features is obtained (e.g., ...). ), until the shallowest level of specific features (e.g. ).

[0063] Furthermore, a multi-class classifier is used to classify the most superficial level of specific features to obtain a semantic segmentation mask.

[0064] Furthermore, both the binary classifier and the multi-class classifier are trained using a joint loss function, which includes cross-entropy loss. L 1. Binary cross-entropy loss L 2. Cosine distance loss L 3 and mean square error loss L 4.

[0065] The above joint loss function Loss As shown in formula (1):

[0066] Loss = L 1+ L 2+ L 3+L 4(1)

[0067] Cross-entropy loss L 1 in Figure 5 This is represented as CE Loss in Chinese. L 1. The definition is shown in formula (2), where n Represents the number of categories. and They represent the first i The true label and predicted probability value for each category. i Loss is calculated starting from 1, excluding the influence of category 0 (the unchanged region) in the loss calculation.

[0068] L 1= (2)

[0069] Binary cross-entropy loss L 2 in Figure 5 This is represented as BCE Loss. L 2. The calculation method is shown in formula (3), where y and These represent the true label and the predicted probability value (i.e., y=0 or y=1), respectively.

[0070] (3)

[0071] Cosine distance loss L 3 in Figure 5 This is represented as Cosine Embedding Loss. L 3. Loss is used to enhance the generation X 1. X 2 and X The semantic consistency between 3 is calculated using formulas (4), (5) and (6).

[0072] + (4)

[0073] (5)

[0074] (6)

[0075] in, and They represent X 1 and X Cosine distance loss between 2 and X 2 and X The cosine distance loss between 3 and 3, cos() represents the calculation of the cosine distance. and They represent X 1 and X Between 2 and X 2 and X The probability value of the true label between 3 and 3.

[0076] Mean square error loss L 4 in Figure 5 This is represented as MSE Loss. L 4. The losses are shown in formulas (7), (8), and (9), and are calculated separately. X 1, X 2) and ( X 2, X 3) Euclidean distance, thereby reducing the distance between identical samples and increasing the distance between different samples.

[0077] + (7)

[0078] (8)

[0079] (9)

[0080] in, and They represent X 1 and X The mean square error loss between 2 and X 2 and X The mean square error loss between 3, , and These represent the semantic segmentation masks for each time phase.

[0081] After establishing the joint loss function, the training of the two classifiers is gradually transitioned from simple samples (single-phase) to complex multi-phase data. Furthermore, a cosine distance loss strategy is used to schedule the learning rate during the training process to avoid local optima.

[0082] S140, multiply the binary change mask with the semantic segmentation mask of the first phase to obtain the semantic change map of the first phase; multiply the semantic segmentation mask of the next phase with the binary change mask to obtain the semantic change map of the next phase.

[0083] For example, in Figure 5 middle, Y 1 is a semantic change diagram for the first time phase. Y 2 and Y The three are respectively related to time-series remote sensing images I 2 and IThe semantic change graph corresponding to point 3. The semantic segmentation mask will be output simultaneously with the semantic change graph. Y 1. Y 2 and Y 3 is also called Figure 1 In SCD 1. SCD 2 and SCD 3.

[0084] experiment

[0085] 1. Evaluation Indicators

[0086] This experiment primarily used four evaluation metrics to analyze the model's performance: Overall Accuracy (OA), Mean Intersection over Union (mIoU), Separated Kappa (SeK), and F-axis. scd score (F1-score, F scd ).

[0087] OA, or Optimal Analysis, is the ratio of the number of correctly classified samples to the total number of samples. It effectively represents the overall effectiveness of the model. Its definition is as follows:

[0088] (10)

[0089] In Formula 10, TP represents a true positive, TN represents a true negative, FP represents a false positive, and FN represents a false negative.

[0090] mIoU is an important evaluation metric, primarily used to measure the intersection-union ratio between predicted results and ground truth labels in image segmentation or object detection tasks. It is used to measure the segmentation quality of each class, and in this experiment, it is mainly used to detect changed and unchanged classes.

[0091] (11)

[0092] (12)

[0093] (13)

[0094] in, For confusion matrix, The sample is represented in the matrix, and s or t takes a value of 0 to represent the invariant class.

[0095] SeK primarily evaluates the performance of semantic change detection models in detecting semantic change categories, and is more sensitive to change types. Its calculation formula is as follows:

[0096] (14)

[0097] (15)

[0098] (16)

[0099] It emphasizes the accuracy of the results, especially its performance within areas of change. Therefore, it effectively captures the semantic category capabilities in semantic change detection tasks. Its definition is as follows:

[0100] (17)

[0101] (18)

[0102] (19)

[0103] 2. Ablation experiment

[0104] (1) Validation of the fusion module

[0105] By comparing ablation experiments of Base Mamba (without fusion module and STM Block) with Ver.1 (with only fusion module) and Ver.2 (with only STM Block), the temporal feature capture capability of the fusion module in multi-temporal remote sensing images was verified. Tables 1 and 2 show the evaluation metrics of each model on the two datasets.

[0106] Table 1 Quantitative results of ablation experiments on the Hi-UCD dataset

[0107]

[0108] Table 2 Quantitative results of ablation experiments on the WUSU dataset

[0109]

[0110] As shown in Tables 1 and 2, Ver.1 significantly outperforms Base Mamba. This is because the fusion module stitches together different sequence features along the temporal dimension and then captures the temporal dependencies in multi-temporal images through SS2D omnidirectional scanning. Therefore, Ver.1 demonstrates a significant performance improvement over Base Mamba, fully proving the effectiveness of the fusion module.

[0111] (2) Validation of STM Block

[0112] By comparing the ablation experiments of Base Mamba (without fusion module and STM Block) and Ver.2, the multi-frequency feature decomposition capability of STMBlock was verified. The evaluation metrics of the two models on various datasets are shown in Table 1 and Table 2.

[0113] Experiments show that Ver.2 significantly outperforms BaseMamba on all evaluation metrics across both datasets. This is because the STM Block, through the effective combination of wavelet transform and MHSA, enables the model to decompose features into low-frequency and high-frequency components. Compared to BaseMamba, Ver.2 demonstrates improvements across all performance metrics, proving the effectiveness of the STM Block in enhancing the model's ability to capture spatiotemporal dependencies and improving overall model performance.

[0114] Based on this, the complete model proposed in this application (a model integrating the fusion module and STM Block) performs optimally on multiple datasets.

[0115] (3) Loss L 3 and L Validity verification of 4

[0116] By comparing Base Mamba (using both) L 3 and L 4) with WoL3 (no L 3) WoL4 (None) L 4) The joint loss was verified to promote few-shot learning, and the data are shown in Table 3.

[0117] Table 3 Quantitative results of the experiment on the effectiveness of measuring loss on Hi-UCD

[0118]

[0119] 3. Comparative Experiment

[0120] (1) Three multi-temporal SCD task frameworks and Base Mamba were selected as comparative experiments:

[0121] HRSCD-4 (High Resolution Semantic Change Detection Model): Shares features between the semantic encoder and the change detection decoder, and improves performance through feature reinforcement.

[0122] SSCD-1 (Semantic Mask Change Detection Model): It abandons the binary change detection encoder and adopts Convs to fuse the semantic features of dual temporal images.

[0123] TED (Triple Encoder-Decoder Model): Based on SSCD-1, the Neck module is added to enhance the spatiotemporal information modeling capability.

[0124] Base Mamba: A simplified version based on the user framework, without the full optimization modules.

[0125] (2) Results of comparative experiments on the Hi-UCD dataset

[0126] Table 4 shows the quantitative evaluation results of each model on the Hi-UCD dataset. Experimental results indicate that the SCD framework based on dual-temporal remote sensing imagery does not perform ideally, with all metrics lower than those based on multi-temporal frameworks. The method in this application achieves better performance in mIoU, Sek, and... F scd Good experimental results were achieved on all key indicators.

[0127] Table 4 Quantitative Evaluation Results of the Hi-UCD Dataset

[0128]

[0129] like Figure 7 The image shows the change detection results on the Hi-UCD dataset. Based on the qualitative experimental results of the Hi-UCD dataset, the chart demonstrates the performance differences of different models in the land cover classification task through comparison of multiple sets of test images. In the (a1)-(a3) scenarios, the dual-temporal frame models (such as HRSCD-4 and SSCD-1) missed detections of grassland (light green) and incorrectly labeled buildings below (red), while the TED model showed scattered false detections (yellow / red noise) at the change edges. In contrast, the model proposed in this application accurately covers the grassland change area, with continuous green blocks and clear boundaries, highly consistent with the real labels. In the intermediate transition change scenarios in the (b1)-(b3) groups, the dual-temporal frame model resulted in a large area of ​​undetected areas (white) due to the lack of intermediate images, while the method in this application only had a small number of light green or red false detections at the edges, and the overall classification results of bare land (dark purple) and water (blue) matched the real labels. For groups (c1)-(c3), the dual-phase model completely fails (the pool area is completely white), and Base Mamba shows false positives for the diffusion of buildings (red) and roads (yellow), while the method in this application accurately captures the dynamic process of "water present-water absent-water present" (the blue block changes continuously).

[0130] As can be seen from the comprehensive charts, the prediction results of the method in this application are closest to the real label distribution in various scenarios. The color block boundaries are clear and the categories are concentrated. For example, there is no color mixing in grassland, building and bare land areas, which verifies the effectiveness of improving classification robustness through temporal modeling.

[0131] (3) Dataset WUSU

[0132] Table 5 shows the quantitative evaluation results of each model on the WUSU dataset. The experimental results show that the SCD frame based on dual-temporal remote sensing imagery performs worse than the model based on multi-temporal imagery. The method described in this application performs better on OA, mIoU, Sek, and... F scd The best results were achieved in all areas, especially in Sek and F scd The indicators show a significant improvement compared to the comparison method.

[0133] Table 5 Quantitative Evaluation Results of the WUSU Dataset

[0134]

[0135] like Figure 8 The image shows the change detection results on the WUSU dataset. Based on the qualitative experimental results of the WUSU dataset, in the construction site change scenarios (a1)-(a3), the two-temporal frameworks (HRSCD-4, SSCD-1, TED) neglected the temporal changes between (a2) and (a3), leading to misdetection of construction sites as grassland (misaligned green areas) and missed detection of low-rise buildings (broken red blocks). While Base Mamba outperformed the two-temporal method, it still misclassified some construction sites as grassland in (a3), whereas the method in this application accurately distinguished between buildings (continuous red) and construction sites (concentrated dark yellow), with clear and unmixed boundaries. For the continuous change scenarios (b1)-(b3), the two-temporal framework suffered from large-area missed detections (white areas) due to insufficient temporal modeling, while Base Mamba, in the multi-temporal framework, detected the main change area (continuous blue blocks), but there were scattered false detections (red noise). The method in this application fully covers the changing area (continuous blue blocks), with only slight false detections at the edges (yellow spots), and the overall result is highly consistent with the real labels. In the complex change scenarios (c1)-(c3), the two-temporal method misdetects bare land (light brown) as buildings (red diffusion), and TED even fails completely in (c3) (white area); Base Mamba performs better than the two-temporal framework in (c3), but is sensitive to the transition from grassland to construction site in (c1), leading to false detection of grassland (broken green area). The method in this application successfully captures the dynamics of bare land (concentrated light brown blocks) and the changes in grassland (continuous green blocks), and clearly classifies construction sites (dark yellow) and buildings (red), verifying the robustness of its multi-temporal time series modeling.

[0136] As can be seen from the comprehensive charts, the method of this application maintains high accuracy in detecting complex types of changes such as buildings, construction sites, and bare land. The color blocks are densely distributed and there is no category overlap, which is significantly better than the dual-temporal frame and Base Mamba.

[0137] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0138] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba, characterized in that, include: The temporal remote sensing images from different time phases are input into the corresponding encoder. The encoder includes four cascaded visual state space blocks. The visual state space blocks first use depth-separable convolution to perform lightweight local feature extraction on the temporal remote sensing images. Then, the extracted features are used to capture dynamic changes on the land surface using a two-dimensional selective scanning module to obtain multi-scale features. The multi-scale features are input into the fusion module, which splices the multi-scale features of the same level in the channel dimension, length dimension and width dimension. Then, the visual state space block is used to extract features from the spliced ​​features. The extracted features are spliced ​​across levels in order from deep to shallow to obtain the fused features. The fused features are classified using a binary classifier to obtain a binary variation mask; The multi-scale features are input into the decoder. The decoder extracts specific features for each multi-scale feature using a spatiotemporal multimodal module. The specific features of each time phase are sequentially spliced ​​across layers in order from deep to shallow to obtain the semantic segmentation mask for each time phase. The binary change mask is multiplied by the semantic segmentation mask of the first time phase to obtain the semantic change map of the first time phase; the semantic segmentation mask of the next time phase is multiplied by the binary change mask to obtain the semantic change map of the next time phase. The method for extracting the specific features by the spatiotemporal multimodal module includes: The multi-scale features are superimposed with dynamic location encoding; The multi-scale features superimposed with the dynamic position encoding are decomposed into low-frequency sub-bands and high-frequency sub-bands using discrete wavelet transform; Features in the high-frequency subbands are extracted using depthwise separable convolution. The low-frequency subband and the features extracted from the high-frequency subband are generated by wavelet transform, and the low-frequency subband and the features extracted from the high-frequency subband are concatenated using multi-head self-attention according to the cross-scale attention weights. The specific features are obtained by performing inverse wavelet transform on the features spliced ​​using multi-head self-attention.

2. The method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba according to claim 1, characterized in that, Before inputting the time-series remote sensing image into the encoder, the time-series remote sensing image undergoes the following preprocessing: The pixel values ​​of the time-series remote sensing images are normalized to [0,1]. Dynamic scene enhancement is performed on the time-series remote sensing image after pixel value normalization; The enhanced time-series remote sensing image is cropped into tiles of a set size.

3. The method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba according to claim 1, characterized in that, Before processing using the first level of the four cascaded visual state space blocks, preliminary feature extraction is performed using a convolutional layer with a kernel size of 3×3.

4. The method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba according to claim 1, characterized in that, In the channel dimension, the fusion module stitches the multi-scale features along the channel axis to expand the number of feature channels; in the length dimension, the fusion module stitches the multi-scale features along the spatial height direction to enhance the vertical temporal correlation; in the width dimension, the fusion module stitches the multi-scale features along the spatial width direction and swaps the multi-scale features with the earliest and latest temporal phases to capture lateral dynamic changes.

5. The method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba according to claim 1, characterized in that, Before the fusion module extracts features from the stitched features using the visual state space blocks, it performs preliminary feature extraction on the stitched features using a convolutional layer with a kernel size of 1×1 to adjust the number of channels and suppress redundant information.

6. The method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba according to claim 1, characterized in that, The fusion module performs cross-level stitching on the extracted features in descending order of depth, and then upsamples the cross-level stitched features to gradually restore the spatial resolution of the features.

7. The method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba according to claim 1, characterized in that, When the decoder performs cross-level splicing of the specific features of each time phase, it uses the spatiotemporal multimodal module to extract specific features again for the specific features of the deeper level to obtain intermediate specific features. The intermediate specific features are then spliced ​​with the specific features of the deeper level. The spliced ​​features are upsampled to obtain the specific features of the shallowest level, until the specific features of the shallowest level are obtained.

8. The method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba according to claim 7, characterized in that, The semantic segmentation mask is obtained by classifying the specific features at the shallowest level using a multi-class classifier.

9. The method for detecting semantic changes in remote sensing images based on time-series remote sensing Mamba according to claim 8, characterized in that, Both the binary classifier and the multi-class classifier are trained using a joint loss function, which includes cross-entropy loss, binary cross-entropy loss, cosine distance loss, and mean squared error loss.

Images