A dual-stream remote sensing image change detection method integrated with Mamba enhancement

Abstract

This invention discloses a dual-stream remote sensing image change detection method that incorporates Mamba enhancement. The method belongs to the field of image processing and includes the following steps: image acquisition to obtain a remote sensing image dataset; preprocessing the acquired dual-temporal remote sensing images and dividing them into a training set and a test set; designing a dual-stream change detection network model that incorporates Mamba enhancement; training the constructed dual-stream change detection network model using the training set data until the entire model converges, and saving the optimal model; inputting the test set data into the trained optimal model to predict the changed regions in the test set. The method introduces MambaBlock and a semantic segmentation aggregation module to enhance global feature modeling capabilities under linear complexity. By integrating feature information from different levels and learning multi-scale features, the accuracy and robustness of detecting changed regions are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a dual-stream remote sensing image change detection method integrated with Mamba enhancement. Background Art

[0002] With the rapid development of remote sensing technology, change detection plays a vital role in urban planning, disaster monitoring, and other fields. Traditional change detection methods rely primarily on manual feature extraction and threshold judgment, resulting in low efficiency and limited accuracy. In recent years, deep learning has made significant progress in the field of change detection. However, existing methods often use traditional attention mechanisms for feature enhancement, which is computationally complex and difficult to capture long-range dependencies. Furthermore, insufficient temporal feature modeling of bi-temporal images limits the accuracy of identifying changed regions. Change detection accuracy is low in edge regions, making it difficult to accurately locate the boundaries of changed regions.

[0003] To address these issues, researchers have proposed various improvements, such as using lightweight backbone networks to reduce computational complexity and employing multi-scale feature fusion to improve detection accuracy. However, these methods still fail to adequately address the issues of feature representation and temporal modeling, and face challenges in accuracy and efficiency in practical applications. Existing change detection models exhibit limitations when processing complex scenes, particularly when processing detailed, high-resolution remote sensing imagery, which struggles to accurately capture local change features. Furthermore, the high computational overhead of traditional attention mechanisms limits their effectiveness in practical applications. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a dual-stream remote sensing image change detection method integrated with Mamba enhancement, which has a simple algorithm and high detection accuracy.

[0005] The technical solution of the present invention to solve the above technical problems is: a dual-stream remote sensing image change detection method integrated with Mamba enhancement, comprising the following steps:

[0006] S1, Image acquisition: Use remote sensing image acquisition equipment to obtain remote sensing images of the same area at different times, namely, dual-temporal remote sensing images, to obtain a remote sensing image dataset;

[0007] S2, data preprocessing: preprocess the acquired dual-temporal remote sensing images and divide them into training and test sets;

[0008] S3, Model Building: Design a dual-stream change detection network model RFAMNet that integrates Mamba enhancement. RFAMNet uses a lightweight MobileNet as the backbone network, constructs a dual-stream feature preprocessor LightWeightEncoder and a feature enhancement module FRM, and then generates a change detection map through the semantic segmentation aggregation module SSAM, the channel interaction module CIM, and the global decoder Decoder. The feature enhancement module FRM includes Mamba sequence modeling, enhanced global feature modeling and edge perception processing capabilities. The specific process of model building is as follows:

[0009] S31: Build a dual-stream feature preprocessor LightWeight Encoder to input dual-temporal remote sensing images into a lightweight MobileNet to extract multi-scale feature representations;

[0010] S32: Construct a feature enhancement module (FRM) that integrates Mamba sequence modeling to dynamically capture long-range dependencies and edge perception.

[0011] S33: Establish a semantic segmentation aggregation module SSAM to integrate multi-scale difference features and mine global semantic information;

[0012] S34: Establish a channel interaction module CIM to enhance the interaction between features of different sizes and highlight the features of the changing area;

[0013] S35: Build a global decoder, restore the spatial resolution, and generate a change detection map;

[0014] S4, model training: Use the training set data to train the constructed two-stream change detection network model until the entire model converges, and save the optimal model;

[0015] S5, model validation reasoning: input the test set data into the trained optimal model, predict the change area in the test set, and calculate the intersection and union ratio between the predicted change area and the actual change area. , and perform statistics based on all categories of the change area to obtain the average intersection-union ratio ,Evaluate the prediction accuracy of change detection.

[0016] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, in step S1, the remote sensing image dataset S obtained is:

[0017]

[0018] in, express area Image data of the period, express area Image data of the period.

[0019] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, the specific process of step S2 is as follows:

[0020] S21: Image annotation;

[0021] Label the target categories of the changed areas of the remote sensing image dataset to obtain the labeled remote sensing image dataset for:

[0022]

[0023] in, Represents remote sensing images 、 Comparative change area category label map;

[0024] S22: Image segmentation;

[0025] right The sliding window method is used to segment the image into 1024×1024 size images and label datasets:

[0026]

[0027]

[0028] in, express The first A 1024×1024 sample image, express The first A 1024×1024 sample image, is with 、 Corresponding contrast change area label map;

[0029] S23: Dataset division: The sample dataset is randomly divided into training set, validation set and test set in the ratio of 8:1:1;

[0030] S24: Data preprocessing: Normalize and enhance the original dual-temporal remote sensing images.

[0031] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, in step S22, the sliding window method is specifically operated as follows:

[0032] First, the remote sensing image is divided into multiple 1024×1024 pixel unit grids. If the remote sensing image cannot be divided proportionally, it is filled. The step length of each unit grid is recorded as ; Then the marked area is sampled through the sliding window while retaining the relative position information of the window center; the obtained sliding window sampling image is :

[0033]

[0034] in 、 Indicates the horizontal and vertical coordinates of the center position of the sliding window, Indicates the sliding step length;

[0035] For the edge areas of the image, a mirror filling method is used to ensure that the edge areas are completely sampled, and the image size after filling remains at 1024×1024.

[0036] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, the specific process of step S31 is as follows:

[0037] Images of the period 、 Images of the period And the corresponding label data After the dual-stream feature preprocessor Lightweight Encoder, preprocessing features are extracted , then preprocess the features Extract multi-scale features through lightweight MobileNet to obtain global feature maps ,in for The global feature map obtained by the image of the period, for The global feature map obtained from the image of the period;

[0038] The dual-stream feature preprocessor Lightweight Encoder extracts features through a lightweight MobileNet. The network uses a weight-sharing mechanism and is composed of a Conv3×3 convolution operation, BatchNorm normalization, and a ReLU activation function. MobileNet extracts multi-scale features through deep separable convolution DSConv and outputs multi-scale feature representations. , , are features of 5 different scales, where ;

[0039] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, the specific process of step S32 is as follows:

[0040] The preprocessed multi-scale features Input to the feature enhancement module FRM, with specific level features As a boundary, the features below this specific level are and First, downsample to the target resolution through maximum pooling, then align the channels through Conv1×1 convolution and extract local features through DSConv3×3 depth-separable convolution; then, features above the specific level are added. and Through Conv1×1 convolution alignment channel, DSConv3×3 depth separable convolution, BatchNorm normalization and ReLU activation function processing, bilinear upsampling to the target resolution; then each level feature is input into the MambaBlock module for sequence modeling enhancement; serialization processing, the two-dimensional feature map Rearrange into a one-dimensional sequence , , , is the field of real numbers, All are dimensions; then state space modeling is performed, and the long-range dependencies are captured by parameterized state space model SSM, the formula is:

[0041]

[0042] in 、 、 、 is a learnable parameter, is the input signal, is the hidden state, k represents the position in the feature sequence;

[0043] Then output the sequence Inverse mapping to a two-dimensional feature map , , ; By normalizing the enhanced features and connecting them with the input residual, the output Level Enhancement Features , the enhanced features of each level Splicing by channel dimension, sequentially through Conv1×1 convolution and DSConv3×3 depth separable convolution, further fusing cross-scale feature information, and performing residual connection between the fused features and the original features of the target level, and finally outputting the enhanced Period Level Features , , .

[0044] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, the specific process of step S33 is as follows:

[0045] The difference calculation of the dual-phase multi-scale features enhanced by the feature enhancement module FRM is performed through the formula Get the first Level difference characteristics ,in It represents the operation of taking the absolute value after element-by-element subtraction. express Period Level features, express Period Level 3 difference features As a low-level differential feature , Level 4 difference features As a high-level differentiating feature ;

[0046] Level 4 difference features and Level 5 difference features Input to the semantic segmentation aggregation module SSAM, first Perform bilinear upsampling to make Resolution and The two are then directly added element by element to obtain the coarse global semantic features; then the number of channels is expanded through the Conv3×3 convolution operation, and the channel dimension is evenly divided into four sub-features. Each sub-feature is subjected to multi-scale feature learning through dilated convolution with different void ratios; finally, the four sub-features after multi-scale feature learning are concatenated and feature fused through Conv1×1 convolution and DSConv3×3 depth-separable convolution to obtain the global semantic feature. .

[0047] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, the specific process of step S34 is as follows:

[0048] The obtained low-level difference features , advanced differentiation features and global semantic features Input to the channel interaction module CIM for processing; first 、 、 Perform global average pooling GAP and Conv1×1 convolution operations respectively to obtain the corresponding vectors for calculating attention weights 、 and ; then 、 and Reshape the dimensions so that and 、 Perform matrix multiplication separately to obtain two similarity matrices, add the two similarity matrices and generate attention weights through the sigmoid function ;Will After remodeling Perform matrix multiplication, then process through Conv1×1 convolution, batch BatchNorm normalization and ReLU activation function, and Establish residual connections to obtain enhanced difference features .

[0049] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, the specific process of step S35 is as follows:

[0050] Will 、 and Input to the decoding unit DU for processing; first and After Conv1×1 convolution, and then bilinear upsampling and The resolution and Same, will 、 and After element-by-element addition, the result is input into DSConv3×3 depth-separable convolution, and finally the prediction result of the current level is output through Conv1×1 convolution.

[0051] In the above-mentioned dual-stream remote sensing image change detection method integrated with Mamba enhancement, the specific process of step S4 is as follows:

[0052] S41: Given a training set of dual-temporal remote sensing images and their corresponding real label data;

[0053] S42: Input the paired remote sensing images into RFAMNet, and after processing by RFAMNet, output the prediction results of change detection ;

[0054] S43: Using binary cross entropy loss function and the dice loss function Constructing the total loss function for:

[0055]

[0056] in Indicates the level of the decoder, corresponding to arrive The prediction results after decoding the difference features; 、 Decoders Layer-level binary cross entropy loss function and dice loss function;

[0057] Binary cross entropy loss function formula for:

[0058]

[0059] in and are the width and height of the input remote sensing image, Representing coordinates The true label of the pixel at Representing coordinates The predicted value of the pixel at ;

[0060] Dice loss function formula for:

[0061]

[0062] in Represents the true label and prediction results The intersection size of express The size of express size;

[0063] Minimize the total loss function through the back-propagation algorithm , iteratively optimize the model parameters until The value converges and the model reaches the optimal value;

[0064] During the iterative optimization training process, the validation set is used to verify the model training accuracy in real time, and the model weights with the highest accuracy are saved.

[0065] The beneficial effects of the present invention are as follows: the method proposed in the present invention improves accuracy compared with other algorithms, specifically: 1) using a lightweight MobileNet as the backbone network for feature extraction, reducing model parameters and computational complexity, and improving the operating efficiency of the model while ensuring detection accuracy; 2) introducing MambaBlock and the semantic segmentation aggregation module SSAM to enhance the global feature modeling capability under linear complexity, and by fusing feature information at different levels and multi-scale feature learning, improves the detection accuracy and robustness of the changed area; 3) the channel interaction module CIM highlights the features of the changed area by exploring the correlation between features at different levels in the channel dimension, suppresses background interference, and further improves detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is the overall flow chart of the present invention.

[0067] Figure 2 This is a structural block diagram of the dual-stream change detection network model RFAMNet of the present invention.

[0068] Figure 3 It is a structural diagram of the feature enhancement module FRM of the present invention.

[0069] Figure 4 for area Schematic diagram of 5 remote sensing images of the period.

[0070] Figure 5 for area Schematic diagram of 5 remote sensing images of the period.

[0071] Figure 6 The figure is a schematic diagram of a predicted image of a contrast change region obtained by using the present invention. DETAILED DESCRIPTION

[0072] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0073] like Figure 1 As shown, a dual-stream remote sensing image change detection method integrated with Mamba enhancement includes the following steps:

[0074] S1, Image Acquisition: Use remote sensing image acquisition equipment to obtain remote sensing images of the same area at different times, namely, dual-temporal remote sensing images, to obtain a remote sensing image dataset.

[0075] The obtained remote sensing image dataset S is:

[0076]

[0077] in, express area Image data of the period, express area Image data of the period.

[0078] S2, data preprocessing: preprocess the acquired dual-temporal remote sensing images and divide them into training set and test set.

[0079] The specific process of step S2 is:

[0080] S21: Image annotation;

[0081] Label the target categories of the changed areas of the remote sensing image dataset to obtain the labeled remote sensing image dataset for:

[0082]

[0083] in, Represents remote sensing images 、 Comparative change area category label map;

[0084] S22: Image segmentation;

[0085] right The sliding window method is used to segment the image into 1024×1024 size images and label datasets:

[0086]

[0087]

[0088] in, express The first A 1024×1024 sample image, express The first A 1024×1024 sample image, is with 、 Corresponding contrast change area label map;

[0089] The specific operation of the sliding window method is as follows:

[0090] First, the remote sensing image is divided into multiple 1024×1024 pixel unit grids. If the remote sensing image cannot be divided proportionally, it is filled. The step length of each unit grid is recorded as ; Then the marked area is sampled through the sliding window while retaining the relative position information of the window center; the obtained sliding window sampling image is :

[0091]

[0092] in 、 Indicates the horizontal and vertical coordinates of the center position of the sliding window, Indicates the sliding step length;

[0093] For the edge area of ​​the image, a mirror filling method is used to ensure that the edge area is completely sampled, and the image size after filling remains 1024×1024;

[0094] To improve the generalization and robustness of the model, data augmentation is performed on the segmented samples. Data augmentation methods include horizontal and vertical flipping, random cropping of images while maintaining the original resolution, and adjusting the brightness, contrast, and saturation of the images. For each pair of dual-temporal remote sensing images, the same data augmentation operations are maintained to ensure that the changes in information are not affected by the augmentation operations.

[0095] S23: Dataset division: The sample dataset is randomly divided into training set, validation set and test set in the ratio of 8:1:1;

[0096] S24: Data preprocessing: Normalize and enhance the original dual-temporal remote sensing images.

[0097] S3, Model Building: Design a dual-stream change detection network model RFAMNet that integrates Mamba enhancement. RFAMNet uses a lightweight MobileNet as the backbone network, constructs a dual-stream feature preprocessor LightWeightEncoder and a feature enhancement module FRM, and then generates a change detection map through the semantic segmentation aggregation module SSAM, the channel interaction module CIM, and the global decoder Decoder. The feature enhancement module FRM includes Mamba sequence modeling, enhanced global feature modeling and edge perception processing capabilities. The specific process of model building is as follows:

[0098] S31: Build a dual-stream feature preprocessor LightWeight Encoder to input dual-temporal remote sensing images into a lightweight MobileNet to extract multi-scale feature representations.

[0099] The specific process of step S31 is:

[0100] Images of the period 、 Images of the period And the corresponding label data After the dual-stream feature preprocessor Lightweight Encoder, preprocessing features are extracted , then preprocess the features Extract multi-scale features through lightweight MobileNet to obtain global feature maps ,in for The global feature map obtained by the image of the period, for The global feature map obtained from the image of the period;

[0101] The dual-stream feature preprocessor Lightweight Encoder extracts features through a lightweight MobileNet. The network uses a weight-sharing mechanism and is composed of a Conv3×3 convolution operation, BatchNorm normalization, and a ReLU activation function. MobileNet extracts multi-scale features through deep separable convolution DSConv and outputs multi-scale feature representations. , , are features of 5 different scales, where .

[0102] S32: Construct a feature enhancement module FRM, integrate Mamba sequence modeling, and dynamically capture long-range dependencies and edge perception.

[0103] The specific process of step S32 is:

[0104] The preprocessed multi-scale features Input to FRM, with specific level features As a boundary, the features below this specific level are and First, downsample to the target resolution through maximum pooling, then align the channels through Conv1×1 convolution and extract local features through DSConv3×3 depth-separable convolution; then, features above the specific level are added. and Through Conv1×1 convolution alignment channel, DSConv3×3 depth separable convolution, BatchNorm normalization and ReLU activation function processing, bilinear upsampling to the target resolution; then each level feature is input into the MambaBlock module for sequence modeling enhancement; serialization processing, the two-dimensional feature map Rearrange into a one-dimensional sequence , , , is the field of real numbers, All are dimensions; then state space modeling is performed, and the long-range dependencies are captured by parameterized state space model SSM, the formula is:

[0105]

[0106] in 、 、 、 is a learnable parameter, is the input signal, is the hidden state, k represents the position in the feature sequence;

[0107] Then output the sequence Inverse mapping to a two-dimensional feature map , , ; By normalizing the enhanced features and connecting them with the input residual, the output Level Enhancement Features , the enhanced features of each level Splicing by channel dimension, sequentially through Conv1×1 convolution and DSConv3×3 depth separable convolution, further fusing cross-scale feature information, and performing residual connection between the fused features and the original features of the target level, and finally outputting the enhanced Period Level Features , , .

[0108] S33: Establish a semantic segmentation aggregation module SSAM to integrate multi-scale difference features and mine global semantic information.

[0109] The specific process of step S33 is:

[0110] The difference calculation of the dual-phase multi-scale features enhanced by the feature enhancement module FRM is performed through the formula Get the first Level difference characteristics ,in It represents the operation of taking the absolute value after element-by-element subtraction. express Period Level features, express Period Level 3 difference features As a low-level differential feature , Level 4 difference features As a high-level differentiating feature ;

[0111] Level 4 difference features and Level 5 difference features Input to the semantic segmentation aggregation module SSAM, first Perform bilinear upsampling to make Resolution and The two are then directly added element by element to obtain the coarse global semantic features; then the number of channels is expanded through the Conv3×3 convolution operation, and the channel dimension is evenly divided into four sub-features. Each sub-feature is subjected to multi-scale feature learning through dilated convolution with different void ratios; finally, the four sub-features after multi-scale feature learning are concatenated and feature fused through Conv1×1 convolution and DSConv3×3 depth-separable convolution to obtain the global semantic feature. .

[0112] S34: Establish a channel interaction module CIM to enhance the interaction between features of different sizes and highlight the features of the changing area.

[0113] The specific process of step S34 is:

[0114] The obtained low-level difference features , advanced differentiation features and global semantic features Input to the channel interaction module CIM for processing; first 、 、 Perform global average pooling GAP and Conv1×1 convolution operations respectively to obtain the corresponding vectors for calculating attention weights 、 and ; then 、 and Reshape the dimensions so that and 、 Perform matrix multiplication separately to obtain two similarity matrices, add the two similarity matrices and generate attention weights through the sigmoid function ;Will After remodeling Perform matrix multiplication, then process through Conv1×1 convolution, batch BatchNorm normalization and ReLU activation function, and Establish residual connections to obtain enhanced difference features .

[0115] S35: Build a global decoder, restore the spatial resolution, and generate a change detection map.

[0116] The specific process of step S35 is:

[0117] Will 、 and Input to the decoding unit DU for processing; first and After Conv1×1 convolution, and then bilinear upsampling and The resolution and Same, will 、 and After element-by-element addition, the result is input into DSConv3×3 depth-separable convolution, and finally the prediction result of the current level is output through Conv1×1 convolution.

[0118] S4, model training: Use the training set data to train the constructed two-stream change detection network model until the entire model converges, and save the optimal model.

[0119] The specific process of step S4 is as follows:

[0120] S41: Given a training set of dual-temporal remote sensing images and their corresponding real label data;

[0121] S42: Input the paired remote sensing images into RFAMNet, and after processing by RFAMNet, output the prediction results of change detection ;

[0122] S43: Using binary cross entropy loss function and the dice loss function Constructing the total loss function for:

[0123]

[0124] in Indicates the level of the decoder, corresponding to arrive The prediction results after decoding the difference features; 、 Decoders Layer-level binary cross entropy loss function and dice loss function;

[0125] Binary cross entropy loss function formula for:

[0126]

[0127] in and are the width and height of the input remote sensing image, Representing coordinates The true label of the pixel at Representing coordinates The predicted value of the pixel at ;

[0128] Dice loss function formula for:

[0129]

[0130] in Represents the true label and prediction results The intersection size of express The size of express size;

[0131] Minimize the total loss function through the back-propagation algorithm , iteratively optimize the model parameters until The value converges and the model reaches the optimal value;

[0132] During the iterative optimization training process, the validation set is used to verify the model training accuracy in real time, and the model weights with the highest accuracy are saved.

[0133] S5, model validation reasoning: input the test set data into the trained optimal model, predict the change area in the test set, and calculate the intersection and union ratio between the predicted change area and the actual change area. , and perform statistics based on all categories of the change area to obtain the average intersection-union ratio ,Evaluate the prediction accuracy of change detection.

[0134]

[0135] in Indicates the number of categories for change detection, represents the predicted change area, Indicates the changing area of ​​the true label.

[0136] Example

[0137] Step S1: Use the WHU dataset, which contains two aerial images with a resolution of 0.2 m / pixel and 15354×32507. The dataset contains 22,000 independent buildings, that is, a single category. This completes the image acquisition operation.

[0138] Step S2: Based on the remote sensing images of the same area in two different periods that have been marked in the WHU dataset ,The sliding window method is used to divide the image into blocks, and 32×15=480 images of size 1024×1024 are obtained.

[0139]

[0140]

[0141]

[0142] The training set Train, validation set Val and test set Test are divided into 384, 48 and 48 images respectively according to the ratio of 8:1:1, which are recorded as:

[0143]

[0144]

[0145]

[0146] This completes the data set preprocessing operation.

[0147] Step S3, design a dual-stream remote sensing image change detection model RFAMNet that integrates Mamba enhancement, such as Figure 2 The specific steps are as follows:

[0148] S31: Build a dual-stream feature preprocessor LightWeight Encoder to input dual-temporal remote sensing images into a lightweight MobileNet to extract multi-scale feature representations.

[0149] Images of the period 、 Images of the period And the corresponding label data After the dual-stream feature preprocessor Lightweight Encoder, preprocessing features are extracted , then preprocess the features Extract multi-scale features through lightweight MobileNet to obtain global feature maps ,in for The global feature map obtained by the image of the period, for The global feature map obtained from the image of the period;

[0150] The dual-stream feature preprocessor Lightweight Encoder extracts features through a lightweight MobileNet. The network uses a weight-sharing mechanism and is composed of a Conv3×3 convolution operation, BatchNorm normalization, and a ReLU activation function. MobileNet extracts multi-scale features through deep separable convolution DSConv and outputs multi-scale feature representations. , ,in ; After Conv convolution and downsampling operations, The dimensions become 512×512×48, The dimensions become 256×256×96, The dimensions become 128×128×192, The dimensions become 64×64×384, The dimensions become 32×32×768.

[0151] S32: Construct feature enhancement module FRM, such as Figure 3 As shown, Mamba sequence modeling is integrated to dynamically capture long-range dependencies and edge perception.

[0152] The preprocessed multi-scale features Input to the feature enhancement module FRM, with specific level features As a boundary, the features below this specific level are and First, downsample to the target resolution through maximum pooling, then align the channels through Conv1×1 convolution and extract local features through DSConv3×3 depth-separable convolution; then, features above the specific level are concatenated. and Through Conv1×1 convolution to align channels, DSConv3×3 depth-separable convolution, BatchNorm normalization and ReLU activation function processing, bilinear upsampling is performed to the target resolution, and the dimension becomes 128×128×192.

[0153] Then, the features of each level are input into the MambaBlock module for sequence modeling enhancement; serialization processing is performed to convert the two-dimensional feature map Rearrange into a one-dimensional sequence , , , is the field of real numbers, All are dimensions; then state space modeling is performed, and the long-range dependencies are captured by parameterized state space model SSM, the formula is:

[0154]

[0155] in 、 、 、 is a learnable parameter, is the input signal, is the hidden state, k represents the position in the feature sequence;

[0156] Then output the sequence Inverse mapping to a two-dimensional feature map , , ; By normalizing the enhanced features and connecting them with the input residual, the output Level Enhancement Features , the enhanced features of each level Splicing by channel dimension, sequentially through Conv1×1 convolution and DSConv3×3 depth separable convolution, further fusing cross-scale feature information, and performing residual connection between the fused features and the original features of the target level, and finally outputting the enhanced Period Level Features , , .

[0157] The difference calculation of the dual-phase multi-scale features enhanced by the feature enhancement module FRM is performed through the formula Get the first Level difference characteristics ,in It represents the operation of taking the absolute value after element-by-element subtraction. express Period Level features, express Period Level 3 difference features As a low-level differential feature , Level 4 difference features As a high-level differentiating feature ;

[0158] Level 4 difference features and Level 5 differential features Input to the semantic segmentation aggregation module SSAM, first Perform bilinear upsampling to make Resolution and The two are then directly added element by element to obtain the coarse global semantic features; then the number of channels is expanded through the Conv3×3 convolution operation, and the channel dimension is evenly divided into four sub-features. Each sub-feature is subjected to multi-scale feature learning through dilated convolution with different void ratios; finally, the four sub-features after multi-scale feature learning are concatenated and feature fused through Conv1×1 convolution and DSConv3×3 depth-separable convolution to obtain the global semantic feature. .

[0159] S34: Establish a channel interaction module CIM to enhance the interaction between features of different sizes and highlight the features of the changing area.

[0160] The obtained low-level difference features , advanced differentiation features and global semantic features Input to the channel interaction module CIM for processing; first 、 、 Perform global average pooling GAP and Conv1×1 convolution operations respectively to obtain the corresponding vectors for calculating attention weights 、 and ; then 、 and Reshape the dimensions so that and 、 Perform matrix multiplication separately to obtain two similarity matrices, add the two similarity matrices and generate attention weights through the sigmoid function ;Will After remodeling Perform matrix multiplication, then process through Conv1×1 convolution, batch BatchNorm normalization and ReLU activation function, and Establish residual connections to obtain enhanced difference features .

[0161] S35: Build a global decoder, restore the spatial resolution, and generate a change detection map.

[0162] Will 、 and Input to the decoding unit DU for processing; first and After Conv1×1 convolution, and then bilinear upsampling and The resolution and Same, will 、 and After element-by-element addition, the result is input into DSConv3×3 depth-separable convolution, and finally the prediction result of the current level is output through Conv1×1 convolution.

[0163] Step S4: Model training. The specific process is as follows:

[0164] Train the established two-stream change detection network model, given a training set Dual-temporal remote sensing images and their corresponding real label data;

[0165] Input the paired remote sensing images into RFAMNet, and after processing by RFAMNet, output the prediction results of change detection ;

[0166] Using binary cross entropy loss function and the dice loss function Constructing the total loss function for:

[0167]

[0168] in Indicates the level of the decoder, corresponding to arrive The prediction results after decoding the difference features; 、 Decoders Layer-level binary cross entropy loss function and dice loss function;

[0169] Binary cross entropy loss function formula for:

[0170]

[0171] in and are the width and height of the input remote sensing image, Representing coordinates The true label of the pixel at Representing coordinates The predicted value of the pixel at ;

[0172] Dice loss function formula for:

[0173]

[0174] in Represents the true label and prediction results The intersection size of express The size of express size;

[0175] Minimize the total loss function through the back-propagation algorithm , iteratively optimize the model parameters until The value converges and the model reaches the optimal value;

[0176] During the iterative optimization training process, the validation set is used to verify the model training accuracy in real time, and the model weights with the highest accuracy are saved.

[0177] Step S5, model inference test, the specific process is: test set Input the trained model with paired images and label data, predict the change area in the test set, and calculate the intersection-over-union ratio between the predicted change area and the actual change area. , and perform statistics based on all categories of the change area to obtain the average intersection-union ratio ,Evaluate the prediction accuracy of change detection.

[0178]

[0179] in Indicates the number of categories for change detection, represents the predicted change area, Indicates the changing area of ​​the true label, The higher the value, the higher the model accuracy.

[0180] like Figure 4-Figure 6 As shown in the figure, the contrast images of the same area at different times are input, and after the trained RFAMNet, the predicted images of the contrast change area are output, thus completing the model test and inference operation.

Claims

1. A dual-stream remote sensing image change detection method integrated with Mamba enhancement, characterized in that: The following steps are involved: S1, Image acquisition: Use remote sensing image acquisition equipment to obtain remote sensing images of the same area at different times, namely, dual-temporal remote sensing images, to obtain a remote sensing image dataset; S2, data preprocessing: preprocess the acquired dual-temporal remote sensing images and divide them into training and test sets; S3, Model Building: Design a dual-stream change detection network model RFAMNet that integrates Mamba enhancement. RFAMNet uses lightweight MobileNet as the backbone network, builds a dual-stream feature preprocessor LightWeight Encoder and a feature enhancement module FRM, and then generates a change detection map through the semantic segmentation aggregation module SSAM, the channel interaction module CIM and the global decoder Decoder. The feature enhancement module includes Mamba sequence modeling, enhanced global feature modeling and edge perception processing capabilities. The specific process of model establishment is as follows: S31: Build a dual-stream feature preprocessor to input dual-temporal remote sensing images into a lightweight MobileNet to extract multi-scale feature representations; The dual-stream feature preprocessor is extracted through a lightweight MobileNet, which uses a weight sharing mechanism and is composed of Conv3×3 convolution operations, BatchNorm normalization, and ReLU activation functions. MobileNet extracts multi-scale features through deep separable convolution DSConv and outputs multi-scale feature representations. , , are features of 5 different scales, where ; S32: Construct a feature enhancement module that integrates Mamba sequence modeling to dynamically capture long-range dependencies and edge perception; Input each level of features into the MambaBlock module for sequence modeling enhancement; serialization processing, the two-dimensional feature map Rearrange into a one-dimensional sequence , , , is the field of real numbers, All are dimensions; Then, state space modeling is performed to capture long-range dependencies through parameterized state space model SSM; Then output the sequence Inverse mapping to a two-dimensional feature map , , ; By normalizing the enhanced features and connecting them with the input residual, the output Level Enhancement Features , the enhanced features of each level Splicing by channel dimension, sequentially through Conv1×1 convolution and DSConv3×3 depth separable convolution, further fusing cross-scale feature information, and performing residual connection between the fused features and the original features of the target level, and finally outputting the enhanced Period Level Features , , ; S33: Establish a semantic segmentation aggregation module to integrate multi-scale difference features and mine global semantic information; S34: Establish a channel interaction module to enhance the interaction between features of different sizes and highlight the features of the changing area; S35: Build a global decoder, restore spatial resolution, and generate a change detection map; S4, model training: Use the training set data to train the constructed two-stream change detection network model until the entire model converges, and save the optimal model; S5, model validation reasoning: input the test set data into the trained optimal model, predict the change area in the test set, and calculate the intersection and union ratio between the predicted change area and the actual change area. , and perform statistics based on all categories of the change area to obtain the average intersection-union ratio ,Evaluate the prediction accuracy of change detection.

2. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 1 is characterized in that: In step S1, the remote sensing image dataset S obtained is: ； in, express area Image data of the period, express area Image data of the period.

3. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 2 is characterized in that: The specific process of step S2 is: S21: Image annotation; Label the target categories of the changed areas of the remote sensing image dataset to obtain the labeled remote sensing image dataset for: ； in, Represents remote sensing images 、 Comparative change area category label map; S22: Image segmentation; right The sliding window method is used to segment the image into 1024×1024 size images and label datasets: ； ； in, express The first A 1024×1024 sample image, express The first A 1024×1024 sample image, is with 、 Corresponding contrast change area label map; S23: Dataset division: The sample dataset is randomly divided into training set, validation set and test set in the ratio of 8:1:1; S24: Data preprocessing: Normalize and enhance the original dual-temporal remote sensing images.

4. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 3 is characterized in that: In step S22, the sliding window method is specifically operated as follows: First, the remote sensing image is divided into multiple 1024×1024 pixel unit grids. If the remote sensing image cannot be divided proportionally, it is filled. The step length of each unit grid is recorded as ; Then the marked area is sampled through the sliding window while retaining the relative position information of the window center; the obtained sliding window sampling image is : ； in 、 Indicates the horizontal and vertical coordinates of the center position of the sliding window, Indicates the sliding step length; For the edge areas of the image, a mirror filling method is used to ensure that the edge areas are completely sampled, and the image size after filling remains at 1024×1024.

5. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 4 is characterized in that: The specific process of step S31 is as follows: Images of the period 、 Images of the period And the corresponding label data After the dual-stream feature preprocessor Lightweight Encoder, preprocessing features are extracted , then preprocess the features Extract multi-scale features through lightweight MobileNet to obtain global feature maps ,in for The global feature map obtained by the image of the period, for The global feature map obtained from the image of the epoch.

6. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 5 is characterized in that: The specific process of step S32 is as follows: The preprocessed multi-scale features Input to the feature enhancement module FRM, with specific level features As a boundary, the features below this specific level are and First, downsample to the target resolution through maximum pooling, then align channels through Conv1×1 convolution and extract local features through DSConv3×3 depth-wise separable convolution; Then the features above that specific level are and The network is processed by Conv1×1 convolution to align channels, DSConv3×3 depth-separable convolution, BatchNorm normalization and ReLU activation function, and bilinearly upsampled to the target resolution; Perform state space modeling and capture long-range dependencies through parameterized state space model SSM, the formula is: ； in 、 、 、 is a learnable parameter, is the input signal, is the hidden state, and k represents the position in the feature sequence.

7. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 6 is characterized in that: The specific process of step S33 is as follows: The difference calculation of the dual-phase multi-scale features enhanced by the feature enhancement module FRM is performed through the formula Get the first Level difference characteristics ,in It represents the operation of taking the absolute value after element-by-element subtraction. express Period Level features, express Period Level 3 difference features As a low-level differential feature , Level 4 difference features As a high-level differentiating feature ; Level 4 difference features and Level 5 differential features Input to the semantic segmentation aggregation module SSAM, first Perform bilinear upsampling to make Resolution and The two are then directly added element by element to obtain the coarse global semantic features; then the number of channels is expanded through the Conv3×3 convolution operation, and the channel dimension is evenly divided into four sub-features. Each sub-feature is subjected to multi-scale feature learning through dilated convolution with different void ratios; finally, the four sub-features after multi-scale feature learning are concatenated and feature fused through Conv1×1 convolution and DSConv3×3 depth-separable convolution to obtain the global semantic feature. .

8. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 7 is characterized in that: The specific process of step S34 is as follows: The obtained low-level difference features , advanced differentiation features and global semantic features Input to the channel interaction module CIM for processing; first 、 、 Perform global average pooling GAP and Conv1×1 convolution operations respectively to obtain the corresponding vectors for calculating attention weights 、 and ; then 、 and Reshape the dimensions so that and 、 Perform matrix multiplication separately to obtain two similarity matrices, add the two similarity matrices and generate attention weights through the sigmoid function ;Will After remodeling Perform matrix multiplication, then process through Conv1×1 convolution, batch BatchNorm normalization and ReLU activation function, and Establish residual connections to obtain enhanced difference features .

9. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 8, characterized in that: The specific process of step S35 is as follows: Will 、 and Input to the decoding unit DU for processing; first and After Conv1×1 convolution, and then bilinear upsampling and The resolution and Same, will 、 and After element-by-element addition, the result is input into DSConv3×3 depth-separable convolution, and finally the prediction result of the current level is output through Conv1×1 convolution.

10. The dual-stream remote sensing image change detection method integrated with Mamba enhancement according to claim 9 is characterized in that: The specific process of step S4 is as follows: S41: Given a training set of dual-temporal remote sensing images and their corresponding real label data; S42: Input the paired remote sensing images into RFAMNet, and after processing by RFAMNet, output the prediction results of change detection ; S43: Using binary cross entropy loss function and the dice loss function Constructing the total loss function for: ； in Indicates the level of the decoder, corresponding to arrive The prediction results after decoding the difference features; 、 Decoders Layer-level binary cross entropy loss function and dice loss function; Binary cross entropy loss function formula for: ； in and are the width and height of the input remote sensing image, Representing coordinates The true label of the pixel at Representing coordinates The predicted value of the pixel at ; Dice loss function formula for: ； in Represents the true label and prediction results The intersection size of express The size of express size; Minimize the total loss function through the back-propagation algorithm , iteratively optimize the model parameters until The value converges and the model reaches the optimal value; During the iterative optimization training process, the validation set is used to verify the model training accuracy in real time, and the model weights with the highest accuracy are saved.

Images