A remote sensing image change detection method based on fusion state space model

Abstract

The present application relates to the technical field of image detection, and more particularly to a remote sensing image change detection method based on a fusion state space model, wherein the model comprises: a hierarchical feature extraction unit configured to receive an image group input to the hierarchical feature extraction unit, the image group comprising at least two or more observation images corresponding to the same location at different time points, and at least one observation image in the image group being regarded as a base image and the other observation images being regarded as contrast images; and configured to extract hierarchical feature representations of each observation image in the image group, the hierarchical feature representations comprising low-level features, middle-level features, deep features and high-level features; a splicing and fusion unit configured to splice and compress the hierarchical feature representations of each observation image in a hierarchical manner in the channel dimension, and configured to aggregate remote context information of the compressed hierarchical feature representations in a hierarchical manner to obtain fusion features corresponding to each observation image; and an output module configured to output a prediction probability of a contrast image having changed compared with a base image based on the fusion features of the base image and the fusion features of the contrast image in the image group; the model and the method can obtain a remote sensing image change detection result with high accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of image detection, and more specifically, to a remote sensing image change detection method based on a fusion state space model. Background Art

[0002] Change detection involves conducting repeated observations of the same location over two or more periods, comparing the types and ranges of features observed over time to analyze changes in features within the observation area. Remote sensing imagery, with its wide coverage, efficient data acquisition, and rich information content, has become a key technology for monitoring surface changes. High-resolution remote sensing imagery, in particular, is widely used in natural resource monitoring, urban expansion analysis, agricultural safety, and other fields.

[0003] With the continuous development of remote sensing sensors and image processing technology, remote sensing change detection methods represented by deep learning are driving remote sensing change detection towards a more refined direction. However, high-resolution remote sensing images exhibit significant spatiotemporal heterogeneity, resulting in a large number of false changes in detection results.

[0004] Existing methods attempt to enhance the robustness of models in dealing with confounding backgrounds through data augmentation or attention mechanisms. However, these methods can only simulate illumination variations between two-temporal images, and the models still have difficulty coping with temporal perturbations caused by seasonal variations. Recently, some research has begun to focus on learning pseudo-variation features in multi-temporal remote sensing imagery. However, these methods mostly rely on learning global features, lack sensitivity to local details, and have difficulty capturing seasonal sensitivity when processing high-resolution imagery. Summary of the Invention

[0005] In order to overcome the defect of the above-mentioned prior art that the interference of pseudo changes cannot be eliminated, resulting in low accuracy of change detection results, the present invention provides a remote sensing image change detection method based on a fusion state space model that can eliminate pseudo change interference.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] A fusion state space model, comprising: a hierarchical feature extraction unit, a splicing and fusion unit, and an output module;

[0008] The hierarchical feature extraction unit is used to receive an image group input into the hierarchical feature extraction unit, the image group at least including observation images corresponding to two or more time points of the same location, and at least one observation image in the image group is regarded as a basic image, and the other observation images are regarded as reference images; and is used to extract a hierarchical feature representation of each observation image in the image group, the hierarchical feature representation including low-level features, mid-level features, deep features and high-level features;

[0009] The splicing and fusion unit is used to stitch and compress the hierarchical feature representation of each observation image hierarchically in the channel dimension, and to hierarchically aggregate the remote context information of the compressed hierarchical feature representation to obtain the fusion feature corresponding to each observation image;

[0010] The output module is used to output a predicted probability that the control image has changed compared with the basic image based on the fusion features of the basic image in the image group and the fusion features of the control image in the image group.

[0011] The present invention also proposes a remote sensing image change detection method based on a fusion state space model, comprising the following steps:

[0012] Acquiring an image group to be detected, wherein at least one observed image in the image group to be detected is regarded as a basic image, and the other observed images are regarded as control images;

[0013] Inputting the image group to be detected into the fusion state space model;

[0014] The fusion state space model outputs a predicted probability that the control image in the image group to be detected has changed compared with the basic image in the image group to be detected;

[0015] The region in the control image where the predicted probability is greater than or equal to a preset threshold is regarded as the region where the control image has changed compared with the basic image.

[0016] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0017] The present invention utilizes a hierarchical feature extraction unit to extract hierarchical features including low-level features, intermediate features, deep features and high-level features, and utilizes a splicing and fusion unit to hierarchically splice and compress the hierarchical features in the channel dimension, and hierarchically aggregates the remote context information represented by the compressed hierarchical features to obtain information-rich and scale-rich fusion features, thereby better characterizing important information including changes of the input image group, so that the output module has better change detection capabilities. Combined with the remote sensing image change detection method, it can further eliminate the interference of pseudo-changes and obtain accurate change detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of the first structure of the fusion state space model proposed in Example 1;

[0019] Figure 2 This is a schematic diagram of the second structure of the fusion state space model proposed in Example 1;

[0020] Figure 3 This is a flow chart of the remote sensing image change detection method based on the fusion state space model proposed in Example 2;

[0021] Figure 4 This is a schematic diagram of the contrastive learning training strategy based on time-series image expansion proposed in Example 3;

[0022] Figure 5 This is a schematic diagram of the land cover change detection results on the dataset BCDD proposed in Example 3;

[0023] Figure 6 This is a schematic diagram of the land cover change detection results on the dataset CLCD proposed in Example 3. DETAILED DESCRIPTION

[0024] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0025] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0026] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0027] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0028] Example 1

[0029] This embodiment proposes a fusion state space model. Figure 1 This is a schematic diagram of the first structure of the fusion state space model proposed in this embodiment; Figure 2 This is a schematic diagram of the second structure of the fusion state space model proposed in this embodiment. Figure 2 The structures of the hierarchical feature extraction unit and the splicing and fusion unit are shown. DownSampling in the hierarchical feature extraction unit represents a downsampling operation.

[0030] like Figure 1 and Figure 2 As shown, the fusion state space model of this embodiment includes: a hierarchical feature extraction unit, a splicing and fusion unit, and an output module;

[0031] The hierarchical feature extraction unit is configured to receive an image group input into the hierarchical feature extraction unit, the image group comprising at least two or more observation images corresponding to different time points of the same location, and at least one observation image of the image group is regarded as a base image, and other observation images are regarded as contrast images; and the hierarchical feature extraction unit is configured to extract a hierarchical feature representation of each observation image of the image group, the hierarchical feature representation comprising low-level features, middle-level features, deep features and high-level features.

[0032] The splicing and fusion unit is configured to splice and compress the hierarchical feature representation of each observation image in a hierarchical manner in the channel dimension, and to aggregate remote context information of the compressed hierarchical feature representation in a hierarchical manner, to obtain a fusion feature corresponding to each observation image.

[0033] The output module is configured to output a prediction probability of a change occurring in the contrast image compared with the base image based on the fusion feature of the base image in the image group and the fusion feature of the contrast image in the image group.

[0034] In the specific implementation process, the hierarchical feature extraction unit is used to extract hierarchical features comprising low-level features, middle-level features, deep features and high-level features, and the splicing and fusion unit is used to splice and compress the hierarchical features in a hierarchical manner in the channel dimension, and to aggregate remote context information of the compressed hierarchical feature representation in a hierarchical manner, to obtain a fusion feature rich in information and scale, so as to better represent important information including changes in the input image group, so that the output module has better change detection capability, and the remote sensing image change detection method can further exclude the interference of false changes, to obtain an accurate change detection result.

[0035] In an optional embodiment, the hierarchical feature extraction unit is provided with a VMamba backbone network, the VMamba backbone network comprising at least a first stacking module, a second stacking module, a third stacking module and a fourth stacking module connected in sequence, the first stacking module stacking 2 VSS blocks to preliminarily extract low-level features, the second stacking module stacking 2 VSS blocks to extract middle-level features, the third stacking module stacking 5 VSS blocks to extract deep features, and the fourth stacking module stacking 2 VSS blocks to extract high-level features.

[0036] In an optional embodiment, the splicing and fusion unit comprises a first convolutional layer, a second convolutional layer, a first SS1D module, a third convolutional layer, a second SS1D module, a fourth convolutional layer and a third SS1D module connected in sequence.

[0037] Each convolutional layer is configured to reduce the number of channels of the features input into the convolutional layer.

[0038] Each SS1D module includes a two-dimensional image stretching layer, an SSM layer, a feature dimension restoration layer, and a normalization layer connected in sequence; the two-dimensional image stretching layer is used to stretch the received features into one-dimensional feature vectors in the forward and reverse directions in the horizontal direction, or in the downward and upward directions in the vertical direction; the SSM layer is used to selectively scan the one-dimensional feature vectors input by the two-dimensional image stretching layer, filter and retain valid information, and optimize feature expression; the feature dimension restoration layer is used to restore the output of the SSM layer to two-dimensional features by addition and merging; the normalization layer is used to standardize the output of the feature dimension restoration layer and unify the feature distribution of different scales;

[0039] The high-level features are input into the first convolutional layer, and the first convolutional layer reduces the dimension of the high-level features and outputs them. The high-level features after dimension reduction and the depth features are channel-joined, and then pass through the second convolutional layer and the first SS1D module for information aggregation and output; the output of the first SS1D module and the intermediate features are channel-joined, and then pass through the third convolutional layer and the second SS1D module for information aggregation and output; the output of the second SS1D module and the low-level features are channel-joined, and then pass through the fourth convolutional layer and the third SS1D module for information aggregation and output. The output of the third SS1D module is the fused feature.

[0040] As an exemplary illustration, the normalization layer can improve the stability of the training process.

[0041] In an optional embodiment, a classifier and a Softmax layer are provided in the output module, and the classifier includes a fifth convolutional layer, a first batch normalization layer, a first activation function layer and a sixth convolutional layer connected in sequence, and the output of the sixth convolutional layer is input into the Softmax layer, and the Softmax layer outputs the predicted probability that the control image has changed compared with the base image.

[0042] Example 2:

[0043] This embodiment proposes a remote sensing image change detection method based on a fusion state space model as described in Embodiment 1.

[0044] The remote sensing image change detection method based on the fusion state space model comprises the following steps:

[0045] S1: obtaining an image group to be detected, wherein at least one observed image in the image group to be detected is regarded as a basic image, and the other observed images are regarded as control images;

[0046] S2: inputting the image group to be detected into the fusion state space model;

[0047] S3: The fusion state space model outputs a predicted probability that the control image in the image group to be detected has changed compared with the basic image in the image group to be detected.

[0048] S4: The region in the control image whose predicted probability is greater than or equal to a preset threshold is regarded as the region where the control image has changed compared with the basic image.

[0049] In an optional embodiment, before the image group to be detected is input into the fusion state space model, the fusion state space model is trained to obtain a trained fusion state space model, and when the image group to be detected is input into the fusion state space model, the image group to be detected is input into the trained fusion state space model;

[0050] The steps of training the fusion state space model include:

[0051] Obtain an image change detection dataset, where the image change detection dataset includes multiple image groups, where each image group includes multiple observation images taken at multiple time points at the same location, and one observation image is selected from each image group as a base image, and the other observation images in the image group are regarded as control images, where each control image is annotated with a change label indicating a changed region compared to the base image;

[0052] Performing a random color enhancement operation on the basic images in each image group of the image change detection dataset to generate a time-series extended image corresponding to each image group;

[0053] Adding a global projection head and a dense projection head to the fused state space model;

[0054] Inputting each image group of the image change detection dataset and its corresponding time-series extended image into the fusion state space model, the splicing and fusion unit of the fusion state space model outputting fusion features corresponding to the base image, the control image and the time-series extended image;

[0055] The output module outputs a predicted probability that the control image has changed compared with the base image based on the fusion features corresponding to the base image and the control image, and constructs a cross entropy loss function based on the predicted probability and the change label of the control image;

[0056] The global projection head converts the fusion features corresponding to the basic image, the control image and the time-series extended image into global feature vectors corresponding to the basic image, the control image and the time-series extended image respectively;

[0057] The dense projection head converts the fusion features corresponding to the basic image, the control image and the time-series extended image into dense feature vector groups corresponding to the basic image, the control image and the time-series extended image respectively;

[0058] Constructing a global contrast learning loss function based on the global feature vectors corresponding to the basic image, the control image, and the time-series extended image;

[0059] Constructing a dense contrast learning loss function based on the dense feature vector groups corresponding to the base image, the control image, and the time-series extended image;

[0060] Constructing a target loss function of the model based on the cross entropy loss function, the global contrastive learning loss function and the dense contrastive learning loss function;

[0061] During the training process, the target loss function is solved iteratively. When the number of iterations reaches a preset value or the target loss function reaches a minimum value, the training is terminated to obtain a trained fusion state space model.

[0062] In an optional embodiment, the global projection head includes a first adaptive average pooling layer, a Dropout layer, a first fully connected layer, a second activation function layer, a second fully connected layer, a second batch normalization layer and a third activation function layer connected in sequence.

[0063] In an optional embodiment, the dense projection head includes a seventh convolutional layer and a second adaptive average pooling layer connected in sequence.

[0064] As an example, Figure 3 This is a flow chart of the remote sensing image change detection method based on the fusion state space model proposed in this embodiment; Figure 3 The process of contrastive learning training using the fusion state space model is shown, where: represents the fusion feature corresponding to the time series extended image, represents the fusion feature corresponding to the basic image, Represents the fusion feature corresponding to the control image.

[0065] In an optional embodiment, the expression of the objective loss function includes:

[0066]

[0067]

[0068]

[0069]

[0070] ,or

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] Where, represents the target loss function, represents the change detection loss function, represents the global contrast loss function, represents the dense contrast loss function, express The control coefficient, express The control coefficient of Indicates the The labels corresponding to the image groups, Indicates the The change prediction results corresponding to the group image group; represents the total number of image pairs in the image change detection dataset; 、 and Respectively represent The global feature vectors of the time-series extended images, basic images and control images of the group image group, 、 and Respectively represent the image change detection dataset The time series of extended images, basic images and control images of the image group A local feature vector located in the change area extracted by the change label; Indicates the The total number of local feature vectors of the change region corresponding to a time-extended image, base image, or control image in the group image; express and The cosine similarity between represents the temperature parameter; where The basic images corresponding to the group image groups are generated through data enhancement The positive sample of any base image is the enhanced image corresponding to the base image, and the negative sample of any base image includes the control image, the remaining images except the base image, and the A basic image and its corresponding Enhanced View and Zhang control image, Indicates that the image other than the control image The negative sample The global feature vector of the negative sample, Indicates the top sample with the largest cosine similarity to the local positive sample among all local negative samples. M The mth local feature vector in the negative samples, Indicates the number of negative samples selected.

[0078] This embodiment also proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor performs the steps of the remote sensing image change detection method based on the fusion state space model as described in this embodiment.

[0079] Example 3:

[0080] This embodiment provides a specific implementation example based on the fusion state space model proposed in Example 1 and the remote sensing image change detection method based on the fusion state space model proposed in Example 2:

[0081] Step 1: Obtain and preprocess a large, publicly available dataset for change detection based on high-resolution remote sensing imagery. Review existing data sources, organize the sources of existing large, publicly available datasets for change detection based on high-resolution remote sensing imagery, and download relevant data resources. Because the dataset contains samples without change labels, which are required for dense contrastive learning, filter the dataset samples, remove them, and re-divide them into training, validation, and test sets.

[0082] As an example, the specific steps for obtaining a large, publicly available change detection dataset based on high-resolution remote sensing imagery and preprocessing it are as follows: On websites such as IEEE Xplore and Github, relevant papers and literature were consulted to organize the sources of existing publicly available high-resolution remote sensing change detection datasets. Based on the data sources organized, datasets with relatively large data volumes, better data quality, and image types that are closer to the detection task images were selected for download and used for pre-training in subsequent experiments. Change labels refer to image data with changed areas (such as buildings, newly added farmland, etc.) marked. The obtained high-resolution remote sensing change detection dataset was reviewed to verify that the image data and labels in the dataset met the expected format requirements (such as TIFF, JPEG, PNG, etc.) to ensure that the data could be accurately introduced into the training environment. Samples without change labels were manually removed to ensure that all screened samples had change labels. To ensure data balance, the samples that met the requirements were randomly divided into training, validation, and test sets in a ratio of 6:2:2 for subsequent model training and prediction.

[0083] Step 2: Design a fusion state space model. Among them, the hierarchical feature extraction unit and the splicing and fusion unit are mainly used to extract multi-scale features from remote sensing images and fuse these features. First, the hierarchical feature extraction unit uses VMamba as the backbone network, and extracts low-level to high-level features of the image through stacked VSS modules (Visual State Space), and gradually builds a hierarchical feature representation. Secondly, the splicing and fusion unit replaces the traditional simple splicing operation, splices and compresses feature maps of multiple different stages in the channel dimension, and then aggregates remote context information through the SS1D module (1-Direction Selective Scan) designed in this application, which is conducive to improving the fusion effect of multi-scale features. Finally, information-rich and scale-rich feature maps are generated, which can be used as input for subsequent classifiers or contrastive learning branches.

[0084] As an example, the core component of the VMamba backbone network is SS2D (2D-Selective-Scan), which scans from the four corners of the input feature map to achieve high computational efficiency while maintaining a global receptive field. The network is a multi-stage stacked structure, using multiple layers of VSS blocks to extract feature representations at different levels at each stage. In stage 1, two VSS blocks are stacked to initially extract low-level features; in stage 2, two VSS blocks are stacked to extract mid-level features; in stage 3, five VSS blocks are stacked to extract deep features; and in stage 4, two VSS blocks are stacked to extract high-level features and form the output. Each stage requires downsampling the input feature map from the previous stage. The resulting multi-scale feature map provides the basis for subsequent fusion.

[0085] As an example, the main function of the splicing and fusion unit is to fuse the multi-scale feature maps generated by the VMamba backbone network to produce information-rich and scale-consistent feature representations. The core component, the SS1D module, uses a selective scanning approach: first, image features are stretched into 1D vectors in both the forward and reverse directions; then, they are processed through the SSM. Finally, the 1D vectors are restored to 2D feature maps through the Merge module, completing further feature fusion and information extraction.

[0086] As an example, the feature map of stage 4 (high-level features) is spliced ​​with the feature map of stage 3 (deep features) in the channel dimension. Before splicing, the feature map is compressed into 32 channels through 1×1 convolution to reduce computational complexity. The spliced ​​features are input into the SS1D module for aggregation of long-range context information. The output feature map of SS1D is then spliced ​​with the feature map of stage 2 (intermediate features). The above operation is repeated until it is fused with the feature map of stage 1 (low-level features).

[0087] Step 3: Design a contrastive learning training strategy based on time series image expansion. First, the time series extended images are generated by randomly color-enhancing the pre-change remote sensing images to simulate artifacts caused by changing lighting conditions and seasonal variations. The extended time series images, pre-change images, and post-change images are each fed into a fusion state-space model to extract features. The extracted feature maps are fed into the global projection head (GPH) and dense projection head (DPH) in the contrastive learning branch to extract global feature vectors and dense feature vector sets, respectively. The GPH guides the model to learn the global similarities of the time series images to reduce the impact of artifacts on change detection. The DPH focuses on dense features in remote sensing images and guides the model to learn local differences in the time series images to improve the model's sensitivity to local changes. Based on the global feature vectors and dense feature vector sets of the time series images, pre-change images, and post-change images, a global contrastive learning loss and a dense contrastive learning loss (dense contrastive learning loss) are constructed, respectively, leveraging the fact that semantic information changes in the pre-change images while semantic information in the time series extended images remains unchanged. The two work together to implement change detection loss, maximizing the global and local similarities between positive samples and the global and local differences between negative samples, enabling the model to capture both global changes and local details in change detection tasks, improving the ability to recognize pseudo-changes and accurately identifying changed areas at the pixel level.

[0088] As an exemplary illustration, GPH and DPH in the contrastive learning branch share weights.

[0089] As an example, by randomly transforming the brightness, color saturation, contrast, and hue of the pre-change image, we generate a time-extended image that visually changes while the ground remains unchanged, thereby simulating the pseudo-change phenomenon caused by changing lighting conditions and seasonal changes. The time-extended image generated using random color augmentation serves as a positive sample in the contrastive learning loss.

[0090] As an example, GPH guides the model to learn the global similarity of time series images, improving the model's robustness to spurious changes such as illumination and seasonal variations. The feature map output by the fused state-space model is used as the input to the global contrastive learning branch. It is first processed through an adaptive average pooling layer to generate a global feature vector. Then, two linear projection layers are used to map the high-dimensional feature vector to a low-dimensional space for the subsequent calculation of the contrastive learning loss.

[0091] As an example, DPH complements GPH by capturing pixel-level variation features at the local scale, particularly for identifying subtle local changes in an image. The feature maps output by the fused state-space model serve as the input to the dense contrastive learning branch. They first undergo a 1×1 convolution to compress the channel dimension, and then pass through an adaptive pooling layer to generate a set of dense feature vectors to represent the dense features.

[0092] As an example, the change detection loss is calculated using the cross entropy loss function. For each pixel, the change probability predicted by the model is compared with the actual change label to calculate the loss value. Figure 4 This is a schematic diagram of the contrastive learning training strategy based on time series image expansion proposed in this embodiment; Figure 4 The flowchart of contrastive learning training using global contrastive learning loss and dense contrastive learning loss is shown in . Figure 4 As shown in the figure, the global contrastive learning loss is calculated by maximizing the similarity between positive samples and minimizing the similarity between negative samples. Specifically, the cosine similarity of each image pair is calculated and optimized using a formula based on normalized temperature cross entropy loss. The dense contrastive learning loss is optimized by calculating the cosine similarity between dense feature vectors. The most relevant negative samples are selected during the calculation process. This method not only greatly reduces the amount of calculation, but also improves the model's sensitivity to dense feature differences. The fusion objective function is the sum of the change detection loss and the contrastive learning loss. The contrastive learning loss is the sum of the global contrastive learning loss and the dense contrastive learning loss multiplied by the weight coefficient. .

[0093] As an example, after the model extracts the feature maps of the dual-phase image, they are spliced ​​in the channel dimension and input into a simple classifier to output the change results. Among them, the change classifier consists of a convolutional layer, a BN layer, a ReLU activation layer, and a convolutional layer. For the change detection results, the standard binary cross-entropy loss function is used to calculate and optimize. During the training process, the results output by the classifier are processed by Softmax to obtain the change probability, and then the loss is calculated based on the cross-entropy loss formula and the change label. This process can be expressed as:

[0094]

[0095] Where, represents the change detection loss function, Indicates the image pixel labels, Indicates the image The predicted probability of each pixel.

[0096] As an example, the core of contrastive learning loss is to maximize the similarity between positive samples and minimize the similarity between negative samples. In the case of Enhanced view of samples. Each sample takes the enhanced view as a positive sample, and the rest images as negative samples.

[0097] As an example, in the global contrastive learning loss of the fusion state-space model, the positive sample pair is the original T1 moment image and obtained by time series expansion , the negative sample is Remote sensing images of other imaging locations in the same batch and their time-series expansion views The difference from contrastive learning methods such as SimCLR is that dual-temporal remote sensing images There are changes in the terrain between the two, so Global CL can also be constructed 、 Negative sample pair. So for image or , there is 1 positive sample, and 3(N-1)+1 negative samples, the features of the model output , after the global projection head, the global feature vector is obtained 、 , .in is the dimension of the feature vector, which is set to 128 in the experiment. Based on the defined positive and negative samples, the loss is calculated according to the following formula:

[0098]

[0099]

[0100] ,or

[0101]

[0102]

[0103] Where, represents the global contrast loss function, Indicates the first The three phases of remote sensing images of the group, . Indicates the first of the 3(N-1) negative samples except the control image. The global feature vector of the negative sample. is the temperature, which is set to 0.7 in the experiment.

[0104] As an example, dense contrastive learning loss extracts multiple dense feature vectors based on global contrastive learning to form a dense feature vector group. The loss function for dense contrastive learning loss (Dense CL) is constructed based on a group of dense feature vectors (local feature vectors). In remote sensing imagery, changes and invariances typically occur only in local areas, resulting in the possibility of both positive and negative samples within a dense feature vector group, making it difficult to construct the loss using a unified formula. Therefore, Dense CL uses change labels to filter dense feature vector groups, flexibly selecting positive and negative samples based on the occurrence of changes.

[0105] Specifically, the dense projection head outputs a set of dense feature vectors , ,in is the size of the dense feature vector group, which is set to 9 in CLCDNet. d is the dimension of the feature, which is consistent with the global feature vector and is set to 128. The change label is downsampled to a size of The mask is used to extract the dense feature vector used to calculate the dense contrast learning loss. dense feature vectors, The corresponding dense feature vector in is a positive sample, is a negative sample, i represents the i-th group of images in the batch, and j represents the j-th local feature vector extracted by the mask from the group of images. At the same time, negative samples also include those that are not extracted by the mask. dense feature vectors, and the other images in the same batch Compared with global contrastive learning, dense contrastive learning has a larger number of negative samples, which not only places higher demands on video memory but also reduces training efficiency. Among the negative samples, the front sample with the largest cosine similarity is selected. dense feature vectors as negative samples, thereby reducing video memory and improving training efficiency while also enhancing the model's ability to distinguish between positive and negative samples. The calculation formula for dense contrast loss can be expressed as:

[0106]

[0107]

[0108]

[0109]

[0110] in, Indicates the top sample with the largest cosine similarity to the local positive sample among all local negative samples. M The mth local feature vector in the negative samples.

[0111] Finally, the fusion state space model will be optimized by a hybrid loss function. It consists of three parts, including the change detection loss , global contrastive learning loss and dense contrastive learning loss , therefore, the objective loss function of the fusion state space model can be expressed as:

[0112]

[0113] As an example, In the specific experiments, it is set to 0.05.

[0114] Step 4: Train the proposed change detection model based on the public high-resolution remote sensing change detection dataset.

[0115] First, VMamba is pre-trained using the publicly available ImageNet-1K dataset. Then, the pre-trained VMamba is loaded to initialize the model parameters. After that, appropriate parameters are selected and the entire model is trained on a high-resolution remote sensing change detection dataset. Finally, a change detection model that can distinguish between pseudo changes and real changes is obtained.

[0116] As an exemplary illustration, the proposed model and all related experiments are implemented in PyTorch and executed on a GeForce RTX 3090. VMamba is initially pre-trained on the ImageNet-1K dataset. During the training process based on the contrast learning strategy, the channel number of the network encoder output feature map is set to 32, and the feature vector dimension of the global contrast learning branch and the dense contrast learning branch is 128. The temperature scaling factor in the global contrast learning loss and the dense contrast learning loss is set to 0.7. The feature vector group size output by the dense contrast learning is 9x9, and the number of negative samples M selected in the dense contrast learning loss is 50. The contrast learning loss weight coefficient is 0.05. All models are trained using the Adam optimizer with an initial learning rate of 10 -4 and a training period of 100. According to the test experiments of different datasets, the test set in the dataset is input into the trained deep learning model to obtain the corresponding image change map.

[0117] As an exemplary illustration, Figure 5 is a schematic diagram of the land cover change detection result on the dataset BCDD proposed in this embodiment, and the full name of the dataset BCDD in English is Building Change Detection Dataset, Figure 6 is a schematic diagram of the land cover change detection result on the dataset CLCD proposed in this embodiment, and the full name of the dataset CLCD in English is CropLand Change Dection (CLCD) Dataset, Figure 5 and Figure 6 (a) is the I1 time image; (b) is the I2 time image; (c) is the true change label; and (d) is the change detection result. Figure 5 and Figure 6 show the accurate remote sensing image change detection result output by the fusion state space model.

[0118] The beneficial effects of the present application mainly include: the proposed model is suitable for natural resource monitoring, farmland management, urban expansion analysis and other scenarios. Its lightweight design and efficient training strategy make the model have good practicability, and provide technical support for future diversified remote sensing change detection tasks. The fusion state space model can extract multi-level feature representation, and through multi-scale feature fusion and long-distance context information aggregation, it provides a more recognizable feature map for change detection, thereby improving the adaptability and robustness of the model in complex scenarios. The contrast learning training strategy based on time series images effectively reduces the influence of pseudo changes caused by light and seasonal changes, significantly improves the accuracy of high-resolution remote sensing change detection, and ensures accurate capture of real changes.

[0119] The same or similar reference numerals correspond to the same or similar components;

[0120] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent;

[0121] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a fusion state space model, characterized in that: include: Hierarchical feature extraction unit, splicing and fusion unit and output module; The hierarchical feature extraction unit is used to receive an image group input into the hierarchical feature extraction unit, the image group at least including observation images corresponding to two or more time points of the same location, and at least one observation image in the image group is regarded as a basic image, and the other observation images are regarded as reference images; and is used to extract a hierarchical feature representation of each observation image in the image group, the hierarchical feature representation including low-level features, mid-level features, deep features and high-level features; The splicing and fusion unit is used to stitch and compress the hierarchical feature representation of each observation image hierarchically in the channel dimension, and to hierarchically aggregate the remote context information of the compressed hierarchical feature representation to obtain the fusion feature corresponding to each observation image; The output module is used to output the predicted probability that the control image has changed compared with the basic image based on the fusion features of the basic image in the image group and the fusion features of the control image in the image group; The splicing and fusion unit includes a first convolutional layer, a second convolutional layer, a first SS1D module, a third convolutional layer, a second SS1D module, a fourth convolutional layer and a third SS1D module connected in sequence; Among them, each convolutional layer is used to reduce the number of channels of the features of the input convolutional layer; Each SS1D module includes a two-dimensional image stretching layer, an SSM layer, a feature dimension restoration layer, and a normalization layer connected in sequence; the two-dimensional image stretching layer is used to stretch the received features into one-dimensional feature vectors in the forward and reverse directions in the horizontal direction, or in the downward and upward directions in the vertical direction; the SSM layer is used to selectively scan the one-dimensional feature vectors input by the two-dimensional image stretching layer, filter and retain valid information, and optimize feature expression; the feature dimension restoration layer is used to restore the output of the SSM layer to two-dimensional features by addition and merging; the normalization layer is used to standardize the output of the feature dimension restoration layer and unify the feature distribution of different scales; The high-level features are input into the first convolutional layer, and the first convolutional layer reduces the dimension of the high-level features and outputs them. The high-level features after dimension reduction and the depth features are channel-joined, and then pass through the second convolutional layer and the first SS1D module for information aggregation and output; the output of the first SS1D module and the intermediate features are channel-joined, and then pass through the third convolutional layer and the second SS1D module for information aggregation and output; the output of the second SS1D module and the low-level features are channel-joined, and then pass through the fourth convolutional layer and the third SS1D module for information aggregation and output. The output of the third SS1D module is the fused feature.

2. The method for constructing a fusion state space model according to claim 1, characterized in that: A VMamba backbone network is set in the hierarchical feature extraction unit, and the VMamba backbone network includes at least a first stacking module, a second stacking module, a third stacking module and a fourth stacking module connected in sequence. The first stacking module stacks 2 VSS blocks to preliminarily extract low-level features, the second stacking module stacks 2 VSS blocks to extract intermediate features, the third stacking module stacks 5 VSS blocks to extract deep features, and the fourth stacking module stacks 2 VSS blocks to extract high-level features.

3. The method for constructing a fusion state space model according to claim 1, characterized in that: The output module is provided with a classifier and a Softmax layer. The classifier includes a fifth convolutional layer, a first batch normalization layer, a first activation function layer and a sixth convolutional layer connected in sequence. The output of the sixth convolutional layer is input into the Softmax layer. The Softmax layer outputs the predicted probability that the control image has changed compared with the basic image.

4. A remote sensing image change detection method, characterized in that: The following steps are involved: Acquiring an image group to be detected, wherein at least one observed image in the image group to be detected is regarded as a basic image, and the other observed images are regarded as control images; Inputting the image group to be detected into a fusion state space model; wherein the fusion state space model is obtained by the method according to any one of claims 1 to 3; The fusion state space model outputs a predicted probability that the control image in the image group to be detected has changed compared with the basic image in the image group to be detected; The region in the control image where the predicted probability is greater than or equal to a preset threshold is regarded as the region where the control image has changed compared with the basic image.

5. The remote sensing image change detection method according to claim 4, characterized in that: Before inputting the image group to be detected into the fusion state space model, training the fusion state space model to obtain a trained fusion state space model, and when inputting the image group to be detected into the fusion state space model, inputting the image group to be detected into the trained fusion state space model; The steps of training the fusion state space model include: Obtain an image change detection dataset, where the image change detection dataset includes multiple image groups, where each image group includes multiple observation images taken at multiple time points at the same location, and one observation image is selected from each image group as a base image, and the other observation images in the image group are regarded as control images, where each control image is annotated with a change label indicating a changed region compared to the base image; Performing a random color enhancement operation on the basic images in each image group of the image change detection dataset to generate a time-series extended image corresponding to each image group; Adding a global projection head and a dense projection head to the fused state space model; Inputting each image group of the image change detection dataset and its corresponding time-series extended image into the fusion state space model, the splicing and fusion unit of the fusion state space model outputting fusion features corresponding to the base image, the control image and the time-series extended image; The output module outputs a predicted probability that the control image has changed compared with the base image based on the fusion features corresponding to the base image and the control image, and constructs a cross entropy loss function based on the predicted probability and the change label of the control image; The global projection head converts the fusion features corresponding to the basic image, the control image and the time-series extended image into global feature vectors corresponding to the basic image, the control image and the time-series extended image respectively; The dense projection head converts the fusion features corresponding to the basic image, the control image and the time-series extended image into dense feature vector groups corresponding to the basic image, the control image and the time-series extended image respectively; Constructing a global contrast learning loss function based on the global feature vectors corresponding to the basic image, the control image, and the time-series extended image; Constructing a dense contrast learning loss function based on the dense feature vector groups corresponding to the base image, the control image, and the time-series extended image; Constructing a target loss function of the model based on the cross entropy loss function, the global contrastive learning loss function and the dense contrastive learning loss function; During the training process, the target loss function is solved iteratively. When the number of iterations reaches a preset value or the target loss function reaches a minimum value, the training is terminated to obtain a trained fusion state space model.

6. The remote sensing image change detection method according to claim 5, characterized in that: The global projection head includes a first adaptive average pooling layer, a Dropout layer, a first fully connected layer, a second activation function layer, a second fully connected layer, a second batch normalization layer and a third activation function layer connected in sequence.

7. The remote sensing image change detection method according to claim 5, characterized in that: The dense projection head includes a seventh convolutional layer and a second adaptive average pooling layer connected in sequence.

8. The remote sensing image change detection method according to any one of claims 5 to 7, characterized in that: The expression of the objective loss function includes: ,or Where, represents the target loss function, represents the change detection loss function, represents the global contrast loss function, represents the dense contrast loss function, express The control coefficient, express The control coefficient of Indicates the The labels corresponding to the image groups, Indicates the The change prediction results corresponding to the group image groups; represents the total number of image pairs in the image change detection dataset; 、 and Respectively represent The global feature vectors of the time-series extended images, basic images and control images of the group image group, 、 and Respectively represent the image change detection dataset The time series of extended images, basic images and control images of the image group A local feature vector located in the change area extracted by the change label; Indicates the The total number of local feature vectors of the change region corresponding to a time-extended image, base image, or control image in the group image; express and The cosine similarity between represents the temperature parameter; where The basic images corresponding to the group image groups are generated through data enhancement The positive sample of any base image is the enhanced image corresponding to the base image, and the negative sample of any base image includes the control image, the remaining images except the base image, and the A basic image and its corresponding Enhanced View and Zhang control image, Indicates that the images other than the control images The negative sample The global feature vector of the negative sample, Indicates the top sample with the largest cosine similarity to the local positive sample among all local negative samples. M The mth local feature vector in the negative samples, Indicates the number of negative samples selected.

9. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, wherein: When the computer-readable instructions are executed by the processor, the processor performs the steps of the remote sensing image change detection method according to any one of claims 4 to 8.

Images