Domain generalization remote sensing image change detection method based on cross-domain consistency learning

Abstract

The invention discloses a domain generalization remote sensing image change detection method based on cross-domain consistency learning, and the method comprises the following steps: carrying out the geometric enhancement of obtained synthetic dual-phase remote sensing image data, and dividing an obtained enhanced synthetic data set into a training set and a test set; constructing a domain generalization remote sensing image change detection network based on semantics; selecting a change detection algorithm as a feature extractor to obtain an output feature of an original image pair and an output feature of an enhanced image pair which are consistent with the output in spatial size, and respectively inputting the two groups of features into a prediction head of 3 * 3 convolution sharing weight to obtain two detection result images; constructing a space-time consistency comparative learning loss function and a pseudo change correction comparative learning loss function; and setting an overall loss function, iteratively training and optimizing network parameters, and inputting a detection image into the trained neural network after the loss is stable to obtain a final detection result graph. According to the method, the influence of similar category confusion caused by domain offset can be effectively reduced, so that the generalization ability of an existing change detection neural network model is effectively improved.

Description

A Domain-Generalized Remote Sensing Image Change Detection Method Based on Cross-Domain Consistency Learning Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a domain generalization remote sensing image change detection method based on cross-domain consistency learning. Background Technology

[0002] Remote sensing image change detection is a crucial research area in remote sensing applications. Its core objective is to accurately identify changes in targets within images of the same geographic area acquired at different times. This task plays a vital role in urban monitoring, disaster assessment, and environmental protection. Currently, two main categories of methods have been developed: model-driven and data-driven. Unlike model-driven methods, data-driven methods obtain specific or abstract features through multi-layer stacking, exhibiting superior performance. Although data-driven methods have made significant progress, they all rely on the premise that training and testing data are independent and identically distributed. However, differences in feature distribution exist not only between bi-temporal image pairs within the same dataset but also are prevalent and significant across different datasets. When models are deployed across domains or used for generalization, these distribution differences contain easily confused similar categories, leading to a significant decrease in the model's discriminative performance, or even complete failure in some cases.

[0003] To address this issue, researchers have proposed contrastive learning methods, which have proven effective in extracting discriminative features. However, many contrastive learning strategies only construct positive and negative sample pairs within a single style domain, making it difficult to effectively handle cross-domain style shifts caused by different sensors or imaging conditions. To further improve algorithm performance, it is necessary to design new methods that can reduce the degradation of model discriminative performance caused by domain shift, thereby achieving more accurate remote sensing image change detection in complex scenes. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a domain generalization remote sensing image change detection method based on cross-domain consistency learning, so as to effectively reduce the influence of similar category confusion caused by domain offset, thereby effectively improving the generalization ability of existing change detection neural network models.

[0005] To address the aforementioned technical problems, this invention provides a domain-generalized remote sensing image change detection method based on cross-domain consistency learning, comprising the following steps:

[0006] Geometric augmentation was performed on the acquired synthetic dual-temporal remote sensing image data, and the resulting augmented synthetic dataset was divided into training and test sets.

[0007] Construct a semantic-based domain-generalized remote sensing image change detection network;

[0008] The change detection algorithm is selected as the feature extractor to obtain the output features of the original image pair and the output features of the enhanced image pair with the same spatial size as the output. The two sets of features are then input into the prediction head of the 3×3 convolution with shared weights to obtain two detection result images.

[0009] Construct a spatiotemporal consistency contrastive learning loss function and a pseudo-change correction contrastive learning loss function;

[0010] Set the overall loss function, iteratively train and optimize the network parameters, and after the loss stabilizes, input the detected image into the trained neural network to obtain the final detection result image.

[0011] Furthermore, the acquired synthetic dual-temporal remote sensing image data is geometrically enhanced by unifying image size, rotation, and cropping. Then, it is enhanced with different colors and textures: brightness perturbation amplitude of 0.6, contrast perturbation amplitude of 0.6, saturation perturbation amplitude of 0.6, hue perturbation amplitude of 0.3, and random Gaussian blur, to obtain the dual-temporal image input to the original branch; brightness perturbation amplitude of 0.3 (probability of 0.5), contrast perturbation amplitude of 0.3 (probability of 0.5), saturation perturbation amplitude of 0.3 (probability of 0.5), hue perturbation amplitude of 0.1 (probability of 0.5), and random bilateral blur, to obtain the dual-temporal image input to the enhancement branch. The two branches share the same label.

[0012] Furthermore, the remote sensing image change detection network includes feature extractors for the original bi-temporal images and the enhanced bi-temporal images, and segmentation loss functions for the original image pair branches and the enhanced image pair branches. Spatiotemporal consistency contrastive learning loss function Comparison of learning loss function with pseudo-change correction .

[0013] Furthermore, suppose the dataset contains 2N labeled synthetic bi-temporal image pairs. and tags The synthesized bi-temporal image pairs are then enhanced at the image level in terms of color and texture, resulting in enhanced bi-temporal image pairs. The original image pairs and the enhanced image pairs share a single label. The labeled original image pairs and enhanced image pairs are fed into the encoder of the remote sensing image change detection network to extract features of the original image pairs from different time periods. and and enhancing image features and ,in It is a feature extractor for original image pairs and enhanced image pairs, sharing weights, and will and The two-temporal fusion features of the original image pairs are obtained by concatenation. , and The concatenation yields bi-temporal fusion features of the enhanced image pairs. Then, the data is fed into two decoders of the remote sensing image change detection network to decode it into a change prediction map. and The detection results are obtained after feeding the data into a 3×3 convolutional prediction head. and .

[0014] Furthermore, based on the segmentation loss function of the original image branch and the enhanced image branch of the remote sensing image change detection network... Spatiotemporal consistency contrastive learning loss function Comparison of learning loss function with pseudo-change correction Establish the overall loss function :

[0015]

[0016] Among these, a grid search was performed on the hyperparameters, and they were set to [value] in all experiments. .

[0017] Furthermore, the remote sensing image change detection network uses a segmentation loss function for the original data bi-temporal image pairs and the enhanced data bi-temporal image pairs. for:

[0018]

[0019] in, It is the cross-entropy loss function, and the specific calculation method is as follows:

[0020]

[0021] Among them, y i This is the detection result image for the corresponding branch, and M is the corresponding label.

[0022] Furthermore, to construct a discriminative embedding space for each category, a spatiotemporal consistency contrastive learning loss is considered between the fusion features of enhanced image pairs and the fusion features of original image pairs. The InfoNCE loss function is adopted, which is defined as follows:

[0023]

[0024] Where 'a' represents the anchor point and 'p' represents the positive sample. N- represents a negative sample, and N- represents the set of negative samples.

[0025] For dual-temporal raw image pairs and enhanced dual-temporal image pairs Select the fused feature output of the j-th layer of the decoder; for pixels Their fusion features are respectively represented as and ,in This represents the channel dimension of the fused features of this layer, since and Different versions derived from the same image pair will As an anchor point, As positive samples, they correspond to the same spatial location and contain consistent semantic information; to obtain negative samples, the changing label map is... Downsampling is performed, and pixels of a different class than the anchor pixel are selected as negative samples. Before entering the contrastive loss, all fused features pass through the projection head (denoted as...). Processing to generate projected fused features and The spatiotemporal consistency contrastive learning loss is defined as follows:

[0026]

[0027] Set P represents the fused feature map All pixel positions in, with dimensions of These correspond to the height and width of the fused feature map, respectively. This indicates the total number of blocks in the decoder.

[0028] Furthermore, to address the feature confusion between similar categories caused by domain offset, a pseudo-change correction contrastive learning loss is considered between the fused features of enhanced image pairs and the fused features of original image pairs. The InfoNCE loss function is used, and to further ensure the consistency of the feature space and capture discriminative information, the pseudo-change correction contrastive learning loss shares the projection head used in the spatiotemporal consistency contrastive learning loss. For the predicted map of the enhanced image pair (denoted as...) ) was adjusted to The size, too, This represents the corresponding ground truth label map; using these prediction maps and label maps, in Anchor points are established at the locations of the anchor points. Negative samples are extracted from the enhanced image fusion features associated with the misclassified category at each anchor point. All samples are then passed through the projection head. The pseudo-change correction contrastive learning loss function is as follows:

[0029] .

[0030] The beneficial effects of this invention are as follows: (1) This invention proposes a novel domain generalization framework for remote sensing image change detection. By introducing cross-domain consistency learning technology, the model trained only on source domain data can be directly generalized to the unseen target domain. By designing a new network structure and combining it with a cross-domain learning module, the contrastive learning is extended to style enhancement samples, forcing semantic consistency of feature representations between different style domains, thereby constructing a robust and highly discriminative embedding space, fundamentally improving the model's change detection performance in the unseen target domain. (2) This invention designs a spatiotemporal consistency contrastive learning loss function, which aims to use pixels from different categories in the original image as negative samples to construct a discriminative embedding space for each category, thereby achieving a more robust feature representation. (3) Domain shift may cause feature confusion between similar change and non-change categories, thus causing misclassification. A pseudo-change correction contrastive learning loss function is introduced, which is specifically used to separate misclassified features in the fusion representation of enhanced image pairs, making them closer to the correct category and further away from the misclassified category, thereby achieving effective feature untangling. Experimental results show that the framework proposed in this invention can significantly improve the detection performance of existing models in the unseen domain. Attached Figure Description

[0031] Figure 1 is a schematic diagram of the method flow of the present invention.

[0032] Figure 2 is a flowchart of the method framework of the present invention.

[0033] Figure 3 is a schematic diagram of the cross-domain learning module structure of the present invention.

[0034] Figure 4 shows the detection results of this invention on an unseen domain dataset. Detailed Implementation

[0035] As shown in Figures 1 and 2, a domain-generalized remote sensing image change detection method based on cross-domain consistency learning includes the following steps:

[0036] Geometric augmentation was performed on the acquired synthetic dual-temporal remote sensing image data, and the resulting augmented synthetic dataset was divided into training and test sets.

[0037] Construct a semantic-based domain-generalized remote sensing image change detection network;

[0038] The change detection algorithm is selected as the feature extractor to obtain the output features of the original image pair and the output features of the enhanced image pair with the same spatial size as the output. The two sets of features are then input into the prediction head of the 3×3 convolution with shared weights to obtain two detection result images.

[0039] Construct a spatiotemporal consistency contrastive learning loss function and a pseudo-change correction contrastive learning loss function;

[0040] Set the overall loss function, iteratively train and optimize the network parameters, and after the loss stabilizes, input the detected image into the trained neural network to obtain the final detection result image.

[0041] This invention provides a specific embodiment, which uses a synthetic remote sensing image dataset containing 12,000 image pairs for building binary change detection, and three real remote sensing image datasets containing 7,434, 3,601, and 6,091 image pairs respectively. These datasets come from different regions, have different spatial resolutions, different target shapes and sizes, and different change types. The image size is uniformly 256×256. The entire synthetic dataset is divided into training and test sets in a 5:1 ratio, while the real datasets are divided into training, validation, and test sets in ratios of 8:1:1, 6:2:2, and 7:2:1 respectively. The synthetic training set is used to train the network and save the final model. The synthetic and real test sets are used to compare the performance of the method of this invention with other existing methods. This embodiment is implemented on a computer with an AMD Ryzen 99950X CPU, an NVIDIA GeForce RTX 4090 (24G) GPU, and 93.6G of memory, and the programming language is Python.

[0042] During the training phase, the Siamese DeepLabV3+ change detection network was selected as the bi-temporal feature extractor for the original and augmentation branches. For the encoder, ResNet-50, MobileNetV2, and ShuffleNetV2 with shared weights could be chosen as the backbone. The input has four branches, processing the bi-temporal images of the original and augmentation branches respectively. The original fusion features are obtained by concatenating the bi-temporal images of the original branch, and the augmented fusion features are obtained by concatenating the bi-temporal images of the augmentation branch. These are then input into the decoders of the original and augmentation branches, respectively. To ensure fair comparison, all experiments were repeated three times and the average was taken. All models were trained under the same conditions using the SGD optimizer with a momentum of 0.9, a weight decay of 5e-4, for 50 epochs, and a batch size of 8. A multinomial learning rate scheduler was used with an initial learning rate of 1e-2 and an exponent of 0.9. The spatiotemporal consistency comparison learning loss sampled all classes, with 10 negative samples sampled for each fusion feature map. During training, there are a total of 50 epochs. The model trained in the last epoch is saved. During inference, the test image is input, only the original branch is used, the performance of the method is tested, and the experimental results are saved.

[0043] First, the encoder is applied to both the original two-temporal remote sensing images and the enhanced two-temporal versions of the images. The former includes the original two-temporal images. and post-phase original image To extract two independent original feature maps and The latter includes pre-temporal enhanced images. and post-temporal enhancement images To extract two independent enhanced feature maps and This process can be represented in the following mathematical form:

[0044]

[0045] Here, H×W represents the spatial dimensions of the H'×W' images, C represents the feature dimension, and r is the spatial compression ratio, determined by a specific change detection backbone network. A fused feature map is generated by concatenating the features along the channel dimensions, thereby capturing key changes between two temporal states.

[0046] Subsequently, the decoder will convert the original fused feature map and enhanced fusion feature maps Decoded into the original logit graph of change activation respectively And the enhanced version of the logit graph Dimension 2 represents the two categories: changed and unchanged. This operation can be represented as:

[0047]

[0048] Finally, the original logit graph p and the enhanced version of the logit graph p. A After softmax normalization, it is converted into the original probability map P and the enhanced version of the probability map P. A This process ensures that the sum of the probabilities of the changed and unchanged classes at each position is 1. The expression for this process is:

[0049]

[0050] P and P A Cross-entropy loss was used to optimize the change detection model, and the final total cross-entropy loss is:

[0051]

[0052] This ensures that the original and augmented branches of the change detection model can learn from labeled data in the source domain and acquire preliminary change detection capabilities. However, due to domain spacing, the change detection model cannot directly adapt to unseen target domains without domain generalization.

[0053] Style shifts between different change detection datasets can also lead to high similarity between certain change and no-change categories. Therefore, it is necessary to enhance the model's robustness to such domain shifts. For the decoder, a contrastive learning approach is employed, which has proven effective in extracting discriminative features. Specifically, the InfoNCE loss function is used, defined as follows:

[0054]

[0055] Where 'a' represents the anchor point and 'p' represents the positive sample. N- represents a negative sample, and N- represents the set of negative samples.

[0056] To achieve consistency in feature representation across different styles, a cross-domain learning (CDL) module is introduced. Specifically, anchor points are derived from the fused features of the enhanced image pairs, while positive samples are taken from the same pixel positions in the fused features of the corresponding original image pairs.

[0057] As shown in Figure 3, it mainly consists of two types of core losses: spatiotemporal consistency contrastive learning loss and pseudo-change correction contrastive learning loss.

[0058] For dual-temporal raw image pairs and enhanced dual-temporal image pairs Select the fused feature output from the j-th layer of the decoder. For pixels... Their fusion features are respectively represented as and ,in This represents the channel dimension of the fused features at this layer. As mentioned earlier, due to... and Different versions derived from the same image pair will As an anchor point, Positive samples are selected because they correspond to the same spatial location and contain consistent semantic information. Negative samples are obtained by analyzing the changing label graph. Downsampling is performed, and pixels of a different class than the anchor pixel are selected as negative samples. All fused features pass through a projection head (denoted as ) before entering the contrastive loss. Processing to generate projected fused features and The spatiotemporal consistency contrastive learning loss is defined as follows:

[0059]

[0060] Set P represents the fused feature map All pixel positions in, with dimension . These correspond to the height and width of the fused feature map, respectively. Furthermore, This indicates the total number of blocks in the decoder.

[0061] Domain offset can lead to feature confusion between similar changed and unchanged categories, resulting in misclassification. To mitigate this issue, PCRCL is introduced, specifically designed to separate misclassified features in the fused representation of enhanced image pairs, bringing them closer to the correct category while moving them away from the misclassified category, thus achieving effective feature unwrapping. To further ensure the consistency of the feature space and capture discriminative information, pseudo-change correction contrastive learning loss shares the projection head used in spatiotemporal consistency contrastive learning loss. For the predicted map of the enhanced image pair (denoted as...) ) was adjusted to The size. Similarly. This represents the corresponding ground truth label map. Using these predicted and label maps, in... Anchor points are established at the locations of the anchor points. Negative samples are extracted from enhanced image fusion features associated with the misclassified category at each anchor point. All samples are then passed through the projection head. The pseudo-change correction contrastive learning loss function is as follows:

[0062]

[0063] During the testing phase, test samples are input into the original branch of the network to obtain the final extraction results.

[0064] Figure 4 illustrates the change detection effect of this embodiment. The first column shows the previous time-phase image of a certain area, the second column shows the subsequent time-phase image of the same area, the fifth column shows the true change labels of the corresponding buildings in that area, the third column shows the change detection results of the baseline model, and the fourth column shows the detection result image after applying this invention. In the last three columns, white represents the actual changed area, black represents the actual unchanged area, green represents the missed detection area (actually changed), and red represents the falsely detected area (actually unchanged). Based on the change detection results in Figure 4, the domain-generalized remote sensing image change detection network based on cross-domain consistency learning proposed in this invention can effectively segment the changed areas of target objects in remote sensing images when facing unseen target domains, making it an effective domain-generalized remote sensing image change detection method.

Claims

1. A domain-generalized remote sensing image change detection method based on cross-domain consistency learning, characterized in that, The process includes the following steps: geometric augmentation of the acquired synthetic dual-temporal remote sensing image data; dividing the augmented synthetic dataset into training and testing sets; constructing a semantic-based domain-generalized remote sensing image change detection network; selecting a change detection algorithm as a feature extractor to obtain the output features of the original image pair and the augmented image pair with the same spatial dimensions as the output; inputting the two sets of features into a prediction head with shared weights of a 3×3 convolution to obtain two detection result images; constructing a spatiotemporal consistency contrastive learning loss function and a pseudo-change correction contrastive learning loss function; setting the overall loss function, iteratively training and optimizing the network parameters; and after the loss stabilizes, inputting the detected image into the trained neural network to obtain the final detection result image.

2. The domain generalization remote sensing image change detection method based on cross-domain consistency learning as described in claim 1, characterized in that, The acquired synthetic dual-temporal remote sensing image data is geometrically enhanced by unifying image size, rotation, and cropping. Then, it is enhanced with different colors and textures: brightness perturbation amplitude of 0.6, contrast perturbation amplitude of 0.6, saturation perturbation amplitude of 0.6, hue perturbation amplitude of 0.3, and random Gaussian blur, to obtain the dual-temporal image input to the original branch; brightness perturbation amplitude of 0.3, contrast perturbation amplitude of 0.3, saturation perturbation amplitude of 0.3, hue perturbation amplitude of 0.1, and random bilateral blur, to obtain the dual-temporal image input to the enhancement branch. The two branches share the same label.

3. The domain generalization remote sensing image change detection method based on cross-domain consistency learning as described in claim 1, characterized in that, The remote sensing image change detection network includes feature extractors for the original bi-temporal images and the enhanced bi-temporal images, and segmentation loss functions for the original image pair branch and the enhanced image pair branch. Spatiotemporal consistency contrastive learning loss function Comparison of learning loss function with pseudo-change correction 。 4. The domain generalization remote sensing image change detection method based on cross-domain consistency learning as described in claim 1, characterized in that, Suppose the dataset contains 2N labeled synthetic bi-temporal image pairs. and tags The synthesized bi-temporal image pairs are then enhanced at the image level in terms of color and texture, resulting in enhanced bi-temporal image pairs. The original image pairs and the enhanced image pairs share a single label. The labeled original image pairs and enhanced image pairs are fed into the encoder of the remote sensing image change detection network to extract features of the original image pairs from different time periods. and and enhancing image features and ,in It is a feature extractor for original image pairs and enhanced image pairs, sharing weights, and will and The two-temporal fusion features of the original image pairs are obtained by concatenation. ， and The concatenation yields bi-temporal fusion features of the enhanced image pairs. Then, the data is fed into two decoders of the remote sensing image change detection network to decode it into a change prediction map. and The detection results are obtained after feeding the data into a 3×3 convolutional prediction head. and 。 5. The domain generalization remote sensing image change detection method based on cross-domain consistency learning as described in claim 1, characterized in that, The segmentation loss function of the original image branch and the enhanced image branch of the remote sensing image change detection network. Spatiotemporal consistency contrastive learning loss function Comparison of learning loss function with pseudo-change correction Establish the overall loss function ： Among these, a grid search was performed on the hyperparameters, and they were set to [value] in all experiments. 。 6. The domain generalization remote sensing image change detection method based on cross-domain consistency learning as described in claim 5, characterized in that, The remote sensing image change detection network uses a segmentation loss function for raw data bi-temporal image pairs and augmented data bi-temporal image pairs. for: in, It is the cross-entropy loss function, and the specific calculation method is as follows: Among them, y i This is the detection result image for the corresponding branch, and M is the corresponding label.

7. The domain generalization remote sensing image change detection method based on cross-domain consistency learning as described in claim 5, characterized in that, To construct a discriminative embedding space for each category, a spatiotemporal consistency contrastive learning loss is considered between the fusion features of enhanced image pairs and the fusion features of original image pairs. The InfoNCE loss function is adopted, which is defined as follows: Where 'a' represents the anchor point and 'p' represents the positive sample. N- represents negative samples, and N- represents the set of negative samples; for dual-temporal raw image pairs and enhanced dual-temporal image pairs Select the fused feature output from the j-th layer of the decoder; For pixels Their fusion features are respectively represented as and ,in This represents the channel dimension of the fused features of this layer, since and Different versions derived from the same image pair will As an anchor point, As positive samples, they correspond to the same spatial location and contain consistent semantic information; to obtain negative samples, the changing label map is... Downsampling is performed, and pixels of a different category from the anchor pixel are selected as negative samples. All fused features pass through the projection head before entering the contrastive loss. Processing to generate the fused features after projection. and The spatiotemporal consistency contrastive learning loss is defined as follows: Set P represents the fused feature map All pixel positions in, with dimensions of These correspond to the height and width of the fused feature map, respectively. This indicates the total number of blocks in the decoder.

8. The domain generalization remote sensing image change detection method based on cross-domain consistency learning as described in claim 7, characterized in that, Using the InfoNCE loss function, and to further ensure the consistency of the feature space and capture discriminative information, pseudo-change correction contrastive learning loss is used to share the spatiotemporal consistency projection head used in the contrastive learning loss. For the prediction map of enhanced image pairs Adjusted to The size, too, This represents the corresponding ground truth label map; using these prediction maps and label maps, in Anchor points are established at the locations of the anchor points. Negative samples are extracted from the enhanced image fusion features associated with the misclassified category at each anchor point. All samples are then passed through the projection head. The pseudo-change correction contrastive learning loss function is as follows: 。

Images