Domain generalization remote sensing image change detection method based on domain invariant feature extraction

Abstract

The invention discloses a domain generalization remote sensing image change detection method based on domain invariant feature extraction, and the method comprises the following steps: carrying out the geometric enhancement of obtained synthetic dual-phase remote sensing image data, and dividing an obtained enhanced synthetic data set into a training set and a test set; constructing a domain generalization remote sensing image change detection network based on semantics; selecting a change detection algorithm as a feature extractor to obtain an output feature of an original image pair and an output feature of an enhanced image pair which are consistent with the output in spatial size, and respectively inputting the two groups of features into a prediction head of 3 * 3 convolution sharing weight to obtain two detection result images; constructing an auto-covariance matching loss function and a cross-covariance diagonal loss function; and setting an overall loss function, iteratively training and optimizing network parameters, and inputting a detection image into the trained neural network after the loss is stable to obtain a final detection result graph. According to the method, the domain offset between the training domain data set and the unseen domain data set can be effectively reduced, so that the generalization ability of an existing convolutional neural network model is effectively improved.

Description

A Domain-Generalized Remote Sensing Image Change Detection Method Based on Domain-Invariant Feature Extraction Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a domain-generalized remote sensing image change detection method based on domain-invariant feature extraction. Background Technology

[0002] Remote sensing image change detection is a crucial research area in remote sensing applications. Its core objective is to accurately identify changes in targets within images of the same geographic area acquired at different times. This task plays a vital role in urban monitoring, disaster assessment, and environmental protection. Currently, two main categories of methods have been developed: model-driven and data-driven. Unlike model-driven methods, data-driven methods obtain specific or abstract features through multi-layer stacking, exhibiting superior performance. Although data-driven methods have made significant progress, they all rely on the premise that training and testing data are independent and identically distributed. However, differences in feature distribution exist not only between bi-temporal image pairs within the same dataset but also are prevalent and significant across different datasets. When models are deployed across domains or used for generalization, these distribution differences can lead to significant performance degradation, and in some cases, complete failure.

[0003] To address this issue, researchers have proposed domain adaptation methods based on feature distribution alignment and destylization methods to solve cross-domain change detection. However, domain adaptation methods rely on acquiring target domain data during training to ensure effective learning. Destylization methods assume that style differences are the main cause of performance degradation and mitigate this problem by eliminating style differences; however, since style and content are often highly coupled, this process often loses key information, leading to further performance degradation when applied to remote sensing image change detection tasks. To further improve algorithm performance, it is necessary to design new methods that can reduce the domain margin, thereby achieving more accurate remote sensing image change detection in complex scenes. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a domain-generalized remote sensing image change detection method based on domain-invariant feature extraction, so as to effectively reduce the domain offset between the training domain dataset and the unseen domain dataset, thereby effectively improving the generalization ability of existing convolutional neural network models.

[0005] To address the aforementioned technical problems, this invention provides a domain-generalized remote sensing image change detection method based on domain-invariant feature extraction, comprising the following steps:

[0006] Geometric augmentation was performed on the acquired synthetic dual-temporal remote sensing image data, and the resulting augmented synthetic dataset was divided into training and test sets.

[0007] Construct a semantic-based domain-generalized remote sensing image change detection network;

[0008] The change detection algorithm is selected as the feature extractor to obtain the output features of the original image pair and the output features of the enhanced image pair with the same spatial size as the output. The two sets of features are then input into the prediction head of the 3×3 convolution with shared weights to obtain two detection result images.

[0009] Construct the autocovariance matching loss function and the cross-covariance diagonal loss function;

[0010] Set the overall loss function, iteratively train and optimize the network parameters, and after the loss stabilizes, input the detected image into the trained neural network to obtain the final detection result image.

[0011] Furthermore, the acquired synthetic dual-temporal remote sensing image data is geometrically enhanced by unifying image size, rotation, and cropping. Then, it is enhanced with different colors and textures: brightness perturbation amplitude of 0.6, contrast perturbation amplitude of 0.6, saturation perturbation amplitude of 0.6, hue perturbation amplitude of 0.3, and random Gaussian blur, to obtain the dual-temporal image input to the original branch; brightness perturbation amplitude of 0.3, contrast perturbation amplitude of 0.3, saturation perturbation amplitude of 0.3, hue perturbation amplitude of 0.1, and random bilateral blur, to obtain the dual-temporal image input to the enhancement branch. The two branches share the same label.

[0012] Furthermore, the remote sensing image change detection network includes feature extractors for the original bi-temporal images and the enhanced bi-temporal images, and segmentation loss functions for the original image pair branches and the enhanced image pair branches. Autocovariance matching loss function and cross-covariance diagonal loss function The feature extractor can be replaced with various change detection convolutional neural networks and seamlessly integrated into the framework.

[0013] Furthermore, suppose the dataset contains 2N labeled synthetic bi-temporal image pairs. and tags The synthesized bi-temporal image pairs are then enhanced at the image level in terms of color and texture, resulting in enhanced bi-temporal image pairs. The original image pairs and the enhanced image pairs share a single label. The labeled original image pairs and enhanced image pairs are fed into the encoder of the main network to extract features of the original image pairs from different time periods. and and enhancing image features and ,in It is a feature extractor for original image pairs and enhanced image pairs, sharing weights, and will and The two-temporal fusion features of the original image pairs are obtained by concatenation. , and The concatenation yields bi-temporal fusion features of the enhanced image pairs. Then, they are fed into two decoders in the main network to be decoded into a change prediction map. and The detection results are obtained after feeding the data into a 3×3 convolutional prediction head. and .

[0014] Furthermore, based on the segmentation loss function of the original image branch and the enhanced image branch of the remote sensing image change detection network... Autocovariance matching loss function and cross-covariance diagonal loss function Establish the overall loss function :

[0015]

[0016] Among these, a grid search was performed on the hyperparameters, and they were set to [value] in all experiments. .

[0017] Furthermore, the remote sensing image change detection network uses a segmentation loss function for the original data bi-temporal image pairs and the enhanced data bi-temporal image pairs. for:

[0018]

[0019] in, It is the cross-entropy loss function, and the specific calculation method is as follows:

[0020]

[0021] Among them, y i This is the detection result image for the corresponding branch, and M is the corresponding label.

[0022] Furthermore, to minimize the difference in feature distributions from the same temporal phase (content) but different domains (style), the autocovariance matching loss between the enhanced image and the original image is considered. This loss is applied to the original feature F output from the i-th layer of the encoder at the k-th temporal phase. ki First, feature normalization is performed on the enhanced feature FA ki to obtain ki and Given Aki, calculate their respective autocovariance matrices as follows:

[0023]

[0024] Then, in order to align images with the same content but different styles, the following loss function is used:

[0025] .

[0026] Furthermore, in order to remove style influences while preserving the integrity of image content, the cross-covariance diagonal loss between the enhanced image and the original image is considered, and the weights are adjusted accordingly. ki and Given Aki, calculate the cross-covariance matrix as follows:

[0027]

[0028] Then, to maintain the consistency of the network's semantic response, the diagonal elements of the cross-covariance matrix are converged to 1. To preserve the integrity of the content, no operation is performed on the off-diagonal elements. The cross-covariance diagonal loss is as follows:

[0029]

[0030] During the training iteration, minimizing style influence while preserving the integrity of image content helps improve domain alignment and cross-dataset feature recognition capabilities. Furthermore, the images of the augmented branches are only used during the training phase, and only the original branches are used to obtain detection results during the inference phase.

[0031] The beneficial effects of this invention are as follows: (1) This invention proposes a novel single-domain generalized framework for remote sensing image change detection, which aims to enable models trained solely on source domain data to directly generalize to unseen target domains by introducing domain-invariant feature extraction techniques; in addition, this framework can also significantly improve the performance of existing models when used in a single domain. By designing a new network structure and combining content alignment, the model can remove the influence of style while preserving the integrity of image content during training, thereby effectively reducing the domain offset between the training domain and the unseen domain and improving the model's adaptability to data from different sources; (2) This invention designs an autocovariance matching loss function, which enables the model to minimize the difference between images with different styles and the same content. The regularization can stably achieve the focus and extraction of domain-invariant features, which improves the robustness of the model in detecting changes in remote sensing images in different scenarios; (3) In order to solve the problem of damage to image content caused by previous destylation methods, this invention starts from the perspective of cross-covariance matrix, hoping that the model can maintain the semantic response consistency of different channels of the model while preserving the integrity of image content; This invention starts from the perspective of element operation of cross-covariance matrix of image features with different styles and the same content, and designs a simple and effective cross-covariance diagonal loss function. This module preserves the integrity of content while destylating the influence. Experimental results show that the framework proposed in this invention can significantly improve the detection performance of existing models in invisible domains. Attached Figure Description

[0032] Figure 1 is a schematic diagram of the method flow of the present invention.

[0033] Figure 2 is a flowchart of the method framework of the present invention.

[0034] Figure 3 shows the detection results of this invention on an unseen domain dataset. Detailed Implementation

[0035] As shown in Figures 1 and 2, a domain-generalized remote sensing image change detection method based on domain-invariant feature extraction includes the following steps:

[0036] Geometric augmentation was performed on the acquired synthetic dual-temporal remote sensing image data, and the resulting augmented synthetic dataset was divided into training and test sets.

[0037] Construct a semantic-based domain-generalized remote sensing image change detection network;

[0038] The change detection algorithm is selected as the feature extractor to obtain the output features of the original image pair and the output features of the enhanced image pair with the same spatial size as the output. The two sets of features are then input into the prediction head of the 3×3 convolution with shared weights to obtain two detection result images.

[0039] Construct the autocovariance matching loss function and the cross-covariance diagonal loss function;

[0040] Set the overall loss function, iteratively train and optimize the network parameters, and after the loss stabilizes, input the detected image into the trained neural network to obtain the final detection result image.

[0041] This invention provides a specific embodiment, which uses a synthetic remote sensing image dataset containing 12,000 image pairs for building binary change detection, and three real remote sensing image datasets containing 7,434, 3,601, and 6,091 image pairs respectively. These datasets come from different regions, have different spatial resolutions, different target shapes and sizes, and different change types. The image size is uniformly 256×256. The entire synthetic dataset is divided into training and test sets in a 5:1 ratio, while the real datasets are divided into training, validation, and test sets in ratios of 8:1:1, 6:2:2, and 7:2:1 respectively. The synthetic training set is used to train the network and save the final model. The synthetic and real test sets are used to compare the performance of the method of this invention with other existing methods. This embodiment is implemented on a computer with an AMD Ryzen 99950X CPU, an NVIDIA GeForce RTX 4090 (24G) GPU, and 93.6G of memory, and the programming language is Python.

[0042] During the training phase, the Siamese DeepLabV3+ change detection network was selected as the bi-temporal feature extractor for both the original and augmentation branches. For the encoder, ResNet-50, MobileNetV2, and ShuffleNetV2 with shared weights could be chosen as the backbone. The input had four branches, processing the bi-temporal images of the original and augmentation branches respectively. The original fused features were obtained by concatenating the bi-temporal images of the original branch, and the augmented fused features were obtained by concatenating the bi-temporal images of the augmentation branch. These were then input into the decoders of the original and augmentation branches, respectively. To ensure fair comparison, all experiments were repeated three times and the average was taken. All models were trained under the same conditions using the SGD optimizer with a momentum of 0.9, a weight decay of 5e-4, for 50 epochs, and a batch size of 8. A multinomial learning rate scheduler was used with an initial learning rate of 1e-2 and an exponent of 0.9. During training, a total of 50 epochs were completed. The model trained in the last epoch was saved. During inference, test images were input, using only the original branch, to evaluate the performance of the methods, and the experimental results were saved.

[0043] First, the encoder is applied to both the original two-temporal remote sensing images and the enhanced two-temporal versions of the images. The former includes the original two-temporal images. and post-phase original image To extract two independent original feature maps and The latter includes pre-temporal enhanced images. and post-temporal enhancement images To extract two independent enhanced feature maps and This process can be represented in the following mathematical form:

[0044]

[0045] Where H×W represents the spatial dimensions of the H'×W' image, C represents the feature dimension, and r is the spatial compression ratio, determined by a specific change detection backbone network. A fused feature map is generated by concatenating the features along the channel dimensions, thereby capturing key changes between two temporal states.

[0046] Subsequently, the decoder will convert the original fused feature map and enhanced fusion feature maps Decoded into the original logit graph of change activation respectively And the enhanced version of the logit graph Dimension 2 represents the two categories: changed and unchanged. This operation can be represented as:

[0047]

[0048] Finally, the original logit graph p and the enhanced version of the logit graph p. A After softmax normalization, it is converted into the original probability map P and the enhanced version of the probability map P. A This process ensures that the sum of the probabilities of the changed and unchanged classes at each position is 1. The expression for this process is:

[0049]

[0050] P and P A Cross-entropy loss was used to optimize the change detection model, and the final total cross-entropy loss is:

[0051]

[0052] This ensures that the original and augmented branches of the change detection model can learn from labeled data in the source domain and acquire preliminary change detection capabilities. However, due to domain spacing, the change detection model cannot directly adapt to unseen target domains without domain generalization.

[0053] To enable the network to continuously adapt to diverse visual styles while preserving semantic content, this paper proposes an autocovariance matching loss. Specifically, this loss aims to minimize the difference in autocovariance matrices between features from images of different styles originating from the same temporal phase. Given a pair of images from the k-th temporal phase, the features extracted by the encoder in the i-th block are represented as follows: and Furthermore, the covariance matrix is ​​calculated using instance-normalized features to ensure scale consistency among features. The feature maps are normalized and flattened. and The specific definition is as follows:

[0054]

[0055] in, and Let represent the mean and variance of the features, respectively. Using the normalized features, calculate the autocovariance matrix of the original image features and the enhanced image features for the same time phase, as shown in the following formula:

[0056]

[0057] Subsequently, the autocovariance matching loss is designed to align these autocovariance matrices, thereby ensuring that the network maintains feature consistency under different style variations. The autocovariance matching loss is defined as follows:

[0058]

[0059] in, Indicates the number of encoder blocks.

[0060] While the autocovariance matching loss tends to make the feature distributions of images with different styles but the same content more consistent, it does not fully consider the correlation between the feature channels of the two images. Therefore, the cross-covariance diagonal loss is introduced, which aims to utilize the correlation between the feature channels of images with different styles but the same content. The cross-covariance matrix ensures that the image pairs maintain consistent channel responses, thus achieving content alignment. Given normalized feature pairs... The cross-covariance matrix of this feature pair can be expressed as:

[0061]

[0062] The cross-covariance matrix should ideally approximate the identity matrix, as this pair of features should encode the same content information. However, some crucial content information is expressed through the interactions between channels, and forcibly setting the off-diagonal elements of the cross-covariance matrix to zero will ignore this information. Therefore, the proposed cross-covariance diagonal loss only converges the diagonal elements of the cross-covariance matrix to 1. This approach prevents the shortcomings of existing feature normalization methods that remove content information. The cross-covariance diagonal loss function is defined as:

[0063]

[0064] in Indicates inclusion The column vector of diagonal elements, and This represents a column vector where all elements are 1.

[0065] During the testing phase, test samples are input into the original branch of the network to obtain the final extraction results.

[0066] Figure 3 illustrates the change detection effect of this embodiment. The first column shows the previous time-phase image of a certain area, the second column shows the subsequent time-phase image of the same area, the fifth column shows the true change labels of the corresponding buildings in that area, the third column shows the change detection results of the baseline model, and the fourth column shows the detection result image after applying this invention. In the last three columns, white represents the actual changed area, black represents the actual unchanged area, green represents the missed detection area (actually changed), and red represents the falsely detected area (actually unchanged). Based on the change detection results in Figure 3, the domain-generalized remote sensing image change detection network based on cross-domain consistency learning proposed in this invention can effectively segment the changed areas of target objects in remote sensing images when facing unseen target domains, making it an effective domain-generalized remote sensing image change detection method.

Claims

1. A domain-generalized remote sensing image change detection method based on domain-invariant feature extraction, characterized in that, The process includes the following steps: geometric augmentation of the acquired synthetic dual-temporal remote sensing image data; dividing the augmented synthetic dataset into training and testing sets; constructing a semantic-based domain-generalized remote sensing image change detection network; selecting a change detection algorithm as a feature extractor to obtain the output features of the original image pair and the augmented image pair with the same spatial dimensions as the output; inputting the two sets of features into a prediction head with shared weights of a 3×3 convolution to obtain two detection result images; constructing an autocovariance matching loss function and a cross-covariance diagonal loss function; setting the overall loss function, iteratively training and optimizing the network parameters; and after the loss stabilizes, inputting the detected image into the trained neural network to obtain the final detection result image.

2. The domain-generalized remote sensing image change detection method based on domain-invariant feature extraction as described in claim 1, characterized in that, The acquired synthetic dual-temporal remote sensing image data is geometrically enhanced by unifying image size, rotation, and cropping. Then, it is enhanced with different colors and textures: brightness perturbation amplitude of 0.6, contrast perturbation amplitude of 0.6, saturation perturbation amplitude of 0.6, hue perturbation amplitude of 0.3, and random Gaussian blur, to obtain the dual-temporal image input to the original branch; brightness perturbation amplitude of 0.3, contrast perturbation amplitude of 0.3, saturation perturbation amplitude of 0.3, hue perturbation amplitude of 0.1, and random bilateral blur, to obtain the dual-temporal image input to the enhancement branch. The two branches share the same label.

3. The domain-generalized remote sensing image change detection method based on domain-invariant feature extraction as described in claim 1, characterized in that, The remote sensing image change detection network includes feature extractors for the original bi-temporal images and the enhanced bi-temporal images, and segmentation loss functions for the original image pair branch and the enhanced image pair branch. Autocovariance matching loss function and cross-covariance diagonal loss function 。 4. The domain-generalized remote sensing image change detection method based on domain-invariant feature extraction as described in claim 1, characterized in that, Suppose the dataset contains 2N labeled synthetic bi-temporal image pairs. and tags The synthesized bi-temporal image pairs are then enhanced at the image level in terms of color and texture, resulting in enhanced bi-temporal image pairs. The original image pairs and the enhanced image pairs share a single label. The labeled original image pairs and enhanced image pairs are fed into the encoder of the main network to extract features of the original image pairs from different time periods. and and enhancing image features and ,in It is a feature extractor for original image pairs and enhanced image pairs, sharing weights, and will and The two-temporal fusion features of the original image pairs are obtained by concatenation. ， and The connection yields bi-temporal fusion features of the enhanced image pairs. Then, they are fed into two decoders in the main network to be decoded into a change prediction map. and The detection results are obtained after feeding the data into a 3×3 convolutional prediction head. and 。 5. The domain-generalized remote sensing image change detection method based on domain-invariant feature extraction as described in claim 1, characterized in that, The segmentation loss function of the original image branch and the enhanced image branch of the remote sensing image change detection network. Autocovariance matching loss function and cross-covariance diagonal loss function Establish the overall loss function ： Among these, a grid search was performed on the hyperparameters, and they were set to [value] in all experiments. 。 6. The domain-generalized remote sensing image change detection method based on domain-invariant feature extraction as described in claim 5, characterized in that, The remote sensing image change detection network uses a segmentation loss function for raw data bi-temporal image pairs and augmented data bi-temporal image pairs. for: in, It is the cross-entropy loss function, and the specific calculation method is as follows: Among them, y i This is the detection result image for the corresponding branch, and M is the corresponding label.

7. The domain-generalized remote sensing image change detection method based on domain-invariant feature extraction as described in claim 5, characterized in that, The original features F output by the i-th layer of the encoder at the k-th time phase ki First, feature normalization is performed on the enhanced feature FA ki to obtain ki and Given Aki, calculate their respective autocovariance matrices as follows: Then, in order to align images with the same content but different styles, the following loss function is used: 。 8. The domain-generalized remote sensing image change detection method based on domain-invariant feature extraction as described in claim 7, characterized in that, Considering the cross-covariance diagonal loss between the enhanced and original images, the weights are obtained... ki and Given Aki, calculate the cross-covariance matrix as follows: Then, to maintain the consistency of the network's semantic response, the diagonal elements of the cross-covariance matrix are converged to 1. To preserve the integrity of the content, no operation is performed on the off-diagonal elements. The cross-covariance diagonal loss is as follows: 。

Images