A deep network for remote sensing change interpretation using joint distribution sampling and feature decoupling

Abstract

The present invention discloses a remote sensing change interpretation deep network with joint distribution sampling and feature decoupling, which belongs to the field of remote sensing image processing technology. In view of the problem that mixed feature extraction easily leads to blurred semantic boundaries, which in turn causes inaccurate classification of remote sensing change pixels, the mixed features are decoupled into change and invariant features through joint distribution sampling to complete the remote sensing change interpretation. In the training stage, the posterior distribution of the decoupled features is used to train the change prior generator through label calibration learning; the posterior distribution and feature separator are combined to achieve feature decoupling, and the features are further aggregated through prototype learning; an over-expectation push-pull loss regularization term is proposed, which increases the distance between classes by improving the predicted expectation to push and pull the positive and negative sample features to the farther end. In the testing stage, remote sensing image change detection is completed through modules such as feature separator and change detection head without the support of posterior distribution. Experiments have shown that this paper has achieved significant results in both qualitative and quantitative indicators.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and in particular relates to a remote sensing change interpretation deep network combining distribution sampling and feature decoupling. Background Art

[0002] Remote sensing image change detection aims to compare the differences in images of the same area at different time periods and mark the changed areas. It has been widely used in urban expansion analysis, disaster assessment, military strikes, vegetation cover detection and other fields.

[0003] Traditional change detection methods can be roughly divided into three categories: those based on image arithmetic, image transformation, and post-classification. Image arithmetic methods include image differencing, image ratio, and change vector analysis. These algorithms use relevant arithmetic (such as subtraction or division) to generate feature maps and then use segmentation thresholds to distinguish between changed and unchanged information. Clearly, the segmentation threshold is both a challenge and a key point in these methods. Image transformation methods include principal component analysis and multivariate change detection. These methods transform images into a specific feature space, enhancing change detection accuracy by highlighting changed regions and suppressing unchanged regions. Post-classification methods first classify two time-phase images, then compare and analyze the classification results to generate a change map. Since the accuracy of post-classification methods depends on the accuracy of the classification, they suffer from a significant error accumulation effect.

[0004] Advances in machine learning, such as support vector machines and random forests, have significantly improved the accuracy of change detection. However, the selection of machine learning methods and the generalization of models across diverse scenarios pose challenges for practical application. Deep learning methods, due to their powerful feature extraction and nonlinear representation capabilities, have been widely applied to change detection tasks and demonstrated excellent performance.

[0005] Representative deep network frameworks for remote sensing change detection include CNN, RNN, Transformer, Mamba, and their hybrids. Many researchers have accomplished change detection tasks based on FCN and Unet networks. Another mainstream CNN-based change detection network is the Siamese network. These networks utilize a two-stream architecture to extract deep features from bi-temporal remote sensing imagery and fuse and enhance relevant features to produce change detection results. To exploit the temporal correlations within bi-temporal image feature tensors, some researchers have proposed change detection models that incorporate long short-term memory (LSTM). Notably, to enable neural networks to accurately focus on important features and suppress unimportant ones, some research has used attention mechanisms to identify regions of interest in bi-temporal remote sensing imagery. These methods primarily employ various methods (channel-wise, spatially, and correlation-wise) to reweight features, highlighting effective information and ultimately improving the performance of change detection models.

[0006] To identify and correlate regions of change across time and location, some studies have used Transformers to establish long-range dependencies. This approach can learn useful information in parallel across different representation subspaces through a multi-head attention mechanism, regardless of the location of these regions in the image. However, Transformer models typically contain a large number of parameters and complex self-attention mechanisms, which often consumes a lot of computing resources, memory requirements, and time during training and inference.

[0007] Mamba has also been used in remote sensing change detection. This model incorporates an omnidirectional selective scanning module, enabling selective scanning of images in multiple directions, thereby capturing large spatial features from diverse directions. The Mamba model's application in remote sensing image change detection has demonstrated its powerful ability to model spatiotemporal relationships and efficiently capture global context. However, due to the multiple temporal scans involved, remote sensing change detection models based on Mamba also require significant computational resources.

[0008] Change detection tasks based on mainstream deep learning frameworks such as CNN, Transformer, and Mamba typically involve extracting and fusing mixed features, followed by pixel classification. However, mixed features hinder the model's ability to learn the nature of changing and invariant features. Consequently, to improve change detection performance, models tend to be more complex and contain more parameters. Summary of the Invention

[0009] To address the problem that hybrid feature extraction easily leads to blurred semantic boundaries and thus inaccurate classification of remote sensing change pixels, this paper proposes a remote sensing change interpretation deep network that combines joint distribution sampling and feature decoupling. This network aims to decompose bi-temporal remote sensing features into change features and invariant features to achieve change interpretation. First, the posterior probability distributions of change and invariant features are obtained from difference features and labels, and used to guide the prior probability feature generator. Then, a joint feature separator and posterior distribution reparameter sampling are used to obtain change and invariant features, respectively. Prototype learning is used to make the decoupled features more compact. Finally, the cross-entropy and super-expectation push-pull hybrid loss function is used to expand the distance between positive and negative sample classes, making it easier to separate change and invariant pixels.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] A deep network for remote sensing change interpretation with joint distribution sampling and feature decoupling is proposed. The deep network includes a change prior generator, an invariant prior generator, a feature extractor, a feature separator, and a change detection head. The U-Net encoder in the feature extractor serves as the backbone network. During the training phase, the above modules are trained through posterior features and prototype learning. During the testing phase, the weights of the above modules are frozen, and change results are predicted without the support of posterior features.

[0012] Furthermore, the specific operations of the deep network in the training stage are: first, the posterior distribution after feature decoupling is learned through label calibration, which is used to train the variable prior generator and the invariant prior generator; then, the posterior distribution and the feature separator are combined to achieve feature decoupling, and the features are further aggregated through prototype learning; finally, the super-expectation push-pull loss regularization term is used to push and pull the positive and negative sample features to further ends by improving the prediction expectation, thereby achieving the purpose of increasing the inter-class distance.

[0013] Furthermore, the specific operation of using the posterior distribution after feature decoupling through label calibration to train the variable prior generator and the invariant prior generator is as follows:

[0014] Convolution-based feature extraction retains better spatial location information, while features based on distributed sampling can introduce randomness, making the model more robust to data changes. To this end, the present invention combines the advantages of both feature extraction methods to achieve feature decoupling.

[0015] First, using dual-temporal remote sensing images alone The difference map X generates a decoupled feature prior distribution through two independent U-Net encoders: the change feature prior distribution Invariant feature prior distribution The two feature prior distributions are set to multivariate normal distributions with diagonal covariance matrices, denoted as and

[0016] The decoupled feature prior distribution mean and variance are obtained as follows:

[0017]

[0018] Where, and is two learnable prior decoupled distribution generators, such as Figure 2 (a) Shown in the upper left.

[0019] Using U-Net encoder E u1 Encode the dual-phase difference features to obtain X * , and then select the posterior decoupling features through labels, and then obtain the posterior distribution of the decoupled features through learning:

[0020]

[0021] The mean and variance of the posterior distribution of the decoupled features are obtained as follows:

[0022]

[0023] Where, are two learnable posterior decoupled distribution generators, such as Figure 2 (a) Shown in the upper left.

[0024] To train the varying prior generator and the invariant prior generator by decoupling the feature posterior distribution, the Kullback-Leibler divergence (KL) loss is used to obtain the decoupled feature distribution:

[0025]

[0026] Where, represents the prior distribution of the changing characteristics, represents the posterior distribution of the changing features, represents the prior distribution of invariant features, represents the posterior distribution of the invariant features.

[0027] After obtaining the decoupled feature distribution, change detection is achieved by combining distributed sampling features with convolutional extraction features.

[0028] Assuming a given posterior distribution Randomly sample a latent code and further scaled to Where d is the dimension of all latent codes;

[0029] Next, the scaled latent code is combined with the decoupled features {S c ,S nc} and input it into the change detection head In , auxiliary features are used for decoupling and the corresponding decoupling results are generated:

[0030]

[0031] Where, and Represents the probability map of changing and unchanged features based on the posterior distribution prediction; and Detection head networks that represent changing and invariant features respectively;

[0032] Combined with the posterior distribution prediction change probability map Perform pixel-wise argmax operation on its channel dimension to obtain the final change mask.

[0033] In order to train the prior distribution generator to assist feature decoupling and complete the change detection task, the change mask P can also be obtained from the prior distribution sampling features. prior .

[0034] In terms of loss function, the network parameters are optimized by minimizing cross entropy. It is worth noting that the reparameterization technique is used to respectively and the posterior distribution Latent coding Sampling is performed and gradients are back-propagated for model training.

[0035]

[0036] Where, and Represents the prediction results and label Y based on prior and posterior features respectively hw The average cross entropy loss value of Represents the prediction results and label Y based on prior and posterior features respectively hw Perform binary cross entropy loss on each pixel; H represents the number of pixels in the height direction of the image; W represents the number of pixels in the width direction of the image.

[0037] Furthermore, the specific operations of the joint posterior distribution and feature separator to achieve feature decoupling and further aggregate features through prototype learning are as follows:

[0038] Prototype learning uses typical features of data to train network parameters. This method makes the decoupling space more compact by constraining the distance between the decoupled features and their prototypes.

[0039] First, the decoupled features {S c ,S nc} respectively with distribution sampling characteristics and Cascade to obtain the final decoupled features as well as

[0040] Then, the true value is used to locate the changing features and the invariant features in the feature map to obtain the prototype for generating the decoupled feature; the prior feature prototype is as follows:

[0041]

[0042] Among them, Ω c and Ω nc are the sets of changing features and invariant features respectively; |Ω c | and |Ω nc | represents the set Ω c and Ωnc size; represents the i-th change feature based on the decoupling of prior features; represents the jth invariant feature based on the decoupling of prior features, and They represent the decoupling of changing and invariant prototypes based on prior features;

[0043] Repeat the above operations to obtain the decoupled change and invariant prototypes based on the posterior features.

[0044] Because in the final test phase, we rely on the prior distribution to predict changes without relying on the posterior distribution, so the prototype loss here mainly constrains the prior decoupled features. The prototype constraints for sampling the change features obtained by the joint prior features are as follows:

[0045]

[0046] The prototype constraints for sampling the invariant features from the joint prior features are as follows:

[0047]

[0048] Where N c and N un are the set of changing and unchanged pixels respectively; t is the temperature coefficient;

[0049] The overall decoupled feature prototype constraint loss function is as follows:

[0050]

[0051] Furthermore, the use of the over-expectation push-pull loss regularization term aims to increase the inter-class distance by improving the predicted expected push-pull positive and negative sample features to a further end. The specific operations are as follows:

[0052] Most loss functions predict an expected value of 1 for positive samples and 0 for negative samples. Taking cross entropy as an example, when positive and negative samples are misclassified, the cross entropy will calculate a large loss value, which will help the model adjust the network parameters; however, when the positive and negative samples are correctly classified or even close to the expected value, the cross entropy will drop sharply, affecting the convergence speed and even blurring the classification boundaries. Inspired by this, the expected values ​​of the predicted probabilities of positive and negative examples are pushed and pulled to T and τ respectively through the super-expectation push-pull loss function; when T>1 or τ<0, the predicted probability does not reach the corresponding expected value, but a larger loss value will be calculated for each iteration, which will help improve model performance and accelerate convergence within a certain range. The loss function is shown below:

[0053]

[0054] Among them, 0 represents the unchanged pixel, Nun Indicates the number of unchanged pixels; 1 indicates the number of changed pixels, N c Indicates the number of changed pixels; Y k Indicates the kth pixel value of the label that changes or remains unchanged, P k Represents the model's predicted probability value for the kth pixel.

[0055] Furthermore, the specific operations of the deep network in the testing phase are:

[0056] By predicting the change results through the prior distribution, the prior and posterior losses are separated to obtain the final loss function L T :

[0057]

[0058] Where λ1, λ2, λ3 and λ4 represent the regularization coefficients of the corresponding loss function.

[0059] Compared with the prior art, the present invention has the following advantages:

[0060] (1) Learn the posterior probability distribution of change and invariant features and use reparameterized sampling techniques to assist in training the change prior generator and the invariant prior generator. In the test phase, combine the convolutional features and the prior distribution sampling features to predict the change area.

[0061] (2) The feature separator is trained jointly by posterior distribution features and prototype learning. In the test phase, the extracted features are decoupled into invariant and variable features by combining the feature separator, the invariant prior generator, and the variable prior generator.

[0062] (3) A positive and negative sample push-pull loss regularization term is proposed. The mixed loss function composed of the positive and negative sample push-pull loss regularization term and the cross entropy loss regularization term can effectively increase the distance between positive and negative sample classes, which is directly beneficial to pixel classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a visual comparison chart of the network model efficiency and comprehensive index F1 of the present invention;

[0064] Figure 2 A schematic diagram of a remote sensing change interpretation deep network for joint distribution sampling and feature decoupling according to the present invention;

[0065] Figure 3 This is the structure diagram of the prototype loss of the prior decoupling feature;

[0066] Figure 4 Comparison chart of excess expectation loss and cross entropy;

[0067] Figure 5 The results of Levir-cd detection by different methods are shown in Figure 2.

[0068] Figure 6 The results of cdd detection by different methods are shown in Figure 2.

[0069] Figure 7 Figure 2 is a graph showing the test results of the system using different methods;

[0070] Figure 8 A visual representation of the key modules of DSFDcd. DETAILED DESCRIPTION

[0071] To gain a deeper understanding of the present invention, we will provide a comprehensive and detailed description thereof. However, the present invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a comprehensive understanding of the present disclosure.

[0072] To address the problem that hybrid feature extraction easily leads to blurred semantic boundaries and thus inaccurate classification of remote sensing change pixels, this paper proposes a remote sensing change interpretation deep network that combines joint distribution sampling and feature decoupling. This network aims to decompose bi-temporal remote sensing features into change features and invariant features to achieve change interpretation. First, the posterior probability distributions of change and invariant features are obtained from difference features and labels, and used to guide the prior probability feature generator. Then, a joint feature separator and posterior distribution reparameter sampling are used to obtain change and invariant features, respectively. Prototype learning is used to make the decoupled features more compact. Finally, the cross-entropy and super-expectation push-pull hybrid loss function is used to expand the distance between positive and negative sample classes, making it easier to separate change and invariant pixels.

[0073] In order to achieve the above object, the present invention adopts the following technical solutions:

[0074] A deep network for remote sensing change interpretation with joint distribution sampling and feature decoupling is proposed. The deep network includes five modules: a change prior generator, an invariant prior generator, a feature extractor, a feature separator, and a change detection head. The U-Net encoder in the feature extractor serves as the backbone network. During the training phase, the five modules are trained through posterior features and prototype learning. During the testing phase, the weights of the modules are frozen, and change results are predicted without the support of posterior features.

[0075] Furthermore, the specific operations of the deep network in the training stage are: first, the posterior distribution after feature decoupling is learned through label calibration, which is used to train the variable prior generator and the invariant prior generator; then, the posterior distribution and the feature separator are combined to achieve feature decoupling, and the features are further aggregated through prototype learning; finally, an over-expectation push-pull loss regularization term is proposed, which aims to increase the inter-class distance by improving the predicted expectation and pushing and pulling the positive and negative sample features to further ends.

[0076] Furthermore, the specific operation of using the posterior distribution after feature decoupling through label calibration to train the variable prior generator and the invariant prior generator is as follows:

[0077] Convolution-based feature extraction retains better spatial location information, while features based on distributed sampling can introduce randomness, making the model more robust to data changes. To this end, the present invention combines the advantages of both feature extraction methods to achieve feature decoupling.

[0078] First, using dual-temporal remote sensing images alone The difference map X generates a decoupled feature prior distribution through two independent U-Net encoders: the change feature prior distribution Invariant feature prior distribution The two feature prior distributions are set to multivariate normal distributions with diagonal covariance matrices, denoted as and

[0079] The decoupled feature prior distribution mean and variance are obtained as follows:

[0080]

[0081] Where, and is two learnable prior decoupled distribution generators, such as Figure 2 (a) Shown in the upper left.

[0082] Using U-Net encoder E u1 Encode the dual-phase difference features to obtain X * , and then select the posterior decoupling features through labels, and then obtain the posterior distribution of the decoupled features through learning:

[0083]

[0084] The mean and variance of the posterior distribution of the decoupled features are obtained as follows:

[0085]

[0086] Where, are two learnable posterior decoupled distribution generators, such as Figure 2 (a) Shown in the upper left.

[0087] To train the varying prior generator and the invariant prior generator by decoupling the feature posterior distribution, the Kullback-Leibler divergence (KL) loss is used to obtain the decoupled feature distribution:

[0088]

[0089] Where, represents the prior distribution of the changing characteristics, represents the posterior distribution of the changing features, represents the prior distribution of invariant features, represents the posterior distribution of the invariant features;

[0090] After obtaining the decoupled feature distribution, change detection is achieved by combining distributed sampling features with convolutional extraction features.

[0091] Assuming a given posterior distribution Randomly sample a latent code and further scaled to Where d is the dimension of all latent codes;

[0092] Next, the scaled latent code is combined with the decoupled features {S c ,S nc} and input it into the change detection head In , auxiliary features are used for decoupling and the corresponding decoupling results are generated:

[0093]

[0094] Where, and Represents the probability map of changing and unchanged features based on the posterior distribution prediction; and Detection head networks that represent changing and invariant features respectively;

[0095] Combined with the posterior distribution prediction change probability map Perform pixel-wise argmax operation on its channel dimension to obtain the final change mask.

[0096] In order to train the prior distribution generator to assist feature decoupling and complete the change detection task, the change mask P can also be obtained from the prior distribution sampling features. prior .

[0097] In terms of loss function, the network parameters are optimized by minimizing cross entropy. It is worth noting that the reparameterization technique is used to respectively and the posterior distribution Latent coding Sampling is performed and gradients are back-propagated for model training.

[0098]

[0099] Where, and Represents the prediction results and label Y based on prior and posterior features respectively hwThe average cross entropy loss value of Represents the prediction results and label Y based on prior and posterior features respectively hw Perform binary cross entropy loss on each pixel; H represents the number of pixels in the height direction of the image; W represents the number of pixels in the width direction of the image.

[0100] Furthermore, the specific operations of the joint posterior distribution and feature separator to achieve feature decoupling and further aggregate features through prototype learning are as follows:

[0101] Prototype learning uses typical features of data to train network parameters. This method makes the decoupling space more compact by constraining the distance between the decoupled features and their prototypes.

[0102] First, the decoupled features {S c ,S nc} respectively with distribution sampling characteristics and Cascade to obtain the final decoupled features as well as

[0103] Then, the true value is used to locate the changing features and the invariant features in the feature map to obtain the prototype for generating the decoupled feature; the prior feature prototype is as follows:

[0104]

[0105] Among them, Ω c and Ω nc are the sets of changing features and invariant features respectively; |Ω c | and |Ω nc | represents the set Ω c and Ω nc size; represents the i-th change feature based on the decoupling of prior features; represents the jth invariant feature based on the decoupling of prior features, and They represent the decoupling of changing and invariant prototypes based on prior features;

[0106] Repeat the above operations to obtain the decoupled change and invariant prototypes based on the posterior features.

[0107] Because in the final test phase, we rely on the prior distribution to predict changes without relying on the posterior distribution, so the prototype loss here mainly constrains the prior decoupled features. The prototype constraints for sampling the change features obtained by the joint prior features are as follows:

[0108]

[0109] The prototype constraints for sampling the invariant features from the joint prior features are as follows:

[0110]

[0111] Where N c and N un are the set of changing and unchanged pixels respectively; t is the temperature coefficient;

[0112] The overall decoupled feature prototype constraint loss function is as follows:

[0113]

[0114] Furthermore, the use of the over-expectation push-pull loss regularization term aims to increase the inter-class distance by improving the predicted expected push-pull positive and negative sample features to a further end. The specific operations are as follows:

[0115] Most loss functions predict an expected value of 1 for positive samples and 0 for negative samples. Taking cross entropy as an example, when positive and negative samples are misclassified, the cross entropy will calculate a large loss value, which will help the model adjust the network parameters; however, when the positive and negative samples are correctly classified or even close to the expected value, the cross entropy will drop sharply, affecting the convergence speed and even blurring the classification boundaries. Inspired by this, the expected values ​​of the predicted probabilities of positive and negative examples are pushed and pulled to T and τ respectively through the super-expectation push-pull loss function; when T>1 or τ<0, the predicted probability does not reach the corresponding expected value, but a larger loss value will be calculated for each iteration, which will help improve model performance and accelerate convergence within a certain range. The loss function is shown below:

[0116]

[0117] Among them, 0 represents the unchanged pixel, N un Indicates the number of unchanged pixels; 1 indicates the number of changed pixels, N c Indicates the number of changed pixels; Y k Indicates the kth pixel value of the label that changes or remains unchanged, P k Represents the model's predicted probability value for the kth pixel.

[0118] Furthermore, the specific operations of the deep network in the testing phase are:

[0119] By predicting the change results through the prior distribution, the prior and posterior losses are separated to obtain the final loss function L T :

[0120]

[0121] Where λ1, λ2, λ3 and λ4 represent the regularization coefficients of the corresponding loss function.

[0122] To validate the effectiveness of a deep network for remote sensing change interpretation (DSFDcd) that combines distribution sampling and feature decoupling, we conducted experiments on three representative remote sensing image change detection datasets. Each dataset contains two remote sensing images taken at different times in the same area and their change detection results. Details of the three datasets are as follows:

[0123] LEVIR-CD DatasetLEVIR-CD Dataset

[0124] LEVIR-CD is a public, large-scale building CD dataset consisting of 637 pairs of 1024×1024 ultra-high-resolution (0.5 m / pixel) RS images spanning 5-14 years. These images are collected from 20 different areas across multiple cities in Texas, USA, and cover 31,333 building instances with significant changes. Notably, LEVIR-CD covers a wide range of building types with significant land use changes, such as residential villas, high-rise apartments, small garages, and large warehouses. Each image pair is cropped into 16 pairs of 256×256 images without overlap, and 7120 / 1024 / 2048 pairs are used for training / validation / testing, respectively.

[0125] CDD DatasetCDD Dataset

[0126] CDD is a public CD dataset consisting of seasonal satellite imagery with a spatial resolution of 0.03 to 1 meter per pixel. The regions of change in CDD vary in size, including buildings, roads, and vehicles. The image data is partitioned into 10,000, 2,998, and 3,000 pairs of 256×256 patches for training, validation, and testing, respectively.

[0127] SYSU DatasetSYSU Dataset

[0128] The SYSU dataset contains 20,000 pairs of 256×256-pixel aerial images with a spatial resolution of 0.5 meters and covering three wavelength bands. These images were taken between 2007 and 2014 in Hong Kong, China. They depict a wide variety of changes, including the emergence of new urban buildings, suburban sprawl, continuous road widening and extension, and large-scale land reclamation. During data processing, the images were randomly divided into training, validation, and test sets in a 6:2:2 ratio.

[0129] A deep network for remote sensing change interpretation using joint distribution sampling and feature disentanglement, DSFDcd, was implemented in PyTorch and trained and tested on a single NVIDIA RTX4090 GPU. The model was optimized using the SGD optimizer with momentum set to 0.9, weight loss set to 5e-4, and an initial learning rate of 0.0001. The optimizer's learning rate was dynamically adjusted, decaying by a factor of 0.7 every 10 training epochs. The training epochs for each dataset were 200, and the batch size was 6.

[0130] The experimental results are comprehensively evaluated using five evaluation indicators: precision (P), recall (Re), F1, overall accuracy (OA), and mean intersection over union (mIoU). The definitions of the above indicators are as follows:

[0131]

[0132]

[0133] Among them, TP, FP, TN and FN refer to the number of true positives, false positives, true negatives and false negatives respectively.

[0134] Experimental Results

[0135] AComparison Methods (Quantitative Indicators / Qualitative Visual Comparison)

[0136] To validate the effectiveness of DSFDcd on dual-temporal RS images, we compare it with several state-of-the-art CD methods, including DTCDSCN, BIT, Change Former, ICIFNet, DMINet, USSFC-Net, and ChangeMamba. To ensure fairness, all comparisons are performed using the code published by the authors, and their parameters are set according to the original literature.

[0137] Table 1 Performance comparison of different methods on different datasets

[0138]

[0139] *All values ​​are reported in percentage(%).Color convention: best, 2nd best .

[0140] Table 1 shows the quantitative metrics of different models on three datasets (LEVIR, CDD, and SYSU-CD). The proposed method performs well in comprehensive metrics such as F1 score, OA, and mIoU across multiple datasets. For example, it achieves an F1 score of 89.11 on the LEVIR dataset. This is due to mechanisms such as feature decoupling that improve semantic boundary ambiguity and enable precise localization of change regions. Figure 1 This visualization shows a visual comparison of the efficiency of our network model and the comprehensive F1 metric. Larger bubbles indicate higher F1 values. This demonstrates that our network model, through decoupling features from the posterior distribution and prototype learning, achieves high objective evaluation in remote sensing change detection. Notably, feature generation through distributed sampling keeps both model parameters and FLOPS within acceptable ranges.

[0141] The methods are compared intuitively on different datasets. Figure 5-7 In the figure, (a) T1 Image. (b) T2 Image. (c) Ground Truth. (d) DTCDSCN. (e) BIT. (f) ChangeFormer. (g) ICIFNet. (h) DMINet. (l) USSFC-Net. (j) ChangeMamba. (k) DSFDcd; on the LEVIR-CD dataset, BIT, ICIFNet, DMINet, and ChangeMamba have serious missed detections in small target areas, and USSFC-Net has large false detections in large target areas. BIT is prone to false detection and DMINet is prone to missed detection near the edges of complex and densely arranged buildings. When there is light interference, DTCDSCN has obvious false detections, BIT, DMINet, and USSFC-Net have obvious missed detections, and the DSFDcd algorithm has clear boundaries and false detections. The missed detection rate is low; the CDD dataset emphasizes the impact of seasonal changes. BIT, ChangeFormer and ICIFNet can identify areas with significant changes but ignore areas with minor changes. DMINet has made progress but lacks performance in subtle changes. In areas with large changes, ChangeFormer, ICIFNet and USSFCNet have problems with false detection of edge targets and loss of unchanged objects. DSFDcd has excellent performance in detecting areas with large and small changes and is highly resistant to seasonal changes. On the SYSU-CD dataset, DTCDSCN, ICIFNet and ChangeMamba have serious false detections when there is illumination disturbance, and large-scale target change detection in complex scenes is prone to missed detection. The DSFDcd proposed in this invention performs better in terms of target integrity and boundary accuracy.

[0142] Ablation Experiments

[0143] Table 2 Ablation experiments of each module of the model

[0144]

[0145] This paper validates the effectiveness of the proposed modules (i.e., decoupled feature distribution, prototype loss, and over-expectation loss) through ablation experiments. As shown in Table 2, where "√" indicates the adoption of the corresponding module, the experiments demonstrate that each module contributes to improving model performance regardless of the dataset.

[0146] C Parameter Analysis

[0147] 1) Determination of the super-expected loss hyperparameters T and τ

[0148] Table 3. The impact of using different parameters τ and T in the super-expectation loss function on the model on LEVIR-CD

[0149]

[0150] Table 3 shows the impact of different parameters τ and T of the super expectation loss function on the model on LEVIR-CD. At this time, the ratio of cross entropy loss and super expectation push-pull loss is set to 15:1. Under the premise of keeping τ = 0, the impact of different T values ​​on model performance is tested to explore the maximum target threshold to which the positive sample can be pushed and pulled. Through this series of experiments, it is finally determined that when T = 2, the comprehensive performance of the model reaches a better solution. Similarly, another optimal value τ = -1 is obtained. Experiments show that excessive pushing and pulling may cause the loss value to be too large, which is not conducive to parameter stability and optimization. Comparison of cross entropy and super expectation loss Figure 4 As shown in the figure, when false positives or missed positives occur—that is, when the model's prediction differs significantly from the ground truth (GT)—the cross-entropy loss function clearly generates a larger loss, which is more conducive to model convergence and even improved performance. When the prediction is relatively close to the expected value, the cross-entropy loss function generates a sharp decrease in loss. In this case, the over-expectation push-pull loss function is expected to achieve a larger loss by increasing the predicted expectation, thereby improving model performance.

[0151] 2) Regularization Term Coefficients

[0152] Table 4: Effect of regularization coefficient on quantitative expression

[0153]

[0154] To verify the effectiveness of the beyond-expectation push-pull loss function, we experimentally tuned the regularization term of the loss function on the LEVIR-CD, CDD, and SYSU datasets. The results are shown in Table 4. The optimal parameter configuration for overall performance shows that different datasets have slightly different dependencies on the posterior distribution.

[0155] DVisualization

[0156] In order to understand the model DSFDcd more intuitively, a set of representative samples are selected from the LEVIR-CD test set to visualize the feature maps generated by the DSFDcd model at different stages, as shown in Figure 2. Figure 8 As shown, Figure 8 (a) and (b) are dual-phase images; (c) is the difference feature; (d) and (e) are the decoupled features from the feature separator; (f) and (g) are the prior decoupled features under the prototype constraint; (h) and (i) are the posterior decoupled features; (j) is the change detection result based on the posterior distribution; (k) is the change detection result based on the prior distribution; (l) is the true value (Ground Truth, GT); the change feature S is generated by the feature separator c and the invariant feature S nc ( Figure 8 (d) and (e)) show that the model has well focused the attention of the decoupled features on the area of ​​interest and suppressed the interference of background irrelevant to the task. In addition, the extracted decoupled features and the prior distribution sampling features are well matched under the prototype constraint ( Figure 8 (f) and (g)) The attention ratio of the region of interest to the decoupled features and the posterior distribution feature sampling without prototype constraints ( Figure 8 (h) and (i)) are more concentrated. The generated change probability map (8(j)] is better than [ Figure 8 (k)) shows more accurate target positioning and fine boundaries, indicating that the DSFDcd of the present invention achieves feature decoupling and effectively captures the changing targets through feature extraction and joint distribution sampling.

[0157] Any matters not described in detail in this specification are prior art known to those skilled in the art. Although the above description of the present invention is based on specific embodiments to facilitate understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the specific embodiments. As long as various modifications are within the spirit and scope of the present invention as defined and determined by the appended claims, such modifications will be obvious to those skilled in the art, and all inventions and creations utilizing the concepts of the present invention are protected.

Claims

1. A deep network for remote sensing change interpretation with joint distribution sampling and feature decoupling, characterized by: The deep network includes a change prior generator, an invariant prior generator, a feature extractor, a feature separator, and a change detection head; the U-Net encoder in the feature extractor is used as the backbone network, and in the training phase, the change prior generator, the invariant prior generator, the feature extractor, the feature separator, and the change detection head are trained through posterior features and prototype learning; in the testing phase, the weights of the change prior generator, the invariant prior generator, the feature extractor, the feature separator, and the change detection head are frozen, and the change results are predicted without the support of the posterior features; the specific operations of the deep network in the training phase are: first, the posterior distribution after feature decoupling is learned through label calibration, which is used to train the change prior generator and the invariant prior generator; then, the posterior distribution and the feature separator are combined to achieve feature decoupling, and the features are further aggregated through prototype learning; finally, the super-expectation push-pull loss regularization term is used to increase the inter-class distance; the input of the deep network is a dual-temporal remote sensing image.

2. The remote sensing change interpretation deep network with joint distribution sampling and feature decoupling according to claim 1, characterized in that: The specific operation of learning the posterior distribution after feature decoupling through label calibration and using it to train the variable prior generator and the invariant prior generator is as follows: First, the dual-temporal remote sensing image difference map X is used alone to generate the decoupled feature prior distribution through two independent U-Net encoders: the change feature prior distribution Invariant feature prior distribution The two feature prior distributions are set to multivariate normal distributions with diagonal covariance matrices, denoted as and The dual-temporal remote sensing image is The decoupled feature prior distribution mean and variance are obtained as follows: Where, and Decouple the distribution generator for two learnable priors; Using U-Net encoder E u1 Encode the dual-phase difference features to obtain X * , and then select the posterior decoupling features through labels, and then obtain the posterior distribution of the decoupled features through learning: The mean and variance of the posterior distribution of the decoupled features are obtained as follows: Where, are two learnable posterior decoupled distribution generators; To train the varying prior generator and the invariant prior generator by decoupling the feature posterior distribution, the Kullback-Leibler divergence loss is used to obtain the decoupled feature distribution: Where, represents the prior distribution of the changing characteristics, represents the posterior distribution of the changing features, represents the prior distribution of invariant features, represents the posterior distribution of the invariant features; After obtaining the decoupled feature distribution, change detection is achieved by combining distributed sampling features with convolutional extraction features. Assuming a given posterior distribution Randomly sample a latent code and further scaled to Where d is the dimension of all latent codes; Next, the scaled latent code is combined with the decoupled features {S c ,S nc } and input it into the change detection head In , auxiliary features are used for decoupling and the corresponding decoupling results are generated: Where, and Represents the probability map of changing and unchanged features based on the posterior distribution prediction; and Detection head networks that represent changing and invariant features respectively; Combined with the posterior distribution prediction change probability map Perform pixel-level argmax operation on its channel dimension to obtain the final change mask; Using the reparameterization technique, we can distinguish between the prior distribution and the posterior distribution Latent coding Perform sampling and backpropagate gradients for model training: Where, and Represents the prediction results and label Y based on prior and posterior features respectively hw The average cross entropy loss value of and Represents the prediction results and label Y based on prior and posterior features respectively hw Perform binary cross entropy loss on each pixel; H represents the number of pixels in the height direction of the image; W represents the number of pixels in the width direction of the image.

3. The remote sensing change interpretation deep network with joint distribution sampling and feature decoupling according to claim 2, characterized in that: The specific operations of combining the posterior distribution and the feature separator to achieve feature decoupling and further aggregate features through prototype learning are as follows: First, the decoupled features {S c ,S nc } respectively with distribution sampling characteristics and Cascade to obtain the final decoupled features as well as Then, the true value is used to locate the changing features and the invariant features in the feature map, and the prototype for generating the decoupled features is obtained; The prior feature prototype is as follows: Among them, Ω c and Ω nc are the sets of changing features and invariant features respectively; |Ω c | and |Ω nc | represents the set Ω c and Ω nc size; represents the i-th change feature based on the decoupling of prior features; represents the jth invariant feature based on the decoupling of prior features, and They represent the decoupling of changing and invariant prototypes based on prior features; Repeat the above steps to obtain the decoupling of the change and invariant prototypes based on the posterior features; The prototype constraints for sampling the change features with the joint prior features are as follows: The prototype constraints for sampling the invariant features from the joint prior features are as follows: Where, is the temperature coefficient; N un Indicates the number of unchanged pixels; N c Indicates the number of changed pixels; The overall decoupled feature prototype constraint loss function is as follows:

4. The remote sensing change interpretation deep network with joint distribution sampling and feature decoupling according to claim 3, characterized in that: The specific operation of using the super-expected push-pull loss regularization term to increase the inter-class distance is: The expected values ​​of the predicted probabilities of positive and negative examples are pushed and pulled to T and τ respectively through the over-expectation push-pull loss function; when T>1 or τ<0, the predicted probability does not reach the corresponding expected value. The loss function is as follows: Among them, 0 represents unchanged pixels; 1 represents changed pixels; Y k Indicates the kth pixel value of the label that changes or remains unchanged, P k Represents the model's predicted probability value for the kth pixel.

5. The remote sensing change interpretation deep network with joint distribution sampling and feature decoupling according to claim 4, characterized in that: The specific operations of the deep network in the testing phase are: By predicting the change results through the prior distribution, the prior and posterior losses are separated to obtain the final loss function L T : Where λ1, λ2, λ3 and λ4 represent the regularization coefficients of the corresponding loss function.

Images