A collaborative optimization method and system for few-shot remote sensing change detection

Abstract

This invention relates to a collaborative optimization method and system for remote sensing change detection with few samples. The method includes acquiring original remote sensing image data samples and constructing a style feature library; generating enhanced samples using a conditional diffusion generation model; performing quality screening on the enhanced samples using a preset screening strategy to obtain high-quality style-enhanced samples; constructing a contrastive learning network and establishing global and local contrastive learning tasks; constructing a change detection model and locating difficult regions, generating a comprehensive difficult example mask; generating difficult example enhanced samples for difficult regions based on the conditional diffusion generation model; updating the contrastive learning network and the change detection network; and obtaining the updated comprehensive difficult example mask. This invention effectively expands the style distribution of the training data, improving the model's robustness to changes in imaging conditions; learning style-invariant feature representations with change discrimination capabilities without relying on a large number of annotations; and achieving collaborative optimization between the generation model and the detection model.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and computer vision technology, and in particular to a collaborative optimization method and system for detecting changes in remote sensing with few samples. Background Technology

[0002] Remote sensing image change detection refers to the analysis of remote sensing images of the same geographic area acquired at different times to automatically identify changes in land cover or land cover status. This technology has significant application value in areas such as urban expansion monitoring, land use change analysis, natural disaster assessment, and supervision of farmland conversion to non-agricultural uses.

[0003] In recent years, with the development of deep learning technology, remote sensing image change detection methods based on models such as convolutional neural networks and Transformers have achieved good results. Existing methods typically achieve automatic detection of changed regions by constructing a dual-temporal feature extraction network, a feature difference modeling module, and a change discrimination module. However, in practical applications, existing technologies still have the following shortcomings.

[0004] First, change detection tasks typically rely on pixel-level labeled data for supervised training. However, obtaining high-quality pixel-level labels is costly and time-consuming, resulting in a limited number of labeled samples actually available for training. Under small sample conditions, existing deep learning models often struggle to achieve sufficient training, easily leading to overfitting, which in turn affects the model's generalization ability and detection accuracy.

[0005] Secondly, remote sensing images are easily affected by factors such as lighting conditions, atmospheric conditions, sensor parameters, and seasonal changes during the acquisition process at different times, resulting in significant style differences or distribution shifts between two-temporal images. Existing methods generally achieve good results when the distribution of training and test data is relatively consistent, but when the imaging conditions in the test scene vary greatly, the model performance often deteriorates, and the domain generalization ability remains insufficient.

[0006] Furthermore, to improve model robustness, existing techniques typically employ conventional data augmentation methods such as geometric transformations and color perturbations, or use generative models for sample amplification. However, conventional augmentation methods have limited ability to simulate real-world imaging differences and struggle to adequately characterize shifts in complex domains. While generative model-based augmentation methods can expand the sample distribution, in practical applications, if the generated samples are too similar to the original samples, their effect on improving generalization ability is limited; if the generated samples are too offset, they may deviate from the real data distribution, affecting model training stability. Moreover, existing generative methods based on GANs or style transfer, while maintaining spatial structural consistency, often struggle to achieve fine-grained style control.

[0007] Furthermore, change detection tasks commonly encounter challenging samples such as those with blurred boundaries, small targets, and complex textures. Existing methods lack effective mechanisms for mining and utilizing these challenging samples or regions, making it difficult to conduct targeted reinforcement training. At the same time, existing data augmentation methods typically process the entire image or large regions, lacking the ability to finely control the space for challenging regions, thus limiting their ability to improve the performance of complex region detection. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a collaborative optimization method and system for detecting changes in remote sensing with few samples, in order to address the shortcomings of the prior art.

[0009] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A collaborative optimization method for remote sensing change detection with few samples, comprising the following steps:

[0010] We acquire raw remote sensing image data samples and construct a style feature library. Using style vectors as style conditions and edge structure maps as structure conditions, we use a conditional diffusion generation model to generate enhanced samples that maintain spatial structure consistency but have different styles.

[0011] The generated enhanced samples are subjected to quality screening using a preset screening strategy to obtain high-quality style-enhanced samples.

[0012] A contrastive learning network is constructed, and global and local contrastive learning tasks are built using original remote sensing image data samples and selected high-quality style-enhanced samples.

[0013] A change detection model is constructed, and the model is trained using labeled samples. Difficult regions are located based on the change probability and confidence information output by the model, and a comprehensive hard case mask is generated.

[0014] The comprehensive difficult example mask is used as a spatial condition and is input into the conditional diffusion generation model along with the style and structure conditions to generate enhanced difficult example samples for difficult regions. The enhanced difficult example samples are then subjected to quality screening, and the contrastive learning network and change detection network are updated to obtain the updated comprehensive difficult example mask, thereby completing the closed-loop iterative optimization.

[0015] Based on the above technical solution, the present invention can be further improved as follows:

[0016] Further: The steps of acquiring original remote sensing image data samples and constructing a style feature library, and using a conditional diffusion generation model to generate enhanced samples that maintain spatial structure consistency but have different styles, specifically include the following steps:

[0017] Obtain raw remote sensing image data samples containing images of the target domain and multiple adjacent domains, and extract style feature vectors for each image, and construct a style feature library based on the style feature vectors;

[0018] Candidate style vectors are selected from the style feature library whose distance from the target domain style is within a preset style deviation distance range;

[0019] A sampling distribution based on temperature coefficient control is used to sample from the candidate style vector, so that the sampled target style vector is similar to but not the same as the target domain style;

[0020] Given an input image and a sampled target style vector, an enhanced sample with consistent spatial structure but style transfer is generated based on a pre-defined conditional control network as a conditional diffusion generation model.

[0021] Further: For a given input image and a sampled target style vector, generating enhanced samples with consistent spatial structure but undergoing style transfer based on a preset conditional control network as a conditional diffusion generation model specifically includes the following steps:

[0022] Edge maps of the input image are extracted using an edge detection operator, and the edge maps are encoded into multi-scale structural conditional features using an encoder.

[0023] The sampled target style vector is mapped to style condition features through a style encoder. Low-level style features are injected into the shallow network of the style encoder using a feature linear modulation mechanism, and high-level style features are injected into the deep network generated by conditional diffusion using a cross-attention mechanism. The low-level and high-level style features are style conditions obtained by mapping the sampled target style vector to the shallow and deep network layers through the style encoder.

[0024] The structural and style conditional features are received using a conditional diffusion generation model, and enhanced samples with consistent spatial structure but style transfer are generated through iterative denoising.

[0025] Further: The step of using a conditional diffusion generation model to receive the structural conditional features and style conditional features, and generating enhanced samples with consistent spatial structure but style transfer through iterative denoising, specifically includes the following steps:

[0026] Forward diffusion: Gaussian noise is gradually added to the original remote sensing image data samples to obtain noise samples that approximately follow a Gaussian distribution;

[0027] Conditional reverse denoising: Based on the style condition features and structural condition features, the noise samples are input into the conditional denoising network and reverse denoising is performed step by step. The reverse denoising probability distribution of the conditional denoising network under the style condition features and structural condition features is calculated.

[0028] Inference generation: Based on the inverse denoising probability distribution, iterative sampling is performed from standard Gaussian noise to obtain enhanced samples with consistent spatial structure but style transfer.

[0029] Further: The step of using a preset screening strategy to perform quality screening on the generated enhanced samples to obtain high-quality style-enhanced samples specifically includes the following steps:

[0030] Calculate the edge structure similarity between the enhanced sample and the original remote sensing image data sample, and filter out enhanced samples whose structural deviation exceeds the structural consistency threshold;

[0031] Calculate the style deviation distance between the style features of the enhanced sample and the target style distribution, and filter out the enhanced samples whose style deviation distance exceeds the style consistency threshold;

[0032] The semantic segmentation results of the enhanced samples and the original remote sensing image data samples are obtained by using a pre-trained semantic segmentation model. Enhanced samples with semantic consistency lower than a preset semantic consistency threshold are filtered out to obtain high-quality style-enhanced samples.

[0033] Further: The construction of the contrastive learning network, which utilizes the original remote sensing image data samples and the selected high-quality style-enhanced samples to construct global and local contrastive learning tasks, specifically includes:

[0034] Constructing a global contrastive learning task:

[0035] Positive sample pairs are constructed by selecting original remote sensing image data samples and their corresponding high-quality style-enhanced samples;

[0036] The high-quality style-enhanced samples corresponding to the remaining original remote sensing image data samples in the same batch of training, excluding those used to construct the positive sample pairs, are used to construct negative samples.

[0037] The global encoder is used to extract features from the original remote sensing image data samples and their corresponding high-quality style-enhanced samples. The global contrast loss in the entire training batch is calculated. The feature distance between positive sample pairs and the feature distance between negative samples are minimized and the global contrast loss function is maximized to learn image-level global feature representations that are robust to imaging conditions.

[0038] Constructing a local contrastive learning task:

[0039] Obtain the true binary change mask corresponding to the dual-temporal image pair in the original remote sensing image data sample, and construct positive sample pairs based on the features of the corresponding positions across time within the region where the true binary change mask remains unchanged.

[0040] Negative sample pairs are constructed based on the features of the corresponding positions across time within the region where the true binary change mask remains unchanged.

[0041] The local feature encoder is used to extract features from the two-temporal image pairs, calculate the local contrast loss in the entire training batch, and minimize the local contrast loss function so that the local feature encoder can learn local feature representations that have both the ability to distinguish changed regions and are robust to style perturbations.

[0042] Joint comparative learning training:

[0043] The global contrast loss and local contrast loss are used to construct a joint contrast learning total loss. The two temporal image pairs and their corresponding high-quality style enhancement samples are input into the global feature encoder and the local feature encoder, respectively. The global contrast learning loss and local contrast learning loss are calculated, and the joint contrast learning total loss is obtained.

[0044] Using the total loss of the joint contrastive learning as the optimization objective, the gradient backpropagation algorithm is used to jointly update the network parameters of the global feature encoder and the local feature encoder. The gradient backpropagation algorithm is used to iteratively adjust the model parameters according to the gradient information of the network parameters based on the loss function, so as to minimize the total loss of the joint contrastive learning.

[0045] Further: The construction of the change detection model, training the change detection model using labeled samples, locating difficult regions based on the change probability and confidence information output by the model, and generating a comprehensive hard case mask specifically includes the following steps:

[0046] A change detection model based on a twin encoder-decoder structure is constructed, and the network parameters of the global feature encoder and the local feature encoder are loaded.

[0047] The change detection model is trained by inputting a small number of real labeled samples, and a predicted change probability map is output. A change detection loss function is constructed based on the predicted change probability map and the real change mask, and the total change detection loss is calculated.

[0048] Based on the total change detection loss, gradient backpropagation and iterative updates are performed on the parameters of the change detection model to minimize the total change detection loss, thereby completing the training of the change detection model.

[0049] The trained change detection model is used to predict the dual-temporal image pair to obtain a change probability map. Based on the change probability map and the true binary change mask, a comprehensive hard case mask is constructed from two aspects: low confidence region and mispredicted region.

[0050] Further: The construction of a comprehensive hard example mask based on the change probability map and the true binary change mask, from the perspectives of low-confidence regions and mispredicted regions, specifically includes the following steps:

[0051] Read the pixel locations in the probability map where the predicted probability is close to the decision boundary and identify them as low-confidence hard example regions;

[0052] Based on the aforementioned change probability map and the true binary change mask, the error hard case region is determined;

[0053] The low-confidence hard example region and the mispredicted hard example region are fused to generate a comprehensive hard example mask.

[0054] Further: The step of using the comprehensive hard example mask as a spatial condition, along with the style and structural conditions, as input into the conditional diffusion generation model to generate hard example enhancement samples for difficult regions, performing quality screening on the hard example enhancement samples, updating the contrastive learning network and change detection network, and obtaining the updated hard example mask to complete the closed-loop iterative optimization specifically includes the following steps:

[0055] The comprehensive hard example mask is fed back to the conditional diffusion generation model, and the hard example mask is encoded by the spatial conditional encoder to obtain spatial conditional features;

[0056] The spatial condition features, style condition features, and structural condition features are collectively input into the conditional diffusion generation model;

[0057] A spatial adaptive guidance mechanism is adopted to adaptively adjust the guidance intensity according to the spatial distribution of the synthetic hard case mask, and generate hard case enhancement samples in the synthetic hard case region.

[0058] The contrastive learning network is further trained using the hard example augmentation samples, and the global feature encoder and local feature encoder in the contrastive learning network are updated.

[0059] The change detection model is trained using the updated global feature encoder and local feature encoder to obtain an updated change detection model. Based on the updated change detection model, predictions are made for dual-temporal image pairs, and a new comprehensive hard example mask is generated, thereby completing closed-loop iterative optimization.

[0060] Furthermore: the spatial adaptive guidance mechanism includes:

[0061] The integrated hard example mask overspace conditional encoder is encoded into multi-scale conditional features that match the spatial dimensions of each layer of the conditional denoising network.

[0062] Based on the classifier-independent guidance framework, the guidance strength value of each spatial location is determined according to the comprehensive hard example mask, wherein the guidance strength value of the hard example region is greater than the guidance strength value of the non-hard example region;

[0063] Conditional guidance is applied based on the guidance intensity value at each spatial location to achieve pixel-level control over the generation difficulty.

[0064] The present invention also provides a collaborative optimization system for remote sensing change detection with few samples, including a conditional diffusion generation module, a sample quality screening module, a contrastive learning module, a change detection module, and a closed-loop optimization module;

[0065] The conditional diffusion generation module is used to acquire raw remote sensing image data samples and build a style feature library. It uses style vectors as style conditions and edge structure maps as structure conditions to generate enhanced samples that maintain spatial structure consistency but have different styles using the conditional diffusion generation model.

[0066] The sample quality screening module is used to screen the generated enhanced samples using a preset screening strategy to obtain high-quality style enhancement samples.

[0067] The contrastive learning module is used to construct a contrastive learning network, which uses raw remote sensing image data samples and selected high-quality style-enhanced samples to construct global contrastive learning tasks and local contrastive learning tasks.

[0068] The change detection module is used to build a change detection model, train the change detection model using labeled samples, locate difficult regions based on the change probability and confidence information output by the model, and generate a comprehensive hard case mask.

[0069] The closed-loop optimization module takes the comprehensive hard example mask as a spatial condition and inputs it, along with the style condition and structural condition, into the conditional diffusion generation model to generate hard example enhancement samples for difficult regions. The module then performs quality screening on the hard example enhancement samples and updates the contrastive learning network and change detection network to obtain the updated comprehensive hard example mask, thus completing the closed-loop iterative optimization.

[0070] The beneficial effects of this invention are as follows: This invention effectively expands the style distribution of training data and improves the model's robustness to changes in imaging conditions by employing a style distance-driven "near-domain but not co-domain" sample generation strategy; it avoids noise accumulation and confirmation bias by screening generated samples through multi-dimensional quality constraints; it learns style-invariant and change-discriminating feature representations without relying on a large number of annotations through a global and local contrastive learning mechanism; and it achieves collaborative optimization of the generation and detection models through a difficult-example-driven closed-loop feedback mechanism, specifically improving the model's detection capability in difficult regions. In summary, this invention significantly improves the generalization ability and detection accuracy of change detection under conditions of few samples. Attached Figure Description

[0071] Figure 1 This is a flowchart illustrating a collaborative optimization method for few-sample remote sensing change detection according to an embodiment of the present invention.

[0072] Figure 2 This is a schematic diagram of the structure of a collaborative optimization system for detecting changes in remote sensing with few samples, according to an embodiment of the present invention. Detailed Implementation

[0073] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0074] like Figure 1 As shown, a collaborative optimization method for remote sensing change detection with few samples includes the following steps:

[0075] S1: Obtain the original remote sensing image data samples and construct a style feature library. Use style vectors as style conditions and edge structure maps as structure conditions. Use the conditional diffusion generation model to generate enhanced samples that maintain spatial structure consistency but have different styles.

[0076] S2: Use a preset screening strategy to perform quality screening on the generated enhanced samples to obtain high-quality style-enhanced samples;

[0077] S3: Construct a contrastive learning network, using original remote sensing image data samples and selected high-quality style-enhanced samples to construct global and local contrastive learning tasks;

[0078] S4: Construct a change detection model, train the change detection model using labeled samples, locate difficult regions based on the change probability and confidence information output by the model, and generate a comprehensive hard case mask;

[0079] S5: The comprehensive hard example mask is used as a spatial condition and input into the conditional diffusion generation model along with the style condition and structural condition to generate hard example enhancement samples for difficult regions. The hard example enhancement samples are quality-screened and the contrastive learning network and change detection network are updated to obtain the updated comprehensive hard example mask, thereby completing the closed-loop iterative optimization.

[0080] In one or more embodiments of the present invention, step S1, which involves acquiring original remote sensing image data samples and constructing a style feature library, and using a conditional diffusion generation model to generate enhanced samples that maintain spatial structure consistency but have different styles, specifically includes the following steps:

[0081] S11: Obtain raw remote sensing image data samples containing images of the target domain and multiple adjacent domains, and extract style feature vectors from each image. Construct a style feature library based on the style feature vectors. , represented as:

[0082] ;

[0083] in, Represents a style feature library; Indicates the first The style feature vector corresponding to each image; Indicates image index; This represents the total number of images involved in building the style feature library.

[0084] In embodiments of the present invention, intermediate layer feature statistics of a pre-trained VGG-19 network are used as style representations. Specifically, an image is input into a pre-trained VGG-19 network, feature maps of multiple preset intermediate layers are extracted, the channel mean and channel standard deviation of each layer feature map are calculated, and the statistics of each layer are concatenated in a preset order to obtain the style feature vector of the corresponding image. The VGG-19 network is used to extract multi-layer texture and structural representation information of the image; the channel mean is used to represent the average distribution level of the feature response; the channel standard deviation is used to represent the dispersion of the feature response; the multiple preset intermediate layers are preferably relu1_1, relu2_1, relu3_1 and relu4_1 layers, but are not limited to these.

[0085] The style feature library constructed using the above method It is used for target domain style center calculation, candidate style vector selection and style distance measurement in subsequent steps, thereby providing style prior for subsequent generation of style enhancement samples based on conditional diffusion model.

[0086] To generate enhanced samples that are close to but not exactly the same as the style of the target domain, it is necessary to perform distance constraint filtering on the style vectors in the style feature library constructed in step S11.

[0087] S12: Select candidate style vectors from the style feature library whose distance from the target domain style is within a preset style deviation distance range;

[0088] First, define the style distance metric function as follows:

[0089]

[0090] in, Represents style vector With style vectors The stylistic distance between two images is used to measure the degree of difference in stylistic representation between them. and These represent the first in the style feature library. The and the first One style vector; and These represent style vectors respectively. and The corresponding mean statistic; and These represent style vectors respectively. and The corresponding standard deviation statistic; This represents the L2 norm, used to measure the Euclidean distance between two statistical vectors.

[0091] Based on the style feature vector corresponding to the target domain image, calculate the average style vector of the target domain. ,in, It also represents the style center of the target domain, for any style vector in the style feature library. Calculate its relationship with the average style vector. style distance And set the style distance threshold range Filter candidate style vectors that satisfy the following formula:

[0092]

[0093] in, This represents the style vector to be filtered. This represents the lower bound of the style distance, used to ensure that the style vectors obtained through filtering have a certain difference from the target domain; This indicates the upper limit of style distance, used to ensure that the selected style vectors do not deviate too far from the target domain.

[0094] Through the above screening, a set of candidate style vectors is obtained, which is then used for probability sampling in step S13. In this embodiment, The preferred value is 0.1. The preferred value is 0.5.

[0095] S13: A sampling distribution based on temperature coefficient control is used to sample from the candidate style vector, such that the sampled target style vector... Similar in style to the target domain but not the same;

[0096] To control the concentration of the sampling distribution and adjust the style diversity of the generated samples, a temperature coefficient is introduced. For each candidate style vector Calculate its sampling probability , represented as:

[0097]

[0098] in, Represents candidate style vectors The probability of being sampled; Represents the first in the set of candidate style vectors One style vector; This represents the average style vector of the target domain; Represents candidate style vectors With the target domain average style vector Style distance between; This represents the temperature coefficient, used to control the smoothness of the sampling distribution; ∑k represents the exponential function; ∑k represents the summation of the unnormalized weights corresponding to all candidate style vectors in the candidate style vector set; This represents the k-th candidate style vector during the summation process.

[0099] Temperature coefficient The larger the temperature coefficient, the smaller the difference in sampling probabilities among candidate style vectors, and the more uniform the sampling distribution; The smaller the value, the more concentrated the sampling distribution is on candidate style vectors that are closer to the target domain, thus making the style of the generated samples closer to the target domain. In this embodiment, The preferred value is 0.5. Based on the sampling probability... Style vectors are obtained by randomly sampling from the candidate style vector set. The style vector Used for style condition coding in step S14.

[0100] S14: Given an input image I and a sampled target style vector Based on a pre-defined conditional control network, an enhanced sample is generated as a conditional diffusion generation model that maintains the spatial structure but undergoes style transfer.

[0101] This embodiment uses a diffusion model based on a conditional control network as the generation model. The conditional control network preferably uses the ControlNet architecture, but it is not limited to this; other conditional injection network architectures with similar functions are also applicable. Given the input image I and the target style vector obtained from sampling in step S13... By extracting structural conditions and encoding style conditions, structural and style information are jointly injected into the diffusion model to generate enhanced samples that maintain spatial structure consistency and undergo style transfer.

[0102] In one or more embodiments of the present invention, in step S14, the given input image I and the sampled target style vector... The process of generating augmented samples with consistent spatial structure but style transfer based on a pre-defined conditional control network as a conditional diffusion generation model specifically includes the following steps:

[0103] S141: Use an edge detection operator to extract the edge map of the input image, and use an encoder to encode the edge map into multi-scale structural conditional features;

[0104] For input images Perform edge detection and extract its edge map. , represented as:

[0105]

[0106] in, Indicates the input image; Indicates input image The corresponding edge map is used to characterize the spatial structure information in the image; express Edge detection operator. In this embodiment, it is preferred to use... Edge detection operators extract edge maps, but are not limited to this; other edge detection methods such as Sobel, Laplacian, and HED (Holistically-Nested Edge Detection) can also be used.

[0107] edge map The encoder of the input conditional control network encodes multi-scale structural conditional features. .in, Represents the edge graph The extracted structural condition features are used to constrain the spatial structure of the output sample to remain consistent with the input image during the diffusion generation process. In this embodiment, the conditional control network preferably adopts an encoder structure corresponding to the main denoising network U-Net, and is connected to the main denoising network through a zero-convolutional layer to incorporate the structural condition features. The denoising process of the injection diffusion model.

[0108] S142: The sampled target style vector is mapped to style condition features through a style encoder, and low-level style features are injected into the shallow network generated by the conditional diffusion using a feature linear modulation mechanism. High-level style features are injected into the deep network of the style encoder using a cross-attention mechanism. The low-level style features and high-level style features are different forms of style conditions obtained by mapping the sampled target style vector through the style encoder in the shallow network layer and the deep network layer.

[0109] The style encoder employs a multilayer perceptron (MLP) structure, which includes multiple fully connected layers and nonlinear activation functions, and outputs modulation parameters that match the layers of the denoising network.

[0110] Here, the target style vector obtained from step S13 is... Input style encoder, mapped to style conditional features This is used to control the style transfer direction of the output samples during the diffusion generation process. The style encoder preferably adopts a multilayer perceptron (MLP) structure, containing multiple fully connected layers and nonlinear activation functions to output style modulation parameters that match different levels of the denoising network.

[0111] In this embodiment, a hierarchical conditional injection approach is adopted, injecting low-level style features into the shallow network of the style encoder and high-level style features into the deep network. Both the low-level and high-level style features originate from the target style vector sampled in step S13. The style encoder maps the data into different forms of style conditional representations at different network layers. The shallow mapping results mainly represent low-level style features such as hue, brightness, and contrast, while the deep mapping results mainly represent high-level style features such as texture patterns and semantic style.

[0112] In shallow networks, the feature map is modulated channel by channel using a feature linear modulation mechanism, as follows:

[0113]

[0114] in, This represents the input feature map of the shallow layer of the denoising network; This represents the output feature map after style modulation; Represented by style vectors Scaling factor obtained through style encoder mapping; Represented by style vectors The offset factor obtained through style encoder mapping. This is achieved through the scaling factor. and offset factor Affine transformation is performed on the shallow feature map to inject low-level style features.

[0115] In deep networks, high-level style features are injected through a cross-attention mechanism. The style vectors are then... Linear mapping to the key matrix Sum matrix , represented as:

[0116]

[0117] Deep features of the denoising network Linear mapping to query matrix , represented as:

[0118]

[0119] in, Represented by style vectors The key matrix obtained by mapping; Represented by style vectors The value matrix obtained by mapping; Indicates the characteristics of the denoising network The query matrix obtained through mapping; , and These represent the key mapping matrix, value mapping matrix, and query mapping matrix, respectively.

[0120] Based on the query matrix Key matrix Sum matrix Calculate the cross-attention output, expressed as:

[0121]

[0122] in, Indicates cross-attention output; Key matrix Transpose of; This represents the dimension of the key vector, used to scale the dot product result; This represents the normalization function. Through the aforementioned cross-attention calculation, the style vector... The high-level style features represented are injected into the deep features of the denoising network.

[0123] Through the above style condition encoding, style condition features used to control the diffusion generation process are obtained. and in subsequent steps, structural condition features Co-injection conditional diffusion generation model.

[0124] S143: Utilize the conditional diffusion generation model to receive the structural conditional features and style conditional features, and generate enhanced samples with consistent spatial structure but style transfer through iterative denoising.

[0125] The conditional diffusion model includes a forward diffusion process and a conditional backward denoising process. The forward diffusion process is used to progressively add Gaussian noise to the original samples during the training phase, while the conditional backward denoising process is used to progressively remove noise and recover the target samples under the joint constraints of structural conditional features and style conditional features.

[0126] In one or more embodiments of the present invention, step S143, which involves using a conditional diffusion generation model to receive the structural conditional features and style conditional features, and generating enhanced samples with consistent spatial structure but undergoing style transfer through iterative denoising, specifically includes the following steps:

[0127] S1431: Forward diffusion: Gaussian noise is gradually added to the original remote sensing image data samples to obtain noise samples that approximately follow a Gaussian distribution;

[0128] During the forward diffusion process, Gaussian noise is gradually added to the original sample until the preset total number of diffusion steps is reached. End. Let the original sample be... , No. The sample after adding noise is The forward diffusion process can be represented as:

[0129]

[0130] in, Indicates the original sample Generate the first Step noise samples The forward diffusion probability distribution; Represents the original sample; Indicates the first The sample after adding noise; Indicates a Gaussian distribution; Indicates the first The cumulative noise scheduling coefficient of the step; Represents the identity matrix; This represents the diffusion step index; T represents the preset total diffusion steps.

[0131] The cumulative noise scheduling coefficient Noise scheduling sequence Determined, satisfies:

[0132]

[0133] in, Indicates the first The noise intensity of the step; Indicates the first The noise retention factor of the step; This represents a series of multiplication operations; This represents the step index in the chain multiplication process. In this embodiment, a linear noise scheduling strategy is preferably used, and .

[0134] As the diffusion steps As the noise level increases, the noise in the sample gradually increases, affecting the original sample. The structural and textural information contained within is gradually submerged by noise. When the diffusion process reaches the [missing information]... During the step, the final noise sample is obtained. The noise sample It approximately follows a standard Gaussian distribution. This is used as the initial input for the subsequent conditional reverse denoising generation process.

[0135] S1432: Conditional inverse denoising: Based on the style condition features and structural condition features, the noise sample is input into the conditional denoising network and inverse denoising is performed step by step. The inverse denoising probability distribution of the conditional denoising network under the style condition features and structural condition features is calculated.

[0136] In the conditional inverse denoising process, the conditional diffusion generative model learns the conditional probability distribution. To determine the current noise sample Gradually restore the samples from the previous moment , represented as:

[0137]

[0138] in, The parameter is Conditional denoising networks under conditions The inverse denoising probability distribution under; Indicates the inverse denoising first -1 step samples; This represents the mean of the predictions made by the conditional denoising network. Indicates the first Step-predefined standard deviation; This represents the joint condition information, where, This represents the style condition feature obtained in step S142. This represents the structural condition features obtained in step S141.

[0139] S1433: Inference generation: Based on the inverse denoising probability distribution, iterative sampling is performed from standard Gaussian noise to obtain enhanced samples with consistent spatial structure but style transfer.

[0140] During the inference phase, based on the conditional inverse denoising process, standard Gaussian noise is removed. Starting with iterative sampling, the target samples are gradually obtained. The sampling update can be represented as:

[0141]

[0142] in, Indicates the first Step noise retention factor; Represents the cumulative noise dispatch coefficient; This represents the noise term predicted by the conditional denoising network. This represents random noise.

[0143] To improve generation efficiency, this embodiment preferably employs an accelerated sampling strategy for iterative sampling, such as DDIM, DPM-Solver, or UniPC. Taking DDIM as an example, it achieves rapid generation by reducing the number of sampling steps. In this embodiment, the number of denoising steps is preferably set to 50, but it is not limited to this.

[0144] To enhance the effect of conditional control, a classifier-independent guidance strategy is further preferred during the iterative sampling process. Specifically, during the training phase, the conditions are set to null conditions with a preset probability. This allows the model to learn both conditional and unconditional generation simultaneously; during the inference phase, the conditional and unconditional predictions are linearly combined to obtain the guided noise prediction.

[0145]

[0146] in, This indicates the noise prediction result after guidance; This represents the noise prediction results under empty conditions; Indicates condition Noise prediction results; This represents the guiding strength parameter. In this embodiment, The preferred value is 7.5. The guided noise prediction result. Used to replace the noise prediction term in the standard sampling update, thereby enhancing the control of style and structure conditions over the generated results.

[0147] Enhanced samples are obtained through the above conditional diffusion iteration. .in, This indicates the generated enhanced sample, which retains the same appearance as the original image. Consistent spatial structure, and has a sampling style vector The controlled imaging style features. The enhanced sample Used for quality screening of generated samples in subsequent step S2.

[0148] To avoid low-quality generated samples interfering with subsequent model training, the enhanced samples generated in step S14 are subjected to multi-dimensional quality screening. Constraints are imposed on the generated samples from three aspects: structural consistency, style consistency, and semantic consistency, in order to filter out low-quality samples with structural distortion, excessive style shift, or significant changes in semantic category.

[0149] In one or more embodiments of the present invention, step S2, which involves using a preset screening strategy to perform quality screening on the generated enhanced samples to obtain high-quality style-enhanced samples, specifically includes the following steps:

[0150] S21: Calculate the edge structure similarity between the enhanced sample and the original remote sensing image data sample, and filter out the enhanced samples whose structural deviation exceeds the structural consistency threshold;

[0151] First, the original remote sensing image data samples were processed separately. and enhanced samples Perform edge detection to obtain the corresponding edge map. and .in, This represents a sample of the original remote sensing image data. Indicates augmented samples; Represents the original remote sensing image data sample The set of edge pixels corresponding to the edge map; Indicates augmented samples The edge map corresponds to the set of edge pixels. In this embodiment, the Canny edge detection operator is preferably used to extract the edge map.

[0152] Based on the edge pixel set and Calculate the edge intersection-union ratio , represented as:

[0153]

[0154] in, This represents the cross-union ratio (CUI) of the original remote sensing image data sample and the enhanced sample in terms of edge structure. This represents the intersection area between the edges of the original remote sensing image data samples and the edges of the enhanced samples; This represents the union region of the edges of the original remote sensing image data samples and the edges of the enhanced samples; This indicates the number of pixels in the set.

[0155] like If the structural deviation between the enhanced sample and the original remote sensing image data sample is too large, it will be rejected. This represents the structural consistency screening threshold. In this embodiment, The preferred value is 0.7.

[0156] S22: Calculate the style deviation distance between the style features of the enhanced sample and the target style distribution, and filter out the enhanced samples whose style deviation distance exceeds the style consistency threshold;

[0157] Verify whether the style features of the enhanced samples fall within the expected style range. First, extract the enhanced samples. Style feature vectors And calculate its relationship with the target style vector obtained by sampling in step S13. Style distance between , represented as:

[0158]

[0159] in, This represents the distance between the enhanced sample style and the target sample style; Indicates augmented samples The corresponding style feature vector; This represents the target style vector obtained by sampling in step S13; The style distance metric function is represented, preferably using the style distance metric method defined in step S12.

[0160] like If the generated sample fails to achieve the desired style transfer effect, it will be rejected. This represents the style consistency filtering threshold. In this embodiment, The preferred value is 0.2.

[0161] S23: Use a pre-trained semantic segmentation model to obtain the semantic segmentation results of the enhanced samples and the original remote sensing image data samples, and filter out the enhanced samples whose semantic consistency is lower than the preset semantic consistency threshold to obtain high-quality style enhanced samples.

[0162] Using a pre-trained semantic segmentation model, the original remote sensing image data samples were processed. and enhanced samples Perform semantic segmentation to obtain the corresponding semantic segmentation results. and .in, Represents the original remote sensing image data sample The semantic segmentation results; Indicates augmented samples The semantic segmentation results.

[0163] Based on the semantic segmentation results and Calculate the semantic intersection-union ratio , represented as:

[0164]

[0165] in, This represents the intersection-union ratio (IoU) of the original remote sensing image data sample and the enhanced sample in the semantic region. This represents the intersection of the semantic regions of the original remote sensing image data sample and the enhanced sample; This represents the union of the semantic regions of the original remote sensing image data sample and the enhanced sample; This indicates the number of pixels in the region.

[0166] like If the semantic category of the enhanced sample changes significantly, it will be rejected. This represents the semantic consistency filtering threshold. In this embodiment, The preferred value is 0.85.

[0167] The enhanced samples generated in step S14 are screened for quality through the above structural consistency constraints, style consistency constraints, and semantic consistency constraints. High-quality enhanced samples are retained for subsequent contrastive learning task construction and feature representation learning.

[0168] The high-quality enhanced samples obtained in step S2 are not directly used as supervised training samples for the change detection model. Instead, they are used to construct a contrastive learning task to learn feature representations that are robust to style changes and capable of discriminating changes. The contrastive learning task includes two levels: global contrastive learning and local contrastive learning. Global contrastive learning is used to learn image-level robust feature representations, while local contrastive learning is used to learn region-level change-sensitive feature representations.

[0169] In one or more embodiments of the present invention, step S3, which involves constructing a contrastive learning network and using the original remote sensing image data samples and the selected style-enhanced samples to construct a global contrastive learning task and a local contrastive learning task, specifically includes:

[0170] Constructing a global contrastive learning task:

[0171] S31a: Select original remote sensing image data samples and their corresponding style-enhanced samples to construct positive sample pairs;

[0172] S32a: Construct negative samples from the style-enhanced samples corresponding to the remaining original remote sensing image data samples in the same batch of training, excluding those used to construct the positive sample pairs.

[0173] S33a: Use a global encoder to extract features from the original remote sensing image data samples and their corresponding style-enhanced samples, calculate the global contrast loss in the entire training batch, and learn an image-level global feature representation that is robust to imaging conditions by minimizing the feature distance between positive sample pairs and maximizing the feature distance between negative samples through the global contrast loss function.

[0174] Specifically, let the global feature encoder be... Then the first The global feature representations of the original image and its style-enhanced view are as follows:

[0175]

[0176] in, Indicates the first One sample of original remote sensing image data; Indicates a sample of the original remote sensing image data. Corresponding augmented samples; Represents a global feature encoder; Represents raw remote sensing image data samples Global feature representation; Indicates augmented samples The global feature representation.

[0177] Let similarity function The cosine similarity function is used. Let be the temperature coefficient, then the first The global contrastive loss for each sample is defined as:

[0178]

[0179] in, Indicates the first Global contrast loss for each sample; Indicates positive sample pairs Similarity; Represents raw remote sensing image data samples In the same training batch, the first One enhanced sample Feature similarity between them; Indicates the number of samples in the same training batch; Used to adjust the smoothness of the similarity distribution.

[0180] The global contrastive loss is obtained by averaging the sample losses across the entire training batch.

[0181]

[0182] in, This represents the global contrastive loss for the entire training batch. During the global contrastive learning process, and Forming positive sample pairs and This forms a negative sample pair.

[0183] By minimizing the global contrast loss This enables the global feature encoder to learn image-level representations that are insensitive to style changes, providing a robust global feature foundation for subsequent local contrastive learning and change detection feature modeling.

[0184] Constructing a local contrastive learning task:

[0185] S31b: Obtain the true binary change mask corresponding to the dual-temporal image pair in the original remote sensing image data sample, and construct positive sample pairs based on the features of the corresponding positions across time within the region where the true binary change mask remains unchanged.

[0186] Given a pair of temporal images and its corresponding true binary transformation mask ,in, Indicates phase Input image, Indicates phase Input image, Represents the true change mask for a two-phase image pair; when Time indicates spatial location It belongs to the region of change; when 0 indicates spatial location. It belongs to the unchanged region.

[0187] Let the local feature encoder be... The local feature maps corresponding to the two temporal images are represented as follows:

[0188] , .

[0189] in, Image Local feature map; Image Local feature map; These represent the spatial locations of the local feature maps. The eigenvector at that location.

[0190] Based on the changing mask Define the set of unchanged regions and the set of changed regions as follows:

[0191]

[0192] in, Represents the set of locations of unchanged regions; This represents the set of locations of the changing regions.

[0193] Let similarity function The cosine similarity function is expressed as:

[0194]

[0195] in, and This represents two feature vectors to be compared; Represents the dot product of vectors; The L2 norm of a vector is used to represent the vector's L2 norm.

[0196] For any spatial location in the unchanged region , As positive sample pairs, they are used to constrain the feature consistency of unchanged regions between two temporal phases. The local contrast loss of the unchanged region is defined as:

[0197]

[0198] in, This indicates the local contrast loss in the unchanged region; This represents the number of locations in the set of locations within the unchanged region; The temperature coefficient represents the local contrastive learning. Indicates position The corresponding normalization term is defined as:

[0199]

[0200] in, Indicates spatial location The set of neighborhood locations in the local feature map; This represents the position index within the neighborhood. By minimizing... This improves the feature similarity of corresponding positions in unchanged regions between two temporal phases.

[0201] S32b: Construct negative sample pairs based on the features of corresponding positions across time within the region where the true binary change mask remains unchanged;

[0202] For any spatial location in the changing region As negative sample pairs, they facilitate feature separation of the changed regions between the two temporal phases. The contrastive loss of the changed regions is defined as:

[0203]

[0204] in, This indicates the local contrast loss in the area of ​​change; This represents the number of locations in the set of locations within the changing region. This is achieved by minimizing... This reduces the feature similarity of corresponding locations in the changed region between two temporal phases, thereby enhancing the separability between the changed and unchanged regions.

[0205] S33b: The local feature encoder is used to extract features from the two-phase image pairs, calculate the local contrast loss in the entire training batch, and minimize the local contrast loss function so that the local feature encoder can learn a local feature representation that has both the ability to distinguish changed regions and is robust to style perturbations.

[0206] Constrained by both unchanged and changed regions, the local contrast loss is defined as:

[0207]

[0208] in, Indicates local contrast loss; This represents the balance coefficient, used to adjust for losses in the variable region. The weight.

[0209] To enhance the style invariance of local features, the original remote sensing image data samples were processed. Generate corresponding augmented samples And extract its local feature maps:

[0210]

[0211] in, Represents raw remote sensing image data samples The corresponding enhanced view; Indicates augmented samples Local feature map.

[0212] set up Let represent the set of all spatial locations in the local feature map. Then, for any location... ,Will As positive sample pairs, construct the style consistency constraint loss:

[0213]

[0214] in, This represents the loss due to style consistency constraints. Indicates the total number of spatial locations; This represents the temperature coefficient corresponding to the style consistency constraint. Represents a spatial location index. By minimizing... This enhances the invariance of local features to style perturbations.

[0215] Finally, the total loss of local contrastive learning combined with style enhancement can be expressed as:

[0216]

[0217] in, This represents the total loss from local contrastive learning combined with style enhancement; Represents style consistency constraint loss The weighting coefficients.

[0218] By minimizing the total learning loss of the local contrastives This enables the local feature encoder to learn local feature representations that are both capable of distinguishing changed regions and robust to style perturbations. Together with the global feature representations obtained in step S31, these local feature representations form the feature representation basis for the subsequent change detection model.

[0219] Joint comparative learning training:

[0220] S34: Construct a joint contrast learning total loss from the global contrast loss and the local contrast loss. Input the dual temporal image pairs and their corresponding enhanced samples into the global feature encoder and the local feature encoder respectively, calculate the global contrast learning loss and the local contrast learning loss, and calculate the joint contrast learning total loss.

[0221] Based on the global contrast loss obtained in step S31 The total local contrastive learning loss obtained in step S32 Construct the joint contrastive learning total loss, denoted as:

[0222]

[0223] in, This represents the total loss in joint contrastive learning; This represents the global contrastive learning loss; This represents the total loss from local contrastive learning combined with style enhancement; and This represents the weighting coefficients, used to balance the contributions of global and local contrastive learning to feature representation learning. In this embodiment, The preferred value is 0.3. The preferred value is 0.7.

[0224] During the training process, samples of dual-temporal raw remote sensing image data were used. and and its corresponding high-quality style enhancement view and Input to the global feature encoder respectively and local feature encoder Calculate the global contrastive learning loss Local contrastive learning loss And further calculate the total loss of joint contrastive learning. .

[0225] S35: Using the total loss of the joint contrastive learning as the optimization objective, the gradient backpropagation algorithm is used to jointly update the network parameters of the global feature encoder and the local feature encoder. The gradient backpropagation algorithm is used to iteratively adjust the model parameters according to the gradient information of the network parameters based on the loss function, so as to minimize the total loss of the joint contrastive learning.

[0226] Through the above joint contrastive learning training, the global feature encoder learns image-level feature representations that are robust to changes in imaging style, and the local feature encoder learns location-level feature representations that combine style invariance and change discrimination capabilities, thereby obtaining a feature extraction model suitable for subsequent change detection tasks.

[0227] In one or more embodiments of the present invention, step S4, which involves constructing a change detection model, training the change detection model using labeled samples, locating difficult regions based on the change probability and confidence information output by the model, and generating a comprehensive hard case mask, specifically includes the following steps:

[0228] S41: Construct a change detection model based on a twin encoder-decoder structure, and load the network parameters of the global feature encoder and the local feature encoder;

[0229] Here, the change detection model adopts a twin encoder-decoder structure, using dual-temporal input image pairs ( , The twin encoder uses two parameter-shared feature extraction branches, one for each of the two temporal images. and Feature extraction is performed. The encoder section loads the feature extraction parameters obtained from the joint contrastive learning training in step S3, and uses them to extract multi-layer feature representations that possess both style robustness and change discrimination ability. During the change detection model training phase, the encoder parameters can be frozen to remain unchanged, or fine-tuned to participate in the supervised training update of subsequent change detection tasks. The decoder section adopts a multi-scale feature fusion structure, fusing features from different levels of the encoder output to jointly utilize high-level semantic information and low-level spatial detail information, gradually restoring the spatial resolution of the changed region, and outputting a change probability map. ,in, This represents the probability predicted by the change detection model that each spatial location belongs to a changed region.

[0230] By constructing the aforementioned twin encoder-decoder structure, the robust feature representation learned in step S3 can be transferred to the change detection task, providing a basic model structure for subsequent change detection model training and hard example localization.

[0231] S42: Train the change detection model by inputting a small number of real labeled samples, output a predicted change probability map, construct a change detection loss function based on the predicted change probability map and the real change mask, and calculate the total change detection loss.

[0232] Let the change detection model be Input dual-temporal image pairs ( , Output the probability map of change. , represented as:

[0233] .

[0234] in, This represents a change detection model; This represents the probability that the change detection network predicts each spatial location as belonging to a changed region.

[0235] set up For the true binary transformation mask, where, Indicates spatial location It belongs to the area of ​​change. Indicates spatial location This area remains unchanged. (Based on the predicted change probability map) Mask of Real Changes A change detection loss function is constructed. The change detection loss uses a weighted combination of binary cross-entropy loss and Dice loss as the total loss function, expressed as:

[0236]

[0237] in, This represents the total loss from change detection; This represents the binary cross-entropy loss; Indicates Dice loss; This represents the weighting coefficient, used to balance the contributions of the binary cross-entropy loss and the Dice loss to the total loss function.

[0238] The binary cross-entropy loss is defined as:

[0239]

[0240] in, Represents the set of all spatial locations; Indicates the total number of spatial locations; Indicates a spatial location index.

[0241] The Dice loss is defined as follows:

[0242]

[0243] in, Indicates a spatial location index; This represents a smoothing constant, used to prevent the denominator from being zero; This indicates a summation operation performed over all spatial locations.

[0244] S43: Based on the total change detection loss, perform gradient backpropagation and iterative updates on the parameters of the change detection model to minimize the total change detection loss, thereby completing the training of the change detection model;

[0245] During training, a small number of ground-labeled bi-temporal image pairs and their corresponding ground-change masks are input into the change detection model. The predicted change probability map is obtained through forward propagation. And calculate the total loss of change detection. Then, based on the changes, the total loss is detected. Change detection model The parameters are backpropagated and iteratively updated to minimize the total loss of change detection.

[0246] Through the above-mentioned supervised training with few samples, a post-trained change detection model suitable for the task of remote sensing change detection with few samples is obtained, which enables it to effectively predict the change area of ​​dual-temporal remote sensing images under the condition of limited labeled samples.

[0247] S44: The trained change detection model is used to predict the dual-temporal image pair to obtain a change probability map. Based on the change probability map and the true binary change mask, a comprehensive hard case mask is constructed from two aspects: low confidence region and mispredicted region.

[0248] In one or more embodiments of the present invention, step S44, which involves constructing a comprehensive hard example mask based on the change probability map and the true binary change mask from both the low-confidence region and the misprediction region, specifically includes the following steps:

[0249] S441: Read the pixel positions in the probability map where the predicted probability is close to the decision boundary and identify them as low-confidence hard example regions.

[0250] Let the low confidence mask be Its definition is:

[0251]

[0252] in, Indicates pixel position, and The low confidence threshold is preferably set as follows: .

[0253] S442: Determine the error hard case region based on the change probability map and the true binary change mask;

[0254] Transformation probability diagram The predicted change mask is obtained after threshold binarization. Its definition is

[0255]

[0256] in, The threshold value is 0.5, which is preferred for binarization.

[0257] Furthermore, the predicted change mask will be used. Mask of Real Changes By comparison, the misprediction mask is obtained. Its definition is:

[0258]

[0259] in, Indicates pixel position There are erroneous predictions at certain locations; the erroneous prediction areas include false positive areas and false negative areas.

[0260] S443: Merge the low-confidence hard example regions and the incorrectly predicted hard example regions to generate a comprehensive hard example mask:

[0261]

[0262] in, Indicates the complex case mask; This represents a pixel-wise logical OR operation, meaning that when a pixel location belongs to a low-confidence region or an incorrectly predicted region, the hard case mask at the corresponding location is set to 1, otherwise it is set to 0.

[0263] The comprehensive hard case mask This is used to characterize regions where the current change detection model has difficulty detecting, and serves as the spatial condition input for difficult-example-driven sample augmentation in subsequent generative models.

[0264] In one or more embodiments of the present invention, step S5, which involves inputting the synthesized hard example mask as a spatial condition, along with the style condition and structural condition, into the conditional diffusion generation model to generate hard example enhancement samples for difficult regions, performing quality screening on the hard example enhancement samples and updating the contrastive learning network and change detection network to obtain the updated hard example mask, and completing the closed-loop iterative optimization, specifically includes the following steps:

[0265] S51: Feed the comprehensive hard example mask back to the conditional diffusion generation model, and encode the hard example mask through the spatial conditional encoder to obtain spatial conditional features;

[0266] Spatial condition encoder denoted as It preferably employs a lightweight convolutional network structure, containing multiple convolutional layers and downsampling layers, to output multi-scale conditional features that match the spatial dimensions of each layer of the diffusion model's main denoising network, U-Net. The encoding process is expressed as follows:

[0267]

[0268] in, Indicates a spatial condition encoder. This represents the spatial conditional features obtained from the encoding.

[0269] In this embodiment, the spatial condition encoder preferably includes four convolutional blocks. Each convolutional block includes two... The system employs convolutional layers, batch normalization layers, and ReLU activation functions, and downsampling is achieved through convolutions with a stride of 2, thereby progressively extracting spatial conditional information at different scales. After encoding by the spatial conditional encoder, four spatial conditional feature maps at different scales are output, represented as follows:

[0270]

[0271] in, These represent spatial conditional feature maps at different scale levels, used for conditional injection of features at the corresponding level of the main denoising network U-Net of the diffusion model.

[0272] Through the aforementioned difficult example mask feedback and encoding, the difficult example region information identified by the change detection model is converted into spatial condition features, and targeted spatial constraints are applied to the difficult example region during the subsequent diffusion generation process to enhance the relevance and effectiveness of the generated samples in detecting difficult regions.

[0273] S52: The spatial condition features, style condition features, and structural condition features are jointly input into the conditional diffusion generation model, expressed as:

[0274]

[0275] in, Denotes the set of joint conditions; This represents the style condition feature obtained in step S142; This represents the structural condition features obtained in step S141; This represents the spatial condition features obtained in step S51.

[0276] Three conditions are injected in different ways into the main denoising network U-Net of the diffusion model: the structural condition features The style conditional features are injected into the encoder layer and intermediate layers of the U-Net through zero-convolutional layers of the conditional control network; Injection is performed in shallow layers using a feature linear modulation mechanism, and in deep layers using a cross-attention mechanism; the spatial conditional features Injection is performed element-wise by adding the corresponding scale feature map of U-Net to enhance spatial guidance in hard example regions.

[0277] S53: A spatial adaptive guidance mechanism is adopted to adaptively adjust the guidance intensity according to the spatial distribution of the synthetic hard case mask, and generate hard case enhancement samples in the synthetic hard case region;

[0278] In one or more embodiments of the present invention, the spatial adaptive guidance mechanism includes:

[0279] S531: Encode the integrated hard example mask overspace conditional encoder into a multi-scale conditional feature that matches the spatial size of each layer of the conditional denoising network;

[0280] S532: Based on the classifier-independent guidance framework, the guidance intensity value of each spatial location is determined according to the comprehensive hard example mask, wherein the guidance intensity value of the hard example region is greater than the guidance intensity value of the non-hard example region;

[0281] S533: Conditional guidance is performed based on the guidance intensity value at each spatial location to achieve pixel-level control over the generation difficulty.

[0282] In the classifier-agnostic guidance framework, to enhance the generation strength of hard example regions, a spatially adaptive guidance mechanism is introduced, which expands the fixed guidance strength to a spatially correlated guidance weight map. Its definition is:

[0283]

[0284] in, Indicates pixel position Spatial adaptive guiding weights; This indicates the location of the synthetic hard case mask at the pixel position. The value at; Indicates the basic guidance strength in the non-difficult case area; Indicates the enhanced guidance strength in the difficult example region, and satisfies In this embodiment, 7.5 is preferred. It can be configured according to task requirements.

[0285] Based on the spatial adaptive guided weight graph The diffusion model at pixel location The guided noise prediction at the location is defined as:

[0286]

[0287] in, This represents the prediction result of the diffusion model for noise under unconditional input. This indicates that the diffusion model is under three-conditional input. The following are the prediction results for noise; Indicates pixel position The noise prediction result is obtained after spatial adaptive guidance. The guided noise prediction result is used to participate in subsequent iterative denoising updates.

[0288] By integrating the above three conditions and the spatial adaptive guidance mechanism, a greater guidance intensity is given in the difficult example region, so that the generated sample presents a more challenging style perturbation in the region; the basic guidance intensity is used in the non-difficult example region to maintain the stability of the overall sample structure and the generation process in the non-difficult example region.

[0289] S54: The contrastive learning network is further trained using the hard example augmentation samples, and the global feature encoder and local feature encoder in the contrastive learning network are updated.

[0290] S55: The change detection model is trained using the updated global feature encoder and local feature encoder to obtain the updated change detection model. Based on the updated change detection model, the dual-temporal image pairs are predicted, and the comprehensive hard example mask is regenerated to complete the closed-loop iterative optimization.

[0291] The hard example enhancement samples generated in step S53 are used to update the contrastive learning feature encoder in step S3, focusing on enhancing the local feature encoder's ability to represent difficult regions. In local contrastive learning, hard example regions are assigned higher loss weights to improve the model's ability to detect difficult regions.

[0292] Specifically, set Represents the set of all spatial locations. Indicates spatial location The weighting coefficient of the difficult cases, Indicates spatial location The corresponding local contrast loss term is defined as follows: The hard-case weighted local contrast loss is:

[0293]

[0294] in, This represents the local contrast loss weighted by difficult examples.

[0295] The weighting coefficient of the difficult example Defined as:

[0296]

[0297] in, Indicates the spatial location of the complex example mask. The value at; This represents the hard example loss weight coefficient, used to increase the loss contribution of hard example regions in local contrastive learning. In this embodiment, Version 2.0 is preferred.

[0298] Weighted local contrast loss based on difficult examples The local feature encoder is further updated to make it pay more attention to difficult detection areas such as blurred boundaries, small targets and complex textures during training, thereby improving the feature discrimination ability and detection accuracy of the change detection model in difficult areas.

[0299] The updated feature encoder is used for the change detection model construction, training, and hard example localization in the next round of steps S41 to S43.

[0300] The updated feature encoder from step S54 is reused to train the change detection model, resulting in an updated change detection model. This updated model is then used to predict changes in dual-temporal image pairs, generating a new change probability map. Furthermore, new hard example regions are located, yielding a new comprehensive hard example mask. The new integrated hard example mask is then fed back to the conditional diffusion generation model to generate new hard example enhanced samples, and the contrastive learning feature encoder and change detection model are updated accordingly, thus forming a closed-loop iterative optimization process of "generation-contrastive learning-detection".

[0301] The termination conditions for the iterative optimization process include: synthesizing the hard example mask. The proportion of difficult and complex cases decreases to below a preset threshold, or the preset maximum number of iterations is reached. In this embodiment, the maximum number of iterations is preferably set to 5.

[0302] Through the above iterative optimization cycle, the generative model continuously generates more challenging enhanced samples for areas that are difficult to detect, the contrastive learning feature encoder continuously improves its ability to represent difficult areas, and the change detection model achieves round-by-round optimization under the condition of few samples, thereby completing the collaborative optimization of generation-contrast learning-detection for remote sensing change detection with few samples.

[0303] like Figure 2 As shown, the present invention also provides a collaborative optimization system for remote sensing change detection with few samples, including a conditional diffusion generation module, a sample quality screening module, a contrastive learning module, a change detection module, and a closed-loop optimization module.

[0304] The conditional diffusion generation module is used to acquire raw remote sensing image data samples and build a style feature library. It uses style vectors as style conditions and edge structure maps as structure conditions to generate enhanced samples that maintain spatial structure consistency but have different styles using the conditional diffusion generation model.

[0305] The sample quality screening module is used to screen the generated enhanced samples using a preset screening strategy to obtain style-enhanced samples.

[0306] The contrastive learning module is used to construct a contrastive learning network, which uses raw remote sensing image data samples and selected style-enhanced samples to construct global contrastive learning tasks and local contrastive learning tasks.

[0307] The change detection module is used to build a change detection model, train the change detection model using labeled samples, locate difficult regions based on the change probability and confidence information output by the model, and generate a comprehensive hard case mask.

[0308] The closed-loop optimization module takes the comprehensive hard example mask as a spatial condition and inputs it, along with the style condition and structural condition, into the conditional diffusion generation model to generate hard example enhancement samples for difficult regions. The module then performs quality screening on the hard example enhancement samples and updates the contrastive learning network and change detection network to obtain the updated comprehensive hard example mask, thus completing the closed-loop iterative optimization.

[0309] This invention uses a conditional diffusion model as the generative model, which has the following advantages compared with traditional style transfer networks: (1) The randomness of the diffusion model can generate diverse samples for the same condition, enriching the distribution of training data; (2) It supports a unified framework for multi-condition fusion, and style conditions, structural conditions, and hard example masks can be flexibly combined in the same model; (3) It achieves pixel-level hard example enhancement control through a spatial adaptive guidance mechanism, which is more targeted.

[0310] This invention employs a style distance-driven "near-domain but not co-domain" sample generation strategy to effectively expand the style distribution of training data and improve the model's robustness to changes in imaging conditions. It filters generated samples through multi-dimensional quality constraints to avoid noise accumulation and confirmation bias. Through a global and local contrastive learning mechanism, it learns style-invariant feature representations with change-discriminating capabilities without relying on a large number of annotations. Furthermore, through a difficult-example-driven closed-loop feedback mechanism, it achieves collaborative optimization between the generation and detection models, specifically improving the model's detection capabilities in challenging regions. In summary, this invention significantly improves the generalization ability and detection accuracy of change detection under conditions of few samples.

[0311] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A collaborative optimization method for remote sensing change detection with few samples, characterized in that, Includes the following steps: We acquire raw remote sensing image data samples and construct a style feature library. Using style vectors as style conditions and edge structure maps as structure conditions, we use a conditional diffusion generation model to generate enhanced samples that maintain spatial structure consistency but have different styles. The generated enhanced samples are subjected to quality screening using a preset screening strategy to obtain high-quality style-enhanced samples. A contrastive learning network is constructed, and global and local contrastive learning tasks are built using original remote sensing image data samples and selected high-quality style-enhanced samples. A change detection model is constructed, and the model is trained using labeled samples. Difficult regions are located based on the change probability and confidence information output by the model, and a comprehensive hard case mask is generated. The comprehensive difficult example mask is used as a spatial condition and is input into the conditional diffusion generation model along with the style and structure conditions to generate enhanced difficult example samples for difficult regions. The enhanced difficult example samples are then subjected to quality screening, and the contrastive learning network and change detection network are updated to obtain the updated comprehensive difficult example mask, thereby completing the closed-loop iterative optimization.

2. The collaborative optimization method for few-sample remote sensing change detection according to claim 1, characterized in that, The process of acquiring original remote sensing image data samples and constructing a style feature library, and then using a conditional diffusion generation model to generate enhanced samples that maintain spatial structure consistency but have different styles, specifically includes the following steps: Obtain raw remote sensing image data samples containing images of the target domain and multiple adjacent domains, and extract style feature vectors for each image, and construct a style feature library based on the style feature vectors; Candidate style vectors are selected from the style feature library whose distance from the target domain style is within a preset style deviation distance range; A sampling distribution based on temperature coefficient control is used to sample from the candidate style vector, so that the sampled target style vector is similar to but not the same as the target domain style; Given an input image and a sampled target style vector, an enhanced sample with consistent spatial structure but style transfer is generated based on a pre-defined conditional control network as a conditional diffusion generation model.

3. The collaborative optimization method for few-sample remote sensing change detection according to claim 2, characterized in that, The process of generating enhanced samples with consistent spatial structure but style transfer based on a pre-defined conditional control network as a conditional diffusion generation model for a given input image and sampled target style vector specifically includes the following steps: Edge maps of the input image are extracted using an edge detection operator, and the edge maps are encoded into multi-scale structural conditional features using an encoder. The sampled target style vector is mapped to style condition features through a style encoder. Low-level style features are injected into the shallow network generated by the conditional diffusion using a feature linear modulation mechanism, and high-level style features are injected into the deep network of the style encoder using a cross-attention mechanism. The low-level and high-level style features are obtained by mapping the sampled target style vector to different forms of style conditions through the style encoder in the shallow and deep network layers. The conditional diffusion generation model receives the structural conditional features and style conditional features, and generates enhanced samples with consistent spatial structure but style transfer through iterative denoising.

4. The collaborative optimization method for few-sample remote sensing change detection according to claim 3, characterized in that, The step of using a conditional diffusion generation model to receive the structural conditional features and style conditional features, and generating enhanced samples with consistent spatial structure but style transfer through iterative denoising, specifically includes the following steps: Forward diffusion: Gaussian noise is gradually added to the original remote sensing image data samples to obtain noise samples that approximately follow a Gaussian distribution; Conditional reverse denoising: Based on the style condition features and structural condition features, the noise samples are input into the conditional denoising network and reverse denoising is performed step by step. The reverse denoising probability distribution of the conditional denoising network under the style condition features and structural condition features is calculated. Inference generation: Based on the inverse denoising probability distribution, iterative sampling is performed from standard Gaussian noise to obtain enhanced samples with consistent spatial structure but style transfer.

5. The collaborative optimization method for few-sample remote sensing change detection according to claim 1, characterized in that, The step of using a preset screening strategy to perform quality screening on the generated enhanced samples to obtain high-quality style-enhanced samples specifically includes the following steps: Calculate the edge structure similarity between the enhanced sample and the original remote sensing image data sample, and filter out enhanced samples whose structural deviation exceeds the structural consistency threshold; Calculate the style deviation distance between the style features of the enhanced sample and the target style distribution, and filter out the enhanced samples whose style deviation distance exceeds the style consistency threshold; The semantic segmentation results of the enhanced samples and the original remote sensing image data samples are obtained by using a pre-trained semantic segmentation model. Enhanced samples with semantic consistency lower than a preset semantic consistency threshold are filtered out to obtain high-quality style-enhanced samples.

6. The collaborative optimization method for few-sample remote sensing change detection according to claim 1, characterized in that, The construction of the contrastive learning network, which utilizes original remote sensing image data samples and selected high-quality style-enhanced samples to construct global and local contrastive learning tasks, specifically includes: Constructing a global contrastive learning task: Positive sample pairs are constructed by selecting original remote sensing image data samples and their corresponding high-quality style-enhanced samples; The high-quality style-enhanced samples corresponding to the remaining original remote sensing image data samples in the same batch of training, excluding those used to construct the positive sample pairs, are used to construct negative samples. The global encoder is used to extract features from the original remote sensing image data samples and their corresponding high-quality style-enhanced samples. The global contrast loss in the entire training batch is calculated. The feature distance between positive sample pairs and the feature distance between negative samples are minimized and the global contrast loss function is maximized to learn image-level global feature representations that are robust to imaging conditions. Constructing a local contrastive learning task: Obtain the true binary change mask corresponding to the dual-temporal image pair in the original remote sensing image data sample, and construct positive sample pairs based on the features of the corresponding positions across time within the region where the true binary change mask remains unchanged. Negative sample pairs are constructed based on the features of the corresponding positions across time within the region where the true binary change mask remains unchanged. The local feature encoder is used to extract features from the two-temporal image pairs, calculate the local contrast loss in the entire training batch, and minimize the local contrast loss function so that the local feature encoder can learn local feature representations that have both the ability to distinguish changed regions and are robust to style perturbations. Joint comparative learning training: The global contrast loss and local contrast loss are used to construct a joint contrast learning total loss. The two temporal image pairs and their corresponding high-quality style enhancement samples are input into the global feature encoder and the local feature encoder, respectively. The global contrast learning loss and local contrast learning loss are calculated, and the joint contrast learning total loss is obtained. Using the total loss of the joint contrastive learning as the optimization objective, the gradient backpropagation algorithm is used to jointly update the network parameters of the global feature encoder and the local feature encoder. The gradient backpropagation algorithm is used to iteratively adjust the model parameters according to the gradient information of the network parameters based on the loss function, so as to minimize the total loss of the joint contrastive learning.

7. The collaborative optimization method for few-sample remote sensing change detection according to claim 6, characterized in that, The construction of the change detection model, training the model using labeled samples, locating difficult regions based on the change probability and confidence information output by the model, and generating a comprehensive hard example mask specifically includes the following steps: A change detection model based on a twin encoder-decoder structure is constructed, and the network parameters of the global feature encoder and the local feature encoder are loaded. The change detection model is trained by inputting a small number of real labeled samples, and a predicted change probability map is output. A change detection loss function is constructed based on the predicted change probability map and the real change mask, and the total change detection loss is calculated. Based on the total change detection loss, gradient backpropagation and iterative updates are performed on the parameters of the change detection model to minimize the total change detection loss, thereby completing the training of the change detection model. The trained change detection model is used to predict the dual-temporal image pair to obtain a change probability map. Based on the change probability map and the true binary change mask, a comprehensive hard case mask is constructed from two aspects: low confidence region and mispredicted region.

8. The collaborative optimization method for few-sample remote sensing change detection according to claim 7, characterized in that: The construction of a comprehensive hard example mask based on the change probability map and the true binary change mask, from the perspectives of low-confidence regions and mispredicted regions, specifically includes the following steps: Read the pixel locations in the probability map where the predicted probability is close to the decision boundary and identify them as low-confidence hard example regions; Based on the aforementioned change probability map and the true binary change mask, the error hard case region is determined; The low-confidence hard example region and the mispredicted hard example region are fused to generate a comprehensive hard example mask.

9. The collaborative optimization method for few-sample remote sensing change detection according to any one of claims 1-8, characterized in that: The process of using the comprehensive hard example mask as a spatial condition, along with the style and structural conditions, as input into the conditional diffusion generation model to generate hard example enhancement samples for difficult regions, performing quality screening on the hard example enhancement samples, updating the contrastive learning network and change detection network, and obtaining the updated hard example mask to complete the closed-loop iterative optimization specifically includes the following steps: The comprehensive hard example mask is fed back to the conditional diffusion generation model, and the hard example mask is encoded by the spatial conditional encoder to obtain spatial conditional features; The spatial condition features, style condition features, and structural condition features are collectively input into the conditional diffusion generation model; A spatial adaptive guidance mechanism is adopted to adaptively adjust the guidance intensity according to the spatial distribution of the synthetic hard case mask, and generate hard case enhancement samples in the synthetic hard case region. The contrastive learning network is further trained using the hard-example augmented samples, and the global and local feature encoders in the contrastive learning network are updated. The change detection model is trained using the updated global feature encoder and local feature encoder to obtain the updated change detection model. Based on the updated change detection model, the dual-temporal image pairs are predicted, and the comprehensive hard example mask is regenerated to complete the closed-loop iterative optimization. The space adaptive guidance mechanism includes: The integrated hard example mask overspace conditional encoder is encoded into multi-scale conditional features that match the spatial dimensions of each layer of the conditional denoising network. Based on the classifier-independent guidance framework, the guidance strength value of each spatial location is determined according to the comprehensive hard example mask, wherein the guidance strength value of the hard example region is greater than the guidance strength value of the non-hard example region; Conditional guidance is applied based on the guidance intensity value at each spatial location to achieve pixel-level control over the generation difficulty.

10. A collaborative optimization system for remote sensing change detection with few samples, characterized in that: It includes a conditional diffusion generation module, a sample quality screening module, a contrastive learning module, a change detection module, and a closed-loop optimization module; The conditional diffusion generation module is used to acquire raw remote sensing image data samples and build a style feature library. It uses style vectors as style conditions and edge structure maps as structure conditions to generate enhanced samples that maintain spatial structure consistency but have different styles using the conditional diffusion generation model. The sample quality screening module is used to screen the generated enhanced samples using a preset screening strategy to obtain high-quality style enhancement samples. The contrastive learning module is used to construct a contrastive learning network, which uses raw remote sensing image data samples and selected high-quality style-enhanced samples to construct global contrastive learning tasks and local contrastive learning tasks. The change detection module is used to build a change detection model, train the change detection model using labeled samples, locate difficult regions based on the change probability and confidence information output by the model, and generate a comprehensive hard case mask. The closed-loop optimization module takes the comprehensive hard example mask as a spatial condition and inputs it, along with the style condition and structural condition, into the conditional diffusion generation model to generate hard example enhancement samples for difficult regions. The module then performs quality screening on the hard example enhancement samples and updates the contrastive learning network and change detection network to obtain the updated comprehensive hard example mask, thus completing the closed-loop iterative optimization.

Images