A Semi-Supervised Remote Sensing Image Change Detection Method Based on Adaptive Feature Perturbation

Abstract

The present invention relates to a semi-supervised remote sensing image change detection method based on adaptive feature perturbation. A teacher model is trained using a weakly augmented training set, and the predicted certainty value is obtained through maximum value normalization processing to mitigate the adverse effects brought by unreliable pseudo-labels. Then, an adaptive feature perturbation scheme is customized based on each predicted certainty value, which can dynamically adjust the perturbation of sample features according to the sample quality, increasing controllability on the basis of random perturbation, thereby significantly reducing the confirmation bias of the student model and improving the performance of the student model for semi-supervised change detection of remote sensing images.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and change detection, and in particular, to a semi-supervised remote sensing image change detection method based on adaptive feature perturbation. Background Art

[0002] Remote sensing image change detection (RSCD) is a task of identifying surface feature changes by analyzing remote sensing images acquired at different times. This task is essentially a binary classification problem, and its purpose is to identify target change regions of interest (such as buildings, water areas, vegetation, and roads, etc.) in a pair of bi-temporal images taken at different times in the same area. RSCD has wide applications in fields such as urban construction planning, forest environmental protection, rural land management, and natural disaster assessment, which also makes RSCD an important research direction in the remote sensing field. In recent years, change detection methods based on deep learning have made remarkable progress. However, these methods highly rely on a large amount of labeled training data, and when the labeled samples are insufficient, their performance will significantly decline. In addition, the data annotation process for the change detection task is complex, usually requiring high-precision geometric image registration and pixel-level fine annotation, which is time-consuming and costly. At the same time, with the rapid development of earth observation technology, a large amount of unlabeled data has become easily available, providing an opportunity for semi-supervised learning. Semi-supervised learning methods can effectively utilize these unlabeled samples, thereby significantly improving the performance of the model when the labeled data is limited, and it is an effective way to solve the above problems.

[0003] Semi-supervised change detection (SSCD) aims to simultaneously use limited labeled training samples and a large number of unlabeled training samples for training to improve the change detection performance. In this way, the model can not only learn the required semantic information, but also more fully learn the semantic feature distribution, thereby significantly improving the robustness of the model. The current mainstream methods include methods based on pseudo-label self-training, methods based on consistency regularization, and methods based on generative adversarial networks. Among them, the method based on pseudo-label self-training first uses a prediction model to generate pseudo-labels for unlabeled samples, mixes these unlabeled samples and pseudo-labels with the labeled training samples, and conducts supervised training together, so as to provide augmented training data for the model; the method based on consistency regularization uses the output consistency of the model on these input data as a training constraint by applying different degrees of perturbation to the input data; the method based on generative adversarial networks aims to use the generative model to model the distribution of data, so as to infer the potential information of unlabeled data.

[0004] Although the above methods have achieved certain success, there are still some challenges: the method based on pseudo-label self-training has inevitable noise problems, which may accumulate during the training process and seriously affect the performance of the model; in addition, the training process of the generative adversarial network is highly unstable, and the problem of gradient disappearance often makes it difficult for the generator to be continuously optimized, so it is difficult to achieve the ideal optimal training result. Summary of the Invention

[0005] The problem solved by the present invention is how to effectively reduce the influence of noise in model prediction and prevent the problem of excessive confirmation bias caused by random perturbations, so as to improve the reliability and stability of model prediction.

[0006] The present invention provides a semi-supervised remote sensing image change detection method based on adaptive feature perturbation, including:

[0007] Step 1, collect a pair of multi-temporal images, and divide the labeled training set and the unlabeled training set;

[0008] Step 2, initialize the weights of the student model and the teacher model;

[0009] Step 3, perform weak augmentation on the labeled training set and then input it into the student model, and calculate the supervised loss;

[0010] Step 4, perform weak augmentation on the unlabeled training set to obtain a weakly augmented training set, and perform strong augmentation on the weakly augmented training set to obtain a strongly augmented training set;

[0011] Step 5, input the weakly augmented training set into the teacher model to predict the confidence map, evaluate the confidence map to obtain the certainty value, and normalize all the certainty values with the maximum value to obtain the predicted certainty value;

[0012] Step 6, customize an adaptive feature perturbation scheme according to each predicted certainty value;

[0013] Step 7, input the strongly augmented training set into the student model to obtain high-dimensional features, and apply the adaptive feature perturbation scheme to the high-dimensional features; calculate the unsupervised loss between the teacher model and the student model;

[0014] Step 8, use an exponential decay warm-up function to weight the unsupervised loss, and add it to the supervised loss to obtain the overall training loss;

[0015] Step 9, use the stochastic gradient descent method to minimize the overall training loss to update the weights of the student model; update the weights of the teacher model based on the updated weights of the student model and the exponential moving average method.

[0016] Compared with the prior art, the present application has the following advantages: The teacher model is trained using a weakly augmented training set, and the prediction certainty value is obtained through maximum normalization processing to mitigate the adverse effects brought by unreliable pseudo-labels; then, an adaptive feature perturbation scheme is customized based on each prediction certainty value, which can dynamically adjust the perturbation of sample features according to the sample quality, adding controllability on the basis of random perturbation, thereby significantly reducing the confirmation bias of the student model and improving the performance of the student model for semi-supervised change detection of remote sensing images.

[0017] In a possible implementation manner, the pair of dual-temporal images in step 1 includes two remote sensing images of the same area at different times.

[0018] Compared with the prior art, by making full use of the labeled data in the labeled training set, maximizing the value of the unlabeled training set, significantly reducing the dependence on manual annotation, and improving the efficiency and practicality of semi-supervised learning.

[0019] In a possible implementation manner, the calculation formula for the supervised loss in step 3 is:

[0020] ;

[0021] In the formula, represents the supervised loss, represents the total number of labeled samples input into the student model; represents the prediction change probability of the th pixel of the th pair of dual-temporal images by the student model, represents the true label of the th pixel of the th pair of dual-temporal images; represents the length of the remote sensing image in the pair of dual-temporal images, represents the width of the remote sensing image in the pair of dual-temporal images.

[0022] Compared with the prior art, the method of calculating the supervised loss after weakly augmenting the labeled training set effectively utilizes the labeled information in the labeled training set and enhances the feature learning ability of the student model for the labeled training set.

[0023] In a possible implementation manner, the weak augmentation processing of the unlabeled training set in step 4 includes random flipping, random scaling, and random cropping; the strong augmentation processing of the weakly augmented training set in step 4 includes randomly selecting three of identity, contrast, auto-contrast, brightness, color, equalization, sharpness, posterization, and inversion.

[0024] Compared with the prior art, by performing weak augmentation processing and strong augmentation processing on the unlabeled training set respectively to generate diverse training samples, the weak augmentation processing preserves the core features for the teacher model to generate reliable pseudo-labels, and the strong augmentation processing simulates complex scene changes by combining multiple image transformations, enhances the robustness of the student model to feature perturbations, and reduces the risk of overfitting.

[0025] In a possible implementation manner, step 5 specifically includes:

[0026] Step 501, batch-divide the weak augmentation training set and the strong augmentation training set into a number of weak augmentation sample sets and a number of strong augmentation sample sets respectively; the corresponding samples in the weak augmentation sample set and the strong augmentation sample set of the same batch are derived from the same pair of dual-temporal image pairs;

[0027] Step 502, randomly select a weak augmentation sample set and input it into the teacher model to predict the change probability to obtain a confidence map;

[0028] Step 503, evaluate the confidence map to obtain a certainty value, and the calculation formula is:

[0029] ;

[0030] In the formula, represents the certainty value, represents the confidence map, represents taking the mean of the certainty value over all pixels, represents the predicted probability difference map, , represents the absolute value operation; represents the weighted information entropy map, , is the information entropy of the predicted probability, and respectively represent the information entropy of the unchanged class and the changed class; ; In the formula, ; ; represents the predicted change probability of the th pair of dual-temporal image pairs;

[0031] Step 504, perform maximum normalization processing on each certainty value to obtain a predicted certainty value.

[0032] Compared with the prior art, by calculating the certainty value through the confidence map and performing maximum normalization to obtain the predicted certainty value, the credibility of the teacher model prediction is dynamically quantified, the interference of low-quality samples is avoided, the negative impact of unreliable pseudo-labels is alleviated, and the sample quality assessment is standardized through maximum normalization processing, providing a reliable basis for subsequent customization of the adaptive feature perturbation scheme.

[0033] In a possible implementation, the customized adaptive feature perturbation scheme in step 6 includes the number of perturbations and the perturbation intensity, and step 6 specifically includes:

[0034] Step 601: Using as a scaling factor, calculate the number of perturbations for each prediction certainty value, and the calculation formula is:

[0035] ;

[0036] In the formula, represents the number of perturbations of the prediction certainty value , represents the total number of perturbations in the perturbation pool;

[0037] Step 602: Randomly select the number of perturbation methods from the perturbation pool, and calculate the perturbation intensity of each perturbation method. The calculation formula is:

[0038] ;

[0039] In the formula, represents the perturbation intensity, is the predefined parameter of the th perturbation method.

[0040] Compared with the prior art, adopting the above technical solution realizes the differential strategy of low perturbation for high-quality samples and high perturbation for low-quality samples, which can not only avoid the blindness of random perturbation, optimize features targeted, but also reduce the confirmation bias of the student model and improve the tolerance to noisy data.

[0041] In a possible implementation, step 7 specifically includes:

[0042] Step 701: Input the strongly augmented sample set in the same batch as the weakly augmented sample set selected in step 5 into the student model for feature extraction and fusion to obtain high-dimensional features;

[0043] Step 702: Apply the adaptive feature perturbation scheme to the high-dimensional features, and the expression is:

[0044] ;

[0045] In the formula, represents the high-dimensional features after applying perturbations, represents the high-dimensional features extracted by the student model from the th unlabeled sample in the current batch of strongly augmented sample sets, represents the A set of adaptive feature perturbation schemes for unlabeled samples Represents the process of specifically applying perturbations

[0046] Step 703: Use the student model decoder to decode the perturbed high-dimensional features to obtain the change probability. The calculation formula is: , represents the change probability of the student model on the strongly augmented sample set

[0047] Step 704: Calculate the unsupervised loss between the teacher model and the student model. The calculation formula is:

[0048] ;

[0049] where represents the unsupervised loss is the total number of unlabeled samples in the current batch of weakly augmented sample set represents the change probability of the teacher model on the weakly augmented sample set represents applying the th perturbation method and outputting the change probability represents the mean squared error loss function

[0050] Compared with the prior art, by applying the adaptive feature perturbation scheme to the high-dimensional features extracted by the student model and calculating the unsupervised loss, the sensitivity of the student model to feature changes is enhanced. Through the prediction consistency constraint between the teacher model and the student model, the reliability of the pseudo-labels is further improved

[0051] In a possible implementation manner, step 8 specifically includes:

[0052] Step 801: Use an exponential decay warm-up function to weight the unsupervised loss, which is expressed as:

[0053] ;

[0054] ;

[0055] In the formula represents the unsupervised loss is the maximum achievable weight of used to control the speed of the warm-up process represents the current iteration cycle is the total number of steps in the warm-up process is the total number of iterations in the entire training process, and 0 < < 1

[0056] Step 802: Calculate the overall training loss. The calculation formula is:

[0057] ;

[0058] In the formula, represents the overall training loss, represents the weight.

[0059] Compared with the prior art, by using an exponential decay warm-up function to dynamically adjust the weight of the unsupervised loss, the student model focuses on the supervised constraint in step 4, and introduces the unsupervised constraint in step 7 and subsequent steps, balancing the optimization objectives in different training stages, avoiding the interference of early unstable pseudo-labels on the convergence of the student model, and improving the training stability.

[0060] In a possible implementation manner, the calculation formula for updating the weight of the teacher model based on the updated weight of the student model and the exponential moving average method in step 9 is:

[0061] ;

[0062] In the formula, represents the initial weight of the teacher model, represents the updated weight of the teacher model, represents the updated weight of the student model, is the momentum factor of the moving exponential average.

[0063] Compared with the prior art, the weight of the student model is updated by using the stochastic gradient descent method, and then the weight of the teacher model is updated in combination with the exponential moving average method, inheriting the progressive optimization result of the student model, ensuring that the teacher model outputs more stable pseudo-labels, forming a benign teacher-student interaction, and accelerating the convergence of the student model and the teacher model.

[0064] In a possible implementation manner, the pair of dual-temporal images collected in step 1 is further divided into a validation set and a test set;

[0065] It further includes step 10 of validating, testing, and saving the student model with updated weights based on the validation set and the test set; specifically including:

[0066] Step 1001, after the training of the teacher model and the student model on the current batch of weakly augmented sample set and strongly augmented sample set is completed, randomly select another weakly augmented sample set and strongly augmented sample set to train the teacher model and the student model through steps 5 - step 9;

[0067] Step 1002, evaluate the student model after each batch of training by using the mean intersection over union, and save the weights of the student model with the highest mean intersection over union;

[0068] Step 1003, use the test set to test the saved student model Perform tests, calculate the average intersection over union, and visualize the predicted change probability to obtain the change detection results of the dual-temporal image pair.

[0069] Compared with the prior art, using the average intersection over union as an evaluation index can directly reflect the change detection accuracy, ensure the practicality for real scenarios, and the visualization operation further verifies the robustness of the method of this application in change detection for complex scenarios, providing an intuitive basis for actual deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram of the process framework of the present invention;

[0071] Figure 2 It is a visualization comparison diagram of the 5% labeled LEVIR-CD dataset using different semi-supervised change detection methods in the embodiments of this application;

[0072] Figure 3 It is a visualization comparison diagram of the 5% labeled WHU-CD dataset using different semi-supervised change detection methods in the embodiments of this application;

[0073] Figure 4 It is a visualization comparison diagram of the features on the dataset when using random feature perturbation and adaptive feature perturbation respectively using the t-SNE technique in the embodiments of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] First of all, those skilled in the art should understand that these embodiments are only used to explain the technical principles of the embodiments of this application, and are not intended to limit the protection scope of the embodiments of this application. Those skilled in the art can make adjustments according to needs to adapt to specific application scenarios.

[0075] In the description of the embodiments of this application, it should be noted that unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of this application can be understood according to specific situations.

[0076] In the embodiments of the present application, unless otherwise clearly specified and limited, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on top of" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "below" and "beneath" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the horizontal height of the first feature is less than that of the second feature.

[0077] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0078] See Figure 1 As shown, the embodiments of the present application disclose a semi-supervised remote sensing image change detection method based on adaptive feature perturbation, including:

[0079] Step 1, collect a pair of dual-temporal images, and divide the labeled training set , unlabeled training set , validation set and test set; the pair of dual-temporal images includes two remote sensing images of the same area at different times, the labeled training set , the unlabeled training set , where represents the -th pair of dual-temporal images and its true label, represents the -th pair of unlabeled dual-temporal images; represents the image of the first time phase of the pair of dual-temporal images, represents the image of the second time phase of the pair of dual-temporal images, represents the number of labeled samples in the labeled training set , represents the number of unlabeled samples in the unlabeled training set , and ;

[0080] In this specific embodiment, all pairs of dual-temporal images come from 20 different locations in seven cities in Texas, USA, and were taken between 2002 and 2018. The time span of the pairs of dual-temporal images is between 5 and 14 years; in this specific embodiment, the collected pairs of dual-temporal images are cut into -pixel image pairs, where the labeled training set accounts for 3.5% (356 pairs), the unlabeled training set accounts for 66.5% (6764 pairs), the validation set accounts for 10% (1024 pairs), and the test set accounts for the other 20% (2048 pairs).

[0081] Step 2, construct the student model and the teacher model , and initialize the weights of the student model and the weights of the teacher model ; In this specific embodiment, both the student model and the teacher model use ResNet-50 as the backbone network, and the parameters pre-trained on ImageNet with ResNet-50 are used for initialization and .

[0082] Step 3, perform weak augmentation on the labeled training set and then input it into the student model , and calculate the supervised loss ; Perform weak augmentation on the labeled training set . The weak augmentation includes random flipping, random scaling, and random cropping; the labeled weakly augmented training set obtained after processing is denoted as ;

[0083] In this specific embodiment, randomly extract labeled samples from the labeled weakly augmented sample dataset in each round to train the student model ; In this specific embodiment ; The calculation formula for the supervised loss is:

[0084] ;

[0085] In the formula, represents the total number of labeled samples input into the student model in each round; represents the predicted change probability of the th pixel of the dual-temporal image pair by the student model for the th pixel, represents the true label of the th pixel of the dual-temporal image pair for the th pixel; represents the length of the remote sensing image in the dual-temporal image pair, represents the width of the remote sensing image in the dual-temporal image pair.

[0086] Step 4, perform weak augmentation on the unlabeled training set to obtain a weakly augmented training set, and perform strong augmentation on the weakly augmented training set to obtain a strongly augmented training set; among them, perform weak augmentation on the unlabeled training set Performing weak augmentation processing includes random flipping, random scaling, and random cropping; the obtained weakly augmented training set is denoted as ; the strong augmentation processing of the weakly augmented training set in step 4 includes randomly selecting three of Identity, Contrast, AutoContrast, Brightness, Color, Equalize, Sharpness, Posterize, and Solarize, and the obtained strongly augmented training set is denoted as .

[0087] Step 5: Input the weakly augmented training set into the teacher model to predict the change probability of each pixel in the dual-temporal image pair belonging to the changed category or the unchanged category, obtaining a confidence map , and evaluate the confidence map to obtain a certainty value , and perform maximum normalization on all certainty values to obtain a predicted certainty value ; specifically including:

[0088] Step 501: Batch-divide the weakly augmented training set and the strongly augmented training set respectively to obtain a number of weakly augmented sample sets and a number of strongly augmented sample sets; the corresponding samples in the weakly augmented sample set and the strongly augmented sample set of the same batch are from the same pair of dual-temporal image pairs;

[0089] Step 502: Randomly select a batch of weakly augmented sample sets and input them into the teacher model to predict the change probability to obtain a confidence map ;

[0090] Step 503: Evaluate the confidence map to obtain a certainty value , and the calculation formula is:

[0091] ;

[0092] In the formula, represents taking the mean of the certainty value over all pixels, represents the predicted probability difference map, , represents the absolute value operation; represents the weighted information entropy map, , represents the information entropy of the predicted probability, and respectively represent the information entropy of the unchanged category and the changed category; ; In the formula, ; ;

[0093] Step 504, perform maximum normalization on each certainty value to obtain a predicted certainty value , and the calculation formula is:

[0094] ;

[0095] In the formula, represents the maximum value of all certainty values corresponding to the current batch of weakly augmented sample sets, and represents the minimum value of all certainty values

[0096] corresponding to the current batch of weakly augmented sample sets. Step 6, customize an adaptive feature perturbation scheme according to each predicted certainty value

[0097] ; The adaptive feature perturbation scheme includes the number of perturbations and the perturbation intensity, specifically including: As a scaling factor, calculate the number of perturbations for each predicted certainty value , and the calculation formula is:

[0098] ;

[0099] In the formula, represents the number of perturbations of the predicted certainty value , represents the total number of perturbations in the perturbation pool; the parameters of the perturbation pool are shown in Table 1;

[0100] Table 1 Perturbation pool parameter table

[0101] ;

[0102] Step 602, randomly select perturbation methods from the perturbation pool, and calculate the perturbation intensity of each perturbation method, and the calculation formula is:

[0103] ;

[0104] is the predefined parameter of the th perturbation method.

[0105] Step 7, input the strongly augmented training set into the student model Extract and fuse to obtain high-dimensional features, apply an adaptive feature perturbation scheme to the high-dimensional features, and use the student model decoder to decode the perturbed high-dimensional features to obtain the change probability; calculate the teacher model and the student model of the unsupervised loss ; specifically including:

[0106] Step 701, input the strongly augmented sample set in the same batch as the weakly augmented sample set selected in Step 5 into the student model for feature extraction and fusion to obtain high-dimensional features;

[0107] Step 702, apply the adaptive feature perturbation scheme to the high-dimensional features, and the expression is:

[0108] ;

[0109] In the formula, represents the high-dimensional features extracted by the student model from the th unlabeled sample in the current batch of strongly augmented sample sets, represents the set of normal vectors of the adaptive feature perturbation scheme for the th unlabeled sample in the current batch of weakly augmented sample sets, represents the specific process of applying the perturbation;

[0110] Step 703, use the student model decoder to decode the perturbed high-dimensional features to obtain the change probability, and the calculation formula is: ;

[0111] Step 704, calculate the unsupervised loss of the teacher model and the student model , and the calculation formula is:

[0112] ;

[0113] ;

[0114] where is the total number of unlabeled samples in the current batch of weakly augmented sample sets, is the change probability of the teacher model on the weakly augmented sample set, represents the change probability output after applying the th perturbation method.

[0115] Step 8, use the exponential decay warm-up function to weight the unsupervised loss and combine it with the supervised loss Add them up to get the overall training loss, specifically including:

[0116] Step 801: Use an exponential decay warm-up function to weight the unsupervised loss and represent it as:

[0117] ;

[0118] ;

[0119] In the formula, represents the maximum achievable weight of the unsupervised loss, which is used to control the speed of the warm-up process; represents the current iteration cycle; is the total number of steps in the warm-up process, is the total number of iterations in the entire training process, and 0 < < 1;

[0120] Step 802: Calculate the overall training loss , and the calculation formula is:

[0121] ;

[0122] In the formula, represents the weight.

[0123] Step 9: Use the stochastic gradient descent method to minimize the overall training loss and update the weights of the student model ; Based on the updated weights of the student model and the exponential moving average method, update the weights of the teacher model ;

[0124] ;

[0125] In the formula, represents the learning rate, which is used to control the update step size, represents the gradient calculated based on the overall training loss.

[0126] Based on the updated weights of the student model and the exponential moving average method, the formula for updating the weights of the teacher model is:

[0127] ;

[0128] In the formula, It is the momentum factor of the moving exponential average.

[0129] Step 10: Validate, test, and save the student model with updated weights based on the validation set and the test set, specifically including: Proceed as follows:

[0130] Step 1001: After training the teacher model and the student model using the current batch of weakly augmented sample sets and strongly augmented sample sets, randomly select another batch of weakly augmented sample sets and strongly augmented sample sets to train the teacher model and the student model through Steps 5 - 9; in this specific embodiment, validation is performed every 5 epochs, and when the obtained is the current best, update and save the best parameters;

[0131] Step 1002: Evaluate the student model after each batch of training using the mean intersection over union , save the weights of the student model with the highest mean intersection over union ; the calculation formula for the mean intersection over union is:

[0132] ;

[0133] where , and represent the number of correctly identified changed pixels, the number of unchanged pixels misclassified as changed pixels, and the number of changed pixels misidentified as unchanged pixels, respectively;

[0134] Step 1003: Test the saved student model using the test set, calculate the mean intersection over union and visualize the predicted change probability to obtain the change detection result of the dual - temporal image pair.

[0135] Data experiment

[0136] ​Input the dataset composed of randomly collected dual-temporal image pairs into the semi-supervised remote sensing image change detection method based on adaptive feature perturbation of the present application. Table 2 shows the average quantitative comparison of different datasets in different semi-supervised change detection methods. The blue-highlighted part in Table 2 represents the optimal result, and the underlined part represents the sub-optimal result. Generally speaking, the semi-supervised remote sensing image change detection method based on adaptive feature perturbation of the present application has achieved the best effect on the change detection of all datasets composed of dual-temporal image pairs, and all detection results are better than the change detection effect of RCR. It shows that the adaptive feature perturbation consistency of the present application is more suitable for semi-supervised change detection than the random feature perturbation consistency adopted in RCR.

[0137] Table 2 Average Quantitative Comparison of Different Datasets in Different Semi-Supervised Change Detection Methods

[0138] ;

[0139] In addition, due to the very low resolution of the GZ-CD dataset and the relatively rough manual annotation, it poses a huge challenge to the FPA method based on pseudo-label self-training. However, the semi-supervised remote sensing image change detection method based on adaptive feature perturbation of the present application does not rely on binary hard pseudo-labels, but uses the perturbation consistency within a controllable search space as the training constraint, overcoming the difficulty of inaccurate pseudo-labels and obtaining better experimental results.

[0140] Finally, in this experiment, the 5% labeled LEVIR-CD dataset and the 5% labeled WHU-CD dataset were also visualized using different semi-supervised change detection methods in Table 2, as Figure 2 and Figure 3 shown. In addition, the t-SNE technique was used to reduce the dimensions of the features on the dataset when using random feature perturbation and adaptive feature perturbation respectively and project them onto two-dimensional coordinates for visualization, as shown in the figure. According to the comparison results, compared with the random perturbation used in RCR, for the semi-supervised method using adaptive feature perturbation, the decision boundary in the extracted feature space is more obvious, and the features of the changed foreground and background categories are more compact within the class and more dispersed between classes, intuitively verifying the superiority of the adaptive perturbation method.

[0141] In the description of the embodiments of the present application, it should be noted that in the description of the present application, the terms "inner", "outer", etc. indicating the direction or positional relationship are based on the direction or positional relationship shown in the drawings. This is only for convenience of description and does not indicate or imply that the device or component must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present application.

[0142] In the description of the present application, the descriptions referring to terms such as "one embodiment", "some embodiments", "in this embodiment", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0143] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A semi-supervised remote sensing image change detection method based on adaptive feature perturbation, characterized in that Including: Step 1: Collect a pair of dual-temporal images, and divide them into a labeled training set and an unlabeled training set. Step 2: Initialize the weights of the student model and the teacher model. Step 3: After weakly augmenting the labeled training set, input it into the student model, and calculate the supervised loss. Step 4: Weakly augment the unlabeled training set to obtain a weakly augmented training set, and strongly augment the weakly augmented training set to obtain a strongly augmented training set. Step 5: Input the weakly augmented training set into the teacher model to obtain a confidence map, evaluate the confidence map to obtain a certainty value, and normalize all certainty values by the maximum value to obtain a predicted certainty value. The calculation formula is: ； In the formula, represents a deterministic value, represents the maximum value of all the deterministic values corresponding to the current batch of weakly augmented sample sets ; represents the minimum value of all the deterministic values corresponding to the current batch of weakly augmented sample sets ; Step 6: According to each predicted certainty value, match an adaptive feature perturbation scheme. The adaptive feature perturbation scheme includes the number of perturbations and the perturbation intensity, specifically including: Step 601, take as a scaling factor, and calculate the number of perturbations for each predicted certainty value . The calculation formula is: ； In the formula, represents the predicted certainty value of the number of perturbations, and represents the total number of perturbations in the perturbation pool; Step 602: Randomly select the number of perturbation methods from the perturbation pool, and calculate the perturbation intensity of each perturbation method. The calculation formula is: ； In the formula, represents the disturbance intensity, is the predefined parameter of the th disturbance method; Step 7: Input the strongly augmented training set into the student model to obtain high-dimensional features, apply the adaptive feature perturbation scheme to the high-dimensional features, and calculate the unsupervised loss between the teacher model and the student model. Step 8: Use an exponential decay warm-up function to weight the unsupervised loss, and add it to the supervised loss to obtain the overall training loss. Specifically including: Step 801: Using an exponential decay warm-up function to weight the unsupervised loss is expressed as: ； ； wherein, represents the maximum achievable weight of the unsupervised loss and is used to control the speed of the warm-up process; represents the current iteration cycle; is the total number of steps in the warm-up process, is the total number of iterations in the entire training process, and ; ​ Step 802: Calculate the overall training loss. The calculation formula is: ； In the formula, represents the overall training loss, represents the weight; Step 9: Use the stochastic gradient descent method to minimize the overall training loss to update the weights of the student model; then update the weights of the teacher model based on the updated weights of the student model and the exponential moving average method.

2. The semi-supervised remote sensing image change detection method based on adaptive feature perturbation according to claim 1, wherein In the said Step 1, the pair of dual-temporal images includes two remote sensing images of the same area at different times.

3. The semi-supervised remote sensing image change detection method based on adaptive feature perturbation according to claim 2, wherein, The calculation formula of the supervised loss in the said Step 3 is: ； Wherein, represents the supervised loss, represents the total number of labeled samples in the input student model; represents the student model at the predicted change probability of the th pixel of the dual-temporal image pair, represents the true label of the th pixel of the dual-temporal image pair; represents the length of the remote sensing image in the dual-temporal image pair, represents the width of the remote sensing image in the dual-temporal image pair.

4. The semi-supervised remote sensing image change detection method based on adaptive feature perturbation according to claim 2, wherein In the said Step 4, the weak augmentation processing of the unlabeled training set includes random flipping, random scaling, and random cropping; the strong augmentation processing of the weakly augmented training set in the said Step 4 includes randomly selecting three of identity, contrast, auto-contrast, brightness, color, equalization, sharpness, posterization, and inversion.

5. The semi-supervised remote sensing image change detection method based on adaptive feature perturbation according to claim 2, wherein The said Step 5 specifically includes: Step 501: Batch divide the weakly augmented training set and the strongly augmented training set into several weakly augmented sample sets and several strongly augmented sample sets respectively; the corresponding samples in the same batch of weakly augmented sample sets and strongly augmented sample sets are from the same pair of dual-temporal images. Step 502: Randomly select a weakly augmented sample set and input it into the teacher model to predict the change probability to obtain a confidence map. Step 503: Evaluate the confidence map to obtain a certainty value. The calculation formula is: ； In the formula, represents a deterministic value, represents a confidence map, represents taking the mean of the deterministic value over all pixels, represents a predicted probability difference map, , represents the absolute value operation; represents a weighted information entropy map, , is the information entropy of the predicted change probability, and represent the information entropy of the unchanged class and the changed class respectively; ; In the formula, ; ; represents the predicted change probability of the dual-temporal image pair. Step 504, perform maximum normalization on each deterministic value to obtain a predicted deterministic value .

6. The semi-supervised remote sensing image change detection method based on adaptive feature perturbation according to claim 1, wherein The said Step 7 specifically includes: Step 701: Input the strongly augmented sample set in the same batch as the weakly augmented sample set in Step 5 into the student model for feature extraction and fusion to obtain high-dimensional features. Step 702: Apply the adaptive feature perturbation scheme to the high-dimensional features. The expression is: ； Wherein, represents the high-dimensional features after applying perturbations, represents the high-dimensional features extracted from the unlabeled samples by the student model for the in the strongly augmented sample set, represents the set of adaptive feature perturbation schemes for the unlabeled samples in the weakly augmented sample set, represents the specific process of applying perturbations; Step 703: Use the student model decoder to decode the perturbed high-dimensional features to obtain the change probability. The calculation formula is as follows: , represents the predicted change probability of the student model on the strongly augmented sample set; Step 704: Calculate the unsupervised loss between the teacher model and the student model. The calculation formula is: ； Among them, represents the unsupervised loss, is the total number of unlabeled samples in the current batch of weakly augmented sample set, represents the change probability of the teacher model on the weakly augmented sample set, represents applying the th perturbation method and then outputting the change probability, represents the mean square error loss function.

7. The semi-supervised remote sensing image change detection method based on adaptive feature perturbation according to claim 1, characterized in that In step 9, the calculation formula for updating the weights of the teacher model based on the updated weights of the student model and the exponential moving average method is as follows: ； In the formula, represents the initial weights of the teacher model, represents the weights of the teacher model after update, represents the weights of the student model after update, is the momentum factor of the moving exponential average.

8. The semi-supervised remote sensing image change detection method based on adaptive feature perturbation according to claim 2, characterized in that The pair of dual-temporal images collected in step 1 is also divided into a validation set and a test set; It further includes step 10 of validating, testing, and saving the student model with updated weights based on the validation set and the test set; specifically including: Step 1001: After the teacher model and the student model are trained with the current batch of weakly augmented sample sets and strongly augmented sample sets, randomly select another weakly augmented sample set and strongly augmented sample set to train the teacher model and the student model through steps 5 - 9; Step 1002: Evaluate the student model after each batch of training using the mean intersection over union (mIoU), and save the weights of the student model with the highest mIoU; Step 1003: Test the saved student model using the test set, calculate the mIoU, and visualize the predicted change probability to obtain the change detection result of the pair of dual-temporal images.

Images