A semi-supervised remote sensing image change detection method and system

Abstract

The present application discloses a semi-supervised remote sensing image change detection method and system, which relates to the field of image processing technology. The method first introduces channel-level data enhancement technology to improve the strong enhancement effect and improve consistency regularization, thereby solving the problem of insufficient feature representation. Secondly, an adaptive threshold is used to dynamically adjust the threshold during the training process to balance the quality and quantity of pseudo-labels, thereby optimizing the training process. Finally, an adaptive category weight mechanism is proposed to alleviate the impact of category imbalance and effectively enhance the model's learning ability for change categories. Through the above three strategies, the utilization rate of the model for unlabeled data sets is improved, and the change detection performance of the model is further enhanced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a semi-supervised remote sensing image change detection method and system. Background Art

[0002] Remote sensing image change detection specifically refers to the qualitative analysis of dual-time remote sensing images taken at different times for the same surface environment to determine whether the area has changed. With the continuous improvement of satellite sensors, the resolution of remote sensing images has also been continuously improved. High-resolution remote sensing images usually have higher spatial resolution and can provide more information about surface features and spatial distribution, which helps to detect more subtle changes, such as the reconstruction of small buildings.

[0003] At present, a large number of studies focus on how to effectively extract the change area of ​​dual-temporal high-resolution remote sensing images. With the continuous improvement of image segmentation technology, the effectiveness and accuracy of fully supervised change detection models are also constantly improving. Now, the mainstream change detection methods still rely on a large amount of reliable labeled data to train the model. However, there is a challenging problem in this type of method, that is, it takes a lot of manpower and material resources to label pixel-level change detection labels. Especially in the target change detection in the military field, it is very difficult to obtain a large amount of accurate pixel-level labeled data. In order to solve the problem of difficulty in obtaining pixel-level labeled data and high cost, researchers are increasingly paying attention to semi-supervised change detection methods using a small amount of labeled data, weakly supervised change detection methods using images and labels, and unsupervised change detection methods that do not use labeled data. Unsupervised change detection methods are mainly divided into two categories. One is to use a pre-trained model trained with remote sensing images for scene classification, and then use it as a feature extractor for change detection. The other is a combination of classic unsupervised algorithms and supervised learning change detection frameworks. This method relies heavily on prior knowledge, which limits its scope of application. Weakly supervised change detection uses coarse-grained image-level labels for training, which effectively alleviates the problem of high cost of manual labeling. However, the quality of weakly supervised labels still seriously affects the accuracy of the model. In contrast, semi-supervised change detection methods use a small amount of labeled data, which not only reduces the cost of annotating labeled data, but also achieves better change detection results.

[0004] At present, the core idea of ​​semi-supervised change detection is to use a large amount of unlabeled data to train the model. It can be roughly divided into the following three methods: (1) Methods based on adversarial learning: It mainly uses a generative adversarial network to obtain the data distribution of the generator and uses a discriminator to distinguish different images. (2) Methods based on consistency regularization: By applying different forms of perturbations to the data, the prediction results of the model on different enhanced versions of the input data are constrained. (3) Pseudo-label methods: Use the model to predict pseudo-label data, then select pseudo-labels with confidence, mix the pseudo-labels with the true labels, and train the model.

[0005] Although current semi-supervised change detection methods have performed well in reducing data annotation costs, they still face some challenges. First, methods based on adversarial learning have the problem of unstable training, which makes it difficult for the model to achieve the desired effect. Secondly, methods based on consistency regularization are usually affected by the degree and method of perturbations, resulting in limited robustness of the model. In addition, the method of pseudo-label screening has a significant impact on model performance. Generally, a fixed threshold is set to determine the predicted value of each pixel, but this approach may cause some problems. If the threshold is high, the pseudo-label will only contain part of the information of the original remote sensing image, thereby affecting the feature integrity of the changed object; if the threshold is low, a large number of incorrect labels will be introduced, further reducing model performance. Summary of the invention

[0006] The embodiments of the present application provide a semi-supervised remote sensing image change detection method and system to solve the problem of low model performance in the prior art pseudo-labeling method using a fixed threshold.

[0007] On the one hand, the embodiment of the present application provides a semi-supervised remote sensing image change detection method, including:

[0008] Acquire a remote sensing image dataset consisting of dual-temporal remote sensing images;

[0009] The remote sensing image dataset is divided into a labeled dataset and an unlabeled dataset, the labeled dataset is weakly enhanced to obtain a labeled weakly enhanced dataset, the unlabeled dataset is copied into two copies, the first unlabeled dataset is weakly enhanced to obtain an unlabeled weakly enhanced dataset, and the second unlabeled dataset is weakly enhanced and strongly enhanced in sequence to obtain an unlabeled continuously enhanced dataset;

[0010] The labeled weak enhancement dataset, the unlabeled weak enhancement dataset, and the unlabeled continuous enhancement dataset are respectively input into the encoder-decoder network to obtain a change probability prediction map, wherein the change probability prediction map includes a labeled weak enhancement change probability prediction map corresponding to the labeled weak enhancement dataset, an unlabeled weak enhancement change probability prediction map corresponding to the unlabeled weak enhancement dataset, and an unlabeled continuous enhancement change probability prediction map corresponding to the unlabeled continuous enhancement dataset;

[0011] The labeled loss function is established by using the loss between the labeled weak enhancement change probability prediction map and the corresponding label;

[0012] The unlabeled weak enhancement change probability prediction map is used to update the adaptive threshold, and the corresponding pseudo-label is generated according to the type corresponding to the maximum probability in the unlabeled weak enhancement change probability prediction map. The unlabeled loss function is established by using the loss between the unlabeled continuous enhancement change probability prediction map and the pseudo-label. The unlabeled loss function is updated according to the updated adaptive threshold to obtain an updated unlabeled loss function.

[0013] The encoder-decoder network is trained using a labeled loss function and an updated unlabeled loss function to obtain a trained encoder-decoder network;

[0014] Obtain the remote sensing image to be detected, input the remote sensing image to be detected into the trained encoder-decoder network, and obtain the change detection result.

[0015] On the other hand, the embodiment of the present application also provides a semi-supervised remote sensing image change detection system, including:

[0016] A data set acquisition module is used to acquire a remote sensing image data set consisting of dual-time remote sensing images;

[0017] The data set enhancement module is used to divide the remote sensing image data set into a labeled data set and an unlabeled data set, perform weak enhancement processing on the labeled data set to obtain a labeled weakly enhanced data set, copy the unlabeled data set into two copies, perform weak enhancement processing on the first unlabeled data set to obtain an unlabeled weakly enhanced data set, and perform weak enhancement processing and strong enhancement processing on the second unlabeled data set in sequence to obtain an unlabeled continuously enhanced data set;

[0018] An encoder-decoder network is used to process the labeled weak enhancement dataset, the unlabeled weak enhancement dataset, and the unlabeled continuous enhancement dataset respectively to obtain a change probability prediction map, wherein the change probability prediction map includes a labeled weak enhancement change probability prediction map corresponding to the labeled weak enhancement dataset, an unlabeled weak enhancement change probability prediction map corresponding to the unlabeled weak enhancement dataset, and an unlabeled continuous enhancement change probability prediction map corresponding to the unlabeled continuous enhancement dataset;

[0019] A loss function establishment module is used to establish a labeled loss function using the loss between a labeled weak enhancement change probability prediction map and a corresponding label; it is also used to update an adaptive threshold using an unlabeled weak enhancement change probability prediction map, and generate a corresponding pseudo-label according to the type corresponding to the maximum probability in the unlabeled weak enhancement change probability prediction map, establish an unlabeled loss function using the loss between the unlabeled continuous enhancement change probability prediction map and the pseudo-label, and update the unlabeled loss function according to the updated adaptive threshold to obtain an updated unlabeled loss function;

[0020] A network training module is used to train the encoder-decoder network using a labeled loss function and an updated unlabeled loss function to obtain a trained encoder-decoder network;

[0021] The change detection module is used to obtain the remote sensing image to be detected, input the remote sensing image to be detected into the trained encoder-decoder network, and obtain the change detection result.

[0022] A semi-supervised remote sensing image change detection method and system in this application firstly introduces channel-level data augmentation technology to improve the strong enhancement effect and improve consistency regularization, thereby solving the problem of insufficient feature representation. Secondly, an adaptive threshold is used to dynamically adjust the threshold during the training process to balance the quality and quantity of pseudo-labels, thereby optimizing the training process. Finally, an adaptive category weight mechanism is proposed to alleviate the impact of category imbalance and effectively enhance the model's learning ability for change categories. Through the above three strategies, the utilization rate of the model for unlabeled data sets is improved, and the change detection performance of the model is further enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 An architectural diagram of a semi-supervised remote sensing image change detection method provided in an embodiment of the present application.

[0025] Figure 2 This is the original image before weak enhancement processing provided in the embodiment of the present application.

[0026] Figure 3 for Figure 2 The original image is the image after weak enhancement.

[0027] Figure 4 for Figure 3 The image is the image after strong enhancement processing.

[0028] Figure 5The results of change detection using different methods for remote sensing images at different times. (a) and (b) are remote sensing images at different times, respectively, (c) is the change detection result obtained by using only the supervised learning algorithm, (d) is the change detection result obtained by using the S4GAN algorithm, (e) is the change detection result obtained by using the SemiCDNet algorithm, (f) is the change detection result obtained by using the SemiCD algorithm, (g) is the change detection result obtained by using the RCL algorithm, (h) is the change detection result obtained by using the FPA algorithm, and (i) is the change detection result obtained by using the method of this application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] Figure 1 The present invention provides a semi-supervised remote sensing image change detection method, comprising:

[0031] S100, obtaining a remote sensing image dataset consisting of dual-time remote sensing images.

[0032] For example, the remote sensing image dataset used in the embodiment of the present application is the WHU-CD (Wuhan University Change Detect) dataset. The dataset is segmented, and the remote sensing images therein are cropped into 256×256 size pictures, and there is no overlap between each picture, a total of 7434 pairs of images are obtained, and the dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0033] S110, dividing the remote sensing image dataset into a labeled dataset and an unlabeled dataset, performing weak enhancement processing on the labeled dataset to obtain a labeled weakly enhanced dataset, copying the unlabeled dataset into two copies, performing weak enhancement processing on the first unlabeled dataset to obtain an unlabeled weakly enhanced dataset, and sequentially performing weak enhancement processing and strong enhancement processing on the second unlabeled dataset to obtain an unlabeled continuously enhanced dataset.

[0034] For example, after obtaining the training set, 95% of the remote sensing images in the training set are divided into unlabeled data sets. , and the remaining 5% of remote sensing images are used as labeled datasets The labeled dataset can be represented as ,in Representative i For dual-time remote sensing images, represents the remote sensing image at time T0, represents the remote sensing image at time T1, Represents the binary label of each pixel, that is, whether the pixel is changed or unchanged, n represents the size of the labeled dataset. The unlabeled dataset can be represented as ,in Representative j For dual-time remote sensing images, N Represents the size of the unlabeled dataset.

[0035] In the process of enhancement, unlabeled datasets It is copied into two copies, the first copy is only weakly enhanced, and the second copy needs to be strongly enhanced on the basis of weak enhancement. , the embodiment of the present application obtains a labeled weak enhancement dataset through weak enhancement , for unlabeled datasets , after weak enhancement and strong enhancement, we get unlabeled weak enhancement datasets and unlabeled continuous augmentation datasets . Note that data augmentation is only applied to the training set, not the validation set and the test set.

[0036] Weak enhancement includes three enhancement operations performed in sequence, namely scaling, cropping, and horizontal flipping. The image scaling ratio is [0.5, 2.0], the cropping size is randomly generated, and the probability of horizontal flipping is 50%.

[0037] During strong enhancement, the RGB (Red Green Blue) channels of the image are randomly quantized. Specifically, during random quantization, the data of each channel is divided into multiple intervals, such as 8 intervals, and then the original data of each interval is mapped to a randomly sampled value in the interval, so that each interval of the image can have a random quantization value each time, which will increase the diversity of the data.

[0038] like Figure 2-4 As shown, Figure 2 It's the original picture. Figure 3 It is a weak enhancement result after being enlarged, cropped and horizontally flipped. Figure 4 It is a strong enhancement result obtained by performing random quantization operation again on the basis of weak enhancement.

[0039] S120, respectively inputting the labeled weak enhancement dataset, the unlabeled weak enhancement dataset, and the unlabeled continuous enhancement dataset into the encoder-decoder network to obtain a change probability prediction map, wherein the change probability prediction map includes a labeled weak enhancement change probability prediction map corresponding to the labeled weak enhancement dataset, an unlabeled weak enhancement change probability prediction map corresponding to the unlabeled weak enhancement dataset, and an unlabeled continuous enhancement change probability prediction map corresponding to the unlabeled continuous enhancement dataset.

[0040] Exemplarily, the embodiment of the present application uses two encoder-decoder networks to process the data set, wherein the first encoder-decoder network processes the labeled weak enhancement data set, and the other encoder-decoder network processes the unlabeled weak enhancement data set and the unlabeled continuous enhancement data set in sequence. Each encoder-decoder network includes encoders connected in sequence , pyramid pooling module and decoder , where there are two encoders with the same structure. The two encoders are responsible for extracting features from a pair of enhanced remote sensing images in the data set, while the decoder processes the differential features to obtain a change probability prediction map.

[0041] For a labeled weakly enhanced dataset, the processing process of the encoder E is: a pair of images in the dataset As input, they are input into two encoders respectively. and The same size as , and obtain a pair of high-dimensional feature maps of the same size and .in, represents a three-dimensional eigenvector tensor, H / s Indicates the height of the input image H Spatial scale factor s The result after downsampling is W / s Indicates the width of the input image W Spatial scale factor s The result after downsampling is C represents the number of feature space channels, C and s They are set to 2048 and 8 respectively.

[0042] Pyramid Pooling Module (PPM): In order to obtain the feature difference between the two images, use and Perform differential operation, then extract features from the result of differential operation to obtain feature maps captured at different scales .

[0043] Decoder D: The decoder consists of a series of upsampling convolution modules, through which the captured feature map is upsampled and convolved. Transformed into a labeled weak enhancement change probability prediction map , where 2 represents the two categories of change and no change. Next, the softmax function is used to weakly enhance the change probability prediction graph with labels Scaled to [0,1] along the category dimension.

[0044] The calculation formula for the overall network process is as follows:

[0045]

[0046]

[0047]

[0048]

[0049]

[0050] in, E (·) represents the feature extraction operation of the encoder, PPM (·) represents the operation of the pyramid pooling module to capture feature maps at different scales. Represents high-dimensional feature maps and Perform a differential operation, D (·) represents the operation of the decoder to obtain the labeled weak enhancement change probability prediction map, Indicates the probability of label weak enhancement change prediction graph u Line v The predicted probability of the column pixels, represents the scaled predicted probability, softmax (·) indicates processing by the softmax function.

[0051] For the unlabeled weak enhancement dataset and the unlabeled continuous enhancement dataset, the input image and After the same processing flow as the labeled weak enhancement dataset, we obtain the unlabeled weak enhancement change probability prediction graphs. And unlabeled continuous enhancement change probability prediction map .

[0052] Note that the two encoder-decoder networks process the dataset simultaneously.

[0053] S130, establishing a labeled loss function using the loss between the labeled weak enhancement change probability prediction map and the corresponding labels.

[0054] Exemplarily, the training of a labeled weak enhancement dataset is called supervised training, while the training of an unlabeled weak enhancement dataset and an unlabeled continuous enhancement dataset is called unsupervised training, and the two encoder-decoder networks share weight data.

[0055] During supervised training, the cross entropy loss function is used to minimize the labeled weakly enhanced change probability prediction map. and tags The loss between is expressed as:

[0056]

[0057] in, L s Represents a label loss function, CE (·) represents the cross entropy loss, Indicates u Line v Binary labels for columns of pixels.

[0058] S140, using the unlabeled weak enhancement change probability prediction map to update the adaptive threshold, and generating a corresponding pseudo-label according to the type corresponding to the maximum probability in the unlabeled weak enhancement change probability prediction map, using the loss between the unlabeled continuous enhancement change probability prediction map and the pseudo-label to establish an unlabeled loss function, and updating the unlabeled loss function according to the updated adaptive threshold to obtain an updated unlabeled loss function.

[0059] For example, in the unsupervised training process, the unlabeled weakly enhanced change probability prediction map is used Update the adaptive threshold, and binarize the unlabeled weak enhancement change probability prediction map according to the type corresponding to the maximum probability in the unlabeled weak enhancement change probability prediction map, and then generate the corresponding pseudo label .

[0060] Minimize the unlabeled continuous enhancement change probability prediction map using the consistency loss function and pseudo labels The loss between is expressed as:

[0061]

[0062] in, L u represents the unlabeled loss function, where represents the predefined confidence threshold for filtering noise labels, Indicates that the predicted probability is greater than is considered as a high-quality pseudo label and recorded as 1; otherwise, it is set to 0. It represents the first u Line v The predicted probability of the column pixels, max(·) means taking the maximum value, It represents the first u Line v The predicted probability of the column pixels, Indicates the number of unlabeled weak enhancement change probability prediction graphs. u Line v Pseudo labels for columns of pixels.

[0063] The method for updating the adaptive threshold using the unlabeled weak enhancement change probability prediction map includes the following steps:

[0064] The first step is to predict the probability of unlabeled weak enhancement change Calculate the maximum predicted probability for each pixel, and then for each category c The probabilities are averaged to obtain the local prediction confidence :

[0065]

[0066] in, B represents the batch size, Representative k For the image, the position ( u , v ) belongs to c The maximum predicted probability of a class.

[0067] The second step is to calculate c The proportion of pixels corresponding to the maximum predicted probability of the class , and then use this ratio to calculate c Opposite weights of classes . It is expressed as:

[0068]

[0069]

[0070] in, express c The number of class pixels, Indicates k For the image ( u , v ) The category corresponding to the maximum predicted probability of the pixel at the position c .

[0071] The third step is to set the adaptive threshold It is estimated as the exponential moving average (EMA) of each round of confidence. Its initial value is 1 / C’ ,in C’ Indicates the number of classes.

[0072]

[0073] Here is the momentum decay of EMA, ranging from (0,1), and is set to 0.9999 in the experiment. For binary change detection, C’ The value is 2, which is the initial threshold is set to 0.5.

[0074] After updating the adaptive threshold, the confidence mask corresponding to the high-quality and large-quantity pseudo-labels obtained can be expressed as :

[0075]

[0076] right c Resample the number of categories to get c The corresponding resampling rate of the category :

[0077]

[0078] Therefore, the resampling rate of each pixel can be To adjust the weight of the loss function. The weight map of the pixel point can be expressed as :

[0079]

[0080] Here express( u , v ) The category corresponding to the maximum predicted probability of the pixel at the position c .

[0081] According to the above steps, update the unlabeled loss function Can be updated to:

[0082]

[0083] S150, training the encoder-decoder network using the labeled loss function and updating the unlabeled loss function to obtain a trained encoder-decoder network.

[0084] For example, the total loss of the entire network is LIt consists of two parts and can be expressed as:

[0085]

[0086] Using labeled loss and update the unlabeled loss The encoder-decoder network is optimized to obtain the final network model. After the training is completed, the validation set and test set are respectively input into the trained encoder-decoder network to verify and test the trained encoder-decoder network.

[0087] S160, obtaining a remote sensing image to be detected, and inputting the remote sensing image to be detected into a trained encoder-decoder network to obtain a change detection result.

[0088] Exemplarily, the remote sensing image to be detected should also be a dual-time remote sensing image. Before change detection, the remote sensing image to be detected does not need to be processed with weak enhancement and strong enhancement, and can be directly input into the trained encoder-decoder network to obtain the final change detection result.

[0089] The present application also provides a system for semi-supervised remote sensing image change detection method, the system comprising:

[0090] A data set acquisition module is used to acquire a remote sensing image data set consisting of dual-time remote sensing images;

[0091] The data set enhancement module is used to divide the remote sensing image data set into a labeled data set and an unlabeled data set, perform weak enhancement processing on the labeled data set to obtain a labeled weakly enhanced data set, copy the unlabeled data set into two copies, perform weak enhancement processing on the first unlabeled data set to obtain an unlabeled weakly enhanced data set, and perform weak enhancement processing and strong enhancement processing on the second unlabeled data set in sequence to obtain an unlabeled continuously enhanced data set;

[0092] An encoder-decoder network is used to process the labeled weak enhancement dataset, the unlabeled weak enhancement dataset, and the unlabeled continuous enhancement dataset respectively to obtain a change probability prediction map, wherein the change probability prediction map includes a labeled weak enhancement change probability prediction map corresponding to the labeled weak enhancement dataset, an unlabeled weak enhancement change probability prediction map corresponding to the unlabeled weak enhancement dataset, and an unlabeled continuous enhancement change probability prediction map corresponding to the unlabeled continuous enhancement dataset;

[0093] A loss function establishment module is used to establish a labeled loss function using the loss between a labeled weak enhancement change probability prediction map and a corresponding label; it is also used to update an adaptive threshold using an unlabeled weak enhancement change probability prediction map, and generate a corresponding pseudo-label according to the type corresponding to the maximum probability in the unlabeled weak enhancement change probability prediction map, establish an unlabeled loss function using the loss between the unlabeled continuous enhancement change probability prediction map and the pseudo-label, and update the unlabeled loss function according to the updated adaptive threshold to obtain an updated unlabeled loss function;

[0094] A network training module is used to train the encoder-decoder network using a labeled loss function and an updated unlabeled loss function to obtain a trained encoder-decoder network;

[0095] The change detection module is used to obtain the remote sensing image to be detected, input the remote sensing image to be detected into the trained encoder-decoder network, and obtain the change detection result.

[0096] Simulation experiment

[0097] 1. Simulation conditions:

[0098] A single NVIDIA GeForce GTX 3080Ti GPU with 12G memory is used for training and testing. The batch size is set to 4 in the experiment, and the dataset uses the WHU-CD dataset.

[0099] 2. Evaluation indicators:

[0100] In order to accurately evaluate the performance of the proposed semi-supervised change detection method, five indicators, IoU, F1, Kappa, TPR (True Positive Ratio) and TNR (True Negative Ratio), are selected. The detailed explanation is as follows:

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109]

[0110] In the above formula, TP, TN, FP and FN represent the number of correctly predicted changed pixels, the number of correctly predicted unchanged pixels, the number of unchanged pixels incorrectly predicted as changed pixels, and the number of changed pixels incorrectly predicted as unchanged pixels, respectively. P , R , OA and P e IoU can reflect the overlap between the predicted area and the real area. Since the difficulty of predicting changed pixels is much higher than that of predicting unchanged pixels in change detection, this application further uses the IoU of changed pixels. c to replace the global IoU. F1 evaluates the overall performance of the model. Kappa evaluates the consistency between the model and the actual observations. TPR evaluates the model's ability to predict changed pixels, i.e. sensitivity. TNR evaluates the model's ability to predict unchanged pixels, i.e. specificity. The higher the above five indicators, the better the performance of the model.

[0111] 3. Simulation content:

[0112] In order to prove the effectiveness of the method in this application, five popular methods in recent years were used in the experiment for comparison. At the same time, the effects of using only a small number of labels for supervised training (Only-sup) and using all data for supervised training (Fully-sup) were also compared. In order to further illustrate the utilization rate of the method in this application for unlabeled data sets, the experimental results under the ratio of 1%, 5%, 10% and 20% labeled data sets were compared. As shown in Table 1, Table 2, Table 3 and Table 4. Among them:

[0113] 1) S4GAN (Semi-supervised semantic segmentation Generative Adversarial Networks) is introduced in detail in "S. Mittal, M. Tatarchenko, and T.Brox, Semi-supervised semantic segmentation with high-and low-level consistency, IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 4, pp. 1369–1379, 2019."

[0114] 2) SemiCDNet (Semi-supervised Change Detection Networks) is introduced in detail in "D. Peng, L. Bruzzone, Y. Zhang, H. Guan, H. Ding, and X. Huang, Semicdnet: A semisupervised convolutional neural network for change detection in high resolution remote-sensing images, IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 7, pp. 5891–5906, 2020."

[0115] 3) SemiCD (Semi-supervised Change Detection) is introduced in detail in "WGCBandara and VM Patel, Revisiting consistency regularization for semi-supervised change detection in remote sensing images, arXiv preprint arXiv:2204.08454, 2022."

[0116] 4) RCL (Reliable Contrastive Learning) is introduced in detail in "J.-X. Wang, T.Li, S.-B. Chen, J. Tang, B. Luo, and RC Wilson, Reliable contrastive learning for semi-supervised change detection in remote sensing images, IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–13, 2022."

[0117] 5) FPA (Feature-Prediction Alignment) is introduced in detail in "X. Zhang, X. Huang, and J. Li, Semisupervised change detection with feature-prediction alignment, IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–16, 2023."

[0118] Table 1 Results of different methods on the WHU-CD test set with 1% labeled dataset

[0119]

[0120] Table 2 Results of different methods on the WHU-CD test set with 5% labeled dataset

[0121]

[0122] Table 3 Results of different methods on the WHU-CD test set with 10% labeled dataset

[0123]

[0124] Table 4 Results of different methods on the WHU-CD test set with 20% labeled dataset

[0125]

[0126] From Tables 1 to 4, we can see the effectiveness of the method in this application under different proportions of labeled datasets, which has been verified on the WHU-CD dataset. The experimental results show that compared with other existing methods, the method in this application improves the performance of the semi-supervised change detection model. Figure 5 It can also be seen that the method of the present application has better integrity for detecting small-scale targets.

[0127] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0128] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A semi-supervised remote sensing image change detection method, characterized in that: include: Acquire a remote sensing image dataset consisting of dual-temporal remote sensing images; The remote sensing image dataset is divided into a labeled dataset and an unlabeled dataset, the labeled dataset is weakly enhanced to obtain a labeled weakly enhanced dataset, the unlabeled dataset is copied into two copies, the first copy of the unlabeled dataset is weakly enhanced to obtain an unlabeled weakly enhanced dataset, and the second copy of the unlabeled dataset is weakly enhanced and strongly enhanced in sequence to obtain an unlabeled continuously enhanced dataset; Inputting the labeled weak enhancement dataset, the unlabeled weak enhancement dataset, and the unlabeled continuous enhancement dataset into an encoder-decoder network respectively to obtain a change probability prediction map, wherein the change probability prediction map includes a labeled weak enhancement change probability prediction map corresponding to the labeled weak enhancement dataset, an unlabeled weak enhancement change probability prediction map corresponding to the unlabeled weak enhancement dataset, and an unlabeled continuous enhancement change probability prediction map corresponding to the unlabeled continuous enhancement dataset; Establishing a labeled loss function using the loss between the labeled weak enhancement change probability prediction map and the corresponding label; The unlabeled weak enhancement change probability prediction map is used to update the adaptive threshold, and a corresponding pseudo label is generated according to the type corresponding to the maximum probability in the unlabeled weak enhancement change probability prediction map, and an unlabeled loss function is established by using the loss between the unlabeled continuous enhancement change probability prediction map and the pseudo label, and the unlabeled loss function is updated according to the updated adaptive threshold to obtain an updated unlabeled loss function; Training the encoder-decoder network using the labeled loss function and the updated unlabeled loss function to obtain the trained encoder-decoder network; Acquire a remote sensing image to be detected, and input the remote sensing image to be detected into the trained encoder-decoder network to obtain a change detection result; The method of updating the adaptive threshold using the unlabeled weak enhancement change probability prediction map includes: Determine the maximum prediction probability of each pixel in the unlabeled weak enhancement change probability prediction map, and determine the local prediction confidence according to the maximum prediction probability; Determine the proportion of pixels of each category in the unlabeled weak enhancement change probability prediction map, and determine the corresponding opposite weight according to the proportion; Determining the updated adaptive threshold according to the local prediction confidence and the opposite weight; After updating the adaptive threshold, determining a confidence mask according to a magnitude relationship between the updated adaptive threshold and a maximum prediction probability of each pixel in the unlabeled weak enhancement change probability prediction map; After determining the ratio, determining a resampling rate for each category according to the ratio, and then determining a weight map for each pixel according to the resampling rate; After obtaining the weight map, the unlabeled loss function is updated according to the weight map and the confidence mask.

2. A semi-supervised remote sensing image change detection method according to claim 1, characterized in that: The encoder-decoder network includes an encoder, a pyramid pooling module and a decoder connected in sequence. The encoder performs high-dimensional feature extraction on a pair of remote sensing images in an input data set to obtain a high-dimensional feature map. The pyramid pooling module performs differential operation and feature extraction on each pair of the high-dimensional feature maps in sequence to obtain captured feature maps at different scales. The decoder performs upsampling convolution on the captured feature map to obtain a change probability prediction map.

3. The semi-supervised remote sensing image change detection method according to claim 1, characterized in that: When performing weak enhancement processing, the remote sensing images in the input data set are scaled, cropped and horizontally flipped in sequence. When performing strong enhancement processing, the RGB three channels of the input image are randomly quantized.

4. A semi-supervised remote sensing image change detection method according to claim 3, characterized in that: In the random quantization process, the data of each channel in the image is divided into a plurality of intervals, and the original data in each interval is mapped to a randomly sampled value in the interval.

5. The semi-supervised remote sensing image change detection method according to claim 1, characterized in that: After obtaining the change probability prediction graph, the change probability in the change probability prediction graph is scaled to between [0, 1] along the category dimension using a softmax function.

6. The semi-supervised remote sensing image change detection method according to claim 1, characterized in that: The labeled loss function is established based on the cross entropy loss function, and the unlabeled loss function is established based on the consistency loss function.

7. The semi-supervised remote sensing image change detection method according to claim 1, characterized in that: When training the encoder-decoder network, the remote sensing image data set is divided into a training set, a validation set and a test set. After the training set is subjected to weak enhancement processing and strong enhancement processing, it is input into the encoder-decoder network to train the encoder-decoder network. After the trained encoder-decoder network is obtained, the validation set and the test set are respectively input into the trained encoder-decoder network to verify and test the trained encoder-decoder network.

8. A system using a semi-supervised remote sensing image change detection method according to any one of claims 1 to 7, characterized in that: include: A data set acquisition module is used to acquire a remote sensing image data set consisting of dual-time remote sensing images; The data set enhancement module is used to divide the remote sensing image data set into a labeled data set and an unlabeled data set, perform weak enhancement processing on the labeled data set to obtain a labeled weakly enhanced data set, copy the unlabeled data set into two copies, perform weak enhancement processing on the first copy of the unlabeled data set to obtain an unlabeled weakly enhanced data set, and perform weak enhancement processing and strong enhancement processing on the second copy of the unlabeled data set in sequence to obtain an unlabeled continuously enhanced data set; An encoder-decoder network is used to process the labeled weak enhancement dataset, the unlabeled weak enhancement dataset, and the unlabeled continuous enhancement dataset respectively to obtain a change probability prediction map, wherein the change probability prediction map includes a labeled weak enhancement change probability prediction map corresponding to the labeled weak enhancement dataset, an unlabeled weak enhancement change probability prediction map corresponding to the unlabeled weak enhancement dataset, and an unlabeled continuous enhancement change probability prediction map corresponding to the unlabeled continuous enhancement dataset; A loss function establishment module is used to establish a labeled loss function using the loss between the labeled weak enhancement change probability prediction map and the corresponding label; it is also used to update the adaptive threshold using the unlabeled weak enhancement change probability prediction map, and generate a corresponding pseudo label according to the type corresponding to the maximum probability in the unlabeled weak enhancement change probability prediction map, establish an unlabeled loss function using the loss between the unlabeled continuous enhancement change probability prediction map and the pseudo label, and update the unlabeled loss function according to the updated adaptive threshold to obtain an updated unlabeled loss function; A network training module, used for training the encoder-decoder network using the labeled loss function and the updated unlabeled loss function to obtain the trained encoder-decoder network; The change detection module is used to obtain the remote sensing image to be detected, input the remote sensing image to be detected into the trained encoder-decoder network, and obtain the change detection result.

Images