Semi-supervised change detection method for ultra-high resolution remote sensing images based on improving pseudo-label boundary quality

Abstract

This invention discloses a semi-supervised change detection method for ultra-high resolution remote sensing images based on improving the quality of pseudo-label boundaries. Change detection aims to discover the regions of change between the later and earlier times in dual-time images. This invention enhances the guidance of semi-supervised learning through high-quality pseudo-labels and includes the following steps: 1. Prepare a semi-supervised learning training dataset; 2. Build a teacher model and a student model; 3. Training begins, the student model infers from the training dataset data, and the teacher model generates corresponding coarse pseudo-labels; 4. Use boundary optimization methods to refine the coarse pseudo-labels, and calculate the consistency loss of the labels and the inference result loss of the student model; 5. Calculate the total loss, and update the parameters of the student and teacher models; 6. Training ends, and the student model performs change detection on the dual-time remote sensing images. This invention improves the performance of semi-supervised change detection, which is limited by coarse pseudo-label boundaries and cannot achieve further improvement.

Description

Technical Field

[0001] This invention belongs to the field of computer vision application technology, specifically relating to a semi-supervised change detection method for ultra-high resolution remote sensing images based on improving the quality of pseudo-label boundaries. Background Technology

[0002] Remote sensing image change detection plays a crucial role in Earth observation and environmental monitoring. With the continuous development of remote sensing sensors and advancements in satellite technology, increasingly richer ultra-high-resolution remote sensing images are becoming available. This data holds immense potential in fields such as agriculture, urban planning, and natural resource management.

[0003] With the development of deep learning, mainstream change detection methods have demonstrated significant performance using fully supervised deep neural networks. However, accurate change detection using deep neural networks typically requires a large amount of labeled data for training. Since image annotation for change detection requires manual registration and detailed annotation, the process of annotating change detection datasets is extremely expensive and time-consuming.

[0004] In early semi-supervised change detection research, the mainstream approaches were virtual adversarial training and generative adversarial networks (GANs). These methods trained unlabeled images by applying random perturbations or generating corresponding adversarial examples. Subsequently, semi-supervised change detection methods based on teacher-student models also made some progress. However, these generative-based methods lacked the stability of generating reliable training data. Furthermore, for teacher-student models, the parameters of both the teacher and student models tended to converge as training progressed, thus limiting the student model's ability to learn new knowledge. In recent years, some consistency regularization methods have been introduced into semi-supervised change detection. This approach forces consistency between strong and weak views. Some methods evaluate the quality of pseudo-labels and propose pseudo-label selection schemes. However, these methods have not fundamentally improved the quality of pseudo-labels, which is crucial for semi-supervised change detection.

[0005] Through research and experiments on semi-supervised change detection methods, it was found that current advanced semi-supervised change detection methods mainly follow consistency regularization. These methods can achieve relatively complete predictions in the central region of pseudo-label contours, but still struggle to align with the actual target at the boundary. Therefore, this invention studies a semi-supervised change detection method based on a teacher-student model. Fully utilizing the boundary knowledge of unlabeled data to generate change detection prediction results with refined boundaries plays an important role in land planning and resource utilization. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention aims to propose a semi-supervised change detection method for ultra-high resolution remote sensing images based on improving the quality of pseudo-label boundaries. This invention preserves the boundary knowledge of remote sensing dual-time images, stabilizes the semi-supervised learning state, and improves the semi-supervised change detection performance that is previously limited by coarse pseudo-label boundaries.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A semi-supervised change detection method for ultra-high resolution remote sensing images based on improving the quality of pseudo-label boundaries includes the following steps:

[0009] Step A: Divide the remote sensing dual-time image dataset into a labeled dataset and an unlabeled dataset; apply weak data augmentation to the labeled dataset, and apply weak data augmentation followed by strong data augmentation and feature augmentation to the unlabeled dataset; concatenate the labeled dataset and the unlabeled dataset to obtain the semi-supervised learning training dataset.

[0010] Step B: Build the teacher model and student model, and initialize the parameters of the teacher model and student model;

[0011] Step C: Training begins. The student model obtained in Step B is used to perform inference on the labeled weakly augmented data, unlabeled strongly augmented data, and unlabeled feature-augmented data in the training dataset generated in Step A. The teacher model obtained in Step B is used to perform inference on the unlabeled weakly augmented data in the training dataset generated in Step A to generate coarse pseudo-labels.

[0012] Step D: Calculate the labeled supervised loss for the inference results of the student model obtained in Step C on labeled weakly augmented data and the corresponding labels in the training dataset generated in Step A; refine the coarse pseudo-labels obtained in Step C using a boundary optimization method designed for remote sensing dual-time image change detection, first calculating the consistency loss between the refined pseudo-labels and the coarse pseudo-labels; then calculate the semi-supervised loss by comparing the refined pseudo-labels with the inference results of the student model obtained in Step C on unlabeled strongly augmented and unlabeled feature augmented data.

[0013] Step E: Calculate the total loss using the labeled supervised loss, consistency loss, and semi-supervised loss obtained in Step D; update the student model parameters using stochastic gradient descent; calculate the stability of the student model and update the teacher model parameters.

[0014] Step F: Training ends. The student model updated with the parameters obtained in step E is used to perform inference on the remote sensing dual-time image to obtain the change detection results.

[0015] The specific steps of step A are as follows:

[0016] Step A01: Divide the remote sensing dual-time image dataset into four semi-supervised learning proportions: ① 5% labeled dataset, 95% unlabeled dataset; ② 10% labeled dataset, 90% unlabeled dataset; ③ 20% labeled dataset, 80% unlabeled dataset; ④ 40% labeled dataset, 60% unlabeled dataset.

[0017] Step A02: Set up a weak data augmentation pool, including scaling, cropping, and horizontal flipping; set up a strong data augmentation pool, including color injection, blurring, cropping and splicing, and grayscale adjustment; set feature augmentation to the Dropout strategy; perform weak data augmentation on the labeled dataset, and perform weak data augmentation on the unlabeled data first, followed by strong data augmentation and feature augmentation; splice the labeled and unlabeled datasets to obtain the semi-supervised learning training dataset.

[0018] The specific steps of step B are as follows:

[0019] Step B01: First, construct a change detection network, using ResNet50 and DeepLabv3+ as the encoder and decoder, respectively. Use ResNet50 to extract features from the remote sensing images at different times, and then feed these features into DeepLabv3+ for change detection.

[0020]

[0021] In the formula, A and B are the images of the previous and next time steps in the input remote sensing dual-time image to be tested, respectively, ε(·) is the constructed change detection network encoder, and (·) * The encoder extracts multidimensional features; the multidimensional features of the remote sensing dual-time images are subtracted and their absolute values ​​are taken, then fed into the decoder. In the process, predictions for change detection are obtained.

[0022] Step B02: Construct student and teacher models using the change detection network built in step B01. The student and teacher models use the same change detection network structure, but the parameters between the student and teacher models are not shared. Initialize the parameters of the teacher and student models using random initial values.

[0023] The specific steps of step C are as follows:

[0024] Step C01: Training begins. Using the student model initialized with parameters from Step B, inference is performed on labeled weakly augmented data, unlabeled strongly augmented data, and unlabeled feature-augmented data in the semi-supervised learning training dataset.

[0025]

[0026] In the formula, and , representing weak data augmentation, strong data augmentation and feature augmentation respectively; x1 and x2 are the previous and next time images in the input labeled remote sensing dual-time image, and u1 and u2 are the previous and next time images in the input unlabeled remote sensing dual-time image. Representing the student model, and These represent labeled prediction, unlabeled strongly augmented prediction, and feature augmented prediction for the student model, respectively.

[0027] Step C02: Using the teacher model initialized with parameters from step B, generate coarse pseudo-labels for inference on the unlabeled weakly augmented data in the semi-supervised learning training dataset:

[0028]

[0029] In the formula, Representing the teacher model, This indicates the generated coarse pseudo-tag.

[0030] The specific steps of step D are as follows:

[0031] Step D01: Calculate the loss between the student model's prediction results on labeled weakly augmented data obtained in Step C and the corresponding labels in the semi-supervised learning training dataset generated in Step A.

[0032]

[0033] In the formula, This indicates a loss due to label supervision. M and represent the labeled dataset and its size, respectively; CE represents the cross-entropy loss. and y i Let represent the student model's prediction for the i-th labeled weakly augmented data point and its corresponding label, respectively.

[0034] Step D02: Refine the coarse pseudo-labels generated in step C using a boundary optimization method designed for remote sensing image change detection: First, apply a 3×3 two-dimensional convolution to the coarse pseudo-label change map to extract the set of coordinates of boundary region points; then, use convolution operations to obtain the coordinates of these boundary points, expressed by the following formula:

[0035]

[0036] In the formula, x i and y i These are the x and y coordinates of the boundary points; is the mask corresponding to the i-th boundary block; K is a 3×3 convolution kernel, s is the convolution stride, p is the convolution padding; w represents the predicted width of the boundary; the operation conv is a two-dimensional convolution operation, and the operation where comes from NumPy;

[0037] The boundary points are expanded along the top left and bottom right corners to form 48×48 boundary blocks; these boundary blocks are distributed along the boundary to depict the changing parts and their surrounding environment, as expressed by the formula:

[0038]

[0039] In the formula, the generated It is an array that records the coordinates of the top left and bottom right corners of the boundary block; bs represents the block size, set to 48; t is used to represent the previous or next time step from the remote sensing dual-time image;

[0040] Based on the characteristics of boundary blocks, a series of boundary block filters were designed. First, an "overlapping" filter was designed to remove highly overlapping boundary blocks, thus reducing computational burden. Since convolution operations may cause boundary blocks to appear within the change map and located at the center of the change map, an "interior" filter was introduced to discard boundary blocks appearing within the change map. To ensure that boundary partitioning predictions are aligned with change detection predictions and to avoid completely incorrect classifications, a "distribution" filter was designed. For each boundary partitioning prediction, a mask-getting operation was performed to generate a mask corresponding to each boundary. Then, the intersection-union ratio (IU) with the original change map was calculated at the corresponding boundary block location. Boundary blocks with significant IU differences were considered unreliable and discarded. This "distribution" filter ensures that the distribution of boundary partitioning predictions is aligned with change detection predictions as much as possible.

[0041] Finally, the boundary delineation prediction and change detection prediction are integrated:

[0042]

[0043] In the formula, σ(·) is the sigmoid function, which is used to obtain the confidence level of the prediction; then, the change detection prediction is... Boundary division prediction Averaging is performed; the decision boundary is pushed towards the side with higher confidence; by applying argmax, the final pseudo-labels with refined boundaries are obtained. The consistency loss is calculated by comparing the refined pseudo-labels with the coarse pseudo-labels obtained in step C.

[0044]

[0045] In the formula, It is a loss of consistency; N and N represent the number of unlabeled datasets, respectively; This represents the j-th coarse pseudo-label generated, while Let represent the pseudo-label after refinement at the j-th boundary; then, perform semi-supervised loss calculation by combining the refined pseudo-label with the unlabeled strong augmentation prediction and the feature augmentation prediction:

[0046]

[0047] In the formula, It is a semi-supervised loss used to constrain the consistency between the student model's predictions for strong data augmentation and feature augmentation and the refined pseudo-labels; and Let represent the j-th unlabeled strong augmentation prediction and the j-th feature augmentation prediction, respectively.

[0048] Preferably, the predicted width w of the boundary is set to 3.

[0049] The specific steps of step E are as follows:

[0050] Step E01: Calculate the total loss using the obtained labeled supervised loss, consistency loss, and semi-supervised loss. Update the student model parameters using the loss calculation results and stochastic gradient descent.

[0051]

[0052] In the formula, The total loss is the sum of the constraint losses of each component.

[0053] Step E02: Calculate the stability of the student model and update the parameters of the teacher model; calculate the intersection-union ratio (IoU) using the student model's output for the unlabeled strongly augmented data and the boundary-refined mask of the unlabeled data:

[0054]

[0055] In the formula, This represents the student model's predicted value for the j-th unlabeled remote sensing two-time image change detection, while... express The mask after boundary refinement; the result after calculating argmax. and The average distance between them is used as a stability score The average distance is measured using IoU (Intersection over Union); if stability score An improvement is considered to indicate that the current student model has reached a more stable state, and its parameters are updated to the teacher model by copying the overall parameters.

[0056] The specific steps of step F are as follows:

[0057] Step F01: Training complete. Use the updated student model to perform change detection on the remote sensing dual-time image under test, and obtain the change detection prediction results:

[0058]

[0059] In the formula, For student models, The result is the change detection prediction; the binary map of change detection, i.e., the change map, is obtained through arg max operation.

[0060] Compared with the prior art, the present invention has the following advantages:

[0061] First, in the semi-supervised learning process, the method of this invention abandons the traditional pseudo-label filtering strategy based on a fixed threshold and improves the boundary quality of pseudo-labels by refining the boundaries. High-quality pseudo-labels are of great significance for guiding semi-supervised learning, enabling the student model to utilize and learn better boundary information, thus enhancing its boundary improvement for change detection in remote sensing dual-time images. This results in complete and sharp boundary details in the change detection results.

[0062] Second, this invention utilizes ResNet50 and DeepLabv3+ as the encoder and decoder, respectively, to achieve state-of-the-art semi-supervised change detection performance.

[0063] Third, this invention proposes a carefully designed change map boundary optimization method based on remote sensing image change detection. This boundary optimization can improve the performance of pseudo-labels in semi-supervised learning and fully activate the boundary knowledge in unlabeled data.

[0064] Fourth, this invention proposes a novel teacher parameter update method. The parameter update strategy of this invention can measure the stability of the current student model. Based on the stability assessment, it determines whether to update the teacher model parameters.

[0065] Fifth, the present invention presents a semi-supervised change detection method for ultra-high resolution remote sensing images based on improving the quality of pseudo-label boundaries, achieving new state-of-the-art performance on semi-supervised settings of the LEVIR-CD, WHU-CD, and DSIFN-CD datasets. Attached Figure Description

[0066] Figure 1 This is an overall flowchart of the method of the present invention.

[0067] Figure 2 This is a flowchart of the semi-supervised learning process proposed in this invention.

[0068] Figure 3 This is a diagram showing the effect of boundary optimization proposed in this invention.

[0069] Figure 4 This is the boundary optimization flowchart proposed in this invention.

[0070] Figure 5 This is a flowchart of the variation matching process in boundary optimization proposed in this invention.

[0071] Figure 6 This is a schematic diagram of overlap and internal filtering in the boundary optimization proposed in this invention.

[0072] Figure 7 This is a schematic diagram of distributed filtering in the boundary optimization proposed in this invention.

[0073] Figure 8 This is a schematic diagram of predictive integration in the boundary optimization proposed in this invention. Detailed Implementation

[0074] The specific details of each step of the present invention will be described in detail below with reference to the accompanying drawings.

[0075] This invention proposes a semi-supervised change detection method for ultra-high resolution remote sensing images based on improving the quality of pseudo-label boundaries. The entire process of this method is as follows: Figure 1 As shown, the specific steps include:

[0076] Step A: Divide the remote sensing dual-time image dataset into a labeled dataset and an unlabeled dataset. Apply weak data augmentation to the labeled dataset, and apply weak data augmentation followed by strong data augmentation and feature augmentation to the unlabeled dataset first. Concatenate the labeled and unlabeled datasets to obtain the semi-supervised learning training dataset.

[0077] Step A involves the following steps:

[0078] Step A01: Divide the remote sensing dual-time image dataset into four semi-supervised learning proportions: ① 5% labeled dataset, 95% unlabeled dataset; ② 10% labeled dataset, 90% unlabeled dataset; ③ 20% labeled dataset, 80% unlabeled dataset; ④ 40% labeled dataset, 60% unlabeled dataset.

[0079] Step A02: As Figure 2 As shown in the lower left corner, a weak data augmentation pool is set up, including scaling, cropping, and horizontal flipping. A strong data augmentation pool is set up, including color injection, blurring, cropping and stitching, and grayscale adjustment. Feature augmentation is set to the Dropout strategy. Weak data augmentation is performed on the labeled dataset, and weak data augmentation is performed on the unlabeled data first, followed by strong data augmentation and feature augmentation. The labeled and unlabeled datasets are then concatenated to obtain the semi-supervised learning training dataset. Figure 2The left half of the data consists of labeled and unlabeled data that have been augmented and then input into the student or teacher model, which is the data preprocessing process.

[0080] Step B: Build the teacher model and student model, and initialize the parameters of the teacher model and student model.

[0081] Step B involves the following steps:

[0082] Step B01: First, construct a change detection network, using ResNet50 and DeepLabv3+ as the encoder and decoder, respectively. For example... Figure 2 As shown in the middle section, the student model and the teacher model share the same network. The student model parameters are updated to those of the teacher model using the proposed teacher parameter update strategy. Features from the remote sensing dual-time images are extracted using ResNet50 and then fed into DeepLabv3+ for change detection.

[0083]

[0084] In the formula, A and B are the images of the previous and next time steps in the input remote sensing dual-time image to be tested, respectively. ε(·) is the constructed change detection network encoder, (·) * The multidimensional features extracted by the encoder are used. The multidimensional features of the remote sensing images at different times are subtracted, and the absolute value is taken. This result is then fed into the decoder g to obtain the change detection prediction.

[0085] Step B02: Construct student and teacher models using the change detection network built in Step B01. Both student and teacher models use the same change detection network structure, but their parameters are not shared. Initialize the parameters of the teacher and student models using random initial values.

[0086] Step C: Training begins. The student model obtained in Step B is used to perform inference on the labeled weakly augmented data, unlabeled strongly augmented data, and unlabeled feature-augmented data in the training dataset generated in Step A. The teacher model obtained in Step B is used to perform inference on the unlabeled weakly augmented data in the training dataset generated in Step A to generate coarse pseudo-labels.

[0087] Step C is as follows:

[0088] Step C01: Training begins. The student model, after parameter initialization in step B, performs inference on labeled weakly augmented data, unlabeled strongly augmented data, and unlabeled feature-augmented data in the semi-supervised learning training dataset.

[0089]

[0090] In the formula, and represents weak data augmentation, strong data augmentation, and feature augmentation, respectively. x1 and x2 are the previous and next time-series images in the input labeled remote sensing dual-time image, respectively, and u1 and u2 are the previous and next time-series images in the input unlabeled remote sensing dual-time image, respectively. Representing the student model, and These represent the labeled prediction, the unlabeled strongly augmented prediction, and the feature augmented prediction of the student model, respectively.

[0091] Step C02: Using the teacher model initialized with parameters from step B, generate coarse pseudo-labels for inference on the unlabeled weakly augmented data in the semi-supervised learning training dataset:

[0092]

[0093] In the formula, Representing the teacher model, This represents the generated coarse pseudo-tags. For example... Figure 2 As shown on the right, the four predictions arranged from top to bottom are the labeled predictions of the student model. Unlabeled strong augmentation prediction Feature Augmentation Prediction coarse pseudo-labels generated by the teacher model

[0094] Step D: Calculate the labeled supervised loss for the student model's inference results on labeled weakly augmented data obtained in Step C and the corresponding labels in the training dataset generated in Step A. Refine the coarse pseudo-labels obtained in Step C using a boundary optimization method designed for remote sensing dual-time image change detection. First, calculate the consistency loss between the refined pseudo-labels and the coarse pseudo-labels. Then, calculate the semi-supervised loss using the refined pseudo-labels and the student model's inference results on unlabeled strongly augmented and unlabeled feature augmented data obtained in Step C.

[0095] Step D involves the following steps:

[0096] Step D01: As Figure 2 As shown in the upper right corner, the loss is first calculated by comparing the student model's prediction results on labeled weakly augmented data obtained in step C with the corresponding labels in the training dataset generated in step A:

[0097]

[0098] In the formula, This indicates a loss due to label supervision. M and M represent the labeled dataset and its size, respectively. CE represents the cross-entropy loss. and y iLet represent the student model's prediction for the i-th labeled weakly augmented data point and its corresponding label, respectively.

[0099] Step D02: Refine the coarse pseudo-labels generated in step C using a boundary optimization method designed for remote sensing image change detection. For example... Figure 4 As shown in the boundary optimization process, a 3×3 two-dimensional convolution is first applied to the coarse pseudo-label change map to extract the set of coordinates of the boundary region points of the change map. Figure 5 This demonstrates the matching relationship between different parts of the change detection results and the remote sensing dual-time images. First, the coordinates of these boundary points are obtained using convolution operations, expressed by the following formula:

[0100]

[0101] In the formula, x i and y i These are the x and y coordinates of the boundary points. This is the mask corresponding to the i-th boundary block. Here, K is a 3×3 convolution kernel, s is the convolution stride, and p is the convolution padding. w represents the predicted width of the boundary, set to 3. The operation `conv` is a two-dimensional convolution operation, and the `where` operation comes from NumPy.

[0102] The boundary points are expanded along the top left and bottom right corners, creating 48×48 boundary blocks. These boundary blocks are distributed along the boundary, depicting the changing parts and their surrounding environment. The formula is as follows:

[0103]

[0104] In the formula, the generated This is an array that records the coordinates of the top-left and bottom-right corners of the boundary block. `bs` represents the block size, set to 48. `t` indicates the previous or next time step from the dual-phase image.

[0105] Based on the characteristics of boundary blocks, a series of boundary block filters were designed. For example... Figure 4 As shown in the preprocessing and postprocessing flow, an "overlap" filter was first designed to remove highly overlapping boundary blocks. Figure 6 The upper center visualizes the process of using an "overlap" filter to remove overlapping boundary blocks, reducing computational overhead. Since convolution operations can cause boundary blocks to appear within the change graph and located at its center, an "internal" filter is introduced to discard these boundary blocks. Figure 6 The lower center visualizes the process of the "internal" filter. This is to ensure that boundary delimitation predictions align with change detection predictions and to avoid completely incorrect classifications. Figure 7As shown, this invention designs a "distribution" filter that performs a mask-getting operation on each boundary partition prediction to generate a mask corresponding to each boundary. Then, the intersection-union ratio (IU) with the original change map is calculated at the corresponding boundary block location. Boundary blocks with significant IU differences are considered unreliable and discarded. This "distribution" filter ensures that the boundary partition prediction distribution is aligned as closely as possible with the change detection prediction. Figure 8 The results demonstrate the improved boundary detection performance after boundary optimization. Compared to the original prediction, the optimized boundary better depicts the object's outline, providing clearer and sharper boundary predictions.

[0106] Finally, the boundary delineation prediction and change detection prediction are integrated:

[0107]

[0108] In the formula, σ(·) is the sigmoid function, which is used to obtain the confidence level of the prediction. Then, the change detection prediction is... Boundary division prediction Averaging is performed. The decision boundary is pushed towards the side with higher confidence. By applying argmax, the final pseudo-labels with refined boundaries are obtained. The consistency loss is calculated by comparing the refined pseudo-labels with the coarse pseudo-labels obtained in step C.

[0109]

[0110] In the formula, It is a loss of consistency. N and N represent the unlabeled dataset and its number, respectively. This represents the j-th coarse pseudo-label generated, while Let represent the pseudo-label after refinement at the j-th boundary. Then, perform semi-supervised loss calculation by combining the refined pseudo-label with the unlabeled strong augmentation prediction and the feature augmentation prediction:

[0111]

[0112] In the formula, It is a semi-supervised loss used to constrain the consistency between the student model's predictions for strong data augmentation and feature augmentation and the refined pseudo-labels. and Let represent the j-th unlabeled strong augmentation prediction and the j-th feature augmentation prediction, respectively.

[0113] Figure 3This is a visualization of the boundary optimization method on different datasets. "Before" and "After" represent the preceding and following times in the remote sensing dual-time image, respectively, while the change map represents the output of change detection. The boundary optimization method helps separate the boundaries of adhered buildings in the WHU-CD dataset, making the edges clearer and sharper. In the LEVIR-CD results, the refined boundaries are even closer to the actual edges than the ground truth. For the overpasses in the DSIFN-CD dataset, the boundary optimization method also provides more refined segmentation results.

[0114] Step E: Calculate the total loss using the labeled supervised loss, consistency loss, and semi-supervised loss obtained in Step D, and update the student model parameters using stochastic gradient descent. Calculate the stability of the student model and update the teacher model parameters accordingly.

[0115] Step E is detailed below:

[0116] Step E01: Calculate the total loss using the obtained labeled supervised loss, consistency loss, and semi-supervised loss. Update the student model parameters using the loss calculation results and stochastic gradient descent.

[0117]

[0118] In the formula, The total loss is the sum of the constraint losses of each component.

[0119] Step E02: Calculate the stability of the student model and update the parameters of the teacher model. Calculate the intersection-union ratio (IoU) using the student model's output for the unlabeled strongly augmented data and the boundary-refined mask of the unlabeled data:

[0120]

[0121] In the formula, This represents the student model's predicted value for the j-th unlabeled remote sensing two-time image change detection, while... express The mask after boundary refinement. The mask after calculating argmax. and The average distance between them is used as a stability score The average distance is measured using IoU, or Intersection over Union (IoU). If stability scores are... An improvement is considered to indicate that the current student model has reached a more stable state, and its parameters are updated to the teacher model by copying the overall parameters.

[0122] Step F: Training ends. The student model updated with the parameters obtained in step E is used to perform inference on the remote sensing dual-time image to obtain the change detection results.

[0123] Step F is as follows:

[0124] Step F01: Training complete. Use the updated student model to perform change detection on the remote sensing dual-time image under test, and obtain the change detection prediction results:

[0125]

[0126] In the formula, For student models, This is the prediction result for change detection. The binary map of change detection, i.e., the change map, is obtained through the arg max operation.

[0127] Furthermore, to quantify the effectiveness of this method, the present invention measures its performance using Intersection over Union (IoU), F1-score, Precision (Pre), and Recall (Rec):

[0128]

[0129] Where TP represents the number of samples that were actually positive but were predicted as positive, FP represents the number of samples that were actually negative but were predicted as positive, FN represents the number of samples that were actually positive but were predicted as negative, and TN represents the number of samples that were actually negative but were predicted as negative.

[0130] The remote sensing change test results of the present invention are shown in Tables 1, 2 and 3.

[0131] Table 1: Experimental test results of the WHU-CD dataset

[0132]

[0133]

[0134] Table 2: Experimental test results of the LEVIR-CD dataset

[0135]

[0136]

[0137] Table 3: Experimental test results of the DSIFN-CD dataset

[0138]

[0139]

[0140] Table 1 shows the semi-supervised change detection results of this invention on the WHU-CD dataset. Using only 5% of the labeled dataset, this invention achieves an Intersection over Union (IoU) of 83.19% in change detection. Table 2 shows the results on the LEVIR-CD dataset. With 5%, 10%, 20%, and 40% labeled dataset settings, this invention improves performance by 2.36%, 1.76%, 1.05%, and 0.86% compared to the C2FNet method, respectively. Notably, using only 5% of the labeled dataset, this invention surpasses the UniMatch method using 40% labeled dataset, achieving an IoU of 83.32%. Even with only 5% of the sample data, this invention can effectively learn boundary information from high-quality pseudo-labels. Table 3 shows the semi-supervised change detection results of this invention on the DSIFN-CD dataset. With 5%, 10%, 20%, and 40% labeled datasets, this invention outperforms the C2FNet method by 6.88%, 3.59%, 2.06%, and 2.34%, respectively. This invention benefits from high-quality pseudo-labels, significantly improving the guidance of semi-supervised learning.

[0141] In summary, this invention addresses the problems of low-quality pseudo-labels generated by teacher models and the inability of student models to acquire new knowledge. This invention proposes a novel semi-supervised change detection method and a boundary refinement technique for bi-temporal image change maps. By refining pseudo-labels, state-of-the-art performance is achieved in semi-supervised change detection. Experiments on three datasets confirm the effectiveness of the proposed method. This invention preserves boundary knowledge of remote sensing bi-temporal images, stabilizes the semi-supervised learning state, and improves the semi-supervised change detection performance that was previously limited by coarse pseudo-label boundaries.

Claims

1. A semi-supervised change detection method for ultra-high resolution remote sensing images based on improving the quality of pseudo-label boundaries, characterized in that: Includes the following steps: Step A: Divide the remote sensing dual-time image dataset into a labeled dataset and an unlabeled dataset; apply weak data augmentation to the labeled dataset, and apply weak data augmentation followed by strong data augmentation and feature augmentation to the unlabeled dataset first; By concatenating the labeled and unlabeled datasets, a semi-supervised learning training dataset is obtained. Step B: Build the teacher model and student model, and initialize the parameters of the teacher model and student model; Step C: Training begins. The student model obtained in Step B is used to perform inference on the labeled weakly augmented data, unlabeled strongly augmented data, and unlabeled feature-augmented data in the training dataset generated in Step A. The teacher model obtained in Step B is used to perform inference on the unlabeled weakly augmented data in the training dataset generated in Step A to generate coarse pseudo-labels. Step D: Calculate the labeled supervised loss for the inference results of the student model obtained in Step C on labeled weakly augmented data and the corresponding labels in the training dataset generated in Step A; refine the coarse pseudo-labels obtained in Step C using a boundary optimization method designed for remote sensing dual-time image change detection, first calculating the consistency loss between the refined pseudo-labels and the coarse pseudo-labels; then calculate the semi-supervised loss by comparing the refined pseudo-labels with the inference results of the student model obtained in Step C on unlabeled strongly augmented and unlabeled feature augmented data. Step E: Calculate the total loss using the labeled supervised loss, consistency loss, and semi-supervised loss obtained in Step D; update the student model parameters using stochastic gradient descent; calculate the stability of the student model and update the teacher model parameters. Step F: Training ends. The student model updated with the parameters obtained in step E is used to perform inference on the remote sensing dual-time image to obtain the change detection results.

2. The semi-supervised change detection method for ultra-high resolution remote sensing images according to claim 1, characterized in that: The specific steps of step A are as follows: Step A01: Divide the remote sensing dual-time image dataset into four semi-supervised learning proportions: ① 5% labeled dataset, 95% unlabeled dataset; ②10% of the dataset is labeled, and 90% is unlabeled. ③ 20% of the dataset is labeled, and 80% is unlabeled; ④ 40% of the dataset is labeled, and 60% is unlabeled. Step A02: Set up the weak data augmentation pool, including scaling, cropping, and horizontal flipping; set up the strong data augmentation pool, including color injection, blurring, cropping and splicing, and grayscale adjustment. The feature augmentation strategy is set to Dropout; weak data augmentation is performed on the labeled dataset, and weak data augmentation is performed on the unlabeled dataset first, followed by strong data augmentation and feature augmentation; the labeled and unlabeled datasets are concatenated to obtain the semi-supervised learning training dataset.

3. The semi-supervised change detection method for ultra-high resolution remote sensing images according to claim 1, characterized in that: The specific steps of step B are as follows: Step B01: First, construct a change detection network, using ResNet50 and DeepLabv3+ as the encoder and decoder, respectively. Use ResNet50 to extract features from the remote sensing images at different times, and then feed these features into DeepLabv3+ for change detection. In the formula, A and B are the images of the previous and next time steps in the input remote sensing dual-time image to be tested, respectively, ε(·) is the constructed change detection network encoder, and (·) * Multidimensional features extracted by the encoder; The multidimensional features of the remote sensing images from two different times are subtracted and their absolute values ​​are taken, then fed into the decoder. In the process, predictions for change detection are obtained. Step B02: Construct student and teacher models using the change detection network built in step B01. The student and teacher models use the same change detection network structure, but the parameters between the student and teacher models are not shared. Initialize the parameters of the teacher and student models using random initial values.

4. The semi-supervised change detection method for ultra-high resolution remote sensing images according to claim 1, characterized in that: The specific steps of step C are as follows: Step C01: Training begins. Using the student model initialized with parameters from Step B, inference is performed on labeled weakly augmented data, unlabeled strongly augmented data, and unlabeled feature-augmented data in the semi-supervised learning training dataset. In the formula, and , representing weak data augmentation, strong data augmentation and feature augmentation respectively; x1 and x2 are the previous and next time images in the input labeled remote sensing dual-time image, and u1 and u2 are the previous and next time images in the input unlabeled remote sensing dual-time image. Representing the student model, and These represent labeled prediction, unlabeled strongly augmented prediction, and feature augmented prediction for the student model, respectively. Step C02: Using the teacher model initialized with parameters from step B, generate coarse pseudo-labels for inference on the unlabeled weakly augmented data in the semi-supervised learning training dataset: In the formula, Representing the teacher model, This indicates the generated coarse pseudo-tag.

5. A semi-supervised change detection method for ultra-high resolution remote sensing images according to claim 1, characterized in that: The specific steps of step D are as follows: Step D01: Calculate the loss between the student model's prediction results on labeled weakly augmented data obtained in Step C and the corresponding labels in the semi-supervised learning training dataset generated in Step A. In the formula, This indicates a loss due to label supervision. M and M represent the labeled dataset and its number, respectively; CE represents cross-entropy loss. and y i Let represent the student model's prediction for the i-th labeled weakly augmented data point and its corresponding label, respectively. Step D02: Refine the coarse pseudo-labels generated in step C using a boundary optimization method designed for remote sensing image change detection: First, apply a 3×3 two-dimensional convolution to the coarse pseudo-label change map to extract the set of coordinates of boundary region points; then, use convolution operations to obtain the coordinates of these boundary points, expressed by the following formula: In the formula, x i and y i These are the x and y coordinates of the boundary points; is the mask corresponding to the i-th boundary block; K is a 3×3 convolution kernel, s is the convolution stride, p is the convolution padding; W represents the predicted width of the boundary; the operation conv is a two-dimensional convolution operation, and the operation where comes from NumPy; The boundary points are expanded along the top left and bottom right corners to form 48×48 boundary blocks; these boundary blocks are distributed along the boundary to depict the changing parts and their surrounding environment, as expressed by the formula: In the formula, the generated It is an array that records the coordinates of the top left and bottom right corners of the boundary block; bs represents the block size, set to 48; t is used to represent the previous or next time step from the remote sensing dual-time image; Based on the characteristics of boundary blocks, a series of boundary block filters were designed. First, an "overlapping" filter was designed to remove highly overlapping boundary blocks, thus reducing computational burden. Since convolution operations may cause boundary blocks to appear within the change map and located at the center of the change map, an "interior" filter was introduced to discard boundary blocks appearing within the change map. To ensure that boundary partitioning predictions are aligned with change detection predictions and to avoid completely incorrect classifications, a "distribution" filter was designed. For each boundary partitioning prediction, a mask-getting operation was performed to generate a mask corresponding to each boundary. Then, the intersection-union ratio (IU) with the original change map was calculated at the corresponding boundary block location. Boundary blocks with significant IU differences were considered unreliable and discarded. This "distribution" filter ensures that the distribution of boundary partitioning predictions is aligned with change detection predictions as much as possible. Finally, the boundary delineation prediction and change detection prediction are integrated: In the formula, σ(·0) is the sigmoid function, which is used to obtain the confidence level of the prediction; then, the change detection prediction is... Boundary division prediction Averaging is performed; the decision boundary is pushed towards the side with higher confidence; by applying arg max, the final pseudo-label with refined boundary is obtained. The consistency loss is calculated by comparing the refined pseudo-labels with the coarse pseudo-labels obtained in step C. In the formula, It is a loss of consistency; N and N represent the number of unlabeled datasets, respectively; This represents the j-th coarse pseudo-label generated, while Let represent the pseudo-label after refinement at the j-th boundary; then, perform semi-supervised loss calculation by combining the refined pseudo-label with the unlabeled strong augmentation prediction and the feature augmentation prediction: In the formula, It is a semi-supervised loss used to constrain the consistency between the student model's predictions for strong data augmentation and feature augmentation and the refined pseudo-labels; and Let represent the j-th unlabeled strong augmentation prediction and the j-th feature augmentation prediction, respectively.

6. A semi-supervised change detection method for ultra-high resolution remote sensing images according to claim 5, characterized in that: The predicted width w of the boundary is set to 3.

7. A semi-supervised change detection method for ultra-high resolution remote sensing images according to claim 1, characterized in that: The specific steps of step E are as follows: Step E01: Calculate the total loss using the obtained labeled supervised loss, consistency loss, and semi-supervised loss. Update the student model parameters using the loss calculation results and stochastic gradient descent. In the formula, The total loss is the sum of the constraint losses of each component. Step E02: Calculate the stability of the student model and update the parameters of the teacher model; The intersection-over-union (IoU) ratio is calculated using the student model's output of the unlabeled strongly augmented data and the boundary-refined mask of the unlabeled data: In the formula, This represents the student model's predicted value for the j-th unlabeled remote sensing two-time image change detection, while... express Mask after boundary refinement; Calculate the result after argmax operation and The average distance between them is used as a stability score The average distance is measured using IoU (Intersection over Union); if stability score An improvement is considered to indicate that the current student model has reached a more stable state, and its parameters are updated to the teacher model by copying the overall parameters.

8. A semi-supervised change detection method for ultra-high resolution remote sensing images according to claim 1, characterized in that: The specific steps of step F are as follows: Step F01: Training complete. Use the updated student model to perform change detection on the remote sensing dual-time image under test, and obtain the change detection prediction results: In the formula, For student models, The result is the change detection prediction; the binary map of change detection, i.e., the change map, is obtained through arg max operation.

Images