Pseudo label generation method of remote sensing image based on prototype perception learning, and training and detection method of weak supervision change detection model

Abstract

The invention discloses a remote sensing image pseudo label generation method based on prototype perception learning, and a weak supervision change detection model training and detection method, and belongs to the field of remote sensing image change detection. The invention aims to solve the problem that a category activation graph is incomplete in existing remote sensing image weak supervision change detection and the problem that an obtained detection model is unstable in change recognition. According to the method, feature maps of different scales before and after change are extracted from remote sensing images before and after change through a pseudo label generation model, change features are obtained, a class activation mapping map and a class activation mapping map of an unchanged class are obtained, and the class activation mapping map is segmented into response area masks through a threshold value and used for performing weighted aggregation on fusion features to obtain a class prototype; generating an enhanced class activation mapping graph based on the fusion feature and the class prototype, and selecting the class with the highest similarity as a pseudo tag of the pixel; and training based on the pseudo label data of the pixels to obtain a change detection model, wherein the model is used for remote sensing image change detection.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image change detection, and in particular relates to a remote sensing image weakly supervised change detection method. Background Art

[0002] As a crucial task for monitoring the spatiotemporal evolution of the Earth's surface, change detection is widely used in diverse fields, including disaster assessment, ecological and environmental monitoring, and urban planning. With the advancement of deep learning technology, change detection methods based on deep neural networks have significantly improved the accuracy and robustness of change detection due to their powerful feature extraction capabilities. However, these methods typically require a large number of pixel-level labels for network training, and fine pixel-by-pixel labeling of remote sensing images is time-consuming and labor-intensive. This challenge limits the applicability of pixel-level supervised change detection methods in real-world scenarios. In recent years, image-level weakly supervised change detection has attracted increasing attention due to its advantages of reducing annotation costs and improving annotation efficiency. Existing weakly supervised change detection methods face several key challenges, including incomplete class activation maps, interference from noisy pseudo-labels during training, and unstable change recognition due to illumination and environmental variations. Therefore, a new weakly supervised change detection method for remote sensing images is urgently needed that can reduce the cost of manual labeling while achieving high detection accuracy, thereby effectively supporting its application in real-world scenarios. Summary of the Invention

[0003] The present invention aims to solve the problem of incomplete category activation maps in existing weakly supervised change detection of remote sensing images and the problem of unstable change recognition of the obtained detection model.

[0004] A pseudo-label generation method for remote sensing images based on prototype-based perceptual learning, comprising:

[0005] Construct a weakly supervised change detection dataset for remote sensing images. The dataset includes remote sensing images before and after the changes, as well as corresponding image-level labels. ;

[0006] The remote sensing images before and after the change are extracted through a pseudo-label generation model based on a deep learning model to extract feature maps of different scales before and after the change. , Indicates before and after changes, Represent feature maps of different scales to obtain changing features ; The deepest changes in characteristics Feed into a classifier to generate class activation maps ; Then, spatial aggregation is performed through the global average pooling operation to finally obtain the image-level classification result ; At the same time, the class activation map of the unchanged category is obtained ;based on 、 , generate high confidence response area mask through threshold segmentation strategy , Indicates the category of change or no change;

[0007] Different scale variation features Splicing to generate fusion features , perform weighted aggregation , Represents the prototype of the changed or unchanged class, and uses exponential moving average to update and obtain the global class prototype ;

[0008] Calculate fusion features With class prototype The cosine similarity between them generates an enhanced class activation map , for each pixel position, select the class with the highest similarity as the pseudo label of the pixel:

[0009]

[0010] in, represents the pixel-level change detection pseudo label, is the threshold coefficient used to balance the background area;

[0011] Based on the change prototype With unchanged prototype Similarity loss between The total loss function is constructed by combining the binary cross entropy loss of the change detection image-level prediction results and the image-level labels. The pseudo-label generation model is trained based on the total loss function. Finally, the pixel-level pseudo-labels corresponding to all remote sensing images are obtained based on the trained pseudo-label generation model. .

[0012] Furthermore, the class activation map , ReLU is the linear rectification function, represents maximum normalization, Represents the weight matrix of the classifier; The deepest changes .

[0013] Furthermore, the high confidence response region mask in, A mask representing the high confidence response region, Indicates the changed category or the unchanged category, Represents the segmentation threshold.

[0014] Furthermore, the enhanced class activation map , represents L2 normalization.

[0015] Furthermore, the prototype is changed With unchanged prototype Similarity loss between , It is the change prototype corresponding to each sample in the Batch With unchanged prototype ; The class prototypes of all samples in each batch are averaged to obtain the batch-level class representation.

[0016] A method for training a weakly supervised change detection model, comprising:

[0017] For the weakly supervised change detection dataset of remote sensing images, a pseudo-label generation method for remote sensing images based on prototype perception learning is used to generate pixel-level pseudo-labels corresponding to all remote sensing images. The data with pixel-level pseudo-labels are subjected to weak and strong data enhancement and then fed into a change detection model based on an encoder-decoder structure:

[0018] The weak data enhancement images before and after the change are respectively subjected to feature extraction by the encoder to obtain multi-scale feature maps. The following processing is performed on the feature maps of each scale before and after the change:

[0019] The feature map of a certain scale before and after the change is recorded as , for the feature maps before and after the change Perform wavelet transform and inverse wavelet transform to obtain a complete change feature map; each scale feature map corresponding to the weak data enhancement image before and after the change obtains a change feature map, and these features are finally sent to the decoder for layer-by-layer upsampling to obtain pixel-level change detection results;

[0020] Strong data augmentation images before and after the change are processed identically;

[0021] For the strong enhancement and weak enhancement samples of the same input sample, their final prediction results are recorded as and ;

[0022] Will As the loss function, the change detection model is trained to obtain the final change detection model for remote sensing image change detection; is the cross entropy loss between pixel-level prediction results and pixel-level pseudo labels for change detection, Dice loss for pixel-level prediction results and pixel-level pseudo labels for change detection, Weakly enhanced prediction results and strong enhancement prediction results The consistency regularization loss between is the balance loss contribution factor.

[0023] Further, based on 、 Differentiation to obtain change characteristics , Represents a convolutional layer with a convolution kernel size of 3.

[0024] Furthermore, for the feature maps before and after the change The process of performing wavelet transform and inverse wavelet transform to obtain a complete change feature map includes:

[0025] First, we transform the Decomposed into four frequency components of different frequency bands, namely ;in is the low-frequency component, is the high-frequency component; calculate the similarity relationship between each pixel and all surrounding pixels in the feature map before and after the change , to obtain potential structural change information ,use Highlight the changes in low-frequency information , Represents element-wise multiplication;

[0026] Will The three high-frequency sub-bands are spliced ​​together along the channel dimension to obtain ,based on 、 Differentiation to obtain change characteristics , along the channel dimension Re-divided into 、 and Three sub-bands; for 、 、 and , and obtain the complete change feature map through inverse wavelet transform.

[0027] Further, The process of training a change detection model as a loss function includes:

[0028] Based on whether the confidence exceeds the confidence threshold, determine whether the pixel belongs to the clean area:

[0029]

[0030]

[0031] in, Indicates a clean area. represents the noise area, represents the spatial coordinates of the pixel, Indicates Pseudo-label category of the location, and Represents the weak enhancement map and the strong enhancement map respectively. The pixel at position is predicted to be of class The confidence level, Represents the confidence threshold; if a pixel does not belong to the clean area, it is considered to belong to the noise area;

[0032] For clean areas, supervised training is performed through cross entropy loss, and Dice loss is combined to enhance the model's sensitivity to changing categories, as shown in the formula:

[0033]

[0034]

[0035]

[0036] in, represents the cross entropy loss, and They represent the Dice loss of weakly enhanced view and the Dice loss of strongly enhanced view, respectively. The calculation method of similar, is the final Dice loss;

[0037] For the noisy area, consistency regularization is used to strengthen the constraints, as shown in the formula:

[0038]

[0039] in, and Represent the prediction results of weakly enhanced samples and strongly enhanced samples respectively.

[0040] A change detection method for remote sensing images, comprising:

[0041] First, two remote sensing images that require remote sensing image change detection are obtained, and then sent to a change detection model trained by the training method of a weakly supervised change detection model for change detection.

[0042] Beneficial effects:

[0043] During the training process, the present invention extracts corresponding multi-scale features from the images before and after the change through a ResNet50 backbone network with pre-trained weights. The extracted deepest features before and after the change are fed into a classifier to obtain image-level change detection prediction results. The present invention can relatively comprehensively extract class activation maps for the change category, obtain high-response region masks for the change and unchanged categories based on the class activation mapping technique, and extract stable prototypes of the change and unchanged categories through a prototype perception module to generate pixel-level pseudo-labels. After obtaining the pixel-level pseudo-labels, during the change detection model training phase, weak and strong data augmentation are used to generate different perspectives of the same input sample before training and these are fed into the change detection model. During the training process, the corresponding multi-scale features are extracted through a network with pre-trained weights. These multi-scale features are fed into a wavelet transform enhancement module to extract robust change features. The decoder then upsamples layer by layer to obtain prediction results for the same sample at different enhanced perspectives. Based on the prediction results and pseudo-labels, a dual-view regularization strategy is then used to reduce the model's fitting of noisy pseudo-labels during training. The change detection model is optimized based on the resulting model loss function. The method of the present invention can effectively improve the detection capability of the detection model. The detection model has a very stable and good recognition effect on change recognition, and achieves excellent results in image-level weakly supervised change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of a weakly supervised change detection method for remote sensing images based on prototype-aware learning.

[0045] Figure 2 Schematic diagram of the model structure in a weakly supervised change detection method for remote sensing images based on prototype perception learning.

[0046] Figure 3 This is a diagram showing the detection effect of the present invention. DETAILED DESCRIPTION

[0047] In the face of the weakly supervised change detection task of remote sensing images, the present invention designs a weakly supervised change detection method for remote sensing images based on prototype perception learning, which can generate pixel-level prediction results using only image-level change detection labels, thereby reducing the cost of manual annotation while having higher detection accuracy, and providing effective support for applications in actual scenarios. The method mainly includes a pseudo-label generation stage and a change detection model training stage. In the pseudo-label generation stage, the image-level labels are used to train the pseudo-label generation model to generate high-quality pseudo-labels. In the change detection model training stage, these pseudo-labels are used to train the change detection model to produce more accurate pixel-level change detection results. The following is a detailed description in conjunction with the specific implementation method.

[0048] Specific implementation method 1: Combination Figure 1 and Figure 2To explain this embodiment,

[0049] This embodiment first proposes a pseudo-label generation method for remote sensing images based on prototype-based perceptual learning (steps 1 to 4), then proposes a training method for a weakly supervised change detection model (steps 5 to 7), and finally proposes a change detection method for remote sensing images (step 8). The present invention as a whole is a weakly supervised change detection method for remote sensing images based on prototype-based perceptual learning, specifically comprising:

[0050] Step 1: Construct a weakly supervised change detection dataset, including a training set, a validation set, and a test set. The dataset includes remote sensing images before and after the change, as well as corresponding image-level labels. ;

[0051] Step 2: Perform data augmentation operations on the remote sensing images before and after the change in the training set by horizontal flipping, vertical flipping, and swapping the order of the images before and after the change;

[0052] In this implementation, for each group of remote sensing images before and after the change and their corresponding image-level labels in the training set, data enhancement is performed by randomly performing horizontal flipping, vertical flipping, and swapping the order before and after the change with a probability of 50%, thereby increasing the diversity of the data.

[0053] Step 3: Feed the training data into the pseudo-label generation model and obtain the image-level prediction results of the input samples through the classifier , and obtain the pixel-level pseudo label of the remote sensing image based on the class activation mapping technology and prototype perception module ;

[0054] Get the pixel-level pseudo label of the remote sensing image The specific process includes:

[0055] For the training set after data enhancement, the remote sensing images before and after the change are extracted through the pseudo-label generation model to extract the feature maps of different scales before and after the change. , Indicates before and after changes, Represents feature maps of different scales. The images before and after the change share weights in the backbone network to increase the model's sensitivity to change information.

[0056] In this embodiment, the pseudo-label generation model uses a pre-trained ResNet50, which is a ResNet50 pre-trained on the open source dataset ImageNet.

[0057] After obtaining the features of different scales before and after the change, the change features are obtained by subtracting the features before and after the change and taking the absolute value. , as shown in the formula:

[0058]

[0059] in, Indicates the absolute value operation.

[0060] After obtaining the characteristics of different scale changes, the deepest change characteristics Feed into a classifier to generate class activation maps , The response strength of each spatial position to different categories is characterized. Then, the spatial aggregation is performed through the global average pooling operation to finally obtain the image-level classification result. , used for supervision with the true labels.

[0061] Among them, the class activation map of the change category is obtained based on the class activation mapping technology , as shown in the formula:

[0062]

[0063] Among them, ReLU is the linear rectification function, represents maximum normalization, Represents the weight matrix of the classifier.

[0064] Class activation map for unchanged classes Can be achieved through To estimate.

[0065] After obtaining the class activation map, a threshold segmentation strategy is used to generate a high-confidence response area mask, as shown in the formula:

[0066]

[0067] in, A mask representing the high confidence response region, Indicates the changed category or the unchanged category, Represents the segmentation threshold.

[0068] In order to make full use of the rich semantic information in the features of different scale changes, the features of different scale changes are combined Perform concatenation to generate fusion features Subsequently, the fused features are weightedly aggregated with the mask of the high-confidence response region as a guide to capture stable category prototypes, as shown in the formula:

[0069]

[0070] in, Represents the prototype of a changing or unchanged class, Indicates the changed category or the unchanged category. With unchanged prototype Similarity loss is introduced , to reduce the similarity between classes and enhance the discrimination ability. Participate in the calculation of total losses.

[0071] Then, for each batch of obtained category prototypes Use exponential moving average to update it, thus obtaining a more stable global class prototype .

[0072] After getting the global class prototype ,Since the prototype essentially reflects the semantic features of the class center, by matching the similarity with the feature map, we can mine the potential change areas that are not activated in the initial class activation map. Specifically, by calculating the fusion feature With class prototype The cosine similarity between them is used to generate the enhanced class activation map, as shown in the formula:

[0073]

[0074] in, The enhanced class activation map representing the changed or unchanged classes, Indicates the changed category or the unchanged category, By performing semantic matching between the stable global class prototype and the fused hierarchical features, the local details of the shallow features and the global semantic information of the deep features are effectively utilized, thus enhancing the integrity of the class activation map.

[0075] Finally, for each pixel position, the class with the highest similarity is selected as the pseudo label of the pixel, as shown in the formula:

[0076]

[0077] in, represents the pixel-level change detection pseudo label, is the threshold factor used to balance the background (unchanged) areas.

[0078] Regarding the loss during the training of the pseudo-label generation model, the prototype is changed With unchanged prototype Similarity loss is introduced between the two classes to reduce the similarity between classes and enhance the ability to distinguish. By minimizing the cosine similarity between the prototypes of the changed class and the unchanged class, it makes them more distinguishable in the feature space, thereby obtaining a more discriminative semantic feature representation and improving the model's ability to identify changed and unchanged areas, as shown in the formula:

[0079]

[0080] in, It is the change prototype corresponding to each sample in the Batch With unchanged prototype ; The class prototypes of all samples in each batch are averaged to obtain the batch-level class representation.

[0081] Then determine the loss function of the total pseudo-label generation stage:

[0082]

[0083] in, is the binary cross entropy loss function between the image-level prediction results and image-level labels for change detection, Prototype for the change category With unchanged class prototype Similarity loss between .

[0084] Will The pseudo label generation model is optimized as the loss function to obtain the optimized pseudo label generation model; the model weight of the last iterative training is selected as the final model weight to obtain the optimal pseudo label generation model;

[0085] Step 4: Input the training set data into the optimal pseudo-label generation model and output the pixel-level pseudo-labels corresponding to all remote sensing images in the training set. .

[0086] Step 5: Perform weak and strong data enhancement on the training set data with pixel-level pseudo-labels and then feed it into the change detection model. The dual-view regularization strategy is used to reduce the network's fitting of noisy pseudo-labels. The wavelet transform enhancement module is used to extract robust features of different scale changes. Finally, the multi-scale change features are fed into the decoder and the prediction results under weak enhancement input are obtained through layer-by-layer upsampling. And the prediction results under strong reinforcement input ; The specific process includes the following steps:

[0087] After obtaining pixel-level pseudo-labels, these pseudo-labels are used to train the change detection model, producing more accurate pixel-level change detection results. During the change detection model training phase, each sample consists of an image pair consisting of the before and after images of the change. Strong and weak data augmentation operations are applied to each image in this image pair, generating two samples with different enhanced perspectives. Both enhanced samples are input into the change detection model.

[0088] The change detection model first uses an encoder to extract feature maps at different scales before and after the change, and then uses a wavelet transform enhancement module to extract robust change features. In this implementation, the encoder uses a pre-trained ResNet18, which is a ResNet18 pre-trained on the open-source ImageNet dataset.

[0089] First, the pre-trained ResNet18 is used to extract feature maps of different scales. In this embodiment, feature maps of four scales are extracted according to the network structure of ResNet18.

[0090] The weak data enhancement images before and after the change are subjected to feature extraction by pre-trained ResNet18 to obtain multi-scale feature maps. The following processing is performed on the feature maps of each scale before and after the change.

[0091] The feature map of a certain scale before and after the change is recorded as , in the wavelet transform enhancement module, for the feature maps before and after the change , first decompose it into four frequency components of different frequency bands through wavelet transform, namely .in is the low-frequency component, is the high frequency component.

[0092] The low-frequency component contains most of the image energy and stores rich global semantic information. This information can be used to better model the low-frequency structural information of the changed area. Specifically, the similarity relationship between each pixel in the feature map before and after the change and all surrounding pixels is first calculated, as shown in the formula:

[0093]

[0094] Then, by comparing and , capturing potential structural change information, as shown in the formula:

[0095]

[0096] exist In , the high confidence regions identified by the network tend to 1 and -1. Therefore, using The accurate change information can be highlighted, as shown in the formula:

[0097]

[0098] in, Indicates the changing characteristics of the low-frequency part, Represents element-wise multiplication.

[0099] The high frequency component mainly captures the texture and edge details in the image. Using this information can better capture the changes in image details. Specifically, The three high-frequency sub-bands are spliced ​​together along the channel dimension, and then the change characteristics are obtained by differential calculation, as shown in the formula:

[0100]

[0101]

[0102] in, Indicates the changing characteristics of the high-frequency part, Represents a convolutional layer with a convolution kernel size of 3.

[0103] Finally, along the channel dimension Re-divided into 、 and Three sub-bands; for 、 、 and , and obtain the complete change feature map through inverse wavelet transform.

[0104] In the change detection model, ResNet18 is used to extract each scale feature map for image samples before and after the change. The change feature maps obtained at different scales are input into the wavelet transform enhancement module to further explore the change information at each scale, thereby obtaining a multi-scale change feature representation. That is, each scale feature map corresponding to the weakly data-enhanced image before and after the change is a change feature map. These features are ultimately fed into the decoder and upsampled layer by layer to obtain pixel-level change detection results. In fact, the encoder's last layer of change feature maps first passes through the decoder's first layer, is upsampled, and then fused with the change feature map corresponding to the encoder's penultimate layer before being fed into the decoder's second layer. Similarly, the decoder undergoes layer-by-layer upsampling to ultimately obtain pixel-level change detection results.

[0105] The strongly data augmented images before and after the change are processed identically.

[0106] For the strong enhancement and weak enhancement samples of the same input sample, their final prediction results are recorded as and To mitigate the network's adaptation to noisy pseudo-labels, the pseudo-labels are divided into clean and noisy regions based on the differences and consistencies between the predictions of the strongly and weakly augmented views. Appropriate regularization strategies are applied to different regions, effectively reducing the impact of noisy pseudo-labels and enhancing the network's robustness.

[0107] Step 6: The change detection model is optimized as the loss function to obtain the optimized change detection model; wherein, is the cross entropy loss between pixel-level prediction results and pixel-level pseudo labels for change detection, Dice loss for pixel-level prediction results and pixel-level pseudo labels for change detection, Weakly enhanced prediction results and strong enhancement prediction results The consistency regularization loss between is the balance loss contribution factor.

[0108] Specifically, for the same pixel, if the predictions from the two views are consistent with the pseudo label and their confidence exceeds the set threshold, the pixel is considered a clean sample with high confidence. Otherwise, it is considered a noise sample, as shown in the formula:

[0109]

[0110]

[0111] in, Indicates a clean area. represents the noise area, represents the spatial coordinates of the pixel, Indicates Pseudo-label category of the location, and Respectively represent the two views (weak enhancement map and strong enhancement map), The pixel at position is predicted to be of class The confidence level, Represents the confidence threshold.

[0112] For clean areas, supervised training is performed through cross entropy loss, and Dice loss is combined to enhance the model's sensitivity to changing categories, as shown in the formula:

[0113]

[0114]

[0115]

[0116] in, represents the cross entropy loss, and They represent the Dice loss of weakly enhanced view and the Dice loss of strongly enhanced view, respectively. The calculation method of similar, is the final Dice loss.

[0117] For the noisy area, consistency regularization is used to strengthen the constraints, as shown in the formula:

[0118]

[0119] in, and Represent the prediction results of weakly enhanced samples and strongly enhanced samples respectively.

[0120] Step 7: During the change detection model training process, the performance of the change detection model is verified through a pre-divided validation set, and the model weight with the highest F1 score in the validation set is taken as the final selected model weight.

[0121] The data is divided into a training set, a validation set, and a test set. The training set is used to train the model, and the validation set is used to evaluate model performance after each round of training. The performance of each model is evaluated by calculating the F1 score on the validation set. The set of weights with the highest F1 score on the validation set is selected as the final model parameters. The F1 score is calculated as shown in the formula:

[0122]

[0123] Step 8: Input the remote sensing images before and after the change to be detected into the selected optimal change detection model, and output the change detection result graph corresponding to the remote sensing images to be detected. This embodiment is tested with a test set. Figure 3 The prediction results on the test set are shown in Figure 2, where (a) represents the image before the change, (b) represents the image after the change, (c) represents the change label, and (d) represents the prediction result. Red represents the misdetected area, and blue represents the missed area.

[0124] The present invention first constructs a dataset by acquiring remote sensing images before and after a change and their corresponding image-level labels for change detection. This dataset is then divided into training, validation, and test sets. The method consists of two phases: pseudo-label generation and change detection model training. In the first phase, the pseudo-label generation model is trained using image-level labels to generate high-quality pseudo-labels. In the second phase, these pseudo-labels are used to train the change detection model, producing more accurate pixel-level change detection results. Specifically, in the pseudo-label generation phase, data augmentation is performed before training by horizontally flipping, vertically flipping, and swapping the order of the two-phase images. During training, the images before and after the change are passed through a ResNet50 backbone network with pre-trained weights to extract corresponding multi-scale features. The deepest extracted pre- and post-change features are then fed into a classifier to obtain image-level change detection prediction results. High-response region masks for the change and invariant classes are obtained using class activation mapping techniques. Stable prototypes for the change and invariant classes are extracted using a prototype perception module and pixel-level pseudo-labels are generated. After obtaining the pixel-level pseudo-labels, the change detection model is trained using weak and strong data augmentation before training to generate different perspectives of the same input sample. These are then fed into the change detection model. During training, a ResNet18 backbone network with pretrained weights extracts corresponding multi-scale features. These features are then fed into a wavelet transform enhancement module to extract robust change features. The decoder then upsamples the image layer by layer to produce predictions for the same sample at different enhanced viewpoints. Based on the predictions and pseudo-labels, a dual-view regularization strategy is then implemented to mitigate the model's adaptation to noisy pseudo-labels during training. The resulting model loss function is then used to optimize the change detection model. This method achieves excellent results in image-level weakly supervised change detection.

[0125] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. A pseudo-label generation method for remote sensing images based on prototype perception learning, characterized in that: include: Construct a weakly supervised change detection dataset for remote sensing images. The dataset includes remote sensing images before and after the changes, as well as corresponding image-level labels. ; The remote sensing images before and after the change are extracted through a pseudo-label generation model based on a deep learning model to extract feature maps of different scales before and after the change. , Indicates before and after changes, Represent feature maps of different scales to obtain changing features ; The deepest changes in characteristics Feed into a classifier to generate class activation maps ; Subsequently, spatial aggregation is performed through a global average pooling operation to finally obtain the image-level classification result. ; At the same time, the class activation map of the unchanged category is obtained ;based on 、 , generate high confidence response area mask through threshold segmentation strategy , Indicates the category of change or no change; Different scale variation features Splicing to generate fusion features , perform weighted aggregation , Represents the prototype of the changed or unchanged class, and uses exponential moving average to update and obtain the global class prototype ; Calculate fusion features With class prototype The cosine similarity between them generates an enhanced class activation map , for each pixel position, select the class with the highest similarity as the pseudo label of the pixel: in, represents the pixel-level change detection pseudo label, is the threshold coefficient used to balance the background area; Based on the change prototype With unchanged prototype Similarity loss between The total loss function is constructed by combining the binary cross entropy loss of the change detection image-level prediction results and the image-level labels. The pseudo-label generation model is trained based on the total loss function. Finally, the pixel-level pseudo-labels corresponding to all remote sensing images are obtained based on the trained pseudo-label generation model. .

2. The method for generating pseudo labels for remote sensing images based on prototype-based perceptual learning according to claim 1, characterized in that: The class activation map , ReLU is the linear rectification function, represents maximum normalization, Represents the weight matrix of the classifier; The deepest changes .

3. The method for generating pseudo labels for remote sensing images based on prototype perception learning according to claim 2, characterized in that: High confidence response region mask in, A mask representing the high confidence response region, Indicates the changed category or the unchanged category, Represents the segmentation threshold.

4. The method for generating pseudo labels for remote sensing images based on prototype-based perceptual learning according to claim 1, characterized in that: Enhanced Class Activation Maps , represents L2 normalization.

5. The method for generating pseudo labels for remote sensing images based on prototype-based learning according to any one of claims 1 to 4, characterized in that: Change Prototype With unchanged prototype Similarity loss between , It is the change prototype corresponding to each sample in the Batch With unchanged prototype ; The class prototypes of all samples in each batch are averaged to obtain the batch-level class representation.

6. A training method for a weakly supervised change detection model, characterized in that: include: For a weakly supervised change detection dataset of remote sensing images, a pseudo-label generation method for remote sensing images based on prototype perception learning according to any one of claims 1 to 5 is used to generate pixel-level pseudo-labels corresponding to all remote sensing images; The data with pixel-level pseudo-labels is weakly and strongly augmented and then fed into the change detection model based on the encoder-decoder structure: The weak data enhancement images before and after the change are respectively subjected to feature extraction by the encoder to obtain multi-scale feature maps. The following processing is performed on the feature maps of each scale before and after the change: The feature map of a certain scale before and after the change is recorded as , for the feature maps before and after the change Perform wavelet transform and inverse wavelet transform to obtain a complete change feature map; each scale feature map corresponding to the weak data enhancement image before and after the change obtains a change feature map, and these features are finally sent to the decoder for layer-by-layer upsampling to obtain pixel-level change detection results; Strong data augmentation images before and after the change are processed identically; For the strong enhancement and weak enhancement samples of the same input sample, their final prediction results are recorded as and ; Will As the loss function, the change detection model is trained to obtain the final change detection model for remote sensing image change detection; is the cross entropy loss between pixel-level prediction results and pixel-level pseudo labels for change detection, Dice loss for pixel-level prediction results and pixel-level pseudo labels for change detection, Weakly enhanced prediction results and strong enhancement prediction results The consistency regularization loss between is the balance loss contribution factor.

7. The training method of a weakly supervised change detection model according to claim 6, characterized in that: based on 、 Differentiation to obtain change characteristics , Represents a convolutional layer with a convolution kernel size of 3.

8. The method for training a weakly supervised change detection model according to claim 6, wherein: For the feature maps before and after the change The process of performing wavelet transform and inverse wavelet transform to obtain a complete change feature map includes: First, we transform the Decomposed into four frequency components of different frequency bands, namely ;in is the low-frequency component, is the high-frequency component; calculate the similarity relationship between each pixel and all surrounding pixels in the feature map before and after the change , to obtain potential structural change information ,use Highlight the changes in low-frequency information , Represents element-wise multiplication; Will The three high-frequency sub-bands are spliced ​​together along the channel dimension to obtain ,based on 、 Differentiation to obtain change characteristics , along the channel dimension Re-divided into 、 and Three sub-bands; for 、 、 and , and obtain the complete change feature map through inverse wavelet transform.

9. A method for training a weakly supervised change detection model according to any one of claims 6 to 8, characterized in that: Will The process of training a change detection model as a loss function includes: Based on whether the confidence exceeds the confidence threshold, determine whether the pixel belongs to the clean area: in, Indicates a clean area. represents the noise area, represents the spatial coordinates of the pixel, Indicates Pseudo-label category of the location, and Represents the weak enhancement map and the strong enhancement map respectively. The pixel at position is predicted to be of class The confidence level, Represents the confidence threshold; if a pixel does not belong to the clean area, it is considered to belong to the noise area; For clean areas, supervised training is performed through cross entropy loss, and Dice loss is combined to enhance the model's sensitivity to changing categories, as shown in the formula: in, represents the cross entropy loss, and They represent the Dice loss of weakly enhanced view and the Dice loss of strongly enhanced view, respectively. The calculation method of similar, is the final Dice loss; For the noisy area, consistency regularization is used to strengthen the constraints, as shown in the formula: in, and Represent the prediction results of weakly enhanced samples and strongly enhanced samples respectively.

10. A method for detecting changes in remote sensing images, characterized in that: include: First, two remote sensing images that require remote sensing image change detection are obtained, and then the images are fed into a change detection model trained using the training method for a weakly supervised change detection model described in claim 9 for change detection.

Images