Remote sensing image change detection method based on semantic guidance and SAM optimization

Abstract

The invention discloses a remote sensing image change detection method based on semantic guidance and SAM optimization, and belongs to the field of image processing, and the method comprises the steps: collecting change detection data to construct a change detection data set; preprocessing the change detection data set and dividing the change detection data set into a training set, a test set and a verification set; constructing a weak supervision change detection network; inputting the training set into a weak supervision change detection network and training; inputting the dual-temporal remote sensing image pair in the training set into the trained weak supervision change detection network to obtain a class activation graph of the training set; an SAM optimization network is constructed; the class activation graph of the training set is introduced into an SAM optimization network, and a training set pseudo tag is generated; constructing a fully supervised change detection network; transmitting the training set and the training set pseudo tag into a fully supervised change detection network, and training the training set and the training set pseudo tag; and reading the test set, and inputting the test set into the trained fully-supervised change detection network for prediction to obtain a change detection result. According to the invention, the precision and reliability of change detection under image-level supervision are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a remote sensing image change detection method based on semantic guidance and SAM optimization. Background Art

[0002] Change detection in remote sensing imagery involves analyzing multi-temporal remote sensing images acquired over the same area at different times to observe and identify surface changes. It has been widely used in fields such as land management, environmental monitoring, disaster assessment, and urban planning. Traditional change detection techniques rely on domain knowledge and require the manual design of spectral, spatial, or contextual features. These techniques struggle with complex scenarios and are prone to numerous false detections and missed detections. Driven by big data and artificial intelligence, deep learning technology, with its hierarchical feature extraction capabilities and powerful nonlinear modeling capabilities, has significantly advanced the automation of remote sensing image interpretation. In this context, network architectures such as convolutional neural networks, visual transformers, and Mamba have been widely applied to change detection tasks, effectively improving change detection performance across diverse scenarios. However, these methods typically rely on pixel-level labels, and the cumbersome and costly process of obtaining high-quality pixel-level annotations severely hinders the further development of fully supervised change detection methods in practical applications.

[0003] Consequently, weakly supervised learning has attracted widespread attention in the field of change detection. This approach utilizes coarse-grained supervision (e.g., image-level, point-level, and bounding box-level labels) to drive model learning, providing an effective solution to the difficulty of pixel-level labeling in complex scenes. Image-level labeling requires only binary annotations to indicate whether a bi-temporal image pair contains a changed pixel, without requiring annotation of the changed location, object shape, or boundary information. This approach is considered the most cost-effective option among weak labeling approaches. Khan et al. first introduced image-level supervision to the change detection task, employing a conditional random field (CRF) model to integrate local differences and contextual information through dense connections between pixels to achieve pixel-level predictions. Numerous methods have been developed to improve the performance of image-level supervised change detection. For example, adversarial learning is employed to optimize the model's ability to model and recognize spectral and spatial variations in multi-temporal imagery through iterative training. Another example is background mixture enhancement, which improves instance recognition accuracy by learning the consistency of objects under background changes. Inspired by physical-informed machine learning, existing techniques use image-level labels to constrain the pixel-level predictions output by the network, imposing a global penalty for incorrect predictions. This method expands prior information at the prediction level, effectively addressing the problems of false detection and missed detection in prediction results. In the aforementioned studies, Class Activation Maps (CAMs) have been widely used due to their significant localization capabilities. However, due to the lack of precise boundary priors, their inherent attentional focus often causes the decision boundary to favor the geometric center of highly responsive regions. This "center bias" effect further leads to activation semantic bias and boundary blurring, compromising object segmentation accuracy. To this end, some studies have incorporated the excellent zero-shot segmentation capabilities of the SegmentAnything Model (SAM) into weakly supervised learning to improve object segmentation. For example, the rough localization of the CAM is converted into a box cue for the target object, which is used to prompt the SAM to segment the changing object. Another example is calculating local peak points from the CAM generated for semantic segmentation as point cues, and then using the SAM to segment the target object. Although these methods effectively improve the boundary information of segmented objects, single cues still have certain limitations. For example, point cue strategies can easily cause cognitive confusion in multi-object scenarios, resulting in a significant expansion of the segmented region; while box cue strategies require that the cue box closely matches the target object's morphology. This places high demands on the quality of the initial mask generated by the class activation map. When the class activation map is incompletely activated or when dense objects are insufficiently separated, it will directly lead to structural distortion of the segmentation results. Summary of the Invention

[0004] The present invention aims to solve the semantic deviation and boundary ambiguity problems caused by the ambiguity of the supervision signal and the positioning defects of the class activation map in the existing weakly supervised change detection task, and provide a remote sensing image change detection method based on semantic guidance and SAM optimization to further improve the accuracy and reliability of change detection under image-level supervision.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A remote sensing image change detection method based on semantic guidance and SAM optimization includes the following steps:

[0007] Step 1: collect change detection data and segment the change detection data to construct a change detection dataset; the change detection data includes T1 phase remote sensing images, T2 phase remote sensing images and real pixel-level change label maps;

[0008] Step 2: Preprocess the change detection dataset and divide it into training set, test set, and validation set; the training set, test set, and validation set all contain dual-temporal remote sensing image pairs and corresponding image-level labels;

[0009] Step 3: Build a weakly supervised change detection network;

[0010] Step 4: Input the training set into the weakly supervised change detection network to train it;

[0011] Step 5: Only the dual-temporal remote sensing image pairs in the training set are input into the trained weakly supervised change detection network to obtain the class activation map of the training set;

[0012] Step 6: Build a SAM optimization network;

[0013] Step 7: The class activation map of the training set is passed into the SAM optimization network to generate pseudo labels for the training set;

[0014] Step 8: Build a fully supervised change detection network;

[0015] Step 9: Pass the training set and the training set pseudo labels into the fully supervised change detection network to train it;

[0016] Step 10: Read the test set and input it into the trained fully supervised change detection network for prediction to obtain the change detection result.

[0017] To optimize the above technical solutions, specific measures taken also include:

[0018] Furthermore, in step 1, the segmentation of the change detection data to construct the change detection data set is specifically as follows:

[0019] The T1 phase remote sensing image, T2 phase remote sensing image and true pixel-level change label map are cropped into several image blocks. Image-level labels are generated based on whether the segmented change label map contains changed pixels. An image-level label value of 1 indicates that the image pair contains a changed area and is recorded as a changed bi-temporal remote sensing image pair. A label value of 0 indicates that the image pair does not contain a changed area and is recorded as an unchanged bi-temporal remote sensing image pair.

[0020] Furthermore, in step 2, the preprocessing of the change detection dataset specifically includes data normalization and tensor conversion.

[0021] Furthermore, step 3 specifically includes:

[0022] Step 3.1, construct the feature extraction network, specifically: adopt the teacher-student network architecture, both the teacher network and the student network use the twin ResNet50 residual network, the twin structure internal weights are shared, the teacher network and the student network do not share weights, and are updated independently. The teacher network and the student network both include encoders, feature fusion modules and classifiers, and the encoder is used to extract the features of the dual-phase remote sensing image pair. After the features are combined, the fusion features are generated. The fusion features generated by the teacher network are recorded as G T , the fusion feature generated by the student network is recorded as G S , expressed as follows:

[0023] G T =BN(Conv 1×1 (abs(F1,F2)))

[0024] G s =BN(Conv 1×1 (cat(F1,F2)))

[0025] In the formula, F1 and F2 represent the T1 and T2 phase features extracted by the encoder, abs(·) represents absolute difference calculation, cat(·) represents channel dimension splicing, BN(·) represents batch normalization, and Conv 1×1 It is a 1×1 convolution block used to generate change probability;

[0026] G T and G s After inputting the classifier respectively and normalizing, the class activation map CAM of the teacher network is obtained. T and the class activation map CAM of the student network S , the formula is as follows:

[0027] CAM S =σ(G s )

[0028]

[0029] Where σ(·) represents the Sigmoid activation function, ReLU(·) represents the linear rectification activation function, and max(·) represents the maximum value operation;

[0030] G T Input to the projection head to obtain the projection feature Z T , used for feature space modeling in subsequent semantic learning modules;

[0031] To G T Perform global average pooling and calculate image-level classification loss with image-level labels As shown in the following formula:

[0032]

[0033] Among them, N is the total number of samples, y i represents the image-level label, σ(·) represents the activation function, GAP represents the global average pooling, G T,i represents the i-th fusion feature generated by the teacher model;

[0034] Step 3.2: Construct a semantic learning module. The semantic learning module is set in the teacher network. Specifically, semantic learning includes two processes: cross-image consistency learning and cross-pixel similarity learning.

[0035] In the cross-image consistency learning part, for the set of class activation maps of all invariant dual-temporal remote sensing image pairs output by the teacher network, pixels with Bottom-K activation values are selected, and the weighted average pixel-level projection embedding is calculated based on the invariant probability value of the pixel as the global invariant semantic prototype. The calculation process is as follows:

[0036]

[0037] Where Ω0 represents the set of pixels with Bottom-K activation values whose image-level label is 0 in the class activation map of all invariant dual-temporal remote sensing image pairs output by the teacher network, represents the probability of pixel change, represents the teacher network feature embedding corresponding to the specified pixel position, Z label＝0 Represents the set of projected features with image-level labels of 0, and K is a hyperparameter used to control the screening weight;

[0038] Use exponential moving average method to update the global invariant semantic prototype

[0039] The updated global invariant semantic prototype is used to calculate the similarity loss between the pixel features that satisfy Bottom-K in all dual-temporal remote sensing image pairs. Use it as a shared feature to guide the alignment of the overall feature space and similarity loss The calculation formula is as follows:

[0040]

[0041] Where N K represents the number of pixels with Bottom-K activation values that are filtered, z k Indicates the projection feature corresponding to the j-th pixel;

[0042] Preserve pixel-level binary cross entropy loss As a global scale prior:

[0043]

[0044] Among them, N 0 Indicates the number of samples with image-level label 0, y p,i represents the pixel-level label of the i-th sample, Represents the class activation map output by the teacher network for the bi-temporal remote sensing image pair with the i-th image-level label 0;

[0045] Total loss for learning consistency across images is the similarity loss With binary cross entropy loss The sum is expressed as:

[0046]

[0047] In the cross-pixel similarity learning part, reliable seeds are collaboratively selected based on the preliminary prediction results of the teacher network and the student network, and the local semantic prototype is calculated based on the seed point features. The formula is as follows:

[0048]

[0049] Where, represents the projection feature extracted by the teacher network for the i-th seed of category c, N c represents the number of seeds of category c, is the local semantic prototype of category c, Z 1 Represents the image projection feature with an image-level label of 1;

[0050] A Gaussian distribution is established based on the distance from other pixel features to the local semantic prototype. After the argmax operation, the most reliable semantic category is retained as the classification result. The formula is as follows:

[0051]

[0052] Where, is the predicted category after argmax operation, represents the projection feature extracted by the teacher network for the i-th seed of category c, is the local semantic prototype of category c;

[0053] Set contrast affinity loss based on classification results To constrain cross-pixel similarity learning, the expression of contrast affinity loss is as follows:

[0054]

[0055] In the formula, C represents the total number of categories, is the pixel projection feature predicted to be category c, and Respectively represent the number of positive pixels and negative pixels of category c in the prediction results;

[0056] Step 3.3, set up a multi-stage knowledge distillation strategy, which is performed at the initial output of the teacher network and the end of semantic learning. The first distillation is oriented to all samples, by minimizing CAM T and CAM s The MSE loss between the initial distillation loss The calculation process is as follows:

[0057]

[0058] Where, CAM T,i represents the class activation map output by the teacher network for the i-th dual-temporal remote sensing image pair, CAM s,i represents the class activation map output by the student network for the i-th dual-temporal remote sensing image pair, and N represents the total number of samples;

[0059] Secondary distillation is used for changing dual-temporal remote sensing image pairs. It adopts the self-supervision idea, uses the probability distribution after semantic learning module modeling as soft label, uses seed points as hard label, and uses self-supervision loss. The calculation process is as follows:

[0060]

[0061] Where, represents the soft label supervision loss, N 1 Indicates the number of samples with image-level label 1, represents the probability distribution of the output of the semantic learning module modeling for the i-th sample, represents the image projection feature with the image-level label of 1, Represents the class activation map output by the student network for the bi-temporal remote sensing image pair with the i-th image-level label of 1; represents the hard label supervision loss, N Seedsis the number of samples containing seeds, Indicates the category label indicated by the seed point.

[0062] Furthermore, in step 4, the total loss when training the weakly supervised change detection network is the image-level classification loss Total loss for learning consistency across images First distillation loss Comparative affinity loss With self-supervised loss The total loss function expression when training the weakly supervised change detection network is:

[0063]

[0064] Where, is the total loss when training the weakly supervised change detection network;

[0065] Input the training set into the weakly supervised change detection network and calculate the total loss Through continuous iterations of backpropagation, the parameters of the weakly supervised change detection network are gradually optimized until the weakly supervised change detection network converges.

[0066] Furthermore, step 5 is specifically as follows: only the dual-temporal remote sensing image pairs in the training set are input into the trained weakly supervised change detection network, the dual-temporal remote sensing image pairs are scaled, and the class activation map of the training set is obtained through multi-scale inference. The calculation process is as follows:

[0067]

[0068] Among them, scale is the serial number of the scaling scale, CAM scale Represents the class activation map of the training set obtained by inference after scaling by the scale-th scaling scale, CAM fuesd Represents the class activation map of the final training set obtained through multi-scale inference.

[0069] Furthermore, step 7 specifically includes:

[0070] Step 7.1, calculate the segmentation threshold;

[0071] Step 7.2: Use the segmentation threshold to perform the initial segmentation on the class activation map of the training set to obtain the GMM mask

[0072] Step 7.3: Calculate the local peak points of the training set class activation map and use the GMM mask Filtering available peak points from local peak points as point prompts; the point prompts are used to locate the accurate position of the segmented object;

[0073] Step 7.4: Expand the area to be segmented outward according to the distribution of the point hints to generate a box hint, which is used to constrain the segmentation range of the SAM;

[0074] Step 7.5: SAM segments the class activation map of the training set based on the box prompt to obtain the pseudo label of the training set.

[0075] Furthermore, in step 8, the fully supervised change detection network is a twin network based on DeepLabV3+, including an encoder, a void space pyramid pooling module and a network output head; the encoder adopts ResNet50.

[0076] Furthermore, in step 9, the loss function used to train the fully supervised change detection network is the binary cross entropy loss function, which is expressed as:

[0077]

[0078] Where, is the binary cross entropy loss, p (i) is the predicted probability of the i-th sample, is the pseudo label, and N is the number of samples.

[0079] Furthermore, in step 7.1, the segmentation threshold is calculated as follows:

[0080] A two-component mixed Gaussian model is used to establish a Gaussian distribution for the class activation map of the training set, and the mean of the foreground and background of the class activation map of the training set is obtained. The segmentation threshold is calculated using the mean of the foreground and background. The formula is as follows:

[0081]

[0082] Where, T GMM is the segmentation threshold, μ1 is the mean of the foreground, μ2 is the mean of the background, and α is the enhancement factor.

[0083] The beneficial effects of the present invention are:

[0084] 1. A weakly supervised change detection method based on semantic guidance and SAM optimization strategy is proposed, which can effectively improve the problems of category cognition ambiguity and pseudo-label boundary information loss. It can obtain pixel-level change detection results using only image-level labels, thereby improving the efficient use of labels.

[0085] 2. In the semantic guidance part, a semantic learning module combining cross-image consistency learning (CIC) and cross-pixel similarity learning (CPS) was constructed based on prior knowledge. It optimizes global and local features respectively, deeply mines the reliable information of "unchanged" images, and independently models the "changing" images after globally constraining and optimizing the feature space with invariant semantic prototypes, thereby optimizing the network's ability to recognize semantic information.

[0086] 3. In the SAM optimization phase, a hint generation module was designed to convert the information contained in the class activation map into point and box hints that can be used by SAM, thereby achieving pixel-level pseudo-label conversion based on SAM. The combination of these two hints can enhance SAM's understanding of target information and effectively reduce its dependence on the initial mask. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It is the overall framework diagram of the method proposed in the present invention.

[0088] Figure 2 is a flowchart for training a weakly supervised change detection network.

[0089] Figure 3 This is the structure diagram of the semantic learning module.

[0090] Figure 4 It is a flow chart of SAM optimization network.

[0091] Figure 5 These are some of the detection results of the present invention on the WHU-CD dataset. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0093] Example 1

[0094] This paper proposes a remote sensing image change detection method based on semantic guidance and SAM optimization. The overall framework of this method is as follows: Figure 1 As shown, the following steps are included:

[0095] Step 1: collect change detection data and segment the change detection data to construct a change detection dataset; the change detection data includes T1 phase remote sensing images, T2 phase remote sensing images, and real pixel-level change label maps; segmenting the change detection data to construct the change detection dataset is specifically as follows:

[0096] The T1 and T2 temporal remote sensing images, along with the ground-truth pixel-level change label map, were cropped into 256×256 image blocks. Image-level labels were generated based on whether the segmented change label map contained changed pixels. Image-level labels range from ∈ {0, 1}, where a value of 1 indicates that the image pair contains a changed region, denoted as a changed bi-temporal remote sensing image pair; a label of 0 indicates that no changed region exists, denoted as an unchanged bi-temporal remote sensing image pair. After segmentation, a total of 7434 bi-temporal remote sensing image pairs were generated, including 5544 unchanged bi-temporal remote sensing image pairs and 1890 changed bi-temporal remote sensing image pairs.

[0097] Step 2: Preprocess the change detection dataset and divide it into training, test, and validation sets. Preprocessing specifically includes data normalization and tensor transformations. The training, test, and validation sets all contain pairs of bi-temporal remote sensing images and their corresponding image-level labels. The training, validation, and test sets contain 5,947, 743, and 744 pairs of samples, respectively. The training set is used for network training, the validation set is used to verify model convergence during training and is not involved in network training. The test set is used to test the model after training and evaluate its generalization ability.

[0098] Step 3: Build a weakly supervised change detection network. The weakly supervised change detection network structure is as follows: Figure 2 As shown, step 3 specifically includes:

[0099] Step 3.1, construct the feature extraction network, specifically: adopt the teacher-student network architecture, both the teacher network and the student network use the twin ResNet50 residual network, the twin structure internal weights are shared, the teacher network and the student network do not share weights, and are updated independently. The teacher network and the student network both include encoders, feature fusion modules and classifiers, and the encoder is used to extract the features of the dual-phase remote sensing image pair. After the features are combined, the fusion features are generated. The fusion features generated by the teacher network are recorded as G T , the fusion feature generated by the student network is recorded as G S , expressed as follows:

[0100] G T =BN(Conv 1×1 (abs(F1,F2)))

[0101] G s =BN(Conv 1×1(cat(F1,F2)))

[0102] In the formula, F1 and F2 represent the T1 and T2 phase features extracted by the encoder, abs(·) represents absolute difference calculation, cat(·) represents channel dimension splicing, BN(·) represents batch normalization, and Conv 1×1 It is a 1×1 convolution block used to generate change probability;

[0103] G T and G S After inputting the classifier respectively and normalizing, the class activation map CAM of the teacher network is obtained. T and the class activation map CAM of the student network S , the formula is as follows:

[0104] CAM S =σ(G s )

[0105]

[0106] Where σ(·) represents the Sigmoid activation function, ReLU(·) represents the linear rectification activation function, and max(·) represents the maximum value operation;

[0107] G T Input to the projection head to obtain the projection feature Z T , used for feature space modeling in subsequent semantic learning modules;

[0108] To G T Perform global average pooling and calculate image-level classification loss with image-level labels As shown in the following formula:

[0109]

[0110] Among them, N is the total number of samples, y i represents the image-level label, σ(·) represents the activation function, GAP represents the global average pooling, G T,i represents the i-th fusion feature generated by the teacher model;

[0111] Step 3.2, construct the semantic learning module, such as Figure 3 As shown, the semantic learning module is set in the teacher network, specifically: semantic learning includes two processes: cross-image consistency learning and cross-pixel similarity learning;

[0112] In the cross-image consistency learning part, for the class activation map set of all invariant dual-temporal remote sensing image pairs (i.e., image pairs with image-level labels of 0) output by the teacher network, pixels with Bottom-K activation values are selected, and the weighted average pixel-level projection embedding is calculated based on the invariant probability value of the pixel as the global invariant semantic prototype. The calculation process is as follows:

[0113]

[0114] Where Ω0 represents the set of pixels with Bottom-K activation values whose image-level label is 0 in the class activation map of all invariant dual-temporal remote sensing image pairs output by the teacher network, represents the pixel change probability, represents the teacher network feature embedding corresponding to the specified pixel position, Z label＝0 represents the projected feature set with an image-level label of 0. K is a hyperparameter used to control the screening weight, which is set to 1 / 8 in this embodiment.

[0115] In order to make full use of the invariant pixel set, the exponential moving average method (EMA) is used to update the global invariant semantic prototype. By providing continuously updated prior information, the model improves its understanding of the relationship between instances. The process of updating the global invariant semantic prototype is as follows:

[0116]

[0117] in, and Respectively represent the global invariant semantic prototypes calculated at the i-th and (i-1)-th iterations during the training process, To update the weight, set it to 0.9.

[0118] The updated global invariant semantic prototype is used to calculate the similarity loss between the pixel features that satisfy Bottom-K in all dual-temporal remote sensing image pairs. Use it as a shared feature to guide the alignment of the overall feature space and similarity loss The calculation formula is as follows:

[0119]

[0120] Where N K represents the number of pixels with Bottom-K activation values that are filtered, z j Indicates the projection feature corresponding to the j-th pixel.

[0121] To ensure that the network clearly maintains the same semantic category, the pixel-level binary cross entropy loss is retained. As a global scale prior:

[0122]

[0123] Among them, N 0 Indicates the number of samples with image-level label 0, y p,i represents the pixel-level label of the i-th sample, Represents the class activation map output by the teacher network for the bi-temporal remote sensing image pair with the i-th image-level label 0;

[0124] Total loss for learning consistency across images is the similarity loss With binary cross entropy loss The sum is expressed as:

[0125]

[0126] In the cross-pixel similarity learning part, the focus is on the activation details of changing bi-temporal remote sensing image pairs (i.e., image pairs with an image-level label of 1). Based on the preliminary prediction results of the teacher network and the student network, reliable seeds are collaboratively selected, and local semantic prototypes are calculated based on the seed point features. The formula is as follows:

[0127]

[0128] Where, represents the projection feature extracted by the teacher network for the i-th seed of category c, N c represents the number of seeds of category c, is the local semantic prototype of category c, Z 1 Represents the image projection feature with an image-level label of 1;

[0129] A Gaussian distribution is established based on the distance from other pixel features to the local semantic prototype. After the argmax operation, the most reliable semantic category is retained as the classification result. The formula is as follows:

[0130]

[0131] Where, is the predicted category after argmax operation, represents the projection feature extracted by the teacher network for the i-th seed of category c, is the local semantic prototype of category c;

[0132] Set contrast affinity loss based on classification results To constrain cross-pixel similarity learning, the expression of contrast affinity loss is as follows:

[0133]

[0134] In the formula, C represents the total number of categories, which is set to 2. is the pixel projection feature predicted to be category c, and Respectively represent the number of positive pixels and negative pixels of category c in the prediction results;

[0135] Step 3.3, set up a multi-stage knowledge distillation strategy, which is performed at the initial output of the teacher network and the end of semantic learning. The first distillation is oriented to all samples, by minimizing CAM T and CAM S The MSE loss between the initial distillation loss The calculation process is as follows:

[0136]

[0137] Where, CAM T,i represents the class activation map output by the teacher network for the i-th dual-temporal remote sensing image pair, CAM S,i represents the class activation map output by the student network for the i-th dual-temporal remote sensing image pair, and N represents the total number of samples;

[0138] Secondary distillation is oriented towards changing dual-temporal remote sensing image pairs. It adopts the idea of self-supervision, uses the probability distribution after semantic learning module modeling as soft label, and uses seed points as hard label. The synergy of soft and hard strengthens the guidance of reliable information and improves the accuracy of class activation map. The calculation process is as follows:

[0139]

[0140] Where, represents the soft label supervision loss, N 1 Indicates the number of samples with image-level label 1, represents the probability distribution of the output of the semantic learning module modeling for the i-th sample, represents the image projection feature with the image-level label of 1, Represents the class activation map output by the student network for the bi-temporal remote sensing image pair with the i-th image-level label of 1; represents the hard label supervision loss, N Seeds is the number of samples containing seeds, Indicates the category label indicated by the seed point.

[0141] Step 4: Input the training set into the weakly supervised change detection network and train it; the total loss when training the weakly supervised change detection network is the image-level classification loss Total loss for learning consistency across images First distillation loss Comparative affinity loss With self-supervised loss The total loss function expression when training the weakly supervised change detection network is:

[0142]

[0143] Where, is the total loss when training the weakly supervised change detection network;

[0144] Input the training set into the weakly supervised change detection network and calculate the total loss Through continuous iterations of backpropagation, the parameters of the weakly supervised change detection network are gradually optimized until the weakly supervised change detection network converges.

[0145] In step 5, only the dual-temporal remote sensing image pairs in the training set are input into the trained weakly supervised change detection network, the dual-temporal remote sensing image pairs are scaled, and the class activation map of the training set is obtained through multi-scale inference. The calculation process is as follows:

[0146]

[0147] Among them, scale is the serial number of the scaling scale, scaling scale ∈ {0.5, 1.0, 1.5, 2.0}, CAM scale Represents the class activation map of the training set obtained by inference after scaling by the scale-th scaling scale, CAM fuesd Represents the class activation map of the final training set obtained through multi-scale inference.

[0148] Step 6: Construct a SAM (Segment Anything Model) optimization network. The SAM (Segment Anything Model) optimization network includes a prompt generation module and a SAM (Segment Anything Model).

[0149] Step 7: The class activation map of the training set is passed into the SAM optimization network to generate pseudo labels for the training set; Figure 4 Specifically including:

[0150] Step 7.1: Calculate the segmentation threshold; specifically:

[0151] A two-component mixture Gaussian model is used to establish a Gaussian distribution for the class activation map of the training set, and the mean of the foreground and background of the class activation map of the training set is obtained. The probability distribution function of the Gaussian distribution is expressed as:

[0152]

[0153] Where μ1 and μ2 are the means of foreground and background respectively, are the variances of foreground and background respectively, w1 and w2 are weight parameters, satisfying w1+w2=1.

[0154] In order to reduce the influence of edge noise, the enhancement factor α is introduced to adaptively optimize the segmentation threshold according to the mean difference between the foreground and background. The formula is as follows:

[0155]

[0156] Where, T GMM is the segmentation threshold, μ1 is the mean of the foreground, μ2 is the mean of the background, and α is the enhancement factor.

[0157] Step 7.2: Use the segmentation threshold to perform the initial segmentation on the class activation map of the training set to obtain the GMM mask

[0158] Step 7.3: Calculate the local peak points of the training set class activation map and use the GMM mask Filtering available peak points from local peak points as point prompts; the point prompts are used to locate the accurate position of the segmented object;

[0159] Step 7.4: Expand the area to be segmented outward according to the distribution of the point hints to generate a box hint, which is used to constrain the segmentation range of the SAM. Specifically, the following steps are performed:

[0160] Step 7.4.1: Use the peak value range to split the GMM mask twice to obtain the peak mask

[0161] Step 7.4.2: Calculate each peak mask separately The average features of independent objects in the bi-temporal SAM latent space are used as the query vector F query , retrieve the query vector in the bi-temporal feature space F SAM Affinity response to obtain affinity mask The affinity mask is calculated as follows:

[0162]

[0163] Where γ is the similarity threshold.

[0164] Step 7.4.3: Considering that the query vector may be similar to multiple objects in the same image, we take the affinity mask of the dual-temporal remote sensing image. Peak mask with input or GMM mask The result with the highest overlap degree is used to determine the time phase to which the query object belongs; in order to determine the affinity mask of the time phase The independent mask in is used as the initial query object, and the peak mask and GMM mask Find the object mask with the highest overlap as the initial mask for extended area judgment If the query object does not exceed the initial mask If the corresponding expandable object is not found, the input peak mask is retained. Independent objects are used as the basis for generating frame hints. During the frame hint generation process, a distance of at least 25 pixels from the point hint is required to ensure the independence of small target objects.

[0165] The intersection over union (IoU) is used to evaluate the superposition effect. The two superposition judgment processes are the same and the judgment process is as follows:

[0166]

[0167] Where, IoU i represents the intersection-over-union ratio of the i-th object, Indicates the mask to be judged, and Represent the peak mask and GMM mask corresponding to the i-th object respectively, and finally the mask can be expanded The calculation process is as follows:

[0168]

[0169] In the formula, γ is the similarity threshold, area expand For expansion area. is the jth object in the affinity mask

[0170] Step 7.4.4: Based on scalable mask A bounding box is generated for the expanded object, which serves as a box hint for subsequent SAM segmentation. The combination of point hints and box hints can not only reduce the segmentation requirements of the initial coarse mask, but also fully utilize the position information provided by the class activation map to effectively optimize the segmentation results.

[0171] Step 7.5: SAM segments the class activation map of the training set based on the box prompt to obtain the pseudo label of the training set.

[0172] Step 8: Construct a fully supervised change detection network. The fully supervised change detection network is a twin network based on DeepLabV3+, including an encoder, a dilated spatial pyramid pooling module, and a network output head. The encoder uses ResNet50. The encoder is used to extract the features of the dual-temporal remote sensing image, and the output of the first layer of the encoder is used as the low-level feature F1. low and F2low , F1 low Represents the low-level features of remote sensing images in the T1 phase, F2 low Represents the low-level features of the remote sensing image at the T2 phase, and the output of the fourth layer of the encoder is used as the high-level feature F1 high and F2 high , F1 high Represents the high-level features of remote sensing images in the T1 phase, F2 high The high-level features of the remote sensing image in the T2 phase are processed and passed to the dilated spatial pyramid pooling module, which is then connected with the low-level feature differences and passed to the decoder.

[0173] In step 9, the training set and the training set pseudo labels are passed into the fully supervised change detection network for training. The loss function used in training the fully supervised change detection network is the binary cross entropy loss function, which is expressed as:

[0174]

[0175] Where, is the binary cross entropy loss, p (i) is the predicted probability of the i-th sample, is the pseudo label, and N is the number of samples. After training, the pixel-level change detection results can be directly predicted for the dual-temporal image.

[0176] Step 10: Read the test set and input it into the trained fully supervised change detection network for prediction to obtain a change probability map. The category of each pixel is determined based on its probability value to generate the final change map.

[0177] The superiority of the method of the present invention is verified in combination with specific experiments. This experiment uses the WHU-CD dataset as the experimental data. After fixed-scale cropping, the training set, validation set, and test set have 5947, 743, and 744 pairs of 256×256 images, respectively. This experiment was carried out under the PyTorch deep learning framework. The weakly supervised training part was trained for a total of 20 epochs, the batchsize was set to 8, the initial learning rate was 0.001, and a polynomial decay strategy was adopted. The training process used the validation set change intersection over Union (IoU) as the evaluation indicator to save the model. The SAM optimizer weight mode was set to "sam2_hiera_large", and only frozen weights were used for image segmentation. The noise threshold was set to 0.3. The fully supervised change detection network used the AdamW optimizer, iteratively trained for 50 epochs, the initial learning rate was set to 0.0001, and the batchsize was set to 16.

[0178] To verify the superiority of our method, we compared it with nine weakly supervised change detection networks that use image-level labels: FCD-GAN, WSLCD, KD-MSI, MS-Former-256, CS-WSCDNet, BGMix, TransWCD, ESAM-CD, and ACWCD. The accuracy evaluation metrics for the detection results are Precision (Precision, Pre), Recall (Recall, Rec), F1-score (F1-score), Intersection over Union (IoU), and Overall Accuracy (OA). The specific test results are shown in Table 1.

[0179] Table 1 Performance comparison of different change detection methods

[0180]

[0181]

[0182] In Table 1, the data in bold italics represents the optimal result, the data with underlined lines represents the second-best result, and the data with underlined wavy lines represents the third-best result. As can be seen from Table 1, compared with other methods, our method achieves the best results on all four precision metrics and the second-best results on the recall metric, demonstrating the effectiveness and superiority of our method.

[0183] Figure 5 The following figure shows the detection results of different methods on the WHU-CD dataset. Figure 5 The first column is the first phase image, the second column is the second phase image, the third column is the change truth label, the fourth column is the FCD-GAN network detection result, the fifth column is the KD-MSI network detection result, the sixth column is the CS-WSCDNet network detection result, the seventh column is the BGMix network detection result, the eighth column is the WSLCD network detection result, the ninth column is the TransWCD network detection result, the tenth column is the MS-Former-256 network detection result, the eleventh column is the ACWCD network detection result, and the twelfth column is the detection result of the method of the present invention.

[0184] from Figure 5 It can be seen from the results that the method of the present invention achieves the best visual effect, can effectively identify objects of different sizes and densities, effectively suppresses the interference of spectral changes, and can accurately retain the building boundary information and avoid the adhesion of dense building detection results.

[0185] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0186] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A remote sensing image change detection method based on semantic guidance and SAM optimization, characterized in that: The following steps are involved: Step 1: collect change detection data and segment the change detection data to construct a change detection dataset; the change detection data includes T1 phase remote sensing images, T2 phase remote sensing images and real pixel-level change label maps; Step 2: Preprocess the change detection dataset and divide it into training set, test set, and validation set; the training set, test set, and validation set all contain dual-temporal remote sensing image pairs and corresponding image-level labels; Step 3: Build a weakly supervised change detection network; Step 4: Input the training set into the weakly supervised change detection network to train it; Step 5: Only the dual-temporal remote sensing image pairs in the training set are input into the trained weakly supervised change detection network to obtain the class activation map of the training set; Step 6: Build a SAM optimization network; Step 7: The class activation map of the training set is passed into the SAM optimization network to generate pseudo labels for the training set; Step 8: Build a fully supervised change detection network; Step 9: Pass the training set and the training set pseudo labels into the fully supervised change detection network to train it; Step 10: Read the test set and input it into the trained fully supervised change detection network for prediction to obtain the change detection result.

2. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 1, characterized in that: In step 1, the change detection data is segmented to construct a change detection dataset as follows: The T1 phase remote sensing image, T2 phase remote sensing image and true pixel-level change label map are cropped into several image blocks. Image-level labels are generated based on whether the segmented change label map contains changed pixels. An image-level label value of 1 indicates that the image pair contains a changed area and is recorded as a changed bi-temporal remote sensing image pair. A label value of 0 indicates that the image pair does not contain a changed area and is recorded as an unchanged bi-temporal remote sensing image pair.

3. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 1, characterized in that: In step 2, the preprocessing of the change detection dataset specifically includes data normalization and tensor conversion.

4. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 1, characterized in that: Step 3 specifically includes: Step 3.1, construct the feature extraction network, specifically: adopt the teacher-student network architecture, both the teacher network and the student network use the twin ResNet50 residual network, the twin structure internal weights are shared, the teacher network and the student network do not share weights, and are updated independently. The teacher network and the student network both include encoders, feature fusion modules and classifiers, and the encoder is used to extract the features of the dual-phase remote sensing image pair. After the features are combined, the fusion features are generated. The fusion features generated by the teacher network are recorded as G T , the fusion feature generated by the student network is recorded as G s , expressed as follows: G T =BN(Conv 1×1 (abs(F1,F2))) G s =BN(Conv 1×1 (cat(F1,F2))) In the formula, F1 and F2 represent the T1 and T2 phase features extracted by the encoder, abs(·) represents absolute difference calculation, cat(·) represents channel dimension splicing, BN(·) represents batch normalization, and Conv 1×1 It is a 1×1 convolution block used to generate change probability; G T and G S After inputting the classifier respectively and normalizing, the class activation map CAM of the teacher network is obtained. T and the class activation map CAM of the student network S , the formula is as follows: CAM S =σ(G s ) Where σ(·) represents the Sigmoid activation function, ReLU(·) represents the linear rectification activation function, and max(·) represents the maximum value operation; G T Input to the projection head to obtain the projection feature Z T , used for feature space modeling in subsequent semantic learning modules; To G T Perform global average pooling and calculate image-level classification loss with image-level labels As shown in the following formula: Among them, N is the total number of samples, y i represents the image-level label, σ(·) represents the activation function, GAP represents the global average pooling, G T,i represents the i-th fusion feature generated by the teacher model; Step 3.2: Construct a semantic learning module. The semantic learning module is set in the teacher network. Specifically, semantic learning includes two processes: cross-image consistency learning and cross-pixel similarity learning. In the cross-image consistency learning part, for the set of class activation maps of all invariant dual-temporal remote sensing image pairs output by the teacher network, pixels with Bottom-K activation values are selected, and the weighted average pixel-level projection embedding is calculated based on the invariant probability value of the pixel as the global invariant semantic prototype. The calculation process is as follows: Where Ω0 represents the set of pixels with Bottom-K activation values whose image-level label is 0 in the class activation map of all invariant dual-temporal remote sensing image pairs output by the teacher network, represents the pixel change probability, represents the teacher network feature embedding corresponding to the specified pixel position, Z label＝0 Represents the set of projected features with image-level labels of 0, and K is a hyperparameter used to control the screening weight; Use exponential moving average method to update the global invariant semantic prototype The updated global invariant semantic prototype is used to calculate the similarity loss between the pixel features that satisfy Bottom-K in all dual-temporal remote sensing image pairs. Use it as a shared feature to guide the alignment of the overall feature space and similarity loss The calculation formula is as follows: Where N K represents the number of pixels with Bottom-K activation values that are filtered, z j Indicates the projection feature corresponding to the j-th pixel. Preserve pixel-level binary cross entropy loss As a global scale prior: Among them, N 0 Indicates the number of samples with image-level label 0, y p,i represents the pixel-level label of the i-th sample, Represents the class activation map output by the teacher network for the bi-temporal remote sensing image pair with the i-th image-level label 0; Total loss for learning consistency across images is the similarity loss With binary cross entropy loss The sum is expressed as: In the cross-pixel similarity learning part, reliable seeds are collaboratively selected based on the preliminary prediction results of the teacher network and the student network, and the local semantic prototype is calculated based on the seed point features. The formula is as follows: Where, represents the projection feature extracted by the teacher network for the i-th seed of category c, N c represents the number of seeds of category c, is the local semantic prototype of category c, Z 1 Represents the image projection feature with an image-level label of 1; A Gaussian distribution is established based on the distance from other pixel features to the local semantic prototype. After the argmax operation, the most reliable semantic category is retained as the classification result. The formula is as follows: Where, is the predicted category after argmax operation, represents the projection feature extracted by the teacher network for the i-th seed of category c, is the local semantic prototype of category c; Set contrast affinity loss based on classification results To constrain cross-pixel similarity learning, the expression of contrast affinity loss is as follows: In the formula, C represents the total number of categories, is the pixel projection feature predicted to be category c, and Respectively represent the number of positive pixels and negative pixels of category c in the prediction results; Step 3.3, set up a multi-stage knowledge distillation strategy, which is performed at the initial output of the teacher network and the end of semantic learning. The first distillation is oriented to all samples, by minimizing CAM T and CAM S The MSE loss between the initial distillation loss The calculation process is as follows: Where, CAM T,i represents the class activation map output by the teacher network for the i-th dual-temporal remote sensing image pair, CAM S,i represents the class activation map output by the student network for the i-th dual-temporal remote sensing image pair, and N represents the total number of samples; Secondary distillation is used for changing dual-temporal remote sensing image pairs. It adopts the self-supervision idea, uses the probability distribution after semantic learning module modeling as soft label, uses seed points as hard label, and uses self-supervision loss. The calculation process is as follows: Where, represents the soft label supervision loss, N 1 Indicates the number of samples with image-level label 1, represents the probability distribution of the output of the semantic learning module modeling for the i-th sample, represents the image projection feature with the image-level label of 1, Represents the class activation map output by the student network for the bi-temporal remote sensing image pair with the i-th image-level label of 1; represents the hard label supervision loss, N Seeds is the number of samples containing seeds, Indicates the category label indicated by the seed point.

5. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 1, characterized in that: In step 4, the total loss when training the weakly supervised change detection network is the image-level classification loss Total loss for learning consistency across images First distillation loss Comparative affinity loss With self-supervised loss The total loss function expression when training the weakly supervised change detection network is: Where, is the total loss when training the weakly supervised change detection network; Input the training set into the weakly supervised change detection network and calculate the total loss Through continuous iterations of backpropagation, the parameters of the weakly supervised change detection network are gradually optimized until the weakly supervised change detection network converges.

6. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 1, characterized in that: Step 5 is as follows: only the bi-temporal remote sensing image pairs in the training set are input into the trained weakly supervised change detection network, the bi-temporal remote sensing image pairs are scaled, and the class activation map of the training set is obtained through multi-scale inference. The calculation process is as follows: Among them, scale is the serial number of the scaling scale, CA scale Represents the class activation map of the training set obtained by inference after scaling by the scale-th scaling scale, CAM fuesd Represents the class activation map of the final training set obtained through multi-scale inference.

7. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 1, characterized in that: Step 7 specifically includes: Step 7.1, calculate the segmentation threshold; Step 7.2: Use the segmentation threshold to perform the initial segmentation on the class activation map of the training set to obtain the GMM mask Step 7.3: Calculate the local peak points of the training set class activation map and use the GMM mask Filtering available peak points from local peak points as point prompts; the point prompts are used to locate the accurate position of the segmented object; Step 7.4: Expand the area to be segmented outward according to the distribution of the point hints to generate a box hint, which is used to constrain the segmentation range of the SAM; Step 7.5: SAM segments the class activation map of the training set based on the box prompt to obtain the pseudo label of the training set.

8. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 1, characterized in that: In step 8, the fully supervised change detection network is a twin network based on DeepLabV3+, including an encoder, a void space pyramid pooling module and a network output head; the encoder adopts ResNet50.

9. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 1, characterized in that: In step 9, the loss function used to train the fully supervised change detection network is the binary cross entropy loss function, which is expressed as: Where, is the binary cross entropy loss, p (i) is the predicted probability of the i-th sample, is the pseudo label, and N is the number of samples.

10. The remote sensing image change detection method based on semantic guidance and SAM optimization according to claim 7, characterized in that: In step 7.1, the segmentation threshold is calculated as follows: A two-component mixed Gaussian model is used to establish a Gaussian distribution for the class activation map of the training set, and the mean of the foreground and background of the class activation map of the training set is obtained. The segmentation threshold is calculated using the mean of the foreground and background. The formula is as follows: Where, T GMM is the segmentation threshold, μ1 is the mean of the foreground, μ2 is the mean of the background, and α is the enhancement factor.

Images