A remote sensing change detection method, system, device and storage medium

Abstract

The application discloses a remote sensing change detection method, system, device and storage medium, and belongs to the field of remote sensing image processing, and comprises the following steps: feature extraction is performed on double-time-phase remote sensing images based on a twin residual encoder to obtain a plurality of feature pairs of different scales; pixel-mixed processing is performed on the feature pairs of different scales to obtain mixed pixel feature pairs; foreground-background separation is performed on the mixed pixel feature pairs to obtain feature pairs after foreground extraction, and difference features of the feature pairs after foreground extraction are calculated; attention processing is performed on the mixed pixel features to obtain attention features; and the attention features and the difference features are fused to obtain fused features; the pixel mixing and foreground-background difference processing proposed in the application achieve effective foreground-background separation effects, can significantly suppress the influence of non-target ground object change pixels on the detection results, and can improve the problem of poor change detection precision caused by irrelevant background changes and light differences of double-time-phase high-resolution remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing, and specifically to a remote sensing change detection method, system, device, and storage medium. Background Technology

[0002] Remote sensing image change detection plays a crucial role in investigating illegal construction in cities, assessing disaster losses, and planning urban and rural development. Utilizing dual-temporal high-resolution remote sensing images and deep learning techniques for change detection is highly effective, and many dual-temporal deep learning methods are currently available for high-resolution remote sensing image change detection.

[0003] In recent years, with the rapid development of deep learning change detection technology, more and more deep learning algorithms have achieved good change detection results. For example, Bandara et al. proposed a Transformer-based siamese network architecture, ChangeFormer, in their paper "A transformer-based siamese network for change detection." IGARSS2022-2022 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2022. This architecture is used for change detection in dual-temporal remote sensing images, thereby effectively presenting the multi-scale remote details required for accurate change detection and achieving a fairly high change detection accuracy. Blurred edges in change detection results are a major problem faced by current deep learning change detection algorithms. Existing methods have proposed solutions to address this issue. For example, Bai et al., in their paper "Edge-guided recurrent convolutional neural network for multitemporal remote sensing image building change detection." (IEEE Transactions on Geoscience and Remote Sensing 60(2021):1-13), proposed an edge-guided recurrent convolutional neural network change detection method, EGRCNN. This method uses LSTM for difference enhancement and an edge-guided probability prediction network to obtain an edge probability map, thereby improving the clarity of ground object edge detection contours. However, deep learning change detection algorithms like ChangeFormer do not incorporate ground object edge information as prior knowledge. Although they have high detection accuracy, the problem of unclear edge contours in the change detection results is significant. While deep learning algorithms like EGRCNN incorporate prior knowledge of ground object edges, resulting in clearer edge contours, the utilization of edge information is still insufficient, and the extracted feature information is not rich enough, limiting the improvement in change detection accuracy. Therefore, this application proposes a remote sensing change detection method, system, device, and storage medium. Summary of the Invention

[0004] To address the issue that existing technologies for change detection in dual-temporal high-resolution remote sensing images are easily affected by irrelevant changes, and that the accuracy of change detection results is limited by feature resolution and it is difficult to obtain clear and regular edges, this invention proposes a remote sensing change detection method, system, device, and storage medium to improve the effect of dual-temporal high-resolution remote sensing change detection.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] Firstly, this application discloses a remote sensing change detection method, comprising:

[0007] Feature extraction was performed on dual-temporal remote sensing images based on a twin residual encoder to obtain several feature pairs at different scales;

[0008] Feature pairs at different scales are pixel-wise blended to obtain blended pixel feature pairs; foreground and background separation is performed on the blended pixel feature pairs to obtain feature pairs after foreground extraction, and the difference features of the feature pairs after foreground extraction are calculated; attention processing is applied to the blended pixel features to obtain attention features; the attention features and difference features are fused to obtain fused features.

[0009] Edge fusion features are obtained by upsampling the fusion features of adjacent layers; all fusion features and edge fusion features are then classified by edge-guided deep supervision to obtain change detection results.

[0010] In some embodiments, the method further includes calculating the cross-entropy loss and edge loss during the processing of change detection results at each scale, and weighting the cross-entropy loss and edge loss to calculate the total loss during the processing of change detection results; training the change detection results using a backpropagation algorithm, iterating and adjusting the model weights during the training process with the goal of minimizing the total loss; performing forward propagation calculations using a validation set to obtain the change detection accuracy; after completing a pre-set number of training rounds, taking the model weight with the highest change detection accuracy as the optimal model weight, and recalculating the change detection results based on the optimal model weight.

[0011] In some embodiments, the edge loss is calculated based on the edge dynamically weighted binary cross-entropy loss function, the function expression of which is:

[0012]

[0013] In the formula, P red To predict the image, T arget The labels are real numbers, n is the number of images in each batch, and L is the number of images in each batch. dwbce The edge-dynamically weighted binary cross-entropy loss is used.

[0014] In the above formula, L pi With L ni The function expression is:

[0015] L pi =w pi ×Canny(T argeti )×log(P redi )

[0016] L ni =Canny(1-T)argeti )×log(1-P redi )

[0017] In the above formula, P redi For the i-th predicted image, T argeti For the i-th real label, Canny(T) argeti ) represents the edge true value, T edgei The edge represents the true label of the i-th image, Canny() is the edge detection operator, and w pi The function expression is:

[0018] w pi =N ui / N ci

[0019] In the above formula, N ci Let T be the true edge value of the i-th image. edgei The middle value is the number of pixels at the edge, N. ui Let T be the true edge value of the i-th image. edgei The number of pixels that are not on the edge.

[0020] In some embodiments, the twin residual encoder consists of two residual encoders; the two residual encoders share weights in their dual branches and are used to extract features from the dual-temporal remote sensing images T1 and T2, respectively.

[0021] Each residual encoder includes a 1×1 convolutional layer, a max pooling layer, and four residual layers. Each residual layer includes several residual blocks, and each residual block consists of a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer. The input of the residual block is processed by a 1×1 convolutional layer and then residually concatenated with the output of the residual block.

[0022] In some embodiments, the pixel blending process includes the following steps:

[0023] Determine whether the height and width of the feature are greater than the threshold values ​​of the height and width of the input remote sensing image;

[0024] If the feature pairs exceed the height and width thresholds of the remote sensing image, then feature pairs are swapped in the height and width plane with a step size s. All channel data corresponding to the pixel location; if it is less than the threshold of the height and width of the remote sensing image, then the feature pairs are swapped in the channel dimension with a step size s. All pixel data from the corresponding channels are used to obtain mixed pixel feature pairs.

[0025] In some embodiments, obtaining the edge blending features includes the following steps:

[0026] For fusion feature F j+1 Upsampling to and fusion feature F jWith the same height and width, the fused feature F j With fusion feature F j+1 The concatenation process yields concatenated result one; concatenated result one is then subjected to feature compression through a 1×1 convolutional layer five to obtain compressed feature one.

[0027] The compressed feature 1 extracts spatial features through a 3×3 convolutional layer 2; the spatial features extract linear features through a 1×3 convolutional layer and a 3×1 convolutional layer 1; the linear features and spatial features are concatenated to obtain concatenated result 2.

[0028] The cascaded result two is processed through a 1×1 convolutional layer six to obtain compressed feature two. Compressed feature two is then added to compressed feature one to obtain the edge fusion feature Fe. j .

[0029] In some embodiments, obtaining the change detection result includes the following steps:

[0030] Edge blending features Fe1~Fe J-1 Fusion feature F J Upsample to the same height and width as the remote sensing image, and obtain preliminary classification results for each feature through a classifier;

[0031] After the edge fusion feature Fe1 is upsampled, it is processed by 3×3 convolutional layer three and 3×3 convolutional layer four to obtain the processing result. The processing result is then passed through the sigmoid activation function to obtain the edge probability map Edge1. The edge probability map Edge1 and the preliminary classification result corresponding to the edge fusion feature Fe1 are concatenated in the channel dimension to obtain concatenated result three. Concatenated result three is then processed by 3×3 convolutional layer five, 3×3 convolutional layer six and sigmoid activation function to obtain the edge probability map Edge2. The edge probability map Edge2 and the preliminary classification result corresponding to the edge fusion feature Fe1 are then concatenated in the channel dimension to obtain concatenated result four. Concatenated result four is then passed through 3×3 convolutional layer seven and 3×3 convolutional layer eight to obtain the change detection result Output.

[0032] Secondly, this application discloses a remote sensing change detection system, including:

[0033] Feature extraction module: Based on the twin residual encoder, feature extraction is performed on the dual-temporal remote sensing images to obtain several feature pairs at different scales;

[0034] Feature fusion module: Performs pixel mixing on feature pairs of different scales to obtain mixed pixel feature pairs; performs foreground-background separation on the mixed pixel feature pairs to obtain feature pairs after foreground extraction, and calculates the difference features of the feature pairs after foreground extraction; performs attention processing on the mixed pixel features to obtain attention features; and fuses the attention features and difference features to obtain fused features.

[0035] Edge feature module: Upsamples the fused features of adjacent layers to obtain edge fused features;

[0036] Output module: All fused features and edge fused features are classified by edge-guided deep supervision to obtain change detection results.

[0037] Thirdly, this application discloses a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor, and when the processor loads and executes the computer program, it employs a remote sensing change detection method as described in the first aspect.

[0038] Fourthly, this application discloses a computer-readable storage medium storing a computer program, characterized in that, when the computer program is loaded and executed by a processor, it employs a remote sensing change detection method as described in the first aspect.

[0039] The beneficial effects of this invention are:

[0040] The pixel blending and foreground-background difference processing proposed in this application achieves effective foreground-background separation, which can significantly suppress the influence of non-target ground object change pixels on the detection results and improve the problem of poor change detection accuracy caused by irrelevant background changes and illumination differences in dual-temporal high-resolution remote sensing images.

[0041] The edge fusion proposed in this application achieves efficient fusion of deep semantic information and shallow texture information, resulting in edge fusion features that combine accurate semantic information and rich texture details.

[0042] The edge-guided deep supervised classification proposed in this application, combined with the edge-dynamically weighted binary cross-entropy loss, can fully utilize edge information and perform edge recognition based on the deep supervised classification results. This effectively eliminates irrelevant edge features and improves edge extraction efficiency. The use of the edge-dynamically weighted binary cross-entropy loss can solve the severe sample imbalance problem encountered in edge classification in edge-guided deep supervised classifiers, thereby improving the utilization of edge information. Attached Figure Description

[0043] The invention will now be further described with reference to the accompanying drawings.

[0044] Figure 1 This is a schematic diagram of the structure of the method according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the hybrid pixel foreground difference module structure proposed in an embodiment of the present invention;

[0046] Figure 3This is a schematic diagram of the foreground attention difference module structure proposed in an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the edge fusion module structure proposed in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of the edge-guided deep supervision classifier structure proposed in an embodiment of the present invention;

[0049] Figure 6 This is a comparison chart of test results between the method of this invention embodiment and other existing methods;

[0050] Figure 7 This is a schematic diagram comparing the change detection results of the method of this invention embodiment and other existing methods on the WHU-CD dataset;

[0051] Figure 8 This is a schematic diagram comparing the change detection results of the method of this invention and other existing methods on the LEVIR-CD dataset. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0054] A remote sensing change detection method includes the following steps:

[0055] Step 1: Extract features from the dual-temporal high-resolution remote sensing images T1 and T2 using a twin residual encoder to obtain J feature pairs at different scales. in The j-th layer features are extracted from the 1st and 2nd time phases, respectively; specifically, the following steps are included:

[0056] The twin residual encoder consists of two branches that share weights. The two branches are used to extract features from remote sensing images T1 and T2, respectively. Each residual encoder includes a 1×1 convolutional layer, a max-pooling layer, and residual layers HRE1 to HRE4. j Let represent the j-th residual layer. Each residual layer consists of n residual blocks. Each residual block is composed of a 1×1 convolutional layer 2, a 3×3 convolutional layer 1, and a 1×1 convolutional layer 3. The input of the residual block is processed by a 1×1 convolutional layer 4 and then residually connected with the output of the residual block.

[0057] Step 2: Process feature pairs at each layer Perform pixel blending, and apply the blended pixel features to... Perform foreground-background separation and extract foreground features. Calculate the difference feature d j Simultaneously, for mixed pixel features... After attention processing, it is compared with the difference feature d. j The fusion yields the fusion feature F j For j∈(1, J), the specific steps include:

[0058] The pixel fusion process first determines whether the feature height and width are greater than the thresholds of the input remote sensing image. If they are greater than the thresholds, the feature pairs are swapped in the height-width plane with a stride of s. All channel data corresponding to the pixel location; if it is less than the threshold of the height and width of the remote sensing image, then the feature pairs are swapped in the channel dimension with a step size s. All pixel data from the corresponding channels are used to obtain mixed pixel feature pairs. The threshold range is (1 / 4, 1 / 2).

[0059] Foreground / background separation processing refers to the processing of mixed pixel features. Attention processing is performed separately to obtain attention feature pairs. Attention feature pairs Perform n dilated convolutions, and then combine the dilated convolutions with the attention feature pairs. Perform residual connection to obtain the difference feature d j .

[0060] Attention processing refers to the processing of mixed pixel features Simultaneously, the maximum and average values ​​of all pixels in each channel are calculated. The two results are then added together, and the channel attention probability is calculated using the sigmoid activation function. This channel attention probability is then combined with the mixed pixel features. Multiply the results, then calculate the maximum and average values ​​of all channels for each pixel. Concatenate the two results and use the sigmoid activation function to calculate the spatial attention probability. Multiply this probability by the multiplication result to obtain the attention feature.

[0061] Step 3: Analyze the fusion features of adjacent layers, i.e., fusion feature F. j With fusion feature F j+1 Edge fusion upsampling is performed on j∈(1, J-1) to obtain edge fusion features Fe. j , j∈(1, J-1);

[0062] Edge fusion upsampling refers to first processing the fusion feature F j+1 Upsampling to and fusion feature F j With the same height and width, the fused feature F j With fusion feature F j+1 The concatenation process yields concatenated result one, which is then processed by a 1×1 convolutional layer five to obtain compressed feature one.

[0063] The compressed feature is extracted into spatial features through a 3×3 convolutional layer 2. The spatial features are input into a 1×3 convolutional layer 1 and a 3×1 convolutional layer 1 to extract linear features.

[0064] The linear and spatial features are concatenated to obtain concatenated result two. Concatenated result two is then compressed through a 1×1 convolutional layer to obtain compressed feature two. Finally, compressed feature two is added to compressed feature one to obtain the edge fusion feature Fe. j .

[0065] Step 4: Fuse the features F J and edge fusion features Fe1~Fe J-1 The change detection result Output is obtained through edge-guided deep supervised classification.

[0066] Edge-guided deep supervised classification refers to classifying edge fusion features Fe1 to Fe2 respectively. J-1 Fusion feature F J The image is upsampled to the same height and width as the remote sensing image. Each feature is then processed by a different classifier to obtain a preliminary classification result, i.e., preliminary classification results are obtained by classifiers one through four respectively.

[0067] After upsampling, the edge fusion feature Fe1 is input into 3×3 convolutional layer three and 3×3 convolutional layer four to obtain the processing result. The processing result is then processed by the sigmoid activation function to obtain the edge probability map Edge1. The edge probability map Edge1 and the preliminary classification result corresponding to the edge fusion feature Fe1 are concatenated in the channel dimension to obtain concatenated result three. Concatenated result three is then processed by 3×3 convolutional layer five, 3×3 convolutional layer six and sigmoid activation function to obtain the edge probability map Edge2. The edge probability map Edge2 and the preliminary classification result corresponding to the edge fusion feature Fe1 are then concatenated in the channel dimension. Finally, concatenated result four is input into 3×3 convolutional layer seven and 3×3 convolutional layer eight to obtain the change detection result Output.

[0068] Step 5: Use the change detection dataset to perform deep supervised edge-guided training on steps 1 to 4, save the best model, input the remote sensing image to be detected into the model, and obtain the change detection results.

[0069] Deep supervised edge-guided training refers to calculating the cross-entropy loss (L) for the preliminary classification results at each scale, the change detection results (Output), and the true change labels obtained in step 4. ce1 ~L ceJ and L ceout Calculate the edge dynamic weighted binary cross-entropy loss L for Edge1 and Edge2 and the actual change labels respectively. dwbce1 L dwbce2 For L ce1 ~L ce J, L ceout L dwbce1 L dwbce2 The total loss L is obtained by weighted summation. total .

[0070] During training, the model is trained using the backpropagation algorithm, iterating and adjusting the model weights with the goal of minimizing the total loss. After a predetermined number of training epochs, the model is switched to validation mode for validation. Forward propagation is performed on the validation set to calculate the change detection accuracy, and then the model is switched back to training mode to continue training. After completing the pre-set number of training epochs, the model weights with the highest change detection accuracy are saved as the optimal model weights.

[0071] The aforementioned edge-dynamically weighted binary cross-entropy loss L dwbce It is an improvement on the binary cross-entropy loss. The improvement lies in applying dynamic weights to the positive and negative sample losses. These weights change dynamically with the actual changes in the input. The number of edge pixels and background pixels in the label edge extraction result changes dynamically, increasing the weight of the positive sample loss and decreasing the weight of the negative sample loss.

[0072] The method for detecting sensory changes provided in this application will be further described below with reference to the accompanying drawings and embodiments; for example Figure 1 As shown, the remote sensing change detection method of this embodiment includes the following steps:

[0073] Step 1: Feature extraction is performed on the dual-temporal remote sensing images T1 and T2 using a twin residual encoder to obtain feature pairs at four different scales. in These are the j-th layer features extracted from the 1st and 2nd time phases, respectively, where j∈(1,4);

[0074] In this embodiment, the number of residual blocks in residual layers HRE1 to HRE4 are [3, 4, 6, 4], respectively. The stride of the 3×3 convolutional layer in the first residual block of HRE2 and HRE3 is 2. The dilation rates of the 3×3 convolutional layers corresponding to the four residual blocks in residual layer HRE4 are [6, 8, 12, 8], respectively. To maintain the feature height and width unchanged, the padding is set to [6, 8, 12, 8]. In this embodiment, the size of the dual-temporal remote sensing images T1 and T2 is 3×256×256. After feature extraction by the residual encoder, the features are obtained. in The size is 256×128×128. The size is 512×64×64. The size is 1024×32×32. The size is 2048×32×32.

[0075] Step 2: Process feature pairs at each layer Perform pixel blending, and apply the blended pixel features to... Perform foreground-background separation and extract foreground features. Calculate the difference feature d j Simultaneously, for mixed pixel features... After attention processing, the feature is fused with the difference feature to obtain the fused feature F. j , j∈(1,4);

[0076] like Figure 2 In this embodiment, the hybrid pixel foreground difference module consists of a hybrid pixel module (MPM) and a foreground attention difference module (FDM). This embodiment uses the hybrid pixel module (MPM) to perform feature pair analysis. Pixel blending is performed. In this embodiment, the feature shape is defined as c×h×w, where c corresponds to the channel dimension, h corresponds to the height dimension, and w corresponds to the width dimension. The blending pixel unit judges the h×w of the input feature. When h×w is greater than 64×64, the feature is blended in the pixel plane with a blending step size s. and characteristics Feature blending is performed to obtain blended pixel features. and mixed pixel features

[0077]

[0078] When h×w is less than 64×64, the features are paired in the channel dimension with a mixed step size s. and characteristics Feature blending is performed to obtain blended pixel features. and mixed pixel features

[0079]

[0080] This embodiment uses the Foreground Attention Difference (FDM) module to process mixed pixel feature pairs. Perform foreground / background separation processing. See also Figure 3 In step 2 of this embodiment, the foreground attention difference module (FDM) consists of two 1×1 convolutional layers, batch normalization, a corrected linear unit, and eight 3×3 dilated convolutional layers. The FDM has two branches that process... and The two branches share weights. The feature pairs of the blended pixels from each layer are then combined. Input attention module AM ​​to obtain attention feature pairs The attention feature pairs are compressed to one-eighth of their original number of channels using a 1×1 convolutional layer. After performing four dilated convolutions on the compressed features, each convolution is then compared with the compressed features. The results are summed, with each of the 3×3 dilated convolutional layers having a dilation rate of 3 and a padding of 3. The difference feature d is obtained by performing a difference calculation on the above results. j See also Figure 2 In this embodiment, the difference feature d is calculated. j Simultaneously, attention processing is applied to the mixed pixel features to obtain supplementary feature pairs. Then, on the channel dimension, the difference feature d j Supplementary features Cascading yields fusion feature F j .

[0081] This embodiment uses the attention module AM ​​to analyze mixed pixel feature pairs. Perform attention processing. See also Figure 3 First of all Channel-by-channel average pooling, 1×1 convolution, modified linear unit, and 1×1 convolution are performed to obtain the mean feature (avg), and then... Channel-by-channel max pooling, 1×1 convolution, modified linear unit, and 1×1 convolution are performed to obtain the maximum value feature max. Then, the average value and max are summed and passed through the sigmoid activation function to obtain the channel attention weight c. a Then to The mean (avg) and maximum (max) are calculated pixel by pixel. After concatenating avg and max along the channel dimension, the number of channels is compressed to half of the original number through a 1×1 convolution. Then, the spatial attention weights s are obtained through a sigmoid activation function. a c a s a and Attention features are obtained by multiplying using a broadcast mechanism.

[0082] Step 3: Analyze the fusion feature F j With fusion feature F j+1 Edge fusion upsampling is performed to obtain edge fusion features Fe. j ;

[0083] This embodiment uses the edge blending module EGFM to perform edge blending upsampling processing. See also Figure 4 The edge fusion module EGFM consists of two 1×1 convolutional layers, one 3×3 convolutional layer, one 1×3 convolutional layer, and one 3×1 convolutional layer. The edge fusion module first processes the fusion feature F... j+1 Bilinear interpolation upsampling to fused feature F j The height and width dimensions are the same, and then the fused features F j+1 With fusion feature F j The channels are cascaded, and then a 1×1 convolutional layer is used to compress the number of channels by half. This feature is denoted as F. res1 , will F res1 Spatial features are extracted using a 3×3 convolutional layer while keeping the number of channels constant. This feature is denoted as F. res2 , for F res2 Edge convolution processing is performed, which involves first applying a 1×3 convolution followed by a 3×1 convolution, keeping the number of channels constant. The edge convolution result is then compared with F... res2 Concatenation is performed along the channel dimension. The concatenated result is then passed through a 1×1 convolutional layer to compress the number of channels to half of the original number, and then combined with F. res1 The edge blending feature Fe is obtained by performing an additive process. j In this embodiment, edge blending was performed three times to obtain edge blending features Fe1, Fe2, and Fe3, with feature shapes of 120×128×128, 192×64×64, and 288×32×32, respectively.

[0084] Step 4: Analyze the fusion feature Fj and edge fusion features Fe1~Fe J-1 The input edge-guided deep supervised classifier is used to obtain the change detection result Output; specifically, in this embodiment, the following steps are included:

[0085] Step 4.1: Upsample the edge fusion features Fe1, Fe2, Fe3, and fusion feature F4 to the same height and width as the remote sensing image, and compress the number of channels to 2 for each feature using a corresponding 1×1 convolutional layer. See [link to relevant documentation] Figure 5 The edge fusion features Fe1, Fe2, Fe3 and fusion feature F4 are passed through the upper bilinear sampling layer to upsample the height and width of the features to 256×256. They are then input into four 1×1 convolutional layers to compress the number of channels of Fe1, Fe2, Fe3 and F4 to 2. After passing through the softmax activation function, the classification results ds1, ds2, ds3 and ds4 are obtained. The shape of the classification results is 2×256×256.

[0086] Step 4.2: Input the edge fusion feature Fe1 into two 3×3 convolutional layers, changing the number of channels from 120 to 60 and then to 1. After passing through the sigmoid activation function, it becomes the edge probability map Edge1. Concatenate Edge1 and ds1 and pass them through two 3×3 convolutional layers, changing the number of channels from 3 to 32 and then to 1. After passing through the sigmoid activation function, it becomes the edge probability map Edge2. Concatenate Edge2 and ds1 in the channel dimension, pass them through two 3×3 convolutional layers, changing the number of channels from 3 to 32 and then to 2. After passing through the softmax activation function, the change detection result Output is obtained.

[0087] Step 5: Use the change detection dataset to perform deep supervised edge-guided training on the change detection methods from Step 1 to Step 4, save the optimal model weights, and input the remote sensing image to be detected into the model to obtain the change detection results. In this embodiment, Steps 1 to 4 are called the Edge-Guided Multi-Scale Foreground Attention-Based Change Detection Network, abbreviated as EMAF-CD;

[0088] In this embodiment, step 5 specifically includes the following steps: calculating the cross-entropy loss L from the preliminary classification results at each scale, the change detection results Output, and the true change labels. ce1 ~L cej and L ceout Calculate the edge dynamic weighted binary cross-entropy loss L for Edge1 and Edge2 and the actual change labels respectively. dwbce1 L dwbce2 For L ce1 ~L ceJ L ceout L dwbce1 Ldwbce2 The total loss L is obtained by weighted summation. total During training, each iteration is based on the total loss L. total Backpropagation adjusts the model weights using an optimizer. These weights are the kernel parameters of all convolutional layers in steps 1 to 4 of the change detection method. After a predetermined number of training epochs, the model is switched to validation mode for validation. Forward propagation is performed on the validation set to calculate the change detection accuracy, and then the model is switched back to training mode to continue training. After completing the predetermined number of training epochs, the model weights with the highest change detection accuracy are saved as the optimal model weights. In this embodiment, the total number of training iterations is set to 102,400, with forward propagation calculating the F1 score on the validation set every 5,120 iterations. When the F1 score reaches its maximum, the EMAF-CD model weights are saved.

[0089] Referring to the previous description, the cross-entropy loss L ce As a commonly used loss function, it will not be elaborated upon here. The edge-dynamically weighted binary cross-entropy loss L... dwbce The function expression is:

[0090]

[0091] In the above formula, P red To predict the image, T arget The labels are real numbers, n is the number of images in each batch, and L is the number of images in each batch. pi With L ni The function expression is:

[0092] L pi =w pi ×Canny(T argeti )×log(P redi )

[0093] L ni =Canny(1-T) argeti )×log(1-P redi )

[0094] In the above formula, P redi For the i-th predicted image, T argeti For the i-th real label, Canny(T) argeti ) represents the edge true value, T edgei The edge represents the true label of the i-th image, Canny() is the edge detection operator, and w pi The function expression is:

[0095] w pi =N ui / N ci

[0096] In the above formula, Nci Let T be the true edge value of the i-th image. edqei The middle value is the number of pixels at the edge, N. ui Let T be the true edge value of the i-th image. edgei The number of pixels that are not on the edge.

[0097] Referring to the above description, the total loss L total The function expression is:

[0098] L total =L ceout +(L ce1 +L ce2 +L ce3 +L ce4 ) / 4+(L dwbce1 +L dwbce2 ) / 2

[0099] In the verification and testing phases, dual-temporal remote sensing images are input into EMAF-CD to obtain change detection results. The change detection results are then binarized to obtain a binary change map.

[0100] In steps 1 to 4, the 1×1 convolutional layer operation involves first performing a 2D convolution with a kernel size of 1, then batch normalization, and finally ReLU activation. The purpose is to compress the feature dimensions and apply non-linear activation. The 3×3 convolutional layer operation involves first performing a 2D convolution with a kernel size of 3, then batch normalization, and finally ReLU activation. The purpose is to extract spatial features, change the number of feature channels, and apply non-linear activation. The 3×1 convolutional layer operation involves first performing a 2D convolution with a kernel size of 3×1, then batch normalization, and finally ReLU activation. The purpose is to extract linear features, change the number of feature channels, and apply non-linear activation. The 1×3 convolutional layer operation involves first performing a 2D convolution with a kernel size of 1×3, then batch normalization, and finally ReLU activation. The purpose is to extract linear features, change the number of feature channels, and apply non-linear activation.

[0101] In step 4, the classifier operation steps are as follows: first, a two-dimensional convolution operation with a kernel size of 1 is performed, and the number of output channels is compressed to 2. Then, Softmax activation is performed to transform the input features into a probability feature map.

[0102] This embodiment uses the publicly available Levir-CD dataset to train, validate, and test EMAF-CD to verify its effectiveness. The Levir-CD dataset consists of 637 pairs of 1024×1024 bi-temporal remote sensing images with a spatial resolution of 0.5 meters. It includes complex variations in villas, small garages, high-rise apartments, and large warehouses. The images acquired during the dataset construction process are inconsistent across different regions, and variations caused by seasonal and illumination differences are introduced into the dataset, making remote sensing change detection on this dataset a challenging task. All bi-temporal high-resolution remote sensing image pairs are labeled with binary labels. The bi-temporal high-resolution remote sensing image pairs and labels are cropped to 256×256 pixels without overlap, and divided into training, validation, and test sets with an image pair ratio of 7120:1024:2048.

[0103] This embodiment also uses the publicly available dataset WHU-CD to train, validate, and test EMAF-CD to verify its effectiveness. The WHU-CD dataset consists of high-resolution bi-temporal remote sensing image pairs of 32507×15354 pixels, with a spatial resolution of 0.2 meters, and includes complex variations in roads, residences, factories, warehouses, and commercial buildings. All bi-temporal remote sensing image pairs are labeled with binary labels. The bi-temporal remote sensing image pairs and labels are cropped to a size of 256×256 without overlap, and divided into training, validation, and test sets, with an image pair ratio of 5947:744:744.

[0104] The algorithm proposed in this embodiment is compared with four state-of-the-art change detection methods: BiT, Changeformer, EGRCNN, and AMTNet-50. Specific results are as follows: Figure 6As shown. The evaluation metrics are Precision, Recall, F1 score, and Intersection over Union (IoU), with F1 score and IoU being the core metrics. Precision (P) represents the ratio of correctly identified changed region samples to all samples identified as true changed regions. Recall (R) represents the ratio of correctly identified changed region samples to all samples of true changed regions, which can also be understood as recall. The F1 score considers both precision and recall, and is a harmonic average of the model's precision and recall. Its value ranges from 0 to 1 and is commonly used to measure the results of binary classification. The IoU represents the ratio of the intersection and union of the true and predicted values ​​of changed regions. F1 and IoU are the core metrics, and the bold text in the figure indicates the maximum value in this column. The method in this embodiment (EMAF-CD) outperforms other methods in the core metrics F1 score and IoU on the LEVIR-CD dataset. Its precision P and recall R are slightly lower than ChangeFormer and EGRCNN, respectively. Overall, EMAF-CD performs the best. The method in this embodiment (EMAF-CD) also significantly outperforms other methods in the core metrics F1 score, IoU, and recall R on the WHU-CD dataset. Its precision P is slightly lower than ChangeFormer. Overall, EMAF-CD performs the best. Figure 7 , Figure 8 These are schematic diagrams comparing the change detection results of the method of this invention and other existing methods on the WHU-CD and LEVIR-CD datasets. Figure 7 As can be seen, EMAF-CD significantly suppresses the influence of non-target feature changes on the detection results, improves the poor detection accuracy caused by irrelevant background changes and illumination differences in dual-temporal remote sensing images, and successfully detects the protruding details of changed features while maintaining clear edge contours. Figure 8 As can be seen, EMAF-CD achieves efficient fusion of deep semantic information and shallow texture information, resulting in change detection results that combine accurate semantic information with rich edge details. It successfully detects the gap details between changed features, and the edge contours of the change detection results are smooth and clear. In summary, the method in this embodiment can significantly suppress the influence of non-target feature change pixels on the detection results, obtain change detection results that combine accurate semantic information and rich edge details by integrating deep semantic information and shallow texture information, effectively improve the unclear edge contours of change detection results, and significantly improve change detection accuracy.

[0105] This application discloses a remote sensing change detection system, including:

[0106] Feature extraction module: Based on the twin residual encoder, feature extraction is performed on the dual-temporal remote sensing images to obtain several feature pairs at different scales;

[0107] Feature fusion module: Performs pixel mixing on feature pairs of different scales to obtain mixed pixel feature pairs; performs foreground-background separation on the mixed pixel feature pairs to obtain feature pairs after foreground extraction, and calculates the difference features of the feature pairs after foreground extraction; performs attention processing on the mixed pixel features to obtain attention features; and fuses the attention features and difference features to obtain fused features.

[0108] Edge feature module: Upsamples the fused features of adjacent layers to obtain edge fused features;

[0109] Output module: All fused features and edge fused features are classified by edge-guided deep supervision to obtain change detection results.

[0110] This application also discloses a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein when the processor executes the computer program, it employs any of the remote sensing change detection methods described in the above embodiments.

[0111] The terminal device can be a computer device such as a desktop computer, a laptop computer, or a cloud server. The terminal device includes, but is not limited to, a processor and a memory. For example, the terminal device may also include input / output devices, network access devices, and buses.

[0112] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it in this regard.

[0113] The memory can be an internal storage unit of the terminal device, such as a hard disk or RAM of the terminal device, or an external storage device of the terminal device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) equipped on the terminal device. Furthermore, the memory can be a combination of internal storage units and external storage devices of the terminal device. The memory is used to store computer programs and other programs and data required by the terminal device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0114] In this terminal device, any one of the remote sensing change detection methods in the above embodiments can be stored in the memory of the terminal device and loaded and executed on the processor of the terminal device for convenient use.

[0115] This application also discloses a computer-readable storage medium, which stores a computer program, wherein when the computer program is executed by a processor, it employs any of the remote sensing change detection methods described in the above embodiments.

[0116] The computer program can be stored in a computer-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The computer-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.

[0117] In this embodiment, any one of the remote sensing change detection methods described above can be stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the above methods.

[0118] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A remote sensing change detection method, characterized in that, include: Feature extraction was performed on dual-temporal remote sensing images based on a twin residual encoder to obtain several feature pairs at different scales; Feature pairs at different scales are pixel-wise blended to obtain blended pixel feature pairs; foreground and background separation is performed on the blended pixel feature pairs to obtain feature pairs after foreground extraction, and the difference features of the feature pairs after foreground extraction are calculated. Attention features are obtained by performing attention processing on the mixed pixel features; The attention features and difference features are fused to obtain the fused features; Edge fusion features are obtained by upsampling the fusion features of adjacent layers; all fusion features and edge fusion features are then classified by edge-guided deep supervision to obtain change detection results. The pixel blending process includes the following steps: Determine whether the height and width of the feature are greater than the threshold values ​​of the height and width of the input remote sensing image; If the value exceeds the height and width thresholds of the remote sensing image, then increment the height and width plane by a step size. Exchange feature pairs , All channel data corresponding to the pixel location; if it is less than the threshold of the height and width of the remote sensing image, then in the channel dimension with a step size. Exchange feature pairs , All pixel data from the corresponding channels are used to obtain mixed pixel feature pairs. , ; The acquisition of the edge fusion features includes the following steps: Fusion features Upsampling to fusion features Same height and width will merge features With fusion features Cascading yields cascade result one; cascade result one passes... Convolutional layer 5 performs feature compression to obtain compressed feature 1; Compression feature 1 Convolutional layer 2 extracts spatial features; spatial features are obtained through... Convolutional layers and Convolutional layer one extracts linear features, and concatenates the linear features with spatial features to obtain concatenated result two; Cascade Result 2 Pass Convolutional layer 6 performs feature compression to obtain compressed feature 2. Compressed feature 2 is added to compressed feature 1 to obtain the edge fusion feature. .

2. The remote sensing change detection method according to claim 1, characterized in that, It also includes calculating the cross-entropy loss and edge loss in the process of processing change detection results at each scale, and weighting the cross-entropy loss and edge loss to calculate the total loss in the process of processing change detection results; the backpropagation algorithm is used to train the change detection results, and the model weights are adjusted iteratively with the goal of minimizing the total loss during the training process; The change detection accuracy is obtained by performing forward propagation calculations on the validation set. After completing a pre-set number of training rounds, the model weight with the highest change detection accuracy is taken as the optimal model weight, and the change detection results are recalculated based on the optimal model weight.

3. The remote sensing change detection method according to claim 2, characterized in that, The edge loss is calculated based on the edge dynamically weighted binary cross-entropy loss function, the function expression of which is: In the formula, To predict the image, For real labels, The number of images in each batch. For edge-dynamically weighted binary cross-entropy loss; In the above formula and The function expression is: In the above formula For the first Image prediction For the first A real label, For edge true values, Representing the The edge of a real label, For edge detection operators, The function expression is: In the above formula, For the first True value of amplitude edge The middle represents the number of pixels at the edge. For the first True value of amplitude edge The number of pixels that are not on the edge.

4. The remote sensing change detection method according to claim 3, characterized in that, The twin residual encoder consists of two residual encoders; the two residual encoders share weights in their dual branches and are used to extract dual-temporal remote sensing images respectively. and Feature extraction; Each residual encoder includes One convolutional layer, one max pooling layer, and four residual layers; each residual layer consists of several residual blocks, and each residual block is composed of... Convolutional layer 2 Convolutional layer 1 A convolutional layer consists of three parts; the input of the residual block is passed through... After processing by the fourth convolutional layer, a residual connection is performed with the output of the residual block.

5. The remote sensing change detection method according to claim 4, characterized in that, The acquisition of the change detection results includes the following steps: Edge blending features ~ Fusion characteristics Upsample to the same height and width as the remote sensing image, and obtain preliminary classification results for each feature through a classifier; Edge fusion features Input after upsampling Convolutional Layer 3 The fourth convolutional layer processes the data to obtain the final result, which is then activated by a sigmoid function to obtain the edge probability map. Marginal probability map Features of edge blending The corresponding preliminary classification results are concatenated in the channel dimension to obtain concatenated result three; concatenated result three is then passed through... Convolutional layer 5 The sixth convolutional layer uses the sigmoid activation function to obtain the edge probability map. Then the marginal probability map Features of edge blending The corresponding preliminary classification results are concatenated in the channel dimension to obtain concatenated result four. Concatenated result four is then input. Convolutional layer seven The change detection results were obtained from the eighth convolutional layer. .

6. A remote sensing change detection system, characterized in that, include; Feature extraction module: Based on the twin residual encoder, feature extraction is performed on the dual-temporal remote sensing images to obtain several feature pairs at different scales; Feature fusion module: performs pixel-level fusion processing on feature pairs at different scales to obtain mixed pixel feature pairs; performs foreground-background separation on the mixed pixel feature pairs to obtain feature pairs after foreground extraction, and calculates the difference features of the feature pairs after foreground extraction; Attention features are obtained by performing attention processing on the mixed pixel features; The attention features and difference features are fused to obtain the fused features; Edge feature module: Upsamples the fused features of adjacent layers to obtain edge fused features; Output module: Performs edge-fused features and edge-fused features into change detection results through edge-guided deep supervised classification; The pixel blending process includes the following steps: Determine whether the height and width of the feature are greater than the threshold values ​​of the height and width of the input remote sensing image; If the value exceeds the height and width thresholds of the remote sensing image, then increment the height and width plane by a step size. Exchange feature pairs , All channel data corresponding to the pixel location; if it is less than the threshold of the height and width of the remote sensing image, then in the channel dimension with a step size. Exchange feature pairs , All pixel data from the corresponding channels are used to obtain mixed pixel feature pairs. , ; The acquisition of the edge fusion features includes the following steps: Fusion features Upsampling to fusion features Same height and width will merge features With fusion features Cascading yields cascade result one; cascade result one passes... Convolutional layer 5 performs feature compression to obtain compressed feature 1; Compression feature 1 Convolutional layer 2 extracts spatial features; spatial features are obtained through... Convolutional layers and Convolutional layer one extracts linear features, and concatenates the linear features with spatial features to obtain concatenated result two; Cascade Result 2 Pass Convolutional layer 6 performs feature compression to obtain compressed feature 2. Compressed feature 2 is added to compressed feature 1 to obtain the edge fusion feature. .

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor. When the processor loads and executes the computer program, it employs a remote sensing change detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs a remote sensing change detection method according to any one of claims 1 to 5.

Images