An edge enhancement-based remote sensing image change detection method

Abstract

The present application belongs to the field of computer vision and remote sensing image intelligent recognition, and particularly relates to a remote sensing image change detection method based on edge enhancement, comprising: obtaining a remote sensing image to be detected, performing Fourier transform and inverse transform on the input image by using a Fourier domain feature pull-in module, and performing feature extraction on two images by using a feature extraction encoder; fusing the extracted features by using a feature fusion module; perceiving the edges of the input image by using an edge supervision perception module to obtain an edge map; injecting the edge map into the fused feature map by using an edge-guided enhancement module to obtain a fused feature map with enhanced edge information; and decoding the fused feature map with enhanced edge information by using a feature decoder to obtain a detection result; the present application designs an edge-guided enhancement module to effectively combine the edge information obtained by the edge supervision perception module and the features of each layer of the model together, thereby enhancing the edge expression capability and generalization capability of the model.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent recognition of remote sensing images, specifically relating to a method for detecting changes in remote sensing images based on edge enhancement. Background Technology

[0002] With the widespread use of various satellite imaging technologies in daily life, remote sensing image datasets of various terrains have become increasingly abundant. In the fields of computer vision and intelligent remote sensing image recognition, change detection in remote sensing images is an important research direction. Current research primarily focuses on extracting and processing overall features such as the change detection area, but there is a significant lack of work on extracting and processing fine-grained features such as edges. Therefore, tasks related to the extraction and processing of fine-grained features such as edges are of great research significance for change detection in remote sensing images.

[0003] Research on change detection methods for remote sensing images has yielded significant results over the past few decades, with numerous researchers both domestically and internationally achieving remarkable progress. The principles and data processing methods for change detection based on remote sensing images include traditional methods and deep learning-based methods. Traditional change detection methods primarily rely on handcrafted features and statistical analysis. Deep learning-based methods can be broadly categorized into two types: CNN-based models and Transformer-based models. CNNs utilize convolutional neural networks to extract abstract and robust features, thereby enhancing the overall performance of change detection. Examples include deepened network architectures, dilated convolutional strategies, and attention mechanisms. However, CNNs lack the ability to model long-range dependencies; furthermore, current change detection methods do not address the direct interaction between images at different times before obtaining differential features. Transformer models not only rival convolutional models in performance but also possess robustness and adaptability. Their outstanding performance across various tasks underscores their potential as powerful tools for advancing computer vision research and applications. For instance, one study proposed an asymmetric multi-head cross-attention module that retains the most representative labels by introducing an adaptively updated semantic graph through nonlinear mapping. The quadratic computational complexity of self-attention is reduced to linear computational complexity. This module deeply integrates the advantages of both CNN and Transformer. Similarly, some studies have integrated self-attention and cross-attention into a unified module, fusing representations of dual-time images to improve the information interaction capability of global attention. However, in research on remote sensing image change detection, the aforementioned existing technologies mainly focus on improving the ability to extract features and express differential features, resulting in problems such as unclear fine-grained feature representation and blurred edges. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes a remote sensing image change detection method based on edge enhancement. The method includes: acquiring the remote sensing image to be detected; inputting the remote sensing image to be detected into a trained remote sensing image change detection model to obtain a detection result; training the remote sensing image change detection model includes: acquiring the original dataset; preprocessing the images in the original dataset to obtain a training set; inputting the data in the training set into the remote sensing image change detection model; using a Fourier domain feature augmentation module to perform Fourier transform and inverse transform on the input image to obtain two images after domain augmentation; and using a feature augmentation module to perform Fourier transform and inverse transform on the input image. The feature extraction encoder extracts features from the two images after domain-wise magnification. A feature fusion module fuses the extracted features to obtain a fused feature map. An edge-supervised perception module detects the edges of the input image to obtain an edge map. An edge-guided enhancement module injects the edge map into the fused feature map to obtain an edge-enhanced fused feature map. A feature decoder decodes the edge-enhanced fused feature map to obtain the detection result. Based on the detection result, the model's loss function is calculated, and the AdamW optimizer optimizes the model's parameters. When the loss function converges, the model training is complete.

[0005] The beneficial effects of this invention are:

[0006] This invention employs Fourier transform and inverse transform to map images to the frequency domain, separating the high-frequency and low-frequency components of two images and performing cross-fusion to reduce the domain differences between the two images. This allows the model to effectively suppress background differences and learn more interesting variation features. This invention designs an edge-supervised perception module. To enhance the model's edge learning ability, this module uses high-level features rich in semantic information, low-level features rich in edge information, and the Sobel operator for effective edge extraction. Finally, Diceloss is used for deep supervision of the edge information guided by high-level semantic information, ensuring that the model can learn more effective edge information. This invention also designs an edge-guided enhancement module that effectively combines the edge information acquired by the edge-supervised perception module with the features of each layer of the model, thereby enhancing the model's edge representation ability and generalization ability. Attached Figure Description

[0007] Figure 1 This is a network framework diagram of the remote sensing image change detection method based on edge enhancement according to the present invention;

[0008] Figure 2 This is a structural diagram of the Fourier module of the present invention;

[0009] Figure 3 This is a structural diagram of the feature fusion module of the present invention;

[0010] Figure 4 This is a structural diagram of the edge supervision sensing module of the present invention;

[0011] Figure 5 This is a structural diagram of the edge guidance enhancement module of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] A remote sensing image change detection method based on edge enhancement, such as Figure 1 As shown, the first step is to prepare the cropped dataset. Simultaneously, edge labels are generated using the label image from the dataset for edge depth supervision, and corresponding data preprocessing is performed, including random scale cropping, Gaussian blur, random flipping, random rescaling, and random color jittering. Figure 2 , 3 The network model is constructed as shown in Figures 4 and 5. It mainly includes modules such as a Fourier Transform Feature Boosting (FFT) module, a feature extraction encoder, a feature fusion module, an Edge Supervision Awareness (ESAM) module, an Edge Guidance Enhancement (EGEM) module, and a feature decoder. The remote sensing image change detection network model is trained; multiple loss functions and the Adam optimizer are combined for training and optimization, and the network parameters are updated. After training, the optimal model is obtained. The optimal model is used to generate prediction results on the test set, and the Interchange of Units (IOU) metric is calculated between the model and the label graph of the test set to evaluate the model's performance.

[0014] A remote sensing image change detection method based on edge enhancement includes: acquiring the remote sensing image to be detected; inputting the remote sensing image to be detected into a trained remote sensing image change detection model to obtain a detection result; training the remote sensing image change detection model includes: acquiring the original dataset; preprocessing the images in the original dataset to obtain a training set; inputting the data in the training set into the remote sensing image change detection model; using a Fourier domain feature pulling module to perform Fourier transform and inverse transform on the input image to obtain two images after domain pulling; using a feature extraction encoder to extract features from the two images after domain pulling; using a feature fusion module to fuse the extracted features to obtain a fused feature map; using an edge-supervised perception module to perceive the edges of the input image to obtain an edge map; using an edge-guided enhancement module to inject the edge map into the fused feature map to obtain a fused feature map with enhanced edge information; using a feature decoder to decode the fused feature map with enhanced edge information to obtain a detection result; calculating the loss function of the model based on the detection result; using the AdamW optimizer to optimize the parameters of the model; and completing the model training when the loss function converges.

[0015] In this embodiment, the original dataset is a high-resolution dataset with a resolution of 1024×1024. To adapt to the change detection task, it needs to be cropped to a size of 256×256. For the label images in the cropped dataset, we subtract the average pooled image from the original image to obtain the edges of the label images. Dataset expansion operations are then performed on the cropped dataset, such as random scale cropping, Gaussian blur, random flipping, random rescaling, and random color dithering.

[0016] In this embodiment, the remote sensing image change detection model mainly includes a Fourier domain feature pull-in module (FFT), a feature extraction encoder, a feature fusion module, an edge-supervised perception module (ESAM), an edge-guided enhancement module (EGEM), and a feature decoder.

[0017] The network model consists of a Siamese-based hierarchical encoder with four distinct stages, a fusion module (FM), an edge-supervised acquisition module (ESAM), an edge-guided enhancement module (EGEM), and a decoder for predicting the binary transformation map M. The encoder computes features at different spatial resolutions in each of the four stages. In each stage, the features are first spatially downsampled through convolutional layers and then fed into an SSA attention layer. The EA-CDNet network consists of a Fourier transform module and two parallel, identical encoder streams with shared weights, generating the pre-transformation and post-transformation features f in the i-th stage of the multi-stage network, respectively. pre i f posti Our design focuses on introducing a Fourier transform module to map the inputs of two images to the frequency domain. A novel edge supervision acquisition module (ESAM) and edge feature enhancement module (EGEM) are employed, using ESAM to generate edges and injecting them into the fused features at each feature level via EGEM to enhance edge representations.

[0018] Fourier Domain Feature Shifting Module (FFT): This module maps two images with significant domain differences to the Fourier domain for a domain shift, thereby reducing the false positive rate caused by excessive domain differences. The FFT module takes two images as input and uses a three-dimensional Fourier transform to derive the image's spectrum. The spectrum is then centered to generate corresponding high-frequency and low-frequency components. The central portion is designated as the low-frequency component, separating the high and low frequencies. Subsequently, high-frequency and low-frequency information is exchanged between the two images in the frequency domain. Finally, an inverse Fourier transform is performed to convert from the frequency domain to the spatial domain, resulting in the two images after domain shifting. Figure 2 As shown, the conversion formula is as follows:

[0019] IFFT(fusion(split(FFT(α pre ,α post ))))

[0020] Where, α pre For the features of the image before the change at time point 1, α post The features after the change at time point 2 are: FFT is the Fourier transform, split is the separation of low-frequency and high-frequency components of the image, fusion is the recombination of the high-frequency and low-frequency components of the two images after separation, and IFFT is the inverse Fourier transform.

[0021] Feature Extraction Encoder: A shuffled sparse attention (SSA) layer is introduced into the encoder to capture the semantic changes between the input image pairs fpre and fpost. The feature extraction encoder consists of a shuffled sparse attention (SSA) layer that performs label mixing, a multilayer perceptron (MLP), and a layer normalization layer.

[0022] Feature Fusion Module (FFM): Inspired by SKNet and SA-Gate, the fusion module is primarily used to fuse features from different images generated at each stage by the backbone network. During the fusion process, it aims to amplify the differences between bi-temporal features and mitigate the influence of noisy features. First, 1×1 convolutions are used to sum the features separately, then global average pooling is used to generate global information, and the global information p... i The data, along with kernel 3, is input into the convolutional activation block to extract common features. These data are then fed into two 1×1 convolutional blocks to obtain the channel attention weights g.i pre and g i After posting, the two weight vectors are then normalized to obtain s. i pre and s i post After that, ensure s i pre +s i post =1. Finally, for the input dual-time features α i pre and α i post The fusion feature α is obtained by reweighting. i fusion The formula is as follows:

[0023]

[0024]

[0025]

[0026] Where GAP is the global pooling layer, conv 1x1 For convolution operations with a 1x1 kernel, The features of the graph before the change in the i-th stage, The characteristics of the graph after the transformation of the i-th stage are as follows: and The channel attention weights are p, respectively. i For global information, Two weight vectors and After normalization, we obtain the normalized weights. The features are the result of fusing the two sets of features.

[0027] Edge-Supervised Awareness Module (ESAM): Due to the lack of prior edge structure information, the detection results of building edge regions are inaccurate. ESAM is introduced to enhance edge information under direct edge supervision. Low-level features contain rich edges. The Sobel operator is used to calculate important edge information in low-level features, and the obtained edge information is fused with low-level features to obtain an enhanced edge representation. However, many edges in low-level features are irrelevant to changes. Therefore, higher-level functions are needed to help locate edges that are relevant to changes. α is chosen. f 2 and α f 4 To explore edge semantics, first, a 1×1 convolution upsamples high-level features to adjust the spatial resolution. Then, α... f 2 and αf 4 After channel-level concatenation, two 3×3 convolutional layers are applied to further integrate semantic information. The formula is as follows:

[0028]

[0029] Where S represents the Sobel operator, U represents upsampling, and Cat represents concatenation along the channel dimension.

[0030] Edge-Guided Enhancement Module (EGEM): EGEM enhances the edge graph generated by ESAM. edge Injected into fusion feature α f i In this process, feature learning can be guided to enhance edge representations. First, element-wise multiplication of upsampled edge maps and upsampled fused features is performed at the corresponding feature levels. Then, feature fusion is performed using residual connections, 3×3 and 1×1 convolutional layers. The resulting edge-enhanced feature representation is then obtained. The formula is as follows:

[0031]

[0032] Feature Decoder: The U-Net decoder is employed. The U-Net decoder, through upsampling and operations symmetrical to the encoder, restores high-level semantic features to the original input image size, thus achieving image reconstruction and information restoration. Since the U-Net decoder incorporates high-level semantic features extracted from the encoder, it preserves detailed information in the input image, helping to accurately recover the image's subtle structure. The U-Net decoder also fuses feature maps from corresponding levels in the encoder with those in the decoder through skip connections, helping the network better learn local details and global information, improving network performance and generalization ability. Furthermore, it facilitates the network's comprehensive utilization of features at different scales, enhancing its perception and representation capabilities of the input image. The U-Net decoder employs a symmetrical structure with the encoder, making parameter sharing and training more efficient while reducing the number of model parameters and mitigating the risk of overfitting. The U-Net decoder effectively recovers detailed information from the input image, helping the network accurately locate and segment target objects, and exhibits good generalization ability and training performance, with a small parameter count; therefore, the U-Net decoder was chosen as the network's decoder.

[0033] The decoder outputs a two-channel prediction map with a spatial resolution of 256×256 to compute metric scores. The model is trained using a pixel-level cross-entropy loss function and a dice loss function. sIt is mainly used to constrain the overlap between the predicted results and the ground truth labels in image segmentation tasks. Its optimization objective is to maximize the similarity of the overlapping regions between the predicted results and the ground truth labels, thereby improving the accuracy and quality of segmentation. In the ESAM module, we use dice loss as the edge loss function for constraint, while the pixel-level cross-entropy loss function is used to constrain the features after decoding and the ground truth labels. These two loss functions are then used in conjunction with the AdamW optimizer for training. The total loss function is:

[0034] L loss =L cross-entropy (F total ,GT)+0.1*L dice (F edge GT edge )

[0035] Among them, L cross-entropy For cross-entropy loss, F total The final output of the model represents the total features, where GT stands for Ground Truth and L. dice For the loss of dice, F edge For edge loss, GT edge GroundTrueth is the edge.

[0036] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An edge enhancement-based remote sensing image change detection method, characterized in that, The method comprises the following steps: obtaining a remote sensing image to be detected, inputting the remote sensing image to be detected into a trained remote sensing image change detection model, and obtaining a detection result; training the remote sensing image change detection model comprises: obtaining an original data set, preprocessing the images in the original data set to obtain a training set; inputting the data in the training set into the remote sensing image change detection model, using a Fourier domain feature pull-in module to perform Fourier transform and inverse transform on the input images to obtain two images after domain pull-in; using a feature extraction encoder to extract features from the two images after domain pull-in respectively; using a feature fusion module to fuse the extracted features to obtain a fused feature map; using an edge supervision perception module to perceive the edges of the input images to obtain an edge map; using an edge-guided enhancement module to inject the edge map into the fused feature map to obtain an edge information enhanced fused feature map; using a feature decoder to decode the edge information enhanced fused feature map to obtain a detection result; calculating a loss function of the model according to the detection result, using an AdamW optimizer to optimize the parameters of the model, and when the loss function converges, the training of the model is completed; using the Fourier domain feature pull-in module to process the input images comprises: using three-dimensional Fourier transform to convert the input two images into frequency spectrum images; centering the frequency spectrum to obtain high-frequency components and low-frequency components; dividing the high-frequency components and the low-frequency components, and exchanging information of the high-frequency components and the low-frequency components of the two images in the frequency domain; performing inverse Fourier transform on the two frequency spectrum images after information exchange, i.e. converting from the frequency domain to the spatial domain, to obtain two images after domain pull-in; the conversion formula is: IFFT(fusion(split(FFT(α pre , α post )))) wherein a pre is a feature of the image before the change at time point 1, a post is a feature after the change at time point 2, FFT is a Fourier transform, split is a separation of the low and high frequency components of the image, fusion is a recombination of the respective high and low frequency components of the two images after separation, and IFFT is an inverse Fourier transform.

2. The edge enhancement based remote sensing image change detection method according to claim 1, wherein, the preprocessing of the images comprises: cropping the image data in the original image data set; obtaining an edge map of a label image according to the cropped image data; performing data expansion on the cropped data to obtain a training set.

3. The edge enhancement based remote sensing image change detection method according to claim 1, wherein, using the feature extraction encoder to extract features from the two images after domain pull-in respectively comprises: the feature extraction encoder is composed of a shuffle sparse attention (SSA) layer, a multi-layer perceptron and a layer normalization layer; using the SSA layer to obtain semantic transformation features between the input images; using the multi-layer perceptron to perceive and enhance the semantic transformation features, and using the normalization layer to normalize the enhanced features to obtain a feature map.

4. The edge enhancement based remote sensing image change detection method of claim 1, wherein, The feature fusion module is adopted to fuse the extracted features, including: adopting 1x1 convolution to sum the input features respectively; performing global average pooling on the summed features to obtain global information; inputting the global information into a convolution activation block with a 3x3 convolution kernel to obtain common features; inputting the common features into corresponding 1x1 convolution blocks respectively to obtain channel attention weights and normalizing the two weight vectors to obtain normalized weights and performing weighted fusion on the input double-time features according to the normalized weights to obtain fused features.

5. The edge enhancement based remote sensing image change detection method according to claim 1, wherein, using the edge supervision perception module to perform edge supervision on the edges of the input images specifically comprises: extracting edge features from the input images to obtain low-level features and high-level features; using a sobel operator to calculate edge information in the low-level features, fusing the edge information with the low-level features to obtain edge enhanced representations; using a 1x1 convolution to up-sample the high-level features; connecting the up-sampled high-level features and the edge enhanced representations in the channel dimension, inputting the spliced edge features into two 3x3 convolution layers to obtain an edge map.

6. The edge enhancement based remote sensing image change detection method according to claim 1, wherein, The edge guiding enhancement module is used to inject the edge graph into the fusion feature graph, including: up-sampling the fusion feature graph and the edge graph; performing element-level multiplication on the up-sampled edge graph and the fusion feature graph; using residual connection, 3*3 and 1*1 convolution layers to fuse the features of each layer to obtain the edge-enhanced feature representation.

7. The edge enhancement based remote sensing image change detection method of claim 1, wherein, The feature decoder is used to decode the edge information enhanced fusion feature graph, including: the edge information enhanced fusion feature graph input by the decoder contains four stage features, the four features are subjected to Concate operation in the channel dimension, then subjected to a two-dimensional convolution and a BatchNorm layer for dimension reduction, the reduced feature graph is subjected to two up-sampling layers and a residual connection layer for deep decoding to obtain a detection result.

Images