Remote sensing image change detection method based on edge enhancement

Abstract

The invention belongs to the field of computer vision and remote sensing image intelligent recognition, and particularly relates to a remote sensing image change detection method based on edge enhancement, which comprises the following steps of: acquiring a to-be-detected remote sensing image, and performing Fourier transform and inverse transform on an input image by adopting a Fourier domain feature pull-in module to obtain a to-be-detected remote sensing image; performing feature extraction on the two images by using a feature extraction encoder; fusing the extracted features by using a feature fusion module; an edge supervision perception module is adopted to perceive the edge of the input image to obtain an edge graph; injecting the edge graph into the fusion feature graph by adopting an edge guide enhancement module to obtain an edge information enhanced fusion feature graph; a feature decoder is adopted to decode the edge information enhanced fusion feature map, and a detection result is obtained; according to the invention, an edge guidance enhancement module is designed to effectively combine edge information obtained by an edge supervision perception module with features of each layer of the model, so that the edge expression ability and generalization ability of the model are enhanced.

Description

Technical Field

[0001] The invention belongs to the field of computer vision and remote sensing image intelligent recognition, and specifically relates to a remote sensing image change detection method based on edge enhancement. Background Art

[0002] With the popularization of various satellite imaging technologies in daily life, remote sensing image data sets of various terrains have also become rich. In the field of computer vision and remote sensing image intelligent recognition, remote sensing image change detection is an important research direction. In various related works, the focus is mainly on the extraction and processing of overall features such as change detection areas, but the extraction and processing of fine-grained features such as edges is very lacking. Tasks such as the extraction and processing of fine-grained features such as edges are of great research significance for remote sensing image change detection.

[0003] At present, many researchers at home and abroad have achieved remarkable results in the past few decades in the study of change detection methods for remote sensing images. The principles and data processing methods of change detection methods based on remote sensing images include traditional methods and deep learning-based methods. Traditional change detection methods mainly rely on manual features and statistical analysis. Deep learning-based methods can be roughly divided into two categories: CNN-based and Transformer-based models. That is, convolutional neural networks (CNNs) are used to extract abstract and robust features to enhance the overall performance of change detection. For example, deepened network architectures, dilated convolution strategies, and attention mechanisms. However, CNNs lack the ability to model long-range dependencies; in addition, current change detection methods do not address the direct interaction between dual-time images before obtaining differential features. The performance of the Transformer model is not only comparable to that of the convolutional model, but also has robustness and adaptability. The ability to perform well in various tasks emphasizes the potential of these models as powerful tools to promote the frontiers of computer vision research and application. For example, a study proposed an asymmetric multi-head cross-attention module that retains the most representative markers by introducing an adaptively updated semantic graph using nonlinear mapping. The quadratic computational complexity of self-attention is reduced to linear computational complexity. This module deeply integrates the advantages of CNN and Transformer. Similarly, there are studies that integrate self-attention and cross-attention in a unified module, integrating the representation of dual-time images, thereby improving the information interaction ability of global attention. However, in the above-mentioned existing technologies, in the research on remote sensing image change detection, the main improvement is in the ability to extract features and the ability to express difference features, which leads to the problem of unclear expression of fine-grained features and blurred edges. Summary of the invention

[0004] In order to solve the problems existing in the above prior art, the present invention proposes a remote sensing image change detection method based on edge enhancement, the method comprising: obtaining a remote sensing image to be detected, inputting the remote sensing image to be detected into a trained remote sensing image change detection model to obtain a detection result; training the remote sensing image change detection model comprises: obtaining an original data set, preprocessing the images in the original data set to obtain a training set; inputting the data in the training set into the remote sensing image change detection model, using a Fourier domain feature pull-in module to perform Fourier transform and inverse transform on the input image, and obtaining two images after domain pull-in; using a feature pull-in module to perform Fourier transform and inverse transform on the input image, and obtaining two images after domain pull-in; using a feature pull-in module to perform Fourier transform and inverse transform on the input image, and obtaining two images after domain pull-in; using a feature pull-in module to perform Fourier transform and inverse transform on the input image, and obtaining two images after domain pull-in. The feature extraction encoder extracts features from the two images after domain pulling; the feature fusion module is used to fuse the extracted features to obtain a fused feature map; the edge of the input image is perceived by the edge supervision perception module to obtain an edge map; the edge guided enhancement module is used to inject the edge map into the fused feature map to obtain a fused feature map with edge information enhancement; the feature decoder is used to decode the fused feature map with edge information enhancement to obtain the detection result; the loss function of the model is calculated according to the detection result, and the parameters of the model are optimized by the AdamW optimizer. When the loss function converges, the model training is completed.

[0005] Beneficial effects of the present invention:

[0006] The present invention uses Fourier transform and inverse transform to map the image to the frequency domain, separate the high-frequency components and low-frequency components of each of the two images, and perform cross-fusion to achieve the effect of reducing the domain difference of the two images, so that the model can effectively suppress background differences and learn more interesting change features. The present invention designs an edge supervision perception module. In order to enhance the model's ability to learn edges, this module is designed using high-level features containing rich semantic information and low-level features containing rich edge information and a sobel operator that effectively extracts edges. Finally, the edge information guided by high-level semantic information is deeply supervised using diceloss, thereby ensuring that the model can learn more effective edge information. The present invention designs an edge guidance enhancement module that effectively combines the edge information obtained by the edge supervision perception module with the features of each layer of the model, thereby enhancing the edge expression and generalization capabilities of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 The network framework structure diagram of the remote sensing image change detection method based on edge enhancement of the present invention;

[0008] Figure 2 It is a structural diagram of the Fourier module of the present invention;

[0009] Figure 3 It is a structural diagram of the feature fusion module of the present invention;

[0010] Figure 4 is a structural diagram of the edge supervision perception module of the present invention;

[0011] Figure 5 It is a structural diagram of the edge guidance enhancement module of the present invention. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] A remote sensing image change detection method based on edge enhancement, such as Figure 1 As shown in , first, we prepare the cropped data set, and use the label map in the data set to generate edge labels to perform deep edge supervision and perform corresponding data preprocessing including random scale cropping, Gaussian blur, random flipping, random rescaling and random color jittering; as shown in Figure 2 , 3 , 4, and 5. The network model is constructed. It mainly includes Fourier domain feature pull-in module (FFT), feature extraction encoder, feature fusion module, edge supervision perception module (ESAM), edge guided enhancement module (EGEM), feature decoder and other modules; train the remote sensing image change detection network model; combine multiple loss functions and Adam optimizer for training and optimization, and update network parameters; after training, the optimal model is obtained, and the test set is used to generate prediction results, and the IOU index is calculated with the label graph of the test set to judge the performance of the model.

[0014] A remote sensing image change detection method based on edge enhancement comprises: obtaining a remote sensing image to be detected, inputting the remote sensing image to be detected into a trained remote sensing image change detection model, and obtaining a detection result; training the remote sensing image change detection model comprises: obtaining an original data set, preprocessing images in the original data set, and obtaining a training set; inputting the data in the training set into the remote sensing image change detection model, using a Fourier domain feature pull-in module to perform Fourier transform and inverse transform on the input image, and obtaining two images after domain pull-in; using a feature extraction encoder to extract features from the two images after domain pull-in respectively; using a feature fusion module to fuse the extracted features, and obtaining a fusion feature map; using an edge supervision perception module to perceive the edge of the input image, and obtaining an edge map; using an edge guidance enhancement module to inject the edge map into the fusion feature map, and obtaining a fusion feature map with edge information enhancement; using a feature decoder to decode the fusion feature map with edge information enhancement, and obtaining a detection result; calculating the loss function of the model according to the detection result, and using an AdamW optimizer to optimize the parameters of the model, and when the loss function converges, the model training is completed.

[0015] In this embodiment, the original dataset is a high-resolution dataset with a resolution of 1024×1024. In order to adapt to the task of change detection, it needs to be cropped to a size of 256×256. For the label image in the cropped dataset, we subtract the average pooled image from the original image to obtain the edge of the label image. The cropped dataset is subjected to dataset expansion operations, such as random scale cropping, Gaussian blur, random flipping, random rescaling, and random color jitter.

[0016] In this embodiment, the remote sensing image change detection model mainly includes a Fourier domain feature pull-in module (FFT), a feature extraction encoder, a feature fusion module, an edge supervision perception module (ESAM), an edge guided enhancement module (EGEM), a feature decoder, etc.

[0017] That is, the network model consists of a Siamese-based hierarchical encoder with four different stages, a fusion module (FM), an edge-supervised acquisition module (ESAM), an edge-guided enhancement module (EGEM), and a decoder for predicting a binary change map M. The encoder computes features at different spatial resolutions in four stages. At each stage, the features are first spatially downsampled through a convolutional layer and then input into the SSA attention layer. The network EA-CDNet consists of a Fourier transform module and two parallel identical encoder streams with shared weights, which generate the before-change and after-change features f in the i-th stage of the multi-stage network, respectively. pre i 、f posti The focus of our design is to introduce a Fourier transform module to map the input of two images into the frequency domain. A novel edge supervision acquisition module (ESAM) and edge feature enhancement module (EGEM) are used to generate edges using ESAM and injected into the fused features at each feature level through EGEM to enhance the edge representation.

[0018] Fourier domain feature pulling module (FFT): This module maps two images with large domain differences to the Fourier domain for domain pulling, thereby reducing the false detection rate caused by large domain differences. The input of the FFT module is two images, and the three-dimensional Fourier transform is used to derive the spectrum of the image. The spectrum is then centered to generate corresponding high-frequency and low-frequency components. The center part is designated as the low-frequency component, and the high and low frequencies are separated. Then, the high-frequency and low-frequency information is exchanged between the two images in the frequency domain. Finally, the inverse Fourier transform is performed to perform the conversion from the frequency domain to the spatial domain, thereby obtaining two images after domain pulling. Figure 2 As shown, the conversion formula is as follows:

[0019] IFFT(fusion(split(FFT(α pre ,α post ))))

[0020] Among them, α pre is the feature of the image before the change at time point 1, α post is the feature after the change at time point 2, FFT is Fourier transform, split is the separation of low-frequency and high-frequency components of the image, fusion is the separation and recombination of the high-frequency and low-frequency components of the two images, and IFFT is the inverse Fourier transform.

[0021] Feature Extraction Encoder: A shuffled sparse attention (SSA) layer is introduced in the encoder to capture the semantic changes between the input image pairs fpre and fpost. The feature extraction encoder consists of a shuffled sparse attention (SSA) that performs label mixing, a multi-layer perceptron (MLP), and layer normalization layers.

[0022] Feature Fusion Module (FFM): Inspired by SKNet and SAGate SA-Gate, the fusion module is mainly used to fuse the features of different images at each stage generated by the backbone network. In the fusion process, it aims to amplify the difference between the dual-time features and mitigate the impact of noise features. First, 1×1 convolution is used to sum them separately, and then global average pooling is used to generate global information. i Together with convolution kernel 3, it is input into the convolution activation block to extract common features. Then this data is input into two 1×1 convolution blocks to obtain the channel attention weight gi pre and g i Then, the two weight vectors are normalized to obtain s i pre and i post After that, make sure i pre +s i post = 1. Finally, the input dual-time feature α i pre and α i post Re-weighted to obtain the fusion feature α i fusion , the formula is as follows:

[0023]

[0024]

[0025]

[0026] Among them, GAP is the global pooling layer, conv 1x1 is a convolution operation with a convolution kernel of 1x1. is the feature of the image before the change of the i-th stage, The characteristics of the graph after the change of the i-th stage are: and The channel attention weights are, respectively, i is global information, are two weight vectors and After normalization, the normalized weights are obtained. It is the feature after the fusion of two sets of features.

[0027] Edge-supervised perception module (ESAM): Due to the lack of prior edge structure information, the detection results of the edge area of ​​the building are inaccurate. ESAM is introduced to enhance the edge information under direct edge supervision. The low-level features contain rich edges. The sobel operator is used to calculate the important edge information in the low-level features, and the obtained edge information is fused with the low-level features to obtain the edge enhanced representation. However, there are many edges in the low-level features that are not related to the changes. Therefore, high-level features are needed to help locate the edges related to the changes. Select α f 2 and α f 4 to explore edge semantics. First, a 1×1 convolution upsamples high-level features to adjust the spatial resolution. Then, α f 2 and αf 4 After the channel dimension is connected, it passes through two 3×3 convolutional layers to further integrate semantic information. The formula is as follows:

[0028]

[0029] Among them, S represents the Sobel operator, U represents upsampling, and Cat represents splicing along the channel dimension.

[0030] Edge-guided Enhancement Module (EGEM): EGEM transforms the edge graph generated by ESAM into edge Inject into fusion feature α f i In , feature learning can be guided to enhance edge representation. First, element-wise multiplication of upsampled edge maps and upsampled fused features is performed on the corresponding feature level. Then, feature fusion is performed using residual connections, 3×3 and 1×1 convolutional layers. After that, the feature representation after edge enhancement is obtained. The formula is as follows:

[0031]

[0032] Feature decoder: U-net decoder is used. U-net decoder restores high-level semantic features to the size of the original input image through upsampling and encoder symmetric operations, thereby realizing image reconstruction and information restoration. Since U-net decoder contains high-level semantic features extracted from the encoder, it can retain the detailed information in the input image and help accurately restore the subtle structure of the image. U-net decoder also fuses the feature maps of the corresponding levels in the encoder with the feature maps in the decoder through jump connections, thereby helping the network to better learn the local details and global information of the image, improving the performance and generalization ability of the network. It also helps the network to make comprehensive use of features of different scales, improving the network's perception and representation ability of the input image. U-net decoder adopts a symmetrical structure with the encoder, making parameter sharing and training in the network more efficient, while also reducing the number of model parameters and the risk of overfitting. The decoder part of U-net can effectively restore the detailed information of the input image, help the network accurately locate and segment the target object, and has good generalization ability and training effect, and has a small number of parameters, so U-net decoder is selected as the decoder of the network.

[0033] The decoder outputs a prediction map with two channels and a spatial resolution of 256×256 to calculate the metric score. The model is trained using the pixel-level cross entropy loss function and the dice loss. sIt is mainly used to constrain the overlap between the predicted results and the true labels in the image segmentation task. Its optimization goal is to maximize the similarity of the overlapping areas between the predicted results and the true labels, thereby improving the accuracy and quality of segmentation. In the ESAM module, we use dice loss as the edge loss function to constrain, and the pixel-level cross entropy loss function is used to constrain the features passed by the decoder and the true label. These two loss functions are combined with the AdamW optimizer for training. The total loss function is:

[0034] L loss =L cross-entropy (F total , GT)+0.1*L dice (F edge ,GT edge )

[0035] Among them, L cross-entropy is the cross entropy loss, F total is the total feature of the final output of the model, GT is GroundTrueth, L dice is the dice loss, F edge is the edge loss, GT edge GroundTrueth for the edge.

[0036] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A remote sensing image change detection method based on edge enhancement, characterized in that: include: Acquire a remote sensing image to be detected, input the remote sensing image to be detected into a trained remote sensing image change detection model, and obtain a detection result; Training the remote sensing image change detection model includes: obtaining the original data set, preprocessing the images in the original data set to obtain the training set; inputting the data in the training set into the remote sensing image change detection model, using the Fourier domain feature pull-in module to perform Fourier transform and inverse transform on the input image to obtain two images after domain pull-in; using the feature extraction encoder to extract features from the two images after domain pull-in respectively; using the feature fusion module to fuse the extracted features to obtain a fused feature map; using the edge supervision perception module to perceive the edge of the input image to obtain an edge map; using the edge guided enhancement module to inject the edge map into the fused feature map to obtain a fused feature map with edge information enhanced; using the feature decoder to decode the fused feature map with edge information enhanced to obtain the detection result; calculating the loss function of the model according to the detection result, using the AdamW optimizer to optimize the parameters of the model, and when the loss function converges, the model training is completed.

2. The remote sensing image change detection method based on edge enhancement according to claim 1 is characterized in that: The image preprocessing includes: cropping the image data in the original image data set; obtaining the edge map of the label map according to the number of cropped images; and performing data expansion on the cropped data to obtain a training set.

3. The remote sensing image change detection method based on edge enhancement according to claim 1 is characterized in that: The input image is processed using the Fourier domain feature pulling-in module, including: using three-dimensional Fourier transform to convert the two input images into a spectrum diagram; centering the spectrum to obtain high-frequency components and low-frequency components; dividing the high-frequency components and the low-frequency components, and exchanging information between the high-frequency components and the low-frequency components of the two images in the frequency domain; performing inverse Fourier transform on the two spectrum diagrams after the information exchange, that is, converting from the frequency domain to the spatial domain, to obtain two images after domain pulling-in.

4. The remote sensing image change detection method based on edge enhancement according to claim 1 is characterized in that: A feature extraction encoder is used to extract features from the two images after domain pulling, including: the feature extraction encoder is composed of a shuffled sparse attention layer, a multi-layer perceptron and a layer normalization layer; the shuffled sparse attention layer is used to obtain the semantic transformation features between the input images; the multi-layer perceptron is used to perceptually enhance the semantic transformation features, and the normalization layer is used to normalize the enhanced features to obtain a feature map.

5. The remote sensing image change detection method based on edge enhancement according to claim 1, characterized in that: The feature fusion module is used to fuse the extracted features, including: using 1×1 convolution to sum the input features respectively; performing global average pooling on the summed features to obtain global information; inputting the global information and the 3x3 convolution kernel into the convolution activation block to obtain common features; inputting the common features into the corresponding 1×1 convolution blocks to obtain the channel attention weight g. i pre and g i post ; Normalize the two weight vectors to get s i pre and i post ; The input dual-time features are weighted fused according to the normalized weights to obtain the fused features.

6. The remote sensing image change detection method based on edge enhancement according to claim 1, characterized in that: The edge supervision perception module is used to perform edge supervision on the edges of the input image, specifically including: extracting edge features of the input image to obtain low-level features and high-level features; using the Sobel operator to calculate the edge information in the low-level features, fusing the edge information with the low-level features to obtain an edge enhanced representation; using 1×1 convolution to upsample the high-level features; connecting the upsampled high-level features and the edge enhanced representation in the channel dimension, and inputting the spliced ​​edge features into two 3×3 convolution layers to obtain an edge map.

7. The remote sensing image change detection method based on edge enhancement according to claim 1 is characterized in that: The edge-guided enhancement module is used to inject the edge map into the fused feature map, including: upsampling the fused feature map and the edge map; element-wise multiplication of the upsampled edge map and the fused feature map; and fusing the features of each layer using residual connections, 3×3 and 1×1 convolutional layers to obtain edge-enhanced feature representation.

8. The remote sensing image change detection method based on edge enhancement according to claim 1, characterized in that: The feature decoder is used to decode the fused feature map with edge information enhancement, including: the fused feature map input by the decoder contains the features of four stages, the four features are concatenated in the channel dimension, and then reduced in dimension through a two-dimensional convolution and BatchNorm layer, and the feature map after dimensionality reduction is deeply decoded through two upsampling layers and a residual connection layer to obtain the detection result.

Images