Method, system, equipment and medium for detecting landslide change area on dual-time-phase remote sensing image

Abstract

The invention discloses a method, a system, equipment and a medium for detecting a landslide change area on a dual-temporal remote sensing image. The method comprises the following steps: constructing a landslide change area detection network on a remote sensing image; collecting a landslide sample data set and a non-landslide sample data set, and dividing the data set into a training set, a verification set and a test set; sequentially inputting the training set and the verification set into the landslide change area detection network on the remote sensing image for training and verification to obtain the trained landslide change area detection network on the remote sensing image, and storing a weight file; inputting the test set into the trained landslide change area detection network on the remote sensing image, loading a weight file of the trained landslide change area detection network on the remote sensing image, outputting a landslide change area detection result on the remote sensing image, and storing a label file; the system, the equipment and the medium are used for implementing the method. The method has the advantages that feature information is fully extracted, and the capturing capability of the model on image global information is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of landslide change area detection on remote sensing images, and particularly relates to a method, system, device and medium for detecting landslide change areas on dual-phase remote sensing images, which are used to solve the problems of large amount of network calculation data for landslides in remote sensing images, slow model detection and low positioning accuracy for multi-scale landslides, and more missed detection and wrong detection of landslide change ranges. Background Art

[0002] Landslide refers to a natural phenomenon in which soil or rock on a slope slides down the slope as a whole or in a dispersed manner along a certain weak surface or weak zone under the influence of factors such as river scouring, groundwater activity, rainwater soaking, earthquakes and human activities. With the intensification of global climate change and the impact of human activities, the frequency of landslides has gradually increased, and their actual hazards and potential risks have become increasingly prominent. Therefore, after a landslide occurs, it is of great significance to use satellite remote sensing technology to timely detect the landslide area and determine the location, scale and distribution of the landslide for disaster reduction, relief and planning and construction.

[0003] Change detection is a common method for detecting landslide areas in remote sensing images. By analyzing remote sensing data at different times, the difference in geometry, spectrum, structure, etc. of objects or phenomena at the same geographical location can be identified. This method has a solid theoretical basis and high detection accuracy. With the development of artificial intelligence technology, researchers have begun to explore landslide detection from the perspective of deep learning change detection.

[0004] The landslide area detection methods based on deep learning in optical remote sensing images are mainly divided into two categories: one is the landslide area detection method based on target detection, such as R-CNN, SSD and YOLO models and their improved models, which are used to determine whether there are landslide targets in the image and determine their locations; the other is the landslide area detection method based on semantic segmentation, such as UNet, DeepLab and PSPNet models and their improved models, which are used to segment landslide objects according to semantic content and realize landslide area detection. The former can quickly detect the location of the landslide, but cannot describe the shape and coverage of the landslide; the latter can accurately depict the contour and coverage of the landslide.

[0005] In the paper "Recognition and Detection of Landslides from High-Resolution Remote Sensing Images Based on DETR" (Bulletin of Surveying and Mapping, 2023, No. 5, pp. 16-20), authors including Du Yufeng and Huang Liang proposed a method for identifying and detecting landslides from high-resolution remote sensing images based on DETR (Detection Transformer). It mainly combines the advantages of convolutional neural networks (CNN) and Transformers, takes Transformers as the main body, and uses DETR networks to realize automatic detection of landslides. Although data enhancement improves detection accuracy, the performance of the model may depend on the quality and quantity of data enhancement, and may have limited recognition capabilities for new types of landslides that have not been seen. In addition, as a Transformer-based model, DETR may require more computing resources, and there may be efficiency issues in processing large-scale data sets.

[0006] In the paper “Landslide detection from bitemporalsatellite imagery using attention-based deep neural networks” (Landslides 2022, Vol. 19, pp. 2459-2471), authors including Amankwah and Guojie Wang introduced a method for automatically detecting landslides from dual-temporal satellite imagery using deep neural networks (DNNs) based on the attention mechanism. The specific network architectures used include the spatial-temporal attention neural network (STANet) and the Siamese nested U-Net (SNUNet). STANet contains a feature extractor, an attention module, and a metric module to extract the features of the front and back landslide images and output a distance map. SNUNet is based on an encoder-decoder network architecture, using a Siamese network as an encoder, and improving feature fusion and positioning accuracy through skip connections and integrated channel attention modules (ECAM). The performance of the proposed method was evaluated in the validation and testing phases and compared with other methods. The disadvantage of this method is that STANet has a longer training time due to its deeper network architecture, which may limit its application in scenarios that require fast response.

[0007] Although existing change detection methods have a certain theoretical basis and have achieved relatively good detection results, there are still many problems:

[0008] 1. Currently, landslide identification and detection based on deep learning mainly rely on the convolutional neural network (CNN) structure. Although CNN is widely used in landslide identification due to its translation invariance and inductive bias, its receptive field is limited, and increasing the receptive field requires increasing the number of network layers. This may not only lead to the loss of valuable information and ignore the association between local and global information, but also cause incomplete identification of large-scale landslides, and is easily affected by the "same spectrum, different objects" phenomenon, increasing the risk of false detection and missed detection;

[0009] Second, the application of deep learning in remote sensing image landslide detection is mainly concentrated in one-stage and two-stage models. In the two-stage model, since the image features need to be processed twice and a large number of target pre-selection boxes need to be generated, this leads to slow model operation, high training costs, and lack of real-time detection capabilities, making it difficult to meet the needs of practical applications. Summary of the invention

[0010] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to propose a method, system, device and medium for detecting landslide change areas on dual-phase remote sensing images. By cleverly fusing dense fast Fourier convolution blocks DFC in the convolution layer and introducing shallow feature fusion modules SFF in each stage of the encoder, efficient fusion of features is achieved; in addition, a dual attention module DAB is integrated in the jump connection, and the model significantly improves the accuracy and efficiency of landslide detection in remote sensing images; this innovative method can effectively solve the problems of large network calculation data volume, slow model detection speed and low multi-scale landslide positioning accuracy faced by landslide detection in remote sensing images, while reducing the missed detection and false detection of the landslide change range, and has the advantages of fully extracting feature information and enhancing the model's ability to capture global image information.

[0011] In order to achieve the above object, the technical solution adopted by the present invention is:

[0012] A method for detecting landslide change areas on dual-temporal remote sensing images comprises the following steps:

[0013] Construct a landslide change area detection network on remote sensing images; collect landslide sample and non-landslide sample data sets, and divide the data sets into a training set, a validation set and a test set; input the training set and the validation set into the landslide change area detection network on remote sensing images for training and validation in sequence, obtain a trained landslide change area detection network on remote sensing images, and save a weight file; input the test set into the trained landslide change area detection network on remote sensing images, load the weight file of the trained landslide change area detection network on remote sensing images, output the landslide change area detection result on remote sensing images and save a label file.

[0014] The specific construction method of the landslide change area detection network on the remote sensing image is:

[0015] Constructing an encoder structure, the encoder structure includes four downsampling blocks, namely downsampling block 1, downsampling block 2, downsampling block 3 and downsampling block 4, wherein each downsampling block includes two dense fast Fourier convolution blocks DFC and one shallow feature fusion SFF module;

[0016] The landslide change area detection network on remote sensing images receives two dual-temporal landslide remote sensing images as input. The input dual-temporal landslide remote sensing images are first sent to downsampling block 1. In downsampling block 1, each dual-temporal landslide remote sensing image is feature extracted through a dense fast Fourier convolution block DFC. Subsequently, the features processed by the dense fast Fourier convolution block DFC are fused through a shallow feature fusion SFF module. The fused features are input to the jump connection structure. At the same time, the features processed by the dense fast Fourier convolution block DFC are used as the output of downsampling block 1 and input to downsampling block 2. In downsampling block 2, , the features are also processed by the dense fast Fourier convolution block DFC for feature extraction, and then fused by the shallow feature fusion SFF module. The fused features are input to the jump connection structure, and the features processed by the dense fast Fourier convolution block DFC in the downsampling block 2 are input to the downsampling block 3; downsampling blocks 3 and downsampling blocks 4 repeat the above process, and each downsampling block further abstracts and integrates features through the dense fast Fourier convolution block DFC and the shallow feature fusion SFF module. Finally, each downsampling block inputs the feature information fused by the shallow feature fusion SFF module into the jump connection structure;

[0017] Constructing a skip connection structure, and introducing a dual attention module DAB into the skip connection structure; inputting the feature information output by the shallow feature fusion SFF module into the dual attention module DAB of the skip connection structure;

[0018] The dual attention module DAB consists of two components working together:

[0019] The first component is based on the position attention module PAM, which first performs a convolution operation on the input features to obtain the intermediate feature α by reducing the number of channels to one sixteenth of the original 1 , then, the intermediate feature α 1 Further processing is performed through the position attention module PAM, and then a convolution is performed to obtain the deepened features.

[0020] α 1 =Conv(input)(2)

[0021]

[0022] The core of the second component is the channel attention module CAM, which also starts with the convolution operation on the input features to obtain α 2 , and then processed by the channel attention module CAM

[0023] Finally, we get

[0024] α 2 =Conv(input)(4)

[0025] After extracting from the position attention module PAM and the channel attention module CAM respectively

[0026] and Afterwards, the dual attention module DAB is and The attention features are aggregated and summed, and then the number of channels is restored through a convolution operation to produce the final output features;

[0027]

[0028] Constructing a decoder structure, the decoder structure including upsampling and feature fusion; passing the features processed by the dual attention module DAB to the decoder structure for upsampling, and fusing the upsampled features with the features processed by the dual attention module DAB to obtain output features;

[0029] After the upsampling process is completed, the features are passed through two 3x3 convolutional layers in sequence. After the convolutional layer, the ReLU activation function is applied. In the final stage of the decoder, a segmentation head is used to map the fused features to the required number of categories to obtain the final output features.

[0030] The specific method of collecting the landslide sample and non-landslide sample data set and dividing the data set into a training set, a validation set and a test set includes:

[0031] Remote sensing images including landslides and non-landslides are collected from publicly available data, cropped, processed into the same size, and flipped for data enhancement to form a data set; the collected data set is divided into training set, validation set and test set.

[0032] The training set and the validation set are sequentially input into the landslide change area detection network on the remote sensing image for training and validation, and a trained landslide change area detection network on the remote sensing image is obtained, and the specific method of saving the weight file includes:

[0033] Set the training parameters, input the remote sensing images in the training set into the landslide change area detection network on the remote sensing images, save the weight file as "best_loss.pth", and use the loss function with binary cross entropy loss function;

[0034] The binary cross entropy loss function is as follows:

[0035] Loss = -(y·log(y * )+(1-y)·log(1-y * )) (11)

[0036] Among them, y ＊ represents the probability that the network predicts that the target is a landslide, y represents the label of the target, if the target belongs to a landslide, y takes the value of 1, if the target does not belong to a landslide, y takes the value of 0;

[0037] The training set and the validation set are sequentially input into the landslide change area detection network on the remote sensing image; the Adam optimizer and the stochastic gradient descent method are used to iteratively update the parameter weights in the landslide change area detection network on the remote sensing image, and then the training effect is verified on the validation set until the loss function converges, thereby obtaining a trained landslide change area detection network on the remote sensing image.

[0038] The specific method of inputting the test set into the landslide change area detection network on the remote sensing image, loading the weight file of the trained landslide change area detection network on the remote sensing image, outputting the landslide change area detection result on the remote sensing image and saving the label file comprises:

[0039] The test set is input into the trained remote sensing image landslide change area detection network, and the trained remote sensing image landslide change area detection network and the weight file "best_loss.pth" are loaded; landslide detection is performed on the remote sensing images in the test set to obtain the height, width, center point position and confidence of the landslide boundary box; the landslide detection result is output and the label file is saved.

[0040] The dense fast Fourier convolution block DFC includes a dense connection block DB and a fast Fourier convolution FFC;

[0041] Assuming a densely connected block DB network has l layers, the lth layer will accept the output features of all previous network graphs, and the output of the lth layer represents x l , then x l is defined as:

[0042] x l =H l ({x0,x1,…,x l-1}) (1)

[0043] Among them, H l (*) represents the nonlinear transformation function. The nonlinear transformation function H l (*) Includes the combined operation of convolutional layer, activation function LeakyReLU, 2D fast Fourier transform (FFT) and its inverse transform;

[0044] Fast Fourier convolution (FFC) includes: convolution layer and LeakyReLU activation function, two-dimensional real fast Fourier transform and two-dimensional real inverse fast Fourier transform;

[0045] The input features first pass through the first convolution layer and the LeakyReLU activation function, then pass through the two-dimensional real fast Fourier transform, the second convolution layer and the LeakyReLU activation function, and finally pass through the two-dimensional real inverse fast Fourier transform. The results of the first convolution layer and the LeakyReLU activation function are added to the results of the two-dimensional real fast Fourier transform for output;

[0046] The shallow feature fusion SFF module includes two branches of the same structure, each of which includes global average pooling, two 1×1 convolution operations and feature weighted summation. The workflow is as follows:

[0047] First, the shallow feature fusion SFF module performs global average pooling on each input feature map after the dense fast Fourier convolution block DFC to aggregate global information; for the i-th shallow feature fusion SFF module, its input channel size is C i , through the global average pooling operation, the spatial information of each channel is compressed into a single global eigenvalue; then, the compressed single global eigenvalue undergoes two convolution operations with a kernel size of 1×1, and the two convolution operations are responsible for the compression and excitation processes of the features respectively; then, the ReLU function and Sigmoid function are applied to perform nonlinear activation and normalization on the features; finally, the features of the two channels are fused by weighted and element-by-element addition to generate the final feature information.

[0048] The position attention module (PAM) first obtains a local feature A, represented by A∈R C×H×W , where C represents the channel, H represents the height, and W represents the width. Then the local feature A is input into a convolutional layer to obtain three new feature maps, namely feature map B, feature map C, and feature map D. The size of each feature map is R C×H×W , add the local feature A and feature map B element by element to obtain the weighted feature E, and reshape E into R C×N , where N = H × W represents the number of pixels; reshape the feature map C into R C×N, a matrix multiplication is performed between the feature map C and the transpose of the weighted feature E, followed by a softmax layer to compute the spatial attention map S∈R N×N :

[0049]

[0050] Among them, S ji Weigh the influence of the i-th position on the j-th position, and then reshape the feature map D into R C×N , multiply it with the parameter α to obtain the final output F∈R C×H×W :

[0051]

[0052] The initial value of weight α is 0 and is gradually learned through training;

[0053] The channel attention module (CAM) directly processes the local feature A, whose dimension is R C×H×W and reshape it into R C ×N , where N = H × W represents the total number of pixels. The local feature A is added element by element after global maximum pooling and global average pooling to obtain the weighted feature E, and the weighted feature E is reshaped into R C×N , perform matrix multiplication between the weighted feature E and the transpose of the local feature A, and pass the softmax layer to obtain the channel attention map X∈R C×C :

[0054]

[0055] Among them, x ji Weigh the influence of the i-th channel on the j-th channel, then perform matrix multiplication between the transposed channel attention map X and the local feature A, and reshape the result into R C×H×W , and then multiply the reshaped result by the scale parameter β and perform an element-wise sum operation with the local feature A to obtain the final output F∈R C×H×W :

[0056]

[0057] The weight β is initially set to 0 and is gradually learned through training.

[0058] The present invention also provides a system for detecting landslide change areas on dual-temporal remote sensing images, comprising:

[0059] Network construction module, used to build a landslide change area detection network on remote sensing images;

[0060] A data set collection and processing module, used to collect landslide sample and non-landslide sample data sets, and divide the data sets into training sets, validation sets and test sets;

[0061] The network training module is used to input the training set and the verification set into the landslide change area detection network on the remote sensing image in sequence for training and verification, obtain the trained landslide change area detection network on the remote sensing image, and save the weight file;

[0062] The network detection module is used to input the test set into the trained landslide change area detection network on the remote sensing image, load the weight file of the trained landslide change area detection network on the remote sensing image, output the landslide change area detection result on the remote sensing image and save the label file.

[0063] The present invention also provides a device for detecting landslide change areas on dual-temporal remote sensing images, comprising:

[0064] Memory: a computer program storing the above method for detecting landslide change area on dual-temporal remote sensing images, which is a computer-readable device;

[0065] Processor: used to implement the method of detecting landslide change area on dual-temporal remote sensing images when executing the computer program.

[0066] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the method for detecting landslide change areas on dual-temporal remote sensing images.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] (1) The present invention designs a dense fast Fourier convolution block DFC based on a densely connected network and fast Fourier convolution when constructing a landslide change area detection network on a remote sensing image, and introduces a shallow feature fusion SFF module at each stage of the encoder structure, thereby enhancing the network model's ability to extract multi-scale landslide features and improving the detection accuracy.

[0069] (2) The present invention introduces a double-layer attention module DAB in the skip connection structure. This design can effectively filter out irrelevant information in the skip connection. In this way, the model can focus more on the key features in the image, thereby improving the quality of image reconstruction. This optimized feature transfer mechanism further enhances the model's ability to express landslide features and provides strong support for accurate detection.

[0070] Compared with the existing technology, the present invention significantly reduces the missed detection and false detection rates of landslide area information while maintaining a low training cost. This is due to the innovations of the model in feature extraction and attention mechanism, which significantly improves the precision and accuracy of landslide area change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 The schematic diagram of the structure of the landslide change area detection network on the remote sensing image constructed by the present invention.

[0072] Figure 2 Schematic diagram of the structure of the dense fast Fourier convolution block DFC in the present invention.

[0073] Figure 3 Schematic diagram of the structure of the convolution layer in the dense fast Fourier convolution block DFC in the present invention.

[0074] Figure 4 It is a structural schematic diagram of the shallow feature fusion SFF module in the present invention.

[0075] Figure 5(a) is a schematic diagram of the structure of the dual attention module DAB in the present invention, Figure 5(b) is a schematic diagram of the structure of the position attention module PAM, and Figure 5(c) is a schematic diagram of the structure of the channel attention module CAM.

[0076] Figure 6 The detection result of the landslide change area detection network on the remote sensing image constructed by the present invention, wherein: Figure 6 (a) is the remote sensing image at time T1, Figure 6 (b) is the remote sensing image at time T2, Figure 6 (c) is the label image, Figure 6 (d) is the detection result of the method of the present invention. DETAILED DESCRIPTION

[0077] The technical scheme and effects of the present invention are described in more detail below in conjunction with the accompanying drawings and embodiments.

[0078] A method for detecting landslide change areas on dual-temporal remote sensing images comprises the following steps:

[0079] Construct a landslide change area detection network based on remote sensing images;

[0080] The landslide change area detection network on remote sensing images is based on the U-Net encoding-decoding architecture, which consists of three core parts: encoder, skip connection and decoder; Figure 1 As shown in the figure, the landslide change area detection network on remote sensing images is explained.

[0081] In the encoder part, the input dual-phase remote sensing image is first subjected to feature extraction through the twin network. In order to more effectively fuse global and local features, the present invention introduces a dense fast Fourier convolution block (DFC) in the convolution layer. This design not only enhances the ability of feature extraction, but also helps to capture subtle changes in the image. At each stage of the encoder, a shallow feature fusion module (SFF) is integrated, which is responsible for fusing features of different scales to retain more spatial information, which is crucial for accurately identifying landslide areas.

[0082] The skip connection part is another innovation of the present invention, in which the present invention introduces a dual attention module (DAB). This module enables the model to adaptively focus on key channels and spatial positions in the input features, significantly improving the recognition performance of the model, especially in complex and changeable landslide scenes.

[0083] In the decoder part, the present invention gradually restores the spatial resolution of the feature map through upsampling operations to facilitate more precise positioning of the landslide area. At the same time, the upsampled feature map is accurately fused with the feature map processed by the dual attention module DAB. In the final stage of the decoder, a segmentation head, i.e., a 1x1 convolutional layer, is used to map the fused feature map to the required number of categories, thereby achieving accurate detection of the landslide change area.

[0084] The specific construction method of the landslide change area detection network on the remote sensing image is:

[0085] Constructing an encoder structure, the encoder structure includes four downsampling blocks, namely downsampling block 1, downsampling block 2, downsampling block 3 and downsampling block 4, wherein each downsampling block includes two dense fast Fourier convolution blocks DFC and one shallow feature fusion SFF module;

[0086] Specifically, the landslide change area detection network on remote sensing images receives two dual-temporal landslide remote sensing images as input. The input dual-temporal landslide remote sensing images are first sent to downsampling block 1. In downsampling block 1, each dual-temporal landslide remote sensing image is subjected to feature extraction through a dense fast Fourier convolution block DFC. Subsequently, the features processed by the dense fast Fourier convolution block DFC are fused through a shallow feature fusion SFF module to enhance the feature expression capability. The fused features are input to the skip connection structure. At the same time, the features processed by the dense fast Fourier convolution block DFC are used as the output of downsampling block 1 and input to downsampling block 2. In downsampling block 2, the features are also processed by dense fast Fourier convolution block DFC for feature extraction, and then fused by shallow feature fusion SFF module. The fused features are input to the skip connection structure, and the features processed by dense fast Fourier convolution block DFC in downsampling block 2 are input to downsampling block 3; downsampling block 3 and downsampling block 4 repeat the above process, and each downsampling block further abstracts and integrates features through dense fast Fourier convolution block DFC and shallow feature fusion SFF module. Finally, each downsampling block inputs the feature information fused by shallow feature fusion SFF module into the skip connection structure.

[0087] like Figure 2 As shown, the dense fast Fourier convolution block DFC structure of the present invention is described.

[0088] The dense fast Fourier convolution block DFC includes a dense connection block DB and a fast Fourier convolution FFC;

[0089] Densely connected blocks DB are the core component of densely connected networks. Densely connected networks are a deep neural network architecture for computer vision and image classification tasks, which are widely used in image classification and object recognition tasks. Densely connected networks are known for their dense connections between layers, which can use network parameters more efficiently and reduce the risk of overfitting. In densely connected networks, each layer receives input from all previous layers, rather than just the previous layer as in traditional CNNs. This dense connection structure allows for stronger feature propagation and alleviates gradient vanishing, which can achieve better performance in image classification tasks. Assuming a densely connected block DB network has l layers, the lth layer will accept the output features of all previous network graphs, and the output of the lth layer represents x l , then x l is defined as:

[0090] x l =H l ({x0,x1,…,x l-1}) (1)

[0091] Among them, H l(*) represents the nonlinear transformation function. The nonlinear transformation function H l (*) Includes the combined operation of convolutional layer, activation function LeakyReLU, 2D fast Fourier transform (FFT) and its inverse transform;

[0092] By introducing densely connected blocks DB into the landslide change area detection network on remote sensing images, feature reuse and gradient flow can be enhanced, thereby improving the performance and generalization ability of the model.

[0093] Fast Fourier convolution (FFC) has the ability to extract global features, such as Figure 3 As shown, the present invention uses a two-dimensional real fast Fourier transform to transform the traditional spatial features into the frequency domain to extract global information, and then converts the features back to the spatial domain through a two-dimensional real inverse fast Fourier transform. Specifically, the fast Fourier convolution (FFC) includes: a convolution layer and a LeakyReLU activation function, a two-dimensional real fast Fourier transform, and a two-dimensional real inverse fast Fourier transform;

[0094] The input features first pass through the first convolution layer and the LeakyReLU activation function, then through the two-dimensional real fast Fourier transform, the second convolution layer and the LeakyReLU activation function, and finally through the two-dimensional real inverse fast Fourier transform. The results of the first convolution layer and the LeakyReLU activation function are added to the result of the two-dimensional real fast Fourier transform for output.

[0095] The introduction of dense fast Fourier convolution block DFC accelerates the convolution process through fast Fourier transform (FFT) and inverse transform (IFFT) while maintaining the expressive power of convolution.

[0096] like Figure 4 As shown, the shallow feature fusion SFF module structure proposed in the present invention is explained.

[0097] The shallow feature fusion SFF module includes two branches of the same structure, each of which includes global average pooling, two 1×1 convolution operations and feature weighted summation. The workflow is as follows:

[0098] First, the shallow feature fusion SFF module performs global average pooling (AvgPool) on each input feature map after the dense fast Fourier convolution block DFC to aggregate global information; for the i-th shallow feature fusion SFF module, its input channel size is C i, through the global average pooling (AvgPool) operation, the spatial information of each channel can be compressed into a single global feature value; then, these compressed single global feature values ​​are subjected to two convolution operations with a kernel size of 1×1, which are responsible for the compression and excitation process of the features respectively; then, the ReLU function and Sigmoid function are applied to perform nonlinear activation and normalization on the features; finally, the features of the two channels are fused by weighted and element-by-element addition to generate the final feature information. This fusion method not only retains feature information of different scales, but also enhances the model's ability to capture local details, which is crucial for accurately identifying landslide change areas.

[0099] By introducing the shallow feature fusion SFF module, the feature expression is further enhanced. By integrating information from different convolutional layers, the model's ability to capture key features is improved. The network of the present invention can effectively fuse shallow features at each stage of the encoder and provide richer and more accurate feature information for subsequent decoder layers, thereby improving the accuracy and robustness of landslide change area detection.

[0100] Constructing a skip connection structure, and introducing a dual attention module DAB into the skip connection structure; inputting the feature information output by the shallow feature fusion SFF module into the dual attention module DAB of the skip connection structure;

[0101] As shown in FIG5 , the dual attention module DAB structure involved in the present invention is described.

[0102] As shown in Figure 5(a), the dual attention module (DAB) cleverly combines the superior capabilities of the position attention module (PAM) in spatial feature extraction and the advantages of the channel attention module (CAM) in channel feature extraction. The design of the dual attention module DAB contains two components that work together, each of which is optimized for a specific attention mechanism.

[0103] The first component is based on the position attention module PAM, which first performs a convolution operation on the input features to obtain the intermediate feature α by reducing the number of channels to one sixteenth of the original 1 , this step effectively simplifies the feature extraction process of the position attention module PAM. Subsequently, the intermediate feature α 1 Further processing is performed by the position attention module PAM,

[0104] After another convolution, we get the deepened features.

[0105] α 1 =Conv(input) (2)

[0106]

[0107] The structure of the second component is similar to the first one. Its core is the channel attention module CAM, which also starts with the convolution operation on the input features to obtain α 2 , and then through the channel

[0108] The attention module CAM is processed and finally obtained through convolution

[0109] α 2 =Conv(input)(4)

[0110] After extracting from the position attention module PAM and the channel attention module CAM respectively

[0111] and Afterwards, the dual attention module DAB is and The attention features are aggregated and summed, and then the number of channels is restored through a convolution operation to produce the final output features;

[0112]

[0113] As a feature extraction module, the dual attention module (DAB) integrates the features of specific positions and channels of the image, especially in the U-Net-shaped architecture, which performs well in both position-based and channel-based feature extraction, and can obtain more detailed and accurate feature sets. Therefore, the present invention integrates it into the encoder and jump connection to enhance the segmentation performance of the model.

[0114] The dual attention module (DAB) includes a position attention module (PAM) and a channel attention module (CAM); as shown in Figure 5(b), the position attention module (PAM) captures the spatial dependency between any two positions of the feature map and updates a specific feature by the weighted sum of all position features, where the weight is determined by the feature similarity between the two positions; therefore, the position attention module (PAM) is very effective in extracting meaningful spatial features. The position attention module (PAM) first obtains a local feature A, represented by A∈R C×H×W , where C represents the channel, H represents the height, and W represents the width. Then the local feature A is input into a convolutional layer to obtain three new feature maps, namely feature map B, feature map C, and feature map D. The size of each feature map is R C×H×W Next, the local feature A and the feature map B are added element by element to obtain the weighted feature E, and E is reshaped into RC×N , where N = H × W represents the number of pixels; reshape the feature map C into R C×N , a matrix multiplication is performed between the feature map C and the transpose of the weighted feature E, followed by a softmax layer to compute the spatial attention map S∈R N ×N :

[0115]

[0116] Among them, S ji Weigh the influence of the i-th position on the j-th position, and then reshape the feature map D into R C×N , multiply it with the parameter α to obtain the final output F∈R C×H×W :

[0117]

[0118] The weight α is initialized to 0 and is gradually learned through training; the position attention module (PAM) has a strong ability to extract spatial features. Since the weighted feature E is the weighted sum of all position features and original features, it has global context features and aggregates context according to the spatial attention map, which ensures that the position features are effectively extracted while maintaining the global context information.

[0119] As shown in Figure 5(c), the channel attention module (CAM) is good at extracting channel features. Different from the position attention module (PAM), the channel attention module (CAM) directly processes the local feature A, whose dimension is R C×H×W and reshape it into R C×N , where N = H × W represents the total number of pixels. The local feature A is added element by element after global maximum pooling and global average pooling to obtain the weighted feature E, and the weighted feature E is reshaped into R C×N , perform matrix multiplication between the weighted feature E and the transpose of the local feature A, and pass the softmax layer to obtain the channel attention map X∈R C×C :

[0120]

[0121] Among them, x ji Weigh the influence of the i-th channel on the j-th channel, then perform matrix multiplication between the transposed channel attention map X and the local feature A, and reshape the result into R C×H×W , and then multiply the reshaped result by the scale parameter β and perform an element-wise sum operation with the local feature A to obtain the final output F∈R C×H×W :

[0122]

[0123] Similar to α, the weight β is initially 0 and is gradually learned through training; in the channel attention module (CAM), the final feature of each channel is the weighted sum of all channel features and original features, which makes the channel attention module (CAM) powerful in extracting channel features and can capture important information and relationships between different channels.

[0124] This complex architecture of the dual attention module DAB not only seamlessly combines the advantages of the position attention module PAM and the channel attention module CAM, but also greatly improves the model's feature extraction capabilities. Through this design, the dual attention module DAB can more effectively capture the key spatial and channel information in the image, thereby significantly improving the overall performance of the model in processing the task of detecting landslide change areas in remote sensing images, and achieving more accurate positioning in landslide detection.

[0125] Constructing a decoder structure, the decoder structure including upsampling and feature fusion; passing the features processed by the dual attention module DAB to the decoder structure for upsampling, and accurately fusing the upsampled features with the features processed by the dual attention module DAB to obtain output features;

[0126] After the jump connection, the primary task of the decoder is to accurately fuse the upsampled feature map with the feature map of the corresponding layer of the encoder to ensure the size consistency of the feature map and lay the foundation for subsequent processing. After the upsampling process is completed, the features will pass through two 3x3 convolutional layers in sequence. These convolutional layers are designed to further extract and refine features and enhance the model's ability to capture key information. After the convolution layer, the ReLU activation function is applied. This step not only introduces nonlinear factors, but also significantly improves the model's expression ability and learning ability for complex features. In the final stage of the decoder, a segmentation head is used. The present invention uses a 1x1 convolutional layer to map the fused features to the required number of categories to obtain the final output features, ensuring that the model output can accurately reflect the details of the landslide change area in the remote sensing image.

[0127] The specific method of collecting the landslide sample and non-landslide sample data set and dividing the data set into a training set, a validation set and a test set includes:

[0128] Collect remote sensing images containing landslides and non-landslides from publicly available data, crop the collected remote sensing images, process them to the same size, and perform data enhancement operations on the remote sensing images to increase the capacity of training samples. Among them, the number of images containing landslides is not less than 6,000, and the number of images not containing landslides is not less than 1,500, which together constitute the data set used in the present invention;

[0129] The collected data set is divided into training set, validation set and test set in a ratio of 7:2:1.

[0130] The training set and the validation set are sequentially input into the landslide change area detection network on the remote sensing image for training and validation, so as to obtain a trained landslide change area detection network on the remote sensing image, and save the weight file; the specific method includes:

[0131] Set training parameters, input the remote sensing images in the training set into the landslide change area detection network on the remote sensing image, set the training parameter batch-size to 8, train for 200 rounds, and the initial learning rate to 0.0001. Each time, randomly and non-repeatedly select 8 remote sensing images from the training set and input them into the landslide change area detection network on the remote sensing image. The weight file is saved as "best_loss.pth", and the loss function uses a binary cross entropy loss function;

[0132] The binary cross entropy loss function is as follows:

[0133] Loss = -(y·log(y * )+(1-y)·log(1-y * )) (11)

[0134] Among them, y ＊ represents the probability that the network predicts that the target is a landslide, y represents the label of the target, if the target belongs to a landslide, y takes the value of 1, if the target does not belong to a landslide, y takes the value of 0;

[0135] The training set and the validation set are sequentially input into the landslide change area detection network on the remote sensing image; the parameter weights in the landslide change area detection network on the remote sensing image of the present invention are iteratively updated by using the Adam optimizer and the stochastic gradient descent method, and then the training effect is verified on the validation set until the loss function converges, thereby obtaining a trained landslide change area detection network on the remote sensing image.

[0136] The test set is input into the trained remote sensing image landslide change area detection network, the weight file of the trained remote sensing image landslide change area detection network is loaded, the remote sensing image landslide change area detection result is output and the label file is saved. The specific method includes:

[0137] The test set is input into the trained remote sensing image landslide change area detection network, and the trained remote sensing image landslide change area detection network and the weight file "best_loss.pth" are loaded; landslide detection is performed on the remote sensing images in the test set to obtain the height, width, center point position and confidence of the landslide boundary box; the landslide detection result is output and the label file is saved.

[0138] like Figure 6 As shown, the detection results of the landslide change area detection network on the remote sensing image constructed by the present invention are further described. Figure 6 (a) is the remote sensing image at time T1, Figure 6 (b) is the remote sensing image at time T2, Figure 6 (c) is the label image, Figure 6 (d) is the detection result of the method of the present invention. It can be seen from the figure that, compared with the original label, the method of the present invention can significantly reduce the missed detection and false detection rate of landslide area information while maintaining a low training cost when performing landslide change area detection on remote sensing images. This is due to the innovation of the model in feature extraction and attention mechanism, which significantly improves the precision and accuracy of landslide area change detection.

[0139] The landslide change area detection network of the present invention has a number of innovative features in design, which significantly improves the accuracy and efficiency of detection:

[0140] (1) Combination of twin network and dense fast Fourier convolution block DFC: In the encoder part, the present invention adopts the twin network structure to extract features from the input image. By introducing the dense fast Fourier convolution block DFC in the convolution layer, not only the network's ability to fuse global and local features is enhanced, but also the feature propagation is strengthened through the dense connection structure, effectively alleviating the gradient disappearance problem. Compared with ordinary convolution, fast Fourier convolution (FFC) shows more excellent performance in capturing global features and provides richer information for the identification of landslide areas.

[0141] (2) Innovative application of shallow feature fusion module (SFF): At each stage of the encoder, the present invention introduces the SFF module. This module is specially designed to fuse the features extracted by the twin structure branch at different scales, ensuring the maximum retention of spatial information. Through the SFF module, the network can effectively integrate features at different resolutions, providing a more comprehensive and accurate image representation for subsequent landslide detection.

[0142] (3) Introduction of Dual Attention Module (DAB): In the skip connection part, the present invention introduces a dual attention module DAB. This module enables the model to adaptively focus on key channels and spatial positions in the input features, thereby significantly improving the recognition performance. The design of DAB allows the network to more accurately identify landslide areas when processing remote sensing images, and maintain high accuracy even under complex terrain and environmental conditions.

[0143] Application prospects of the present invention:

[0144] With the continuous breakthroughs in domestic high-resolution satellite technology, the increasing maturity of deep learning technology, and the significant improvement of computer hardware equipment, the field of intelligent recognition of landslide images has ushered in new development opportunities. These technological advances provide researchers with high-resolution data, more efficient and concise network structure design, and more powerful hardware performance support.

[0145] The present invention is based on the encoder-decoder architecture, detects the dual-phase remote sensing landslide area, uses the collected remote sensing landslide image data and performs data enhancement operations on it to train the model. In the encoder part, the present invention uses a twin network for feature extraction and introduces a dense fast Fourier convolution block (DFC) to fuse global and local features. In addition, at each stage of the encoder, a shallow feature fusion module (SFF) is integrated to fuse features of different scales and enhance the model's ability to capture landslide features. In the jump connection part, the present invention innovatively introduces a dual attention module (DAB), which enables the model to adaptively focus on key channels and spatial positions, further improving recognition performance.

[0146] Landslide detection technology has broad application prospects and important social value. First, through accurate landslide detection, disasters can be prevented in advance to protect people's lives and property safety. Secondly, this technology is of great significance for emergency rescue command after a landslide occurs, which helps to improve the efficiency and effectiveness of emergency response and maintain the stability of the city. In addition, this technology can also play a key role in many fields such as environmental protection and disaster warning, providing strong support and guarantee for urban management and social development.

[0147] The present invention also provides a system for detecting landslide change areas on dual-temporal remote sensing images, comprising:

[0148] Network construction module, used to build a landslide change area detection network on remote sensing images;

[0149] A data set collection and processing module, used to collect landslide sample and non-landslide sample data sets, and divide the data sets into training sets, validation sets and test sets;

[0150] The network training module is used to input the training set and the verification set into the landslide change area detection network on the remote sensing image in sequence for training and verification, obtain the trained landslide change area detection network on the remote sensing image, and save the weight file;

[0151] The network detection module is used to input the test set into the trained landslide change area detection network on the remote sensing image, load the weight file of the trained landslide change area detection network on the remote sensing image, output the landslide change area detection result on the remote sensing image and save the label file.

[0152] The present invention also provides a device for detecting landslide change areas on dual-temporal remote sensing images, comprising:

[0153] Memory: a computer program storing the above method for detecting landslide change area on dual-temporal remote sensing images, which is a computer-readable device;

[0154] Processor: used to implement the method of detecting landslide change area on dual-temporal remote sensing images when executing the computer program.

[0155] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the method for detecting landslide change areas on dual-temporal remote sensing images.

Claims

1. A method for detecting landslide change areas on dual-temporal remote sensing images, characterized in that: The following steps are involved: Construct a landslide change area detection network on remote sensing images; collect landslide sample and non-landslide sample data sets, and divide the data sets into a training set, a validation set and a test set; input the training set and the validation set into the landslide change area detection network on remote sensing images for training and validation in sequence, obtain a trained landslide change area detection network on remote sensing images, and save a weight file; input the test set into the trained landslide change area detection network on remote sensing images, load the weight file of the trained landslide change area detection network on remote sensing images, output the landslide change area detection result on remote sensing images and save a label file.

2. A method for detecting landslide change areas on dual-temporal remote sensing images according to claim 1, characterized in that: The specific construction method of the landslide change area detection network on the remote sensing image is: Constructing an encoder structure, the encoder structure includes four downsampling blocks, namely downsampling block 1, downsampling block 2, downsampling block 3 and downsampling block 4, wherein each downsampling block includes two dense fast Fourier convolution blocks DFC and one shallow feature fusion SFF module; The landslide change area detection network on remote sensing images receives two dual-temporal landslide remote sensing images as input. The input dual-temporal landslide remote sensing images are first sent to downsampling block 1. In downsampling block 1, each dual-temporal landslide remote sensing image is feature extracted through a dense fast Fourier convolution block DFC. Subsequently, the features processed by the dense fast Fourier convolution block DFC are fused through a shallow feature fusion SFF module. The fused features are input to the jump connection structure. At the same time, the features processed by the dense fast Fourier convolution block DFC are used as the output of downsampling block 1 and input to downsampling block 2. In downsampling block 2, , the features are also processed by the dense fast Fourier convolution block DFC for feature extraction, and then fused by the shallow feature fusion SFF module. The fused features are input to the jump connection structure, and the features processed by the dense fast Fourier convolution block DFC in the downsampling block 2 are input to the downsampling block 3; downsampling blocks 3 and downsampling blocks 4 repeat the above process, and each downsampling block further abstracts and integrates features through the dense fast Fourier convolution block DFC and the shallow feature fusion SFF module. Finally, each downsampling block inputs the feature information fused by the shallow feature fusion SFF module into the jump connection structure; Constructing a skip connection structure, and introducing a dual attention module DAB into the skip connection structure; inputting the feature information output by the shallow feature fusion SFF module into the dual attention module DAB of the skip connection structure; The dual attention module DAB consists of two components working together: The first component is based on the position attention module PAM, which first performs a convolution operation on the input features to obtain the intermediate feature α by reducing the number of channels to one sixteenth of the original 1 , then, the intermediate feature α 1 Further processing is performed through the position attention module PAM, and then a convolution is performed to obtain the deepened features. α 1 =Conv(input) (2) The core of the second component is the channel attention module CAM, which also starts with the convolution operation on the input features to obtain α 2 , and then processed by the channel attention module CAM, and finally obtained by convolution α 2 =Conv(input) (4) After extracting from the position attention module PAM and the channel attention module CAM respectively and Afterwards, the dual attention module DAB is and The attention features are aggregated and summed, and then the number of channels is restored through a convolution operation to produce the final output features; Constructing a decoder structure, the decoder structure including upsampling and feature fusion; passing the features processed by the dual attention module DAB to the decoder structure for upsampling, and fusing the upsampled features with the features processed by the dual attention module DAB to obtain output features; After the upsampling process is completed, the features are passed through two 3x3 convolutional layers in sequence. After the convolutional layer, the ReLU activation function is applied. In the final stage of the decoder, a segmentation head is used to map the fused features to the required number of categories to obtain the final output features.

3. The method for detecting landslide change area on dual-temporal remote sensing images according to claim 1, characterized in that: The specific method of collecting the landslide sample and non-landslide sample data set and dividing the data set into a training set, a validation set and a test set includes: Remote sensing images including landslides and non-landslides are collected from publicly available data, cropped, processed into the same size, and flipped for data enhancement to form a data set; the collected data set is divided into training set, validation set and test set.

4. The method for detecting landslide change area on dual-temporal remote sensing images according to claim 1, characterized in that: The training set and the validation set are sequentially input into the landslide change area detection network on the remote sensing image for training and validation, and a trained landslide change area detection network on the remote sensing image is obtained, and the specific method of saving the weight file includes: Set the training parameters, input the remote sensing images in the training set into the landslide change area detection network on the remote sensing images, save the weight file as "best_loss.pth", and use the loss function with binary cross entropy loss function; The binary cross entropy loss function is as follows: Loss=-(y·log(y * )+(1-y)·log(1-y * )) (11) Among them, y ＊ represents the probability that the network predicts that the target is a landslide, y represents the label of the target, if the target belongs to a landslide, y takes the value of 1, if the target does not belong to a landslide, y takes the value of 0; The training set and the validation set are sequentially input into the landslide change area detection network on the remote sensing image; the Adam optimizer and the stochastic gradient descent method are used to iteratively update the parameter weights in the landslide change area detection network on the remote sensing image, and then the training effect is verified on the validation set until the loss function converges, thereby obtaining a trained landslide change area detection network on the remote sensing image.

5. The method for detecting landslide change area on dual-temporal remote sensing images according to claim 1, characterized in that: The specific method of inputting the test set into the landslide change area detection network on the remote sensing image, loading the weight file of the trained landslide change area detection network on the remote sensing image, outputting the landslide change area detection result on the remote sensing image and saving the label file comprises: Input the test set into the trained remote sensing image landslide change area detection network, load the trained remote sensing image landslide change area detection network, weight file "best_loss.pth"; perform landslide detection on the remote sensing images in the test set, obtain the height, width, center point position and confidence of the landslide boundary box; output the landslide detection result and save the label file.

6. The method for detecting landslide change area on dual-temporal remote sensing images according to claim 2, characterized in that: The dense fast Fourier convolution block DFC includes a dense connection block DB and a fast Fourier convolution FFC; Assuming a densely connected block DB network has l layers, the lth layer will accept the output features of all previous network graphs, and the output of the lth layer represents x l , then x l is defined as: x l =H l ({x0,x1,…,x l-1 }) (1) Among them, H l (*) represents the nonlinear transformation function. The nonlinear transformation function H l (*) Includes the combined operation of convolutional layer, activation function LeakyReLU, 2D fast Fourier transform (FFT) and its inverse transform; Fast Fourier convolution (FFC) includes: convolution layer and LeakyReLU activation function, two-dimensional real fast Fourier transform and two-dimensional real inverse fast Fourier transform; The input features first pass through the first convolution layer and the LeakyReLU activation function, then pass through the two-dimensional real fast Fourier transform, the second convolution layer and the LeakyReLU activation function, and finally pass through the two-dimensional real inverse fast Fourier transform. The results of the first convolution layer and the LeakyReLU activation function are added to the results of the two-dimensional real fast Fourier transform for output; The shallow feature fusion SFF module includes two branches of the same structure, each of which includes global average pooling, two 1×1 convolution operations and feature weighted summation. The workflow is as follows: First, the shallow feature fusion SFF module performs global average pooling on each input feature map after the dense fast Fourier convolution block DFC to aggregate global information; for the i-th shallow feature fusion SFF module, its input channel size is C i , through the global average pooling operation, the spatial information of each channel is compressed into a single global eigenvalue; then, the compressed single global eigenvalue undergoes two convolution operations with a kernel size of 1×1, and the two convolution operations are responsible for the compression and excitation processes of the features respectively; then, the ReLU function and Sigmoid function are applied to perform nonlinear activation and normalization on the features; finally, the features of the two channels are fused by weighted and element-by-element addition to generate the final feature information.

7. The method for detecting landslide change area on dual-temporal remote sensing images according to claim 2, characterized in that: The position attention module (PAM) first obtains a local feature A, represented by A∈R C×H×W , where C represents the channel, H represents the height, and W represents the width. Then the local feature A is input into a convolutional layer to obtain three new feature maps, namely feature map B, feature map C, and feature map D. The size of each feature map is R C×H×W , add the local feature A and feature map B element by element to obtain the weighted feature E, and reshape E into R C×N , where N = H × W represents the number of pixels; reshape the feature map C into R C×N , a matrix multiplication is performed between the feature map C and the transpose of the weighted feature E, followed by a softmax layer to compute the spatial attention map S∈R N×N : Among them, S ji Weigh the influence of the i-th position on the j-th position, and then reshape the feature map D into R C×N , multiply it with the parameter α to obtain the final output F∈R C×H×W : The initial value of weight α is 0 and is gradually learned through training; The channel attention module (CAM) directly processes the local feature A, whose dimension is R C×H×W and reshape it into R C×N , where N = H × W represents the total number of pixels. The local feature A is added element by element after global maximum pooling and global average pooling to obtain the weighted feature E, and the weighted feature E is reshaped into R C×N , perform matrix multiplication between the weighted feature E and the transpose of the local feature A, and pass the softmax layer to obtain the channel attention map X∈R C×C : Among them, x ji Weigh the influence of the i-th channel on the j-th channel, then perform matrix multiplication between the transposed channel attention map X and the local feature A, and reshape the result into R C×H×W , and then multiply the reshaped result by the scale parameter β and perform an element-wise sum operation with the local feature A to obtain the final output F∈R C×H×W : The weight β is initially set to 0 and is gradually learned through training.

8. A system for detecting landslide change areas on dual-temporal remote sensing images based on the method according to any one of claims 1 to 7, characterized in that: include: Network construction module, used to build a landslide change area detection network on remote sensing images; A data set collection and processing module, used to collect landslide sample and non-landslide sample data sets, and divide the data sets into training sets, validation sets and test sets; The network training module is used to input the training set and the verification set into the landslide change area detection network on the remote sensing image in sequence for training and verification, obtain the trained landslide change area detection network on the remote sensing image, and save the weight file; The network detection module is used to input the test set into the trained landslide change area detection network on the remote sensing image, load the weight file of the trained landslide change area detection network on the remote sensing image, output the landslide change area detection result on the remote sensing image and save the label file.

9. A device for detecting landslide change areas on dual-temporal remote sensing images, characterized in that: include: Memory: a computer program storing a method for detecting a landslide change area on a dual-temporal remote sensing image as described in any one of claims 1 to 7, which is a computer-readable device; Processor: used to implement the method for detecting landslide change area on dual-temporal remote sensing images as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement a method for detecting a landslide change area on a dual-temporal remote sensing image as described in any one of claims 1 to 7.

Images