Landslide Identification Method Based on Visual State Space and Frequency Domain Augmentation
By introducing a remote sensing image recognition method that enhances visual state space and frequency domain, the problems of insufficient feature extraction and inadequate detail capture in high-resolution remote sensing images are solved, achieving high-precision and efficient recognition of landslide areas, and applicable to the recognition of landslides with complex shapes.
Patent Information
- Application Number
- CN202511352815.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-22
- Publication Date
- 2026-03-06
- Estimated Expiration
- 2045-09-22
AI Technical Summary
Existing remote sensing image landslide identification methods suffer from insufficient feature extraction and inadequate detail capture in high-resolution image processing, making them difficult to adapt to landslide areas with complex shapes. They also exhibit low computational efficiency and weak detail recovery capabilities.
A remote sensing image recognition method based on visual state space and frequency domain enhancement is adopted. Multi-scale feature extraction and high-frequency information enhancement are performed by the visual state space module VSSBlock and the frequency domain enhancement module FreqEM. Combined with the grouped spatial upsampling module GSU Block, the spatial resolution of the feature map is restored, so as to achieve accurate segmentation and localization of landslide areas.
It significantly improves the recognition accuracy of landslide edges and complex terrain scenes, enhances the integrity and robustness of landslide areas, solves the computational resource problem of traditional methods in high-resolution remote sensing image processing, and provides high-precision and efficient landslide geological hazard identification capabilities.
Smart Images

Figure CN121170486B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of geological disaster monitoring, and specifically relates to a landslide identification method based on visual state space and frequency domain enhancement of remote sensing images. Background Technology
[0002] High-resolution remote sensing imagery has significant application value in geological disaster monitoring, especially in the identification and early warning of landslide geological hazards. However, traditional landslide identification methods mainly rely on manual visual interpretation and simple image processing techniques, which have many limitations. Manual visual interpretation is not only inefficient but also subject to the professional level and experience of the interpreters, making it prone to subjective bias and missed detections. Simple image processing techniques (such as edge detection and threshold segmentation) can often only extract partial features from the image and have poor adaptability to complex and ever-changing landslide geological phenomena, making it difficult to meet the needs of high-precision landslide identification. In recent years, with the rapid development of remote sensing technology, the amount of high-resolution remote sensing imagery data has exploded, which has placed higher demands on the automation and intelligence of landslide identification methods.
[0003] Deep learning technology, especially convolutional neural networks, has achieved remarkable results in image classification, object detection, and semantic segmentation. Deep learning models can automatically learn complex features in images and have powerful feature extraction and pattern recognition capabilities. In the field of landslide identification, researchers have begun to try to use deep learning technology to improve identification accuracy and efficiency. For example, some deep learning-based landslide identification methods have built convolutional neural network models to learn landslide features in high-resolution remote sensing images, achieving automatic segmentation and identification of landslide areas. These methods have overcome the limitations of traditional methods to some extent, but there are still some key problems that need to be solved.
[0004] In their patent "Automatic Geological Hazard Identification Method Based on Multi-Source Data and Deep Learning" (CN202411317906.6, September 20, 2024), Li Dongdong, Jian Haojun, and other authors proposed an improved landslide identification model based on UNet++. This model integrates multi-source remote sensing information to overcome the limitations of information from a single data source, thereby improving the accuracy and efficiency of geological hazard identification. The specific steps of this method include: acquiring multi-source data within a preset area; preprocessing the multi-source data; fusing the preprocessed multi-source data to obtain multi-band images; determining landslide boundaries based on the multi-band images to generate landslide samples; and training the improved UNet++ semantic segmentation model using the landslide samples to obtain a trained improved UNet++ semantic segmentation model. The drawback of this method is that it lacks sufficient exploration of the weight learning mechanism, which may introduce noise features and reduce the accuracy of hazard identification.
[0005] Li Weibin, Kong Yuhui, and other authors proposed an automatic landslide identification method in their patent "An Automatic Landslide Identification Method, System, Device, and Medium Based on Lightweight Convolutional Neural Network and Dual Attention" (CN202310218039.X, March 8, 2023), aiming to improve the accuracy and efficiency of landslide identification. The specific steps of this method are: preprocessing remote sensing images to obtain a landslide remote sensing image dataset; building an Ournetworkframework network model; iteratively training the Ournetworkframework network model and finally saving the weights of the best training iterations; and making predictions using the trained network model to evaluate its performance. The shortcomings of this method are: it involves frequent dimensionality transformations (such as flattening and restoring), which may introduce additional computational overhead; and the dimensionality-upgrading design will significantly increase memory usage, weakening the lightweight advantage.
[0006] In their paper "Research on Automatic Landslide Recognition Based on Faster R-CNN and Mask R-CNN" (Geodesy and Geodynamics 2025), Yu Xianyu et al. proposed an instance segmentation-driven landslide boundary optimization method. Specifically, they compared the two models in the Three Gorges Reservoir area and proved that Mask R-CNN retains more boundary details through pixel-level mask generation, and its recall rate and F1 score are significantly better than the detection framework. The limitation of this method is that pixel-level annotation requires extremely high manual costs, and the model is sensitive to image resolution, with severe loss of boundary details when the input is low resolution.
[0007] In their paper "Research on Landslide Image Detection Method Based on Remote Sensing Imagery" (Computer Science and Applications, Vol. 11, No. 13, 2023, pp. 1204-1212), Hou Mengyuan et al. proposed an improved Faster R-CNN landslide detection method. Specifically, they introduced the Inception module to enhance multi-scale feature extraction capabilities and fused a residual attention mechanism to focus on the main landslide region. On the Bijie City dataset, the accuracy was improved by 2% to 5%, and the identification of blurred boundary regions was significantly optimized. However, this method mainly focuses on the target detection framework and does not fully restore the shape of the region boundary. Moreover, in terms of landslide boundary refinement, it still relies on the accuracy of the detection box and is difficult to achieve pixel-level recognition.
[0008] In their paper "A Study of High-Resolution Remote Sensing ImageLandslide Detection with Optimized Anchor Boxes and Edge Enhancement" (European Journal of Remote Sensing 2023), Wang K. et al. proposed an improved landslide detection method for YOLOv5. Specifically, they combined Gabor filters to enhance image edge features and used K-means++ and ISODATA algorithms to optimize anchor box clustering, thus solving the problem of missed detection of small targets and achieving a recall rate of 86.14%. The limitations of this method are that the Gabor filter parameters need to be manually adjusted, it has poor adaptability to different terrain edge features, and the anchor box optimization does not solve the problem of false detection caused by complex backgrounds (such as bare rock and landslides having similar colors).
[0009] Existing methods for identifying landslide geological hazards based on remote sensing images have the following main drawbacks:
[0010] First, the feature extraction capability is limited, making it difficult to effectively identify the boundaries and details of landslide areas. Existing methods mostly rely on traditional convolutional neural networks (such as U-Net, PSPnet, etc.) to extract local features in the spatial domain. The receptive field is limited, making it difficult to model the long-distance dependencies of landslide areas, resulting in insufficient ability to identify landslide areas with blurred boundaries or complex details.
[0011] Second, it has poor adaptability to high-resolution remote sensing images, low computational efficiency and weak ability to restore details. High-resolution remote sensing images are characterized by large scale, rich texture and varied shapes. However, traditional methods often use simple upsampling or deconvolution in the decoding stage, which makes it difficult to balance computational efficiency and reconstruction of spatial details. This results in problems such as inaccurate positioning or discontinuous boundaries in the segmentation results of landslide areas. Summary of the Invention
[0012] To address the shortcomings of existing technologies, this invention designs a landslide identification method based on visual state space and frequency domain enhancement from remote sensing images. This method aims to solve problems such as insufficient feature extraction, inadequate detail capture, and poor adaptability to complex landslide shapes when processing high-resolution remote sensing images. The method utilizes a visual state space module (VSSBlock) and a frequency domain enhancement module (FreqEM) to perform multi-scale feature extraction and high-frequency information enhancement on the remote sensing images during the encoder stage, effectively capturing key details such as texture and boundaries of the landslide area. In the decoder stage, a grouped spatial upsampling module (GSU Block) is used to gradually restore the spatial resolution of the feature map, achieving accurate segmentation and localization of the landslide area. Through this innovative architecture, this invention can effectively improve the accuracy of landslide geological hazard identification, providing strong technical support for early warning and prevention of geological disasters.
[0013] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0014] A landslide identification method based on visual state space and frequency domain enhancement of remote sensing images includes the following steps:
[0015] Step 1: Construct the coding sub-network of a landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0016] Step 2: Construct a decoding subnetwork for a landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0017] Step 3: Construct a recognition subnetwork for a landslide recognition method based on visual state space and frequency domain enhancement of remote sensing images;
[0018] Step 4: Based on the encoding subnetwork constructed in Step 1, the decoding subnetwork constructed in Step 2, and the recognition subnetwork constructed in Step 3, construct a landslide recognition method network based on visual state space and frequency domain enhancement.
[0019] Step 5: Generate the training set, validation set, and test set;
[0020] Step 6: Use the training set and validation set generated in Step 5 to train the landslide identification method network based on visual state space and frequency domain enhancement constructed in Step 4, and obtain the model weight file.
[0021] Step 7: Using the model weight file obtained from step 6, detect the extent of landslide geological hazards in the remote sensing images of the test set generated in step 5.
[0022] The specific method for step 1 is as follows:
[0023] A coding subnetwork for landslide identification based on visual state space and frequency domain enhancement of remote sensing images is constructed, with the following structure: downsampling 1, downsampling 2, downsampling 3, downsampling 4 and downsampling 5.
[0024] Step 1.1: Construct the downsampling module 1 of the coding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0025] The downsampling module 1 includes a first convolutional layer, a visual state space module (VSS Block), and a max pooling layer. The first convolutional layer is used to adjust the image dimensions to facilitate subsequent calculations. The VSS Block extracts multi-level spatial and semantic features. The max pooling layer reduces the size of the feature map, thereby reducing the parameters and computational cost of subsequent layers and accelerating training.
[0026] The Visual State Space Block (VSS Block) consists of two parallel branches, namely X b and X t ;
[0027] In branch X t In this process, feature X is first processed through a fully connected linear layer, then the feature map is processed using deformable convolution (DConv) and SiLU activation functions; long-range spatial dependency information is extracted through the 2D selection module SS2D; and the feature map is processed using a normalization layer to generate X. t The entire process can be calculated as follows:
[0028] ;
[0029] ;
[0030] The deformable convolution DConv dynamically generates convolution kernel offsets from the input feature map, performs irregular sampling on the feature map, and achieves adaptive feature extraction. It calculates an offset field based on the input feature map, which is a tensor with the same spatial size as the input feature map, representing the number of neighbors for each pixel in space. The offset at each position guides subsequent feature sampling. Irregular sampling is performed on the input feature map; based on its corresponding offset, neighboring pixels are found in the input feature map for sampling, and the feature value at that position is calculated using interpolation and other methods. The features after offset sampling are fused through an aggregation operation to obtain the output feature map.
[0031] In branch X b In this process, the feature map X is first transformed into tensor space through a fully connected linear layer; the output of the fully connected layer is then processed using the SiLU activation function to obtain the feature map X. b This process can be calculated as follows:
[0032] ;
[0033] exist and Element-wise multiplication is performed to fuse the features of the two branches; a fully connected layer is used to process the fused feature map and output feature X'.
[0034] The SS2D module in the Visual State Space Block (VSS Block) is used for feature information capture. Its structure includes: Cross-Scan, Selective Scanning with S6 Blocks, and Cross-Merge. Cross-Scan divides the input image into non-overlapping patches, then expands them into a sequence along four directions (top left to bottom right, bottom right to top left, top right to bottom left, bottom left to top right). Selective Scanning with S6 Blocks... Blocks: Sequences in each direction are processed by independent blocks, extracting features and retaining relevant information while filtering out irrelevant information; Cross-Merge: The processed sequences are recombined to restore a two-dimensional feature map, integrating multi-directional contextual information to achieve a global receptive field; The input image is traversed along four paths in different directions and expanded into a sequence form. The directions of these four paths are: from the top left to the bottom right, from the bottom right to the top left, from the top right to the bottom left, and from the bottom left to the top right; The sequences obtained in each direction are processed in parallel for feature extraction; The feature sequences obtained from the four different directions are expanded, summarized, and merged to obtain a feature map of the same size as the input feature map.
[0035] Step 1.2: Construct the downsampling 2 module of the coding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0036] The downsampling module 2 includes a first convolutional layer, a visual state space module (VSS Block), and a max pooling layer. The first convolutional layer is used to adjust the image dimensions to facilitate subsequent calculations. The VSS Block extracts multi-level spatial and semantic features. The max pooling layer reduces the size of the feature map, thereby reducing the parameters and computational load of subsequent layers and accelerating training.
[0037] Step 1.3: Construct the downsampling module 3 of the coding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0038] The downsampling module 3 includes a first convolutional layer, a visual state space module (VSS Block), and a max pooling layer. The first convolutional layer is used to adjust the image dimensions to facilitate subsequent calculations. The VSS Block extracts multi-level spatial and semantic features. The max pooling layer reduces the size of the feature map, thereby reducing the parameters and computational load of subsequent layers and accelerating training.
[0039] Step 1.4: Construct the downsampling module 4 of the coding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0040] The downsampling module 4 includes a first convolutional layer, a frequency domain enhancement module FreqEM, and a max pooling layer. The first convolutional layer is used to adjust the image dimensions to facilitate subsequent calculations. FreqEM is used to improve the expressive power of frequency domain features and extract deep semantic features of landslide geological hazards. The max pooling layer reduces the size of the feature map, thereby reducing the parameters and computational cost of subsequent layers and accelerating training.
[0041] The specific process of the FreqEM module structure is as follows: the feature map X output by the downsampling 3 module is processed by a linear fully connected layer to obtain the normalized feature X. ln The formula is expressed as:
[0042] ;
[0043] X ln The input is fed into the Visual State Space Block (VSS Block), which extracts multi-level spatial and semantic features to obtain a new feature representation X. ssd ; X ssd The feature is added to the original input feature X to achieve feature fusion. The specific formula is as follows:
[0044] ;
[0045] X add Perform a linear fully connected operation to stabilize the feature distribution and generate X. ln2 The formula is expressed as:
[0046] ;
[0047] The feature map X after linear full connection ln2 The input is fed into the deformable convolutional DConv module to obtain the enhanced feature map X. dconv ; X dconv With X add Adding them together yields the feature map X after deep feature enhancement. out The formula is expressed as:
[0048] ;
[0049] Step 1.5: Construct the downsampling module 5 of the coding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0050] The downsampling module 5 includes a first convolutional layer, a frequency domain enhancement module FreqEM, and a max pooling layer. The first convolutional layer is used to adjust the image dimensions to facilitate subsequent calculations. FreqEM is used to improve the frequency domain feature representation capability and extract deep semantic features of landslide geological hazards. The max pooling layer reduces the feature map size, thereby reducing the parameters and computational load of subsequent layers and accelerating training.
[0051] The specific method for step 2 is as follows:
[0052] A decoding subnetwork for landslide identification based on visual state space and frequency domain enhancement of remote sensing images is constructed, and its structure is as follows: upsampling 1, upsampling 2, upsampling 3, upsampling 4 and upsampling 5;
[0053] Step 2.1: Construct the upsampling module 1 of the decoding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0054] The upsampling module 1 includes a GSU Block and a second convolution. The GSU Block is used to enhance the semantic features of landslide geological hazards extracted by the downsampling module 5. It extracts local and global contextual features through dual-path convolution and restores the size of the feature map.
[0055] The GSU Block module is used to process the input feature map. The first path uses 3×3 grouped spatial convolution GSConv to extract local contextual features:
[0056] ;
[0057] The second path uses 1×1 GSConv to capture global feature associations:
[0058] ;
[0059] Where W3 and W1 are the convolution kernel weight parameters, and the outputs of the two branches are processed by batch normalization and ReLU6 activation function:
[0060] ;
[0061] ;
[0062] The output feature maps of the two paths are added together to obtain the fused multi-scale feature F. f :
[0063] ;
[0064] Multiscale feature map F f The input is fed into the squeeze-excitation attention module (SE), where channel weights are generated through global average pooling and fully connected layers, and the feature maps are weighted accordingly.
[0065] ;
[0066] ;
[0067] ;
[0068] in, Let r be the dimensionality reduction / incrementing matrix, and r be the compression ratio. The weighted feature representation is as follows:
[0069] ;
[0070] To prevent overfitting, a Dropout layer is used to remove some redundant features:
[0071] ;
[0072] The grouped spatial convolution GSConv divides the input feature map into several groups, and performs convolution operations on each group separately, reducing the amount of computation while performing spatial convolution to achieve the fusion of cross-group features.
[0073] The input is a feature map with C channels. The first convolutional unit is a grouped convolutional layer, whose input is the input feature map and whose output channel number is C / 2. The second convolutional unit is a pointwise convolutional layer, whose input is also the input feature map and whose output channel number is C / 2. The outputs of the first and second convolutional units are concatenated along the channel dimension to form a concatenated feature map with C channels. Channel shuffling is then performed on the concatenated feature map to generate an output feature map.
[0074] Step 2.2: Construct the upsampling module 2 of the decoding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0075] The upsampling module 2 includes a skip link module, a GSU block, and a second convolution. The GSU block is used to enhance the semantic features of landslide geological hazards in the upsampling module 1 by extracting local and global contextual features through dual-path convolution and restoring the size of the feature map. The skip link module is used to add the feature map output by the upsampling module 1 and the feature map output by the downsampling module 4 to obtain a fused feature map. The second convolution is used to restore the size of the feature map for subsequent upsampling.
[0076] Step 2.3: Construct the upsampling module 3 of the decoding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0077] The upsampling module 3 includes a skip link module, a GSU block, and a second convolution. The GSU block is used to enhance the semantic features of landslide geological hazards in the upsampling module 2 by extracting local and global contextual features through dual-path convolution and restoring the size of the feature map. The skip link module is used to add the feature map output by the upsampling module 2 and the feature map output by the downsampling module 3 to obtain a fused feature map. The second convolution is used to restore the size of the feature map for subsequent upsampling.
[0078] Step 2.4: Construct the upsampling module 4 of the decoding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0079] The upsampling module 4 includes a skip link module, a GSU block, and a second convolution. The GSU block enhances the semantic features of landslide geological hazards in the upsampling module 3 by extracting local and global contextual features through dual-path convolution and restoring the size of the feature map. The skip link module adds the feature map output by the upsampling module 3 and the feature map output by the downsampling module 2 to obtain a fused feature map. The second convolution restores the size of the feature map for subsequent upsampling.
[0080] Step 2.5: Construct the upsampling module 5 of the decoding subnetwork for a landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0081] The upsampling module 5 includes a GSU Block and a skip link module. The GSU Block is used to enhance the semantic features of landslide geological hazards in the upsampling module 2 by extracting local and global context features through dual-path convolution and restoring the size of the feature map. The skip link module is used to add the feature map output by the upsampling module 2 and the feature map output by the downsampling module 1 to obtain the fused feature map.
[0082] The specific method for step 3 is as follows:
[0083] The recognition subnetwork consists of a convolutional layer used to restore the feature maps of the decoding subnetwork to the original remote sensing image size.
[0084] Step 4: Based on the encoding subnetwork constructed in Step 1, the decoding subnetwork constructed in Step 2, and the recognition subnetwork constructed in Step 3, construct a landslide recognition method network based on visual state space and frequency domain enhancement.
[0085] The specific method for step 4 is as follows:
[0086] Step 4.1: The feature map of the landslide geological hazard range extracted by the encoder in Step 1 and the feature map extracted by the decoder in Step 2 are fused and concatenated by upsampling;
[0087] Step 4.2: Connect the landslide geological hazard feature map extracted from the sampling block in Step 2 with the identification sub-network constructed in Step 3.
[0088] The specific method for step 5 is as follows:
[0089] Step 5.1: Obtain the publicly available remote sensing image dataset for landslide geological hazards, Bijie Landslide.
[0090] Step 5.2: Randomly perform five data augmentation operations on each image in the image dataset from Step 5.1, including histogram equalization, geometric flipping, and contrast enhancement, and divide them into training set, validation set, and test set according to the proportion.
[0091] The specific method for step 6 is as follows:
[0092] Step 6.1: Set training parameters. Each time, randomly and non-repeatedly select remote sensing images from the training set and input them into the network. Use Dice Loss as the loss function.
[0093] The Dice Loss is as follows:
[0094] ;
[0095] Where, p i To predict the probability, g i The label is a real label, and ϵ is used to prevent division by zero;
[0096] Step 6.2: Input the training set and validation set generated in Step 5 into the landslide recognition method network based on visual state space and frequency domain enhancement constructed in Step 4. The specific steps are as follows: input the landslide geological disaster remote sensing image into the coding sub-network to extract landslide geological disaster features; input the feature map output by the feature extraction sub-network into the decoding sub-network to obtain the landslide geological disaster feature map after feature fusion; finally, use the recognition sub-network to restore the feature map after feature fusion to the original image size.
[0097] The specific method for step 7 is as follows:
[0098] Step 7.1: Input the test set generated in Step 5 into the remote sensing image landslide recognition method network trained in Step 6 based on visual state space and frequency domain enhancement, and load the parameter file "best_weight.pth" of the remote sensing image landslide recognition method network trained in Step 6.1 based on visual state space and frequency domain enhancement.
[0099] Step 7.2: Apply a landslide identification method based on visual state space and frequency domain enhancement to the remote sensing images in the test set to obtain the extent of the landslide in the remote sensing images.
[0100] Step 7.3: Output the results of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images and save it as a label file in ".png" format.
[0101] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0102] (1) Traditional landslide identification methods mostly rely on single spatial domain features or static convolution operations, which are difficult to effectively capture high-frequency boundaries and small landslide areas in images. This invention introduces FreqEM to enhance high-frequency features in images from the frequency domain perspective, which significantly improves the identification accuracy of landslide edges and irregular areas, and is especially suitable for complex terrain scenes with blurred boundaries and fine textures.
[0103] (2) Compared with conventional convolutional structures, the VSS Block proposed in this invention, combined with the SS2D selective scanning mechanism, can construct long-distance dependencies across multiple directions, effectively capturing the spatial context information of landslide areas in large-scale remote sensing images, thereby improving the completeness and robustness of recognition;
[0104] (3) The GSConv decoder module of the present invention integrates multi-scale convolution and attention mechanisms, which not only retains the ability to restore image details, but also keeps the computational overhead light, effectively solving the computational resource problem of traditional deep networks in high-resolution remote sensing image processing;
[0105] In summary, compared with the best existing technologies for landslide geological hazard identification on remote sensing images, this invention innovatively introduces frequency domain enhancement and spatial state modeling mechanisms to construct a landslide geological hazard identification network that has both high-precision identification capabilities and computational efficiency. Compared with existing technologies, it has stronger boundary awareness, contextual understanding, and practicality, providing more reliable technical support for intelligent landslide disaster response and emergency response. Attached Figure Description
[0106] Figure 1 This is a schematic diagram of the method flow in a specific embodiment of the present invention;
[0107] Figure 2 This is a schematic diagram of the remote sensing image landslide identification network model structure based on visual state space and frequency domain enhancement in a specific embodiment of the present invention.
[0108] Figure 3 This is a schematic diagram of the Visual State Space Module (VSS Block) structure in a specific embodiment of the present invention;
[0109] Figure 4 This is a schematic diagram of the deformable convolution DConv structure in a specific embodiment of the present invention;
[0110] Figure 5 This is a schematic diagram of the FreqEM frequency domain enhancement module structure in a specific embodiment of the present invention;
[0111] Figure 6 This is a schematic diagram of the grouped spatial upsampling GSU structure in a specific embodiment of the present invention;
[0112] Figure 7 This is a schematic diagram of the grouped spatial convolution GSConv structure in a specific embodiment of the present invention;
[0113] Figure 8 This is a schematic diagram comparing landslide identification results in a specific embodiment of the present invention, wherein, Figure 8 (a) is the original remote sensing image. Figure 8 (b) is the original tagged image. Figure 8 (c) is the identification result of the method of the present invention. Detailed Implementation
[0114] The technical solution and effects of the present invention will be further described below with reference to the accompanying drawings.
[0115] Reference Figure 1 The implementation steps of the embodiments of the present invention will be further described below.
[0116] The remote sensing image building change detection network constructed in this invention comprises four main parts: an encoding subnetwork, a decoding subnetwork, and a recognition subnetwork; (Refer to...) Figure 2 The encoding subnetwork is responsible for extracting the features of landslide geological hazards from remote sensing images; the decoding subnetwork is responsible for feature fusion and image size restoration, restoring the feature map of landslide geological hazards to its original resolution size; the skip link module is responsible for fusing the landslide feature map extracted by each downsampling module of the encoding subnetwork with the feature map obtained by upsampling in the decoding subnetwork, and the feature map obtained by the last downsampling in the encoding subnetwork is fused with the feature map after upsampling and feature enhancement; the recognition subnetwork is responsible for restoring the feature map to the size of the original image.
[0117] Step 1: Construct a coding subnetwork for landslide geological hazards from remote sensing images and extract landslide features from the input remote sensing images;
[0118] Construct an encoding subnetwork with 5 downsampling modules, whose structure is as follows: downsampling 1, downsampling 2, downsampling 3, downsampling 4 and downsampling 5 modules;
[0119] The structures of downsampling 2 to 5 are similar to those of downsampling 1.
[0120] Step 1.1: Construct the downsampling module 1 of the coding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0121] Reference Figure 2 The downsampling module 1 in the coding sub-network structure of the landslide identification method based on visual state space and frequency domain enhancement constructed in this invention will be further described.
[0122] like Figure 2 As shown, the downsampling module 1 includes a first convolutional layer, a visual state space module (VSS Block), and a max pooling layer. The first convolutional layer adjusts the image dimensions for easier subsequent computation, with an input feature map size of 256×256×3 and an output feature map size of 256×256×64. The VSS Block extracts multi-level spatial and semantic features from the feature map output by the first convolutional layer, with an input feature map size of 256×256×64 and an output feature map size of 256×256×64. The max pooling layer reduces the size of the output feature map from the VSS Block, thereby reducing the parameters and computational cost of subsequent layers and accelerating training, with an input feature map size of 256×256×64 and an output feature map size of 128×128×64.
[0123] Reference Figure 3 The VSS Block module in the coding sub-network for landslide geological hazards in remote sensing images constructed in this invention will be further described.
[0124] The Visual State Space Block (VSS Block) consists of two parallel branches, namely X. b and X t ;
[0125] In branch X t In this process, feature X is first processed through a fully connected linear layer, then the feature map is processed using deformable convolution (DConv) and SiLU activation functions; long-range spatial dependency information is extracted through the 2D selection module SS2D; and the feature map is processed using a normalization layer to generate X. t The entire process can be calculated as follows:
[0126] ;
[0127] ;
[0128] In branch X b In this process, the feature map X is first transformed into tensor space through a fully connected linear layer; the output of the fully connected layer is then processed using the SiLU activation function to obtain the feature map X.b This process can be calculated as follows:
[0129] ;
[0130] exist and Element-wise multiplication is performed to fuse the features of the two branches; a fully connected layer is used to process the fused feature map and output feature X'.
[0131] Reference Figure 4 The deformable convolution DConv in the VSS Block module of the remote sensing image landslide geological hazard coding sub-network constructed in this invention will be further described.
[0132] Deformable Convolution (DConv) dynamically generates convolution kernel offsets from the input feature map, performing irregular sampling on the feature map to achieve adaptive feature extraction. It calculates an offset field based on the input feature map, which is a tensor with the same spatial size as the input feature map, representing the number of neighbors for each pixel in space. The offset at each position guides subsequent feature sampling. Irregular sampling is performed on the input feature map; based on the corresponding offset, neighboring pixels are found and sampled in the input feature map, and feature values at that position are calculated using interpolation and other methods. The features after offset sampling are fused through aggregation operations to obtain the output feature map.
[0133] The SS2D module in the Visual State Space (VSS) Block is used for feature information capture. Its structure includes: Cross-Scan, Selective Scanning with S6 Blocks, and Cross-Merge. Cross-Scan divides the input image into non-overlapping patches, then expands them into a sequence along four directions (top left to bottom right, bottom right to top left, top right to bottom left, bottom left to top right). Selective Scanning with S6 Blocks... Blocks: Sequences in each direction are processed by independent blocks, extracting features and retaining relevant information while filtering out irrelevant information; Cross-Merge: The processed sequences are recombined to restore a two-dimensional feature map, integrating multi-directional contextual information to achieve a global receptive field; The input image is traversed along four paths in different directions and expanded into a sequence form. The directions of these four paths are: from the top left to the bottom right, from the bottom right to the top left, from the top right to the bottom left, and from the bottom left to the top right; The sequences obtained in each direction are processed in parallel for feature extraction; The feature sequences obtained from the four different directions are expanded, summarized, and merged to obtain a feature map of the same size as the input feature map.
[0134] Step 1.2: Construct the downsampling 2 module of the coding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0135] The downsampling module 2 includes a first convolutional layer, a visual state space module (VSS Block), and a max pooling layer. The first convolutional layer is used to adjust the image dimensions for easier subsequent computation, with an input feature map size of 128×128×64 and an output feature map size of 128×128×128. The VSS Block extracts multi-level spatial and semantic features from the feature map output by the first convolutional layer, with an input feature map size of 128×128×128 and an output feature map size of 128×128×128. The max pooling layer reduces the size of the output feature map of the VSS Block, thereby reducing the parameters and computational cost of subsequent layers and accelerating training, with an input feature map size of 128×128×128 and an output feature map size of 64×64×128.
[0136] Step 1.3: Construct the downsampling module 3 of the coding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0137] The downsampling module 3 includes a first convolutional layer, a visual state space (VSS) block, and a max-pooling layer. The first convolutional layer adjusts the image dimensions to facilitate subsequent computation, with an input feature map size of 64×64×128 and an output feature map size of 64×64×256. The VSS block extracts multi-level spatial and semantic features from the feature map output by the first convolutional layer, with an input feature map size of 64×64×256 and an output feature map size of 64×64×256. The max-pooling layer reduces the size of the output feature map from the VSS block, thereby reducing the parameters and computational cost of subsequent layers and accelerating training, with an input feature map size of 64×64×256 and an output feature map size of 32×32×256.
[0138] Step 1.4: Construct the downsampling module 4 of the coding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0139] The downsampling module 4 includes a first convolutional layer, a frequency domain enhancement module FreqEM, and a max pooling layer. The first convolutional layer is used to adjust the image dimensions to facilitate subsequent calculations, with an input feature map size of 32×32×256 and an output feature map size of 32×32×512. FreqEM is used to enhance the expressive power of frequency domain features and extract deep semantic features of landslide geological hazards, with an input feature map size of 32×32×512 and an output feature map size of 32×32×512. The max pooling layer reduces the feature map size, thereby reducing the parameters and computational cost of subsequent layers and accelerating training, with an input feature map size of 32×32×512 and an output feature map size of 16×16×512.
[0140] Reference Figure 5 The feature pyramid attention module in the feature fusion subnetwork for building changes in remote sensing images constructed in this invention will be further described.
[0141] The FreqEM module performs a linear fully connected layer on the feature map X output by the downsampling module 3 to obtain the normalized feature X. ln The formula is expressed as:
[0142] ;
[0143] X ln The input is fed into the Visual State Space Block (VSS Block), which extracts multi-level spatial and semantic features to obtain a new feature representation X. ssd ; X ssd The feature is added to the original input feature X to achieve feature fusion. The specific formula is as follows:
[0144] ;
[0145] X add Perform a linear fully connected operation to stabilize the feature distribution and generate X. ln2 The formula is expressed as:
[0146] ;
[0147] The feature map X after linear full connection ln2 The input is fed into the deformable convolutional DConv module to obtain the enhanced feature map X. dconv ; X dconv With X add Adding them together yields the feature map X after deep feature enhancement. out The formula is expressed as:
[0148] ;
[0149] Step 1.5: Construct the downsampling module 5 of the coding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0150] The downsampling module 5 includes a first convolutional layer and a frequency domain enhancement module FreqEM. The first convolutional layer is used to adjust the image dimensions to facilitate subsequent calculations, with an input feature map size of 16×16×512 and an output feature map size of 16×16×1024. FreqEM is used to enhance the expressive power of frequency domain features and extract deep semantic features of landslide geological hazards, with an input feature map size of 16×16×1024 and an output feature map size of 16×16×1024.
[0151] Step 2: Construct a decoding sub-network for a landslide identification method based on visual state space and frequency domain enhancement of remote sensing images, and fuse the feature map output by the downsampling module with the feature map output by the upsampling module;
[0152] Reference Figure 2 The decoding sub-network structure is as follows: upsampling 1, upsampling 2, upsampling 3, upsampling 4 and upsampling 5;
[0153] Step 2.1: Construct the upsampling module 1 of the decoding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0154] The upsampling module 1 consists of a grouped spatial upsampling GSU Block and a second convolutional layer. The GSU Block enhances the semantic features of landslide geological hazards extracted by the downsampling module 5. It extracts local and global contextual features through dual-path convolution and restores the size of the feature map. The input feature map size is 16×16×1024, the transposed convolution has a stride of 2, and the output feature map size is 32×32×512. The second convolutional layer restores the feature map size for subsequent calculations. The input feature map size is 32×32×512, and the output feature map size is 32×32×512.
[0155] Reference Figure 6 The present invention further describes the Group Spatial Upsampling Module (GSU Block) in the decoding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement constructed in this invention.
[0156] GSU Block is the input feature map The first path uses 3×3 grouped spatial convolution GSConv to extract local contextual features:
[0157] ;
[0158] The second path uses 1×1 GSConv to capture global feature associations:
[0159] ;
[0160] Where W3 and W1 are the convolution kernel weight parameters, and the outputs of the two branches are processed by batch normalization and ReLU6 activation function:
[0161] ;
[0162] ;
[0163] The output feature maps of the two paths are added together to obtain the fused multi-scale feature F. f :
[0164] ;
[0165] Multiscale feature map F f The input is fed into the squeeze-excitation attention module (SE), where channel weights are generated through global average pooling and fully connected layers, and the feature maps are weighted accordingly.
[0166] ;
[0167] ;
[0168] ;
[0169] in, Let r be the dimensionality reduction / incrementing matrix, and r be the compression ratio. The weighted feature representation is as follows:
[0170] ;
[0171] To prevent overfitting, a Dropout layer is used to remove some redundant features:
[0172] ;
[0173] Reference Figure 7 The paper further describes the group spatial convolution module in the group spatial upsampling module GSU Block of the decoding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement constructed in this invention.
[0174] Grouped Spatial Convolution (GSConv) divides the input feature map into several groups, performs convolution operations on each group separately, reduces the amount of computation, and performs spatial convolution to achieve the fusion of cross-group features.
[0175] The input is a feature map with C channels. The first convolutional unit is a grouped convolutional layer, whose input is the input feature map and whose output channel number is C / 2. The second convolutional unit is a pointwise convolutional layer, whose input is also the input feature map and whose output channel number is C / 2. The outputs of the first and second convolutional units are concatenated along the channel dimension to form a concatenated feature map with C channels. Channel shuffling is then performed on the concatenated feature map to generate an output feature map.
[0176] Step 2.2: Construct the upsampling module 2 of the decoding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0177] The upsampling module 2 consists of a skip link module, a grouped spatial upsampling GSU Block, and a second convolutional layer. The skip link module fuses the semantic features of landslide geological hazards extracted by the downsampling module 5 with the feature map output by upsampling module 1. The semantic features of landslide geological hazards extracted by the downsampling module 5 are 32×32×512 in size, the feature map output by upsampling module 1 is 32×32×512 in size, and the feature map output by the skip link is 32×32×1024 in size. The GSU Block is used to enhance the feature map after skip linking. It extracts local and global context features through dual-path convolution and restores the size of the feature map. The input feature map size is 32×32×1024, the transposed convolution has a stride of 2, and the output feature map size is 32×32×512 in size. The second convolutional layer is used to restore the size of the feature map for subsequent calculations. The input feature map size is 32×32×512 in size, and the output feature map size is 64×64×256 in size.
[0178] Step 2.3: Construct the upsampling module 3 of the decoding subnetwork of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0179] The upsampling module 3 consists of a skip link module, a grouped spatial upsampling GSU Block, and a second convolutional layer. The skip link module fuses the semantic features of landslide geological hazards extracted by the downsampling module 4 with the feature map output by upsampling module 1. The semantic features of landslide geological hazards extracted by the downsampling module 4 are 64×64×256 in size, the feature map output by upsampling module 2 is 64×64×256 in size, and the feature map output by the skip link is 64×64×512 in size. The GSU Block is used to enhance the feature map after skip linking. It extracts local and global context features through dual-path convolution and restores the size of the feature map. The input feature map size is 64×64×512, the transposed convolution has a stride of 2, and the output feature map size is 64×64×256 in size. The second convolutional layer is used to restore the size of the feature map for subsequent calculations. The input feature map size is 64×64×256 in size, and the output feature map size is 128×128×128 in size.
[0180] Step 2.4: Construct the upsampling module 4 of the decoding subnetwork for the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0181] The upsampling module 4 consists of a skip link module, a grouped spatial upsampling GSU Block, and a second convolutional layer. The skip link module fuses the semantic features of landslide geological hazards extracted by the downsampling module 3 with the feature map output by the upsampling module 1. The semantic features of landslide geological hazards extracted by the downsampling module 3 are 128×128×128 in size, the feature map output by the upsampling module 3 is 128×128×128 in size, and the feature map output by the skip link is 128×128×256 in size. The GSU Block is used to enhance the feature map after the skip link. It extracts local and global context features through dual-path convolution and restores the size of the feature map. The input feature map size is 128×128×256, the transposed convolution has a stride of 2, and the output feature map size is 128×128×128 in size. The second convolutional layer is used to restore the size of the feature map for subsequent calculations. The input feature map size is 128×128×128 in size, and the output feature map size is 256×256×64 in size.
[0182] Step 2.5: Construct the upsampling module 5 of the decoding subnetwork for a landslide identification method based on visual state space and frequency domain enhancement of remote sensing images;
[0183] The upsampling module 5 consists of a skip link module, a grouped spatial upsampling GSU Block, and a second convolutional layer. The skip link module fuses the semantic features of landslide geological hazards extracted by the downsampling module 3 with the feature map output by the upsampling module 1. The semantic features of landslide geological hazards extracted by the downsampling module 3 are 256×256×64 in size, the feature map output by the upsampling module 3 is 256×256×64 in size, and the feature map output by the skip link is 256×256×128 in size. The GSU Block is used to enhance the feature map after the skip link. It extracts local and global context features through dual-path convolution and restores the size of the feature map. The input feature map size is 256×256×128, the transposed convolution is performed, the stride is 2, and the output feature map size is 256×256×64 in size. The second convolutional layer is used to restore the size of the feature map for subsequent calculations. The input feature map size is 256×256×64 in size, and the output feature map size is 256×256×64 in size.
[0184] Step 3: Construct a recognition sub-network for landslide recognition based on visual state space and frequency domain enhancement of remote sensing images, restore the landslide geological hazard feature map obtained in Step 2 to the size of the original remote sensing image, and obtain the final landslide geological hazard area range.
[0185] The recognition sub-network consists of a third convolutional layer, with an input feature map size of 256×256×64, a convolutional kernel size of 1, and an output size of 256×256×1.
[0186] Step 4: Based on the encoding subnetwork constructed in Step 1, the decoding subnetwork constructed in Step 2, and the recognition subnetwork constructed in Step 3, construct a landslide recognition method network based on visual state space and frequency domain enhancement.
[0187] Step 4.1: The feature map of the landslide geological hazard range extracted by the encoder in Step 1 and the feature map extracted by the decoder in Step 2 are fused and concatenated by upsampling;
[0188] Step 4.2: Connect the landslide geological hazard feature map extracted from the sampling block in Step 2 with the identification sub-network constructed in Step 3.
[0189] Step 5: Generate the training set, validation set, and test set;
[0190] Step 5.1: Obtain the Bijie Landslide remote sensing image dataset, which contains 973 images of landslide geological hazards, with an image size of 256×256.
[0191] Step 5.2: Randomly perform five data augmentation operations on each image in the image dataset from Step 5.1, including histogram equalization, geometric flip, and contrast enhancement, resulting in a total of 4865 images after augmentation.
[0192] Step 5.3: Divide the dataset into training set, validation set, and test set in a ratio of 7:1:2, which includes 3405 training images, 487 validation images, and 973 test images.
[0193] Step 6: Use the training set and validation set generated in Step 5 to train the landslide identification method network based on visual state space and frequency domain enhancement constructed in Step 4, and obtain the model weight file.
[0194] Step 6.1: Set training parameters. Each time, randomly and non-repeatedly select remote sensing images from the training set and input them into the network. Use Dice Loss as the loss function.
[0195] The Dice Loss is as follows:
[0196] ;
[0197] Where, p i To predict the probability, g i For true labels, ϵ is used to prevent division by zero.
[0198] Step 6.2: Input the training set and validation set generated in Step 5 into the landslide recognition method network based on visual state space and frequency domain enhancement constructed in Step 4. The specific steps are as follows: input the landslide geological disaster remote sensing image into the coding sub-network to extract landslide geological disaster features; input the feature map output by the feature extraction sub-network into the decoding sub-network to obtain the landslide geological disaster feature map after feature fusion; finally, use the recognition sub-network to restore the feature map after feature fusion to the original image size.
[0199] Step 7: Using the model weight file obtained from step 6, detect the extent of landslide geological hazards in the remote sensing images of the test set generated in step 5.
[0200] Step 7.1: Input the test set generated in Step 5 into the remote sensing image landslide recognition method network trained in Step 6 based on visual state space and frequency domain enhancement, and load the parameter file "best_weight.pth" of the remote sensing image landslide recognition method network trained in Step 6.1 based on visual state space and frequency domain enhancement.
[0201] Step 7.2: Apply a landslide identification method based on visual state space and frequency domain enhancement to the remote sensing images in the test set to obtain the extent of the landslide in the remote sensing images.
[0202] Step 7.3: Output the results of the landslide identification method based on visual state space and frequency domain enhancement of remote sensing images and save it as a label file in ".png" format.
[0203] Reference Figure 8 The recognition results of the landslide recognition method network based on visual state space and frequency domain enhancement constructed in this invention are further described.
[0204] like Figure 8 As shown, where, Figure 8 (a) is the original remote sensing image. Figure 8 (b) is the tagged image. Figure 8 (c) shows the landslide identification results. It can be seen that, due to the addition of the frequency domain enhancement module, the model pays more attention to the feature details of landslides at different scales, and smaller landslides are effectively identified. After the GSConv module, the network's ability to understand landslide features is enhanced, and the ability to identify landslides in complex terrain is improved. It has the technical effect of highlighting details and accurate identification.
[0205] Application prospects of this invention
[0206] The landslide geological hazard identification method on remote sensing images proposed in this invention has significant advantages in identification accuracy, model robustness and computational efficiency, and has broad application prospects and promotion value;
[0207] First, in the field of geological disaster monitoring and emergency response, this method can quickly and automatically identify landslide areas from large-scale remote sensing images, replacing the traditional method that relies on manual interpretation. This significantly improves response speed and the efficiency of disaster information acquisition, and is suitable for rapid disaster assessment and rescue deployment after natural disasters such as earthquakes and rainstorms. Second, in geological risk early warning and land spatial planning, the high-precision landslide identification model constructed by this invention can be integrated with data such as digital elevation models and soil moisture monitoring to assist in the zoning of landslide-prone areas and the investigation of geological hazards. This provides scientific decision-making basis for governments and relevant departments, and enhances the ability of land resource management and disaster prevention and control. In addition, this method has both lightweight and high-resolution processing capabilities, and can be applied to landslide monitoring tasks using UAV remote sensing images, satellite remote sensing data, and multi-source remote sensing information. It has good platform adaptability, making it easy to embed into actual monitoring systems and realize an intelligent and automated geological disaster identification process.
[0208] In summary, this invention has significant practical implications and broad application prospects in multiple fields such as geological disaster monitoring, emergency management, urban planning, and ecological protection, and is expected to become one of the core technological supports in the field of landslide identification and early warning.
Claims
1. A landslide identification method based on visual state space and frequency domain enhancement for remote sensing images, characterized in that, The application relates to a landslide identification method based on a visual state space and a frequency domain enhancement coding subnetwork. The application relates to a landslide identification method based on a visual state space and a frequency domain enhancement coding subnetwork. Step 1, a coding subnetwork of a landslide identification method based on a visual state space and a frequency domain enhancement coding subnetwork is built, and the structure is: down-sampling 1, down-sampling 2, down-sampling 3, down-sampling 4 and down-sampling 5. Step 1.1, a down-sampling 1 module of the coding subnetwork of the landslide identification method based on the visual state space and the frequency domain enhancement coding subnetwork; The down-sampling 1 module comprises a first convolutional layer, a visual state space module VSS Block and a maximum pooling layer; wherein the first convolutional layer is used for adjusting the image dimension, facilitating subsequent calculation; the VSS Block extracts multi-level spatial and semantic features; and the maximum pooling layer reduces the parameter and calculation amount of a subsequent layer by reducing the feature map size, and accelerates training. The visual state space module VSS Block consists of two parallel branches, namely X b and X t ; In branch X t In feature X, the feature map will first be processed by a linear fully connected layer, and then processed by deformable convolution (DConv) and SiLU activation function; long-distance spatial dependency information is extracted by two-dimensional selection module SS2D; the feature map is processed by a normalization layer to generate X t ′, the whole process can be calculated as follows: ; ; The deformable convolution DConv dynamically generates a convolution kernel offset of an input feature map, irregularly samples the feature map, realizes adaptive feature extraction, calculates an offset field according to the input feature map, the offset field is a tensor with the same spatial size as the input feature map, is the number of neighbors of each pixel point in space, and the offset of each position guides subsequent feature sampling; the input feature map is irregularly sampled according to the corresponding offset, neighboring pixels in the input feature map are found for sampling, and the feature value of the position is calculated through interpolation and other methods; the feature after offset sampling is fused through an aggregation operation to obtain an output feature map. In branch X b , the feature X is first converted to the tensor space by a linear fully connected layer Linear; the SiLU activation function is used to process the output of the fully connected layer to obtain the feature map X b ′, which can be calculated as follows: ; In and Element multiplication operation is performed on the two branches to realize feature fusion; a fully connected layer is used to process the fused feature map to output feature X'; The SS2D module in the visual state space module VSS Block is used for feature information capture, and the structure comprises: cross-scan Cross-Scan, selective scanning Selective Scanning with S6 Blocks and cross-merge Cross-Merge; wherein the cross-scan Cross-Scan divides the input image into non-overlapping patches, and then expands into a sequence along four directions (from the upper left to the lower right, from the lower right to the upper left, from the upper right to the lower left and from the lower left to the upper right); the selective scanning Selective Scanning with S6 Blocks: the sequence of each direction is processed by independent blocks, features are extracted and relevant information is retained, and irrelevant information is filtered; the cross-merge Cross-Merge: the processed sequences are recombined to recover into a two-dimensional feature map, the multi-direction context information is integrated, and a global receptive field is realized; the input image is traversed along four paths in different directions, and is expanded into a sequence form; the directions of the four paths are: from the upper left to the lower right, from the lower right to the upper left, from the upper right to the lower left and from the lower left to the upper right; the sequences obtained by parallel processing of each direction are subjected to feature extraction; the feature sequences obtained from the four different directions are expanded, summarized and merged to obtain a feature map with the same size as the input feature map; Step 1.2, a down-sampling 2 module of the coding subnetwork of the landslide identification method based on the visual state space and the frequency domain enhancement coding subnetwork; The downsampling 2 module comprises a first convolutional layer, a visual state space module VSS Block and a maximum pooling layer; wherein the first convolutional layer is used to adjust the image dimension, facilitating subsequent calculation; the VSS Block extracts multi-level spatial and semantic features; and the maximum pooling layer reduces the size of the feature map, thereby reducing the parameters and calculation amount of the subsequent layer and accelerating the training; Step 1.3, a downsampling 3 module of the encoding subnetwork of the landslide identification method based on the visual state space and the frequency domain enhancement; The downsampling 3 module comprises a first convolutional layer, a visual state space module VSS Block and a maximum pooling layer; wherein the first convolutional layer is used to adjust the image dimension, facilitating subsequent calculation; the VSS Block extracts multi-level spatial and semantic features; and the maximum pooling layer reduces the size of the feature map, thereby reducing the parameters and calculation amount of the subsequent layer and accelerating the training; Step 1.4, a downsampling 4 module of the encoding subnetwork of the landslide identification method based on the visual state space and the frequency domain enhancement; The downsampling 4 module comprises a first convolutional layer, a frequency domain enhancement module FreqEM and a maximum pooling layer; wherein the first convolutional layer is used to adjust the image dimension, facilitating subsequent calculation; the FreqEM is used to improve the expression ability of the frequency domain feature and extract deep semantic features of the landslide geological disaster; and the maximum pooling layer reduces the size of the feature map, thereby reducing the parameters and calculation amount of the subsequent layer and accelerating the training; The specific process of the FreqEM module structure is: performing a linear fully connected layer on the feature map X output by the down-sampling 3 module to obtain normalized features X ln , which is expressed by the formula: ; X ln is input to the visual state space module VSS Block, which extracts multi-level spatial and semantic features to obtain a new feature representation X ssd ; X ssd is added to the original input feature X to realize feature fusion, and the specific formula is: ; X add Perform linear fully connected operation, stabilize feature distribution, generate X ln2 , formula is expressed as: ; The linear fully connected feature map X ln2 is input into a deformable convolution DConv module to obtain an enhanced feature map X dconv ; X dconv is added to X add to obtain a feature map X out enhanced by deep layer features, which is expressed by the following formula: ; Step 1.5, a downsampling 5 module of the encoding subnetwork of the landslide identification method based on the visual state space and the frequency domain enhancement; The downsampling 5 module comprises a first convolutional layer, a frequency domain enhancement module FreqEM and a maximum pooling layer; wherein the first convolutional layer is used to adjust the image dimension, facilitating subsequent calculation; the FreqEM is used to improve the expression ability of the frequency domain feature and extract deep semantic features of the landslide geological disaster; and the maximum pooling layer reduces the size of the feature map, thereby reducing the parameters and calculation amount of the subsequent layer and accelerating the training; A decoding subnetwork of the landslide identification method based on the visual state space and the frequency domain enhancement is constructed, and the feature map is restored to the original feature map size, comprising: The decoding subnetwork of the landslide identification method based on the visual state space and the frequency domain enhancement is built, and the structure is in turn: upsampling 1, upsampling 2, upsampling 3, upsampling 4 and upsampling 5; Step 2.1, an upsampling 1 module of the decoding subnetwork of the landslide identification method based on the visual state space and the frequency domain enhancement; The upsampling 1 module comprises a GSU Block and a second convolution; the GSU Block is used to enhance the semantic features of the landslide geological disaster extracted by the downsampling 5 module, to extract local and global context features through double-path convolution, and to restore the size of the feature map; The GSU Block module is to input a feature map The first path adopts a 3x3 grouped spatial convolution (GSConv) kernel to extract local context features: ; The second path uses a 1x1 GSConv to capture global feature correlation: ; Wherein, W3 and W1 are convolution kernel weight parameters, and the outputs of the two branches are processed through batch normalization and ReLU6 activation function: ; ; The output feature maps of the two paths are added to obtain the fused multi-scale feature F f : ; The multi-scale feature map F f is input to the squeeze-and-excitation attention module SE, and a channel weight is generated through global average pooling and a fully connected layer to weight the feature map: ; ; ; wherein, is a dimensionality reduction / increase matrix, r is the compression ratio, and the weighted feature representation is ; In order to prevent overfitting, a Dropout layer is used to delete part of the redundant features: ; The group spatial convolution GSConv is to divide an input feature map into groups, and each group is subjected to convolution operation, so as to reduce the calculation amount and realize spatial convolution and fusion of features across groups; The input feature map has C channels, the first convolution unit is a group convolution layer, the input of which is the input feature map, and the output channel number is C / 2; the second convolution unit is a point-by-point convolution layer, the input of which is also the input feature map, and the output channel number is C / 2; the outputs of the first convolution unit and the second convolution unit are spliced along the channel dimension to form a spliced feature map with C channels; a channel shuffle operation is performed on the spliced feature map to generate an output feature map; Step 2.2, the up-sampling 2 module of the decoding sub-network of the landslide identification method based on visual state space and frequency domain enhancement; The up-sampling 2 module includes a skip link module, a GSU Block and a second convolution; the GSU Block is used to enhance the semantic features of the landslide geological disasters of the up-sampling 1 module, extract local and global context features through double-path convolution, and restore the size of the feature map; the skip link module is used to add the feature map output by the up-sampling 1 module and the feature map output by the down-sampling 4 module to obtain a fused feature map; and the second convolution is used to restore the size of the feature map for subsequent up-sampling; Step 2.3, the up-sampling 3 module of the decoding sub-network of the landslide identification method based on visual state space and frequency domain enhancement; The up-sampling 3 module includes a skip link module, a GSU Block and a second convolution; the GSU Block is used to enhance the semantic features of the landslide geological disasters of the up-sampling 2 module, extract local and global context features through double-path convolution, and restore the size of the feature map; the skip link module is used to add the feature map output by the up-sampling 2 module and the feature map output by the down-sampling 3 module to obtain a fused feature map; and the second convolution is used to restore the size of the feature map for subsequent up-sampling; Step 2.4, the up-sampling 4 module of the decoding sub-network of the landslide identification method based on visual state space and frequency domain enhancement; The up-sampling 4 module includes a skip link module, a GSU Block and a second convolution; the GSU Block is used to enhance the semantic features of the landslide geological disasters of the up-sampling 3 module, extract local and global context features through double-path convolution, and restore the size of the feature map; the skip link module is used to add the feature map output by the up-sampling 3 module and the feature map output by the down-sampling 2 module to obtain a fused feature map; and the second convolution is used to restore the size of the feature map for subsequent up-sampling; Step 2.5, the up-sampling 5 module of the decoding sub-network of the landslide identification method based on visual state space and frequency domain enhancement; The up-sampling 5 module has a structure comprising a GSU Block and a skip link module; the GSU Block is used for enhancing the semantic features of the landslide geological disasters of the up-sampling 2 module, local and global context features are extracted through double-path convolution respectively, and the size of the feature map is restored; the skip link module is used for adding the feature map output by the up-sampling 2 module and the feature map output by the down-sampling 1 module, to obtain a fused feature map; The recognition sub-network of the remote sensing image landslide recognition method based on the visual state space and the frequency domain enhancement is constructed, and the feature map of the landslide on the remote sensing image is restored to the original input image size, comprising: Step 3, based on the encoding sub-network constructed in step 1 and the decoding sub-network constructed in step 2, the recognition sub-network of the remote sensing image landslide recognition method based on the visual state space and the frequency domain enhancement is constructed, comprising: The recognition sub-network is composed of a convolutional layer, which is used to restore the feature map of the decoding sub-network to the original remote sensing image size; The remote sensing image landslide recognition network based on the visual state space and the frequency domain enhancement is constructed, comprising: Step 4, based on the encoding sub-network constructed in step 1, the decoding sub-network constructed in step 2 and the recognition sub-network constructed in step 3, the remote sensing image landslide recognition network based on the visual state space and the frequency domain enhancement is constructed; Step 4.1, the feature map of the landslide geological disaster range extracted by the encoder in step 1 and the feature map extracted by the decoder in step 2 are added and fused through up-sampling cascade; Step 4.2, the landslide geological disaster feature map extracted by the up-sampling block in step 2 and the recognition sub-network constructed in step 3 are connected in series; A training set, a validation set and a test set are generated by using a public landslide recognition data set; The remote sensing image landslide recognition network based on the visual state space and the frequency domain enhancement is trained in combination with the Dice loss function, to obtain a model weight file; The model weight file obtained by training is used to detect the range of the landslide on the remote sensing image.
2. The method of claim 1, wherein, The training set, the validation set and the test set are generated by using a public landslide recognition data set, comprising: Step 5, the training set, the validation set and the test set are generated; Step 5.1, the public data set Bijie Landslide of remote sensing image landslide geological disasters is obtained; Step 5.2, each image in the image data set in step 5.1 is randomly subjected to five data augmentation operations of histogram equalization, geometric flipping, contrast enhancement, etc., and is divided into a training set, a validation set and a test set in proportion.
3. The method of claim 1, wherein, The remote sensing image landslide recognition network based on the visual state space and the frequency domain enhancement is trained in combination with the Dice loss function, to obtain a model weight file, comprising: Step 6, the remote sensing image landslide geological disaster recognition network constructed in step 4 is trained using the training set and the validation set generated in step 5, to obtain a model weight file; Step 6.1, the training parameters are set, the remote sensing image is randomly and non-repeatedly selected from the training set each time and input into the network, and the loss function uses Dice Loss; The Dice Loss is as follows: ; where p i is the predicted probability, g i is the true label, and e is to prevent division by zero; Step 6.2, input the training set and the validation set generated in step 5 into the remote sensing image landslide geological disaster identification network constructed in step 4 in sequence; the specific steps are: input the remote sensing image of landslide geological disaster to the coding sub-network to extract the characteristics of landslide geological disaster; input the feature map output by the feature extraction sub-network into the decoding sub-network to obtain the feature map of the feature fusion of landslide geological disaster; finally, use the identification sub-network to restore the feature fusion feature map to the original image size.
4. The method of claim 1, wherein, The model weight file trained is used to detect the range of landslides on the remote sensing image, comprising: Step 7, using the model weight file trained in step 6, detecting the range of remote sensing image landslide geological disasters in the test set generated in step 5; Step 7.1, input the test set generated in step 5 into the remote sensing image landslide geological disaster identification network trained in step 6, and load the "best_weight.pth" parameter file of the remote sensing image landslide identification network based on visual state space and frequency domain enhancement trained in step 6.1; Step 7.2, remote sensing image landslide geological disaster identification is performed on the remote sensing image in the test set to obtain the range of landslides on the remote sensing image; Step 7.3, output the remote sensing image landslide geological disaster identification result and save it as a ".png" format label file.
Citation Information
Patent Citations
Automatic landslide identification method, system and device based on lightweight convolutional neural network and double attention, and medium
CN116206214A
Geological disaster automatic identification method based on multi-source data and deep learning
CN119314009A
Landslide disaster automatic identification method based on convolutional neural network
CN116030353A
High-resolution remote sensing image landslide identification method, device and system, and storage medium
CN119723333A