A semantic segmentation method, system, device and medium for landslide remote sensing

Abstract

The application relates to the technical field of landslide detection, and discloses a semantic segmentation method, system, device and medium for mountain landslide remote sensing. The method comprises the following steps: acquiring a remote sensing image of a mountain landslide, and extracting a feature map of the remote sensing image; adopting a first attention mechanism to perform feature extraction on the feature map, so as to obtain a first-stage feature map focusing on the main texture of the mountain landslide; adopting a second attention mechanism to perform feature extraction on the first-stage feature map, so as to obtain a second-stage feature map focusing on the edge extension of the mountain landslide; adopting a third attention mechanism to perform feature extraction on the second-stage feature map, so as to obtain a third-stage feature map focusing on the direction of the mountain landslide slope body; adopting a fourth attention mechanism to perform feature extraction on the third-stage feature map, so as to obtain a fourth-stage feature map focusing on the stripping trace of the mountain landslide; and combining the first-stage feature map, the second-stage feature map, the third-stage feature map and the fourth-stage feature map to perform semantic segmentation on the remote sensing image.

Description

Technical Field

[0001] This invention relates to the field of landslide detection technology, and in particular to a semantic segmentation method, system, device and medium for remote sensing of landslides. Background Technology

[0002] Early semantic segmentation of landslide remote sensing mainly relied on traditional machine learning methods, such as Support Vector Machine (SVM) and Random Forest (RF). This method depends on manual feature engineering and high-precision labeled data, and the whole process has limitations such as low efficiency and strong subjectivity, especially in the semantic segmentation of landslide remote sensing in large-scale complex terrain areas, where it faces significant bottlenecks.

[0003] In recent years, deep learning-based semantic segmentation techniques (such as U-Net and DeepLabv3+) have gradually replaced traditional machine learning, demonstrating higher accuracy and automation in landslide remote sensing semantic segmentation. Current satellites can provide a large amount of ground remote sensing imagery, offering ample data support for deep learning methods, which have shown superior performance in landslide remote sensing semantic segmentation across large areas of complex terrain.

[0004] However, current landslide segmentation algorithms are usually based on the Transformer algorithm, which has a computational complexity of quadratic time, making it quite complex. Even though some solutions use local attention to address this issue, they still lack global landslide interaction capabilities, resulting in low semantic segmentation accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a semantic segmentation method, system, device and medium for remote sensing of landslides, which can solve the problem that the current landslide segmentation algorithms lack global landslide interaction capabilities, resulting in low semantic segmentation accuracy.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a semantic segmentation method for remote sensing of landslides, comprising the following steps:

[0007] Acquire remote sensing images of landslides and extract feature maps from the remote sensing images;

[0008] An attention window composed of multiple preset windows is obtained. Each preset window in the attention window is deformed according to the feature map to obtain the first attention window. The first attention mechanism is used to extract features from the feature map using the first attention window to obtain the first-stage feature map focusing on the main texture of the landslide.

[0009] Based on the first-stage feature map, each preset window in the first attention window is deformed to obtain the second attention window. The second attention mechanism is then used to extract features from the first-stage feature map using the second attention window to obtain the second-stage feature map focusing on the extension of the edge of the landslide.

[0010] Based on the second-stage feature map, each preset window in the second attention window is deformed to obtain the third attention window. The third attention mechanism is then used to extract features from the second-stage feature map using the third attention window to obtain the third-stage feature map focusing on the slope direction of the landslide.

[0011] Based on the third-stage feature map, each preset window in the third attention window is deformed to obtain the fourth attention window. The fourth attention mechanism is then used to extract features from the third-stage feature map using the fourth attention window to obtain the fourth-stage feature map focusing on the stripping traces of the landslide.

[0012] By combining the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map, semantic segmentation is performed on the remote sensing image to obtain the location of the landslide.

[0013] Optionally, the first attention mechanism performs attention calculations on all pixels in the first attention window to extract features from the feature map;

[0014] The second attention mechanism performs attention calculations on pixels at the same position in multiple preset windows corresponding to the second attention window in order to extract features from the first-stage feature map;

[0015] The third attention mechanism performs attention calculations on the pixels of a preset window in a fixed row corresponding to the third attention window to extract features from the second-stage feature map.

[0016] The fourth attention mechanism performs attention calculations on the pixels of a preset window in a fixedly arranged column corresponding to the fourth attention window in order to extract features from the second-stage feature map.

[0017] Optionally, the first attention window, the second attention window, the third attention window, and the fourth attention window are obtained according to the following steps:

[0018] Through average pooling layer, The deformation prediction module, composed of activation layers and convolutional layers, determines the deformation ratio and offset of each preset window in the attention window based on the feature map, the first-stage feature map, the second-stage feature map, and the third-stage feature map.

[0019] By using the corresponding deformation ratio and offset, each preset window in the attention window is deformed to obtain the first attention window, the second attention window, the third attention window, and the fourth attention window respectively.

[0020] Optionally, the step of combining the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on the remote sensing image includes:

[0021] The fourth-stage feature map generates a spatial weight map to indicate the distribution of different landforms in the landslide, and the spatial weight map and the feature map are multiplied by the Haddam product to obtain the enhanced feature map.

[0022] The first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map are concatenated in a cascade manner and then fused with the enhanced feature map to obtain a fused feature map;

[0023] Semantic segmentation is performed on remote sensing images based on the fused feature maps.

[0024] Optionally, after extracting the feature map of the remote sensing image, the method further includes:

[0025] Based on the channel dimension of the feature map, the feature map is divided into multiple feature sub-maps;

[0026] For each feature sub-map, the first attention mechanism, the second attention mechanism, the third attention mechanism, and the fourth attention mechanism are used for feature extraction.

[0027] Embodiments of the present invention also provide a semantic segmentation system for remote sensing of landslides, comprising:

[0028] The image acquisition module is used to acquire remote sensing images of landslides and extract feature maps from the remote sensing images;

[0029] The first feature extraction module is used to obtain an attention window composed of multiple preset windows, deform each preset window in the attention window according to the feature map to obtain the first attention window, and use the first attention mechanism to extract features from the feature map with the first attention window to obtain the first stage feature map focusing on the main texture of the landslide.

[0030] The second feature extraction module is used to deform each preset window in the first attention window according to the first stage feature map to obtain the second attention window, and to use the second attention mechanism to extract features from the first stage feature map using the second attention window to obtain the second stage feature map focusing on the extension of the edge of the landslide.

[0031] The third feature extraction module is used to deform each preset window in the second attention window according to the second stage feature map to obtain the third attention window, and to use the third attention mechanism to extract features from the second stage feature map with the third attention window to obtain the third stage feature map focused on the slope direction of the landslide.

[0032] The fourth feature extraction module is used to deform each preset window in the third attention window according to the third stage feature map to obtain the fourth attention window, and to use the fourth attention mechanism to extract features from the third stage feature map using the fourth attention window to obtain the fourth stage feature map focusing on the stripping traces of the landslide.

[0033] The semantic segmentation module is used to combine the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on remote sensing images and obtain the location of landslides.

[0034] Embodiments of the present invention also provide a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described semantic segmentation method for remote sensing of landslides.

[0035] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described semantic segmentation method for remote sensing of landslides.

[0036] The semantic segmentation method for landslide remote sensing provided by this invention has at least the following beneficial effects:

[0037] For remote sensing images of landslides, this invention employs a heterogeneous attention mechanism composed of multiple attention mechanisms. These mechanisms utilize their own attention windows to extract features from the remote sensing image in stages. Each stage focuses on different aspects of the landslide's features, including the main texture of the landslide, its edge extension, slope direction, and erosion marks. The attention window used by each mechanism can adaptively deform according to the feature map of the current stage, adapting it to the currently processed feature map. This allows it to focus on edge objects in the feature map, expanding the perceptual range of the attention window. Combining this adaptive window deformation mechanism with the heterogeneous attention mechanism, the receptive field of traditional attention mechanisms can be expanded globally without increasing computational complexity during remote sensing image processing. This effectively broadens the scope of feature information interaction, thereby improving the accuracy of remote sensing semantic segmentation of landslides. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0039] Figure 1 A flowchart illustrating a semantic segmentation method for remote sensing of landslides provided by this invention;

[0040] Figure 2 A schematic diagram of an architecture based on a dynamic deformable window and a heterogeneous local attention model provided by the present invention;

[0041] Figure 3 A schematic diagram of a dynamically deformable window provided by the present invention;

[0042] Figure 4 A schematic diagram of a dynamic deformable window intervening in a heterogeneous local attention module provided by the present invention;

[0043] Figure 5 A schematic diagram of a heterogeneous local attention structure module provided by the present invention;

[0044] Figure 6 A schematic diagram of a novel progressive landslide feature aggregation decoder structure module provided by the present invention;

[0045] Figure 7 A model comparison effect illustration provided by the present invention Figure 1 ;

[0046] Figure 8 A model comparison effect illustration provided by the present invention Figure 2 ;

[0047] Figure 9 A model comparison effect illustration provided by the present invention Figure 3 . Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] One embodiment of the present invention relates to a semantic segmentation method for remote sensing of landslides. The specific process of the semantic segmentation method for remote sensing of landslides in this embodiment can be as follows: Figure 1 As shown, it includes:

[0051] Step 101: Obtain remote sensing images of the landslide and extract feature maps from the remote sensing images.

[0052] Step 102: Obtain an attention window composed of multiple preset windows. Deform each preset window in the attention window according to the feature map to obtain a first attention window. Use the first attention mechanism to extract features from the feature map using the first attention window to obtain a first-stage feature map focusing on the main texture of the landslide.

[0053] Step 103: Based on the first stage feature map, each preset window in the first attention window is deformed to obtain the second attention window, and the second attention mechanism is used to extract features from the first stage feature map using the second attention window to obtain the second stage feature map focusing on the extension of the edge of the landslide.

[0054] Step 104: Based on the second-stage feature map, each preset window in the second attention window is deformed to obtain the third attention window. The third attention mechanism is then used to extract features from the second-stage feature map using the third attention window to obtain the third-stage feature map focusing on the direction of the landslide slope.

[0055] Step 105: Based on the third-stage feature map, each preset window in the third attention window is deformed to obtain the fourth attention window. The fourth attention mechanism is then used to extract features from the third-stage feature map using the fourth attention window to obtain the fourth-stage feature map focusing on the stripping traces of the landslide.

[0056] Step 106: Combine the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on the remote sensing image to obtain the location of the landslide.

[0057] The following is a detailed description of the implementation details of the semantic segmentation method for landslide remote sensing in this embodiment. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0058] First, an initial feature map of the remote sensing image is obtained, and then the initial feature map is processed in stages using a heterogeneous (hybrid) local attention mechanism. Specifically, regarding the attention window of the heterogeneous local attention mechanism, this embodiment designs a dynamic deformable window mechanism. This mechanism establishes the landslide spatial dependency, and after the window adaptively deforms using a learnable deformation parameter prediction network, four types of local attention from the heterogeneous local attention mechanism are applied.

[0059] In the specific implementation, the window is first divided into local attention computation units according to the heterogeneous local attention mechanism. Then, the self-attention window is dynamically adapted and deformed through a dynamic deformation window mechanism module. When a remote sensing image of a landslide is input into the semantic segmentation model, it first enters the dynamic window deformation module. In this module, the window deforms and shifts according to the landslide features. Then, local attention calculation is performed based on the deformed window. The results of the four local attention calculations are concatenated to obtain a feature map that integrates local and global features. This process is as follows: Figure 2 As shown, the calculation reference formula is as follows:

[0060] ;

[0061] ;

[0062] In the formula, The output of the feature map after dynamic window deformation and heterogeneous local attention; Computation for heterogeneous local attention; For layer normalization function; This is the output of the feedforward neural network; It is a feedforward neural network; This is a dynamic window deformation module.

[0063] The dynamic deformable window mechanism module in this embodiment expands the window's perception range through a learnable deformation process, enabling the window to include window edge objects before attention calculation. This approach allows the window to learn to focus on distant regions and flexibly determine its size and shape based on landslide image features. The size and position of each window on the feature map are adjusted by a corresponding learned offset. and The deformation scale in the direction is adjusted. After deformation, the window will focus more on the main landslide-related features and ignore other irrelevant information.

[0064] For the dynamic deformable window mechanism, please refer to Figure 3 It mainly includes the following steps:

[0065] (1) Determine the prototype window. The feature map is divided into different windows according to each attention mechanism; the four local attention mechanisms encompass four window partitioning methods. It should be noted that the size and position of the prototype window are fixed and do not overlap. The partitioning process of the dynamically deformable window module can be represented as follows:

[0066] ;

[0067] In the formula: Number of windows; The width of the predefined window; Given the predefined window height, we can derive... After segmentation, these windows are assembled according to the shape of the original feature map, thus maintaining the original layout after assembly.

[0068] (2) Determine the deformation scale. After obtaining the prototype windows, these windows will acquire offsets and deformation scales through a learnable deformation prediction module to complete the window deformation. In this embodiment, two learnable quantities are set to control the degree of window deformation: one is the ratio of window deformation to window deviation displacement, which is obtained through the deformation prediction module. The deformation prediction module consists of an average pooling layer, It consists of activation layers and 1×1 convolutional layers with a stride of 1. The kernel size and stride of the pooling layers follow the prototype window size, so that each window will learn the specific values ​​of the deformation scale during training.

[0069] The deformation scale is calculated as follows:

[0070] ;

[0071] In the formula, The deformation scale; This is the offset; It is a convolutional layer; For activation functions; For average pooling layers; ; This refers to the number of feature map channels at different stages. Since both involve horizontal and vertical directions, this is achieved through... and The window can be calculated in the coordinate axes of the feature map.

[0072] (3) Window Sampling. Since each window has a different shape after deformation and cannot be predicted before training, it is almost impossible to directly perform self-attention calculation on these different shapes simultaneously. If forced to do so, it would be necessary to iteratively obtain the deformation and offset of these windows and then calculate them separately, which would make the calculation module very complex and the time complexity would increase dramatically. Therefore, a window sampling technique is proposed. Specifically, before dividing the window, the upper left corner of the feature map is taken as the origin, and the points to the right and down are respectively... Axial direction and Along the axis, the two-dimensional coordinates of each pixel are recorded. After dividing the window, each window also obtains its corresponding coordinate window. This is equivalent to generating a coordinate graph with the same height and width for each prototype window. This is completed within each window. , After calculation, deformation and offset are applied to the coordinate points within the window. The calculation process is shown in the formula below:

[0073] ;

[0074] In the formula, , The first The deformation scale and offset of each window; The original horizontal coordinates; This represents the original vertical coordinate. In the above formula... and After segmentation, these windows are assembled according to the shape of the original feature map, thus maintaining the original layout after assembly.

[0075] This effectively maps the prototype window to a deformed window while maintaining the same number of pixels. Then, sampled values ​​are used to assign the deformed pixels from the coordinate graph to the prototype window, replacing the pixel values ​​in the original window. This achieves window deformation and displacement without changing the window size.

[0076] Based on the above process, it can be seen that the first attention window, the second attention window, the third attention window, and the fourth attention window are obtained according to the following steps: through an average pooling layer, The deformation prediction module, composed of activation layers and convolutional layers, determines the deformation ratio and offset of each preset window in the attention window based on the feature map, the first-stage feature map, the second-stage feature map, and the third-stage feature map. It then deforms each preset window in the attention window using the corresponding deformation ratio and offset to obtain the first attention window, the second attention window, the third attention window, and the fourth attention window, respectively.

[0077] After the dynamic window transformation module, then... Figure 4The heterogeneous local attention module is the core module of the landslide segmentation algorithm. Its main purpose is to enable the model to better extract landslide features through four different local attention mechanisms. For details on the heterogeneous local attention module, please refer to the above description.

[0078] In its implementation, the heterogeneous local attention mechanism comprises four modules, such as... Figure 5 As shown, the first part is basic static local attention (i.e., the first attention mechanism), which means that windows are stacked together in a non-overlapping manner, like tiling bricks, focusing on the texture consistency of the main landslide area. The second part is cross-domain expansion attention (i.e., the second attention mechanism), which requires grouping all pixels at the same location in the window block together for self-attention computation to track the radial extension of edge cracks. The next quarter is axial sparse divide-and-conquer attention (i.e., the third attention mechanism), which uses a set of column pixels arranged at the same intervals for local attention computation. Similarly, the last quarter, vertical sparse divide-and-conquer attention (i.e., the fourth attention mechanism), will use windows grouped row by row at the same intervals. The latter two are used to extract the directional features of slope slip and layered detachment traces, respectively.

[0079] That is, the first attention mechanism performs attention calculation on all pixels in the first attention window to extract features from the feature map; the second attention mechanism performs attention calculation on pixels at the same position in multiple preset windows corresponding to the second attention window to extract features from the first-stage feature map; the third attention mechanism performs attention calculation on pixels in preset windows in a fixed row corresponding to the third attention window to extract features from the second-stage feature map; and the fourth attention mechanism performs attention calculation on pixels in preset windows in a fixed column corresponding to the fourth attention window to extract features from the second-stage feature map.

[0080] The basic static local attention mechanism requires decomposing the feature map along the spatial dimension, dividing it into non-overlapping windows. These windows fit together tightly like bricks, forming individual computational units. The size of the map tiles is set to... × The feature map is decomposed into The shape, self-attention in × Execute on the block.

[0081] While basic static local attention eliminates the huge computational overhead of global attention and achieves linear complexity, its windows do not overlap and lack communication between them. To supplement the global interaction capability in the attention mechanism, cross-domain extended local attention is proposed as a global enhancement, building upon basic static local attention.

[0082] Cross-domain extended local attention allows the size of the computational unit to remain within a certain range. × Cross-domain dilatational local attention does not select pixels from contiguous regions of the feature map, but rather selects pixels from the entire feature map, performing attention in the spatial dimension. This embodiment specifies the matrix for performing attention. × The size of the matrix. It should be noted that this matrix differs from the blocks in basic static local attention: although the number of pixels and the window shape are the same, the pixels in the matrix are selected from the entire feature map. The computational units of basic static local attention consist of windows composed of spatially adjacent pixels, while the pixels within a single window matrix in the cross-domain dilatational local attention mechanism are at the same positions from different windows under the basic static local attention mechanism. The feature map is divided into... The shape of the matrix. This method effortlessly blends global markers within the matrix, where the blended window pixels come from the same positions in the original window. Compared to SWin based on shift windows, this attention mechanism is more intuitive, easier to implement, and does not increase the number of parameters or FLOPS.

[0083] Next, this embodiment introduces a direct and effective method to expand the range of a local window globally, called axial sparse divide-and-conquer attention, which refers to selecting a set of rows arranged at fixed distances to form the desired window. Defined as the height of a window in the axial sparse divide-and-conquer attention. To obtain a set of pixels along a specific axis, a row or column of pixels is selected at fixed intervals. The feature map is divided into... Group, N Refers to the number of windows. H This refers to the height of the feature map. This model defines the window height separately. Set the four stages to 1, 2, 7, 7. The window size is... The feature map is decomposed into... at each stage. The shape of the image is determined by the number of rows in the feature map. At each stage, pixels within each window are selected from different rows of the feature map. Considering the divisibility relationship between image shape and the number of window rows, image padding is used in this embodiment to ensure an integer number of windows are obtained. Compared to traditional local attention, the receptive field of each self-attention computation unit is expanded globally, effectively broadening the scope of information interaction.

[0084] Similar to the attention window selection mechanism of axial sparse divide-and-conquer, Defined as the width of a window in vertical sparse divide-and-conquer attention. In vertical sparse divide-and-conquer attention, the feature map is divided into... Group, M Refers to the number of windows. W This refers to the width of the feature map. Window width. Set the four stages to 1, 2, 7, 7. The window size is... The feature map is decomposed into... at each stage. The shape of the window is determined. At each stage, pixels within each window are selected from different columns of the feature map. This method effectively expands the scope of the local window without increasing computational complexity. Compared to traditional local attention, the receptive field of each self-attention computational unit is expanded globally, effectively broadening the scope of information interaction.

[0085] In one example, before performing the above processing using a heterogeneous local attention mechanism, the initial feature map is first divided into multiple feature sub-maps based on the channel dimension of the initial feature map of the remote sensing image. For each feature sub-map, the first attention mechanism, the second attention mechanism, the third attention mechanism, and the fourth attention mechanism are then used for feature extraction (equivalent to performing self-attention in horizontal, vertical, tiling, and expanding windows).

[0086] In some embodiments, after processing the remote sensing image using a heterogeneous local attention mechanism to obtain a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map, a spatial weight map is generated using the fourth-stage feature map to indicate the distribution of different landforms of landslides. The spatial weight map is then multiplied by the initial feature map using the Hadam product to obtain an enhanced feature map. The first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map are then concatenated and fused with the enhanced feature map to obtain a fused feature map. Based on the fused feature map, the remote sensing image is semantically segmented.

[0087] In this implementation, after the encoder work, which includes dynamic deformation windows and heterogeneous local attention, is completed, the output feature map of each stage encoder (an attention mechanism) first passes through the landslide feature extraction module (SFEM) of the decoder. A landslide weight extraction mechanism is constructed to achieve dynamic focusing of the landslide area. In the landslide feature extraction module, high-level features are used as input, and their clear semantics are combined with low-level features to recover detailed landslide information, such as... Figure 6 As shown.

[0088] Specifically, a 1×1 convolution is applied to the feature map to compress the channel dimension to 1, and then a spatial weight map with values ​​between [0,1] is generated using the sigmoid function. This weight map is essentially a probability heatmap, with high-response regions corresponding to pixel locations of key landforms such as landslide boundaries and cracks. Shallow features, containing rich details such as edges and textures, can effectively identify the transition region between the landslide and the background, while the nonlinear mapping of the sigmoid function can transform the ambiguous initial-order features into discriminative spatial weights. This weight map is then subjected to a Hadamard product with downsampled low-level features. This achieves targeted enhancement of deep semantic features, strengthening feature responses in regions with high landslide probability and suppressing redundant noise in background regions. Subsequently, multi-scale features are fused, and through cross-level fusion segmentation, a progressive interaction of multi-level feature pyramids is achieved, completing multi-scale modeling of landslide morphology.

[0089] The landslide feature extraction module cascades features from different stages: First, features from Stage 4 and Stage 3 are input into the primary landslide feature extraction module to generate intermediate-level features with fused landslide weights; these features then interact with Stage 2 features through the secondary landslide feature extraction module to extract enhanced features that balance semantic consistency and spatial accuracy; finally, they are fused with the low-level detail features from Stage 1 in the final landslide feature extraction module to form a multi-level joint representation covering the entire resolution. Ultimately, all enhanced features from each level are input into the segmentation head via channel concatenation, and are gradually restored to the original resolution through 3×3 convolution and bilinear upsampling, while skip connections are used to preserve the discriminative information of each stage.

[0090] In some embodiments, a landslide dataset is first acquired and divided proportionally into a training set, a validation set, and a test set. The training set is used for model training, and the test set is used to test the trained model. The dataset division can be random or based on a preset ratio; this embodiment does not impose a limitation. Then, a dynamic deformation window mechanism module is used to dynamically adapt and deform the self-attention window, and a heterogeneous local attention mechanism is used to calculate the feature map. See the above processing steps for further details. Next, the landslide feature extraction module uses a landslide weight extraction mechanism to achieve dynamic focusing of the landslide area. Furthermore, this embodiment uses the above training set to train the landslide semantic segmentation model using a batch-based stochastic gradient descent method. Finally, the trained model is used to perform semantic segmentation on the landslide test set to obtain the landslide area recognition result.

[0091] In summary, the semantic segmentation method for landslide remote sensing proposed in this invention presents a landslide segmentation algorithm based on dynamic deformable windows and heterogeneous local attention. This algorithm can more effectively adapt to different regions and types of landslides, capturing the unified features of landslides. After inputting the image, the network first passes through a window deformation module for window deformation and segmentation. This window deformation and segmentation allows the local attention in the network to establish a complete dependency on the landslide target. Four different local attention mechanisms are used in the hybrid local attention module to extract different landslide features, which are then cascaded into the same feature map. Subsequently, the feature map passes through a novel progressive landslide feature aggregation decoder, which enhances the landslide features through the landslide feature extraction module. This gives the model a strong global modeling capability and greater adaptability to landslide segmentation in different scenes and terrains.

[0092] The dynamic window deformation mechanism expands the window's perception range, enabling the window to include edge objects before attention calculations are performed. This approach allows the window to learn to focus on distant areas and flexibly determine its size and shape based on landslide image features.

[0093] The heterogeneous local attention module effectively expands the scope of the local window without increasing computational complexity. Compared with traditional local attention, the receptive field of each self-attention computation unit is expanded to the global scope, effectively broadening the range of information interaction. This enables the model to learn both the spatial and channel features of the input image simultaneously, thereby better segmenting landslide areas and accurately identifying the location of landslides even in complex geographical environments.

[0094] The progressive landslide feature aggregation decoder convolves the output of each stage encoder as fine-grained features and fuses them into the coarse-grained features of the next stage. Before inputting the decoder, an upsampling module is used to assign the coarse-grained features to the fine-grained features of the next stage. Ultimately, the features of each stage have a global semantic vision. This not only allows the decoder input of each stage of the model to have small-sized, coarse-grained semantic information, but also large-sized, fine-grained semantic information. The model has a larger semantic vision that focuses on long-range dependencies, and has better generalization to different terrains, achieving the best performance in landslide segmentation.

[0095] Ultimately, the feature map after passing through the feature pyramid decoder contains not only coarse-grained and fine-grained semantic information but also multi-scale global information, enabling the method to achieve better results in the landslide segmentation task.

[0096] The following tests were conducted on three landslide datasets, and the experimental results show that the present invention significantly improves the model's ability to identify landslide areas.

[0097] Table 1 shows a comparison of precision, recall, F1-score, mean accuracy (mACC), and mean intersection-over-union (mIoU) values ​​on the first landslide dataset. This invention compares the precision, recall, F1-score, mean accuracy (mACC), and mean intersection-over-union (mIoU) values ​​with other benchmark methods on the landslide dataset. The benchmark methods compared include publicly published methods such as ViT-CoMer, RMT, Segformer, SWin-Transformer, SeaFormer, and CSWin-Transformer. Small, Base, Tiny, Large, B4, and B5 in the table below represent the specific specifications of the selected models. The quantitative results in Table 1 show that the model using this invention significantly outperforms other methods in all five metrics. This dataset contains a total of 1025 labeled samples. Of these, 820 were training samples, accounting for about 80% of the total, while 96 were retained in the validation set and 109 in the test set, each accounting for 10%.

[0098] Table 1

[0099]

[0100] The second landslide dataset contains a total of 275 samples, with 84% used for training and the remaining 16% for validation and testing. The remote sensing images mainly depict landslides caused by earthquakes in a specific region. The experimental results for this dataset are as follows:

[0101] Table 2

[0102]

[0103] The third landslide dataset consisted of 770 manually segmented landslide samples from satellite imagery, including rockfalls, landslides, and a small number of rock slips. These samples were randomly divided in an 8:1:1 ratio. The training set contained 616 samples, the validation set contained 77 samples, and the test set contained 77 samples. The experimental results are as follows:

[0104] Table 3

[0105]

[0106] Figure 7 This paper presents landslide regions in the first dataset identified by the aforementioned benchmark method and the landslide semantic segmentation model proposed in this invention. Figure 7It can be seen that the semantic segmentation model for landslides proposed in this invention has a very good segmentation effect on landslides of different scales in various geographical environments, with clear edges and accurate recognition.

[0107] Figure 8 The paper presents the segmentation results of different comparison methods and the method of this invention on a second dataset. The second landslide dataset is characterized by low image resolution, a very small proportion of landslide pixels in the image, and minimal color difference between the landslide area and the background, posing a significant challenge to the model's segmentation. The method of this invention is closest to the label in shape. Visual comparison results show that this method performs excellently even under conditions of low image resolution, a very small proportion of landslide pixels, and interference from external cloud and fog conditions.

[0108] Figure 9 The image shows the segmentation results of different comparison methods and the method of this invention on a third landslide dataset. This third landslide dataset also has low image resolution. Although the terrain is similar, the landslide shapes are very diverse, posing a significant challenge to the model's segmentation. Two images show the segmentation results for strip and square landslides, demonstrating that the method of this invention has a strong ability to capture landslide shapes.

[0109] The segmentation results show that the method of this invention provides the best control over the shape of the landslide area. Other comparative methods misidentify landslides, resulting in overly large segmented areas. They also exhibit significant adhesion issues; the method of this invention is the closest in shape to the label. Visual comparison results indicate that the method of this invention operates at lower image resolutions, with landslide pixels occupying a large portion of the image.

[0110] Meanwhile, the visualization comparison results show that the method of this invention exhibits robust segmentation capabilities on all three datasets, and its overall performance is superior to existing comparative methods. The three datasets provide various landslide scenarios, including deserts, Gobi deserts, forests, farmland, and reservoirs, as well as various landslide types such as strip-shaped, band-shaped, and block-shaped landslides. Even under complex environments, cloud and fog interference, and low resolution conditions, the model can still maintain good segmentation results, demonstrating strong generalization ability and anti-interference capability.

[0111] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, without changing the core design of the algorithm and process, are also within the protection scope of this invention.

[0112] Another embodiment of the present invention relates to a semantic segmentation system for remote sensing of landslides. The implementation details of the semantic segmentation system for remote sensing of landslides in this embodiment are described below. The following content is only for ease of understanding and is not essential for implementing this solution. The semantic segmentation system for remote sensing of landslides in this embodiment includes:

[0113] The image acquisition module is used to acquire remote sensing images of landslides and extract feature maps from the remote sensing images;

[0114] The first feature extraction module is used to obtain an attention window composed of multiple preset windows, deform each preset window in the attention window according to the feature map to obtain the first attention window, and use the first attention mechanism to extract features from the feature map with the first attention window to obtain the first stage feature map focusing on the main texture of the landslide.

[0115] The second feature extraction module is used to deform each preset window in the first attention window according to the first stage feature map to obtain the second attention window, and to use the second attention mechanism to extract features from the first stage feature map using the second attention window to obtain the second stage feature map focusing on the extension of the edge of the landslide.

[0116] The third feature extraction module is used to deform each preset window in the second attention window according to the second stage feature map to obtain the third attention window, and to use the third attention mechanism to extract features from the second stage feature map with the third attention window to obtain the third stage feature map focused on the slope direction of the landslide.

[0117] The fourth feature extraction module is used to deform each preset window in the third attention window according to the third stage feature map to obtain the fourth attention window, and to use the fourth attention mechanism to extract features from the third stage feature map using the fourth attention window to obtain the fourth stage feature map focusing on the stripping traces of the landslide.

[0118] The semantic segmentation module is used to combine the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on remote sensing images and obtain the location of landslides.

[0119] It is not difficult to see that this embodiment is a system embodiment corresponding to the above method embodiments, and this embodiment can be implemented in conjunction with the above method embodiments. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0120] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0121] Another embodiment of the present invention relates to a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the semantic segmentation method for landslide remote sensing in the above embodiments.

[0122] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0123] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0124] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0125] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A semantic segmentation method for remote sensing of landslides, characterized in that, The method includes: Acquire remote sensing images of landslides and extract feature maps from the remote sensing images; An attention window composed of multiple preset windows is obtained. Each preset window in the attention window is deformed according to the feature map to obtain the first attention window. The first attention mechanism is used to extract features from the feature map using the first attention window to obtain the first-stage feature map focusing on the main texture of the landslide. Based on the first-stage feature map, each preset window in the first attention window is deformed to obtain the second attention window. The second attention mechanism is then used to extract features from the first-stage feature map using the second attention window to obtain the second-stage feature map focusing on the extension of the edge of the landslide. Based on the second-stage feature map, each preset window in the second attention window is deformed to obtain the third attention window. The third attention mechanism is then used to extract features from the second-stage feature map using the third attention window to obtain the third-stage feature map focusing on the slope direction of the landslide. Based on the third-stage feature map, each preset window in the third attention window is deformed to obtain the fourth attention window. The fourth attention mechanism is then used to extract features from the third-stage feature map using the fourth attention window to obtain the fourth-stage feature map focusing on the stripping traces of the landslide. By combining the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map, semantic segmentation is performed on the remote sensing image to obtain the location of the landslide.

2. The semantic segmentation method for remote sensing of landslides according to claim 1, characterized in that, The first attention mechanism performs attention calculations on all pixels in the first attention window to extract features from the feature map; The second attention mechanism performs attention calculations on pixels at the same position in multiple preset windows corresponding to the second attention window in order to extract features from the first-stage feature map; The third attention mechanism performs attention calculations on the pixels of a preset window in a fixed row corresponding to the third attention window to extract features from the second-stage feature map. The fourth attention mechanism performs attention calculations on the pixels of a preset window in a fixedly arranged column corresponding to the fourth attention window in order to extract features from the second-stage feature map.

3. The semantic segmentation method for remote sensing of landslides according to claim 1, characterized in that, The first attention window, the second attention window, the third attention window, and the fourth attention window are obtained according to the following steps: Through average pooling layer, The deformation prediction module, composed of activation layers and convolutional layers, determines the deformation ratio and offset of each preset window in the attention window based on the feature map, the first-stage feature map, the second-stage feature map, and the third-stage feature map. By using the corresponding deformation ratio and offset, each preset window in the attention window is deformed to obtain the first attention window, the second attention window, the third attention window, and the fourth attention window respectively.

4. The semantic segmentation method for remote sensing of landslides according to claim 1, characterized in that, The semantic segmentation of remote sensing images by combining the first-stage feature maps, the second-stage feature maps, the third-stage feature maps, and the fourth-stage feature maps includes: The fourth-stage feature map generates a spatial weight map to indicate the distribution of different landforms in the landslide, and the spatial weight map and the feature map are multiplied by the Haddam product to obtain the enhanced feature map. The first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map are concatenated in a cascade manner and then fused with the enhanced feature map to obtain a fused feature map; Semantic segmentation is performed on remote sensing images based on the fused feature maps.

5. The semantic segmentation method for remote sensing of landslides according to claim 1, characterized in that, Following the extraction of feature maps from the remote sensing images, the process also includes: Based on the channel dimension of the feature map, the feature map is divided into multiple feature sub-maps; For each feature sub-map, the first attention mechanism, the second attention mechanism, the third attention mechanism, and the fourth attention mechanism are used for feature extraction.

6. A semantic segmentation system for remote sensing of landslides, characterized in that, The system includes: The image acquisition module is used to acquire remote sensing images of landslides and extract feature maps from the remote sensing images; The first feature extraction module is used to obtain an attention window composed of multiple preset windows, deform each preset window in the attention window according to the feature map to obtain the first attention window, and use the first attention mechanism to extract features from the feature map with the first attention window to obtain the first stage feature map focusing on the main texture of the landslide. The second feature extraction module is used to deform each preset window in the first attention window according to the first stage feature map to obtain the second attention window, and to use the second attention mechanism to extract features from the first stage feature map using the second attention window to obtain the second stage feature map focusing on the extension of the edge of the landslide. The third feature extraction module is used to deform each preset window in the second attention window according to the second stage feature map to obtain the third attention window, and to use the third attention mechanism to extract features from the second stage feature map with the third attention window to obtain the third stage feature map focused on the slope direction of the landslide. The fourth feature extraction module is used to deform each preset window in the third attention window according to the third stage feature map to obtain the fourth attention window, and to use the fourth attention mechanism to extract features from the third stage feature map using the fourth attention window to obtain the fourth stage feature map focusing on the stripping traces of the landslide. The semantic segmentation module is used to combine the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on remote sensing images and obtain the location of landslides.

7. A computer device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the semantic segmentation method for landslide remote sensing as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the semantic segmentation method for remote sensing of landslides as described in any one of claims 1 to 5.

Images