Landslide remote sensing semantic segmentation method, system, equipment and medium

Abstract

The invention relates to the technical field of landslide detection, and discloses a semantic segmentation method, system and device for landslide remote sensing and a medium. The method comprises the following steps: acquiring a remote sensing image of a landslide, and extracting a feature map of the remote sensing image; performing feature extraction on the feature map by adopting a first attention mechanism to obtain a first-stage feature map focused on the landslide main body texture; performing feature extraction on the first-stage feature map by adopting a second attention mechanism to obtain a second-stage feature map focused on landslide edge extension; performing feature extraction on the second-stage feature map by adopting a third attention mechanism to obtain a third-stage feature map focused in the landslide slope body direction; performing feature extraction on the third-stage feature map by adopting a fourth attention mechanism to obtain a fourth-stage feature map focused on the landslide stripping trace; and performing semantic segmentation on the remote sensing image by combining the first-stage feature map, the second-stage feature map, the third-stage feature map and the fourth-stage feature map.

Description

Technical Field

[0001] The present invention relates to the technical field of landslide detection, and in particular to a semantic segmentation method, system, equipment and medium for remote sensing of landslides. Background Art

[0002] Early efforts to segment landslides using remote sensing primarily relied on traditional machine learning methods, such as support vector machines (SVMs) and random forests (RFs). These methods, which rely on manual feature engineering and highly annotated data, suffer from limitations such as low efficiency and high subjectivity. This presents a significant bottleneck in the segmentation of landslides using remote sensing over large, complex terrain.

[0003] In recent years, deep learning-based semantic segmentation techniques (such as U-Net and DeepLabv3+) have gradually replaced traditional machine learning, demonstrating higher accuracy and automation in landslide remote sensing semantic segmentation. Current satellites provide a vast amount of ground-based remote sensing imagery, which provides ample data support for deep learning methods, resulting in their superior performance in landslide remote sensing semantic segmentation across large, complex terrain areas.

[0004] However, current landslide segmentation algorithms are usually implemented based on the Transformer algorithm. The calculation of this method reaches quadratic complexity and is relatively complex. Even though some solutions use local attention to solve this problem, they seriously lack global landslide interaction capabilities, resulting in low semantic segmentation accuracy. Summary of the Invention

[0005] The purpose of the present invention is to provide a semantic segmentation method, system, device and medium for landslide remote sensing, which can solve the problem that the current landslide segmentation algorithm lacks global landslide interaction capability, resulting in low semantic segmentation accuracy.

[0006] To solve the above technical problems, an embodiment of the present invention provides a semantic segmentation method for landslide remote sensing, comprising the following steps: Obtain remote sensing images of landslides and extract feature maps of remote sensing images; An attention window consisting of multiple preset windows is obtained, each preset window in the attention window is deformed according to the feature map to obtain a first attention window, and a first attention mechanism is used to extract features from the feature map using the first attention window to obtain a first-stage feature map focused on the main texture of the landslide. According to the first-stage feature map, each preset window in the first attention window is deformed to obtain a second attention window, and a second attention mechanism is used to extract features from the first-stage feature map with the second attention window to obtain a second-stage feature map focused on the extension of the landslide edge. According to the second-stage feature map, each preset window in the second attention window is deformed to obtain a third attention window, and a third attention mechanism is used to extract features from the second-stage feature map with the third attention window to obtain a third-stage feature map focused on the direction of the landslide slope. According to the third stage feature map, each preset window in the third attention window is deformed to obtain a fourth attention window, and the fourth attention mechanism is used to extract features from the third stage feature map with the fourth attention window to obtain a fourth stage feature map focusing on the landslide peeling traces; Combining the first stage feature map, the second stage feature map, the third stage feature map and the fourth stage feature map, the remote sensing image is semantically segmented to obtain the location of the landslide.

[0007] Optionally, the first attention mechanism performs attention calculation on all pixels in the first attention window to extract features from the feature map; The second attention mechanism performs attention calculation on pixels at the same position in multiple preset windows corresponding to the second attention window to extract features from the feature map of the first stage; The third attention mechanism performs attention calculation on pixels of a preset window in a fixedly arranged row corresponding to the third attention window to extract features from the second stage feature map; The fourth attention mechanism performs attention calculation on the pixels of a preset window in a fixedly arranged column corresponding to the fourth attention window to extract features from the second stage feature map.

[0008] Optionally, the first attention window, the second attention window, the third attention window, and the fourth attention window are obtained according to the following steps: Through the average pooling layer, The deformation prediction module, which consists of activation layers and convolutional layers, determines the deformation ratio and offset of each preset window in the attention window based on the feature map, the first-stage feature map, the second-stage feature map, and the third-stage feature map. Each preset window in the attention window is deformed by a corresponding deformation ratio and offset to obtain a first attention window, a second attention window, a third attention window, and a fourth attention window, respectively.

[0009] Optionally, combining the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on the remote sensing image includes: The fourth stage feature map is used to generate a spatial weight map indicating the distribution of different landforms of landslides, and the spatial weight map is used to perform a Haddam product with the feature map to obtain an enhanced feature map. The first stage feature map, the second stage feature map, the third stage feature map and the fourth stage feature map are connected in series in a cascade manner and then fused with the enhanced feature map to obtain a fused feature map; Based on the fused feature map, the remote sensing image is semantically segmented.

[0010] Optionally, after extracting the feature map of the remote sensing image, the method further includes: According to the channel dimension of the feature map, the feature map is divided into multiple feature sub-maps; For each feature subgraph, the first attention mechanism, the second attention mechanism, the third attention mechanism and the fourth attention mechanism are used for feature extraction.

[0011] An embodiment of the present invention further provides a semantic segmentation system for landslide remote sensing, comprising: An image acquisition module is used to acquire remote sensing images of landslides and extract feature maps of the remote sensing images; A first feature extraction module is used to obtain an attention window composed of multiple preset windows, deform each preset window in the attention window according to the feature map to obtain a first attention window, and use the first attention mechanism to extract features from the feature map with the first attention window to obtain a first-stage feature map focused on the main texture of the landslide; A second feature extraction module is configured to deform each preset window in the first attention window according to the first-stage feature map to obtain a second attention window, and adopt a second attention mechanism to extract features from the first-stage feature map with the second attention window to obtain a second-stage feature map focused on the extension of the landslide edge; A third feature extraction module is used to deform each preset window in the second attention window according to the second stage feature map to obtain a third attention window, and adopt a third attention mechanism to extract features from the second stage feature map with the third attention window to obtain a third stage feature map focused on the direction of the landslide slope; a fourth feature extraction module, configured to deform each preset window in the third attention window according to the third stage feature map to obtain a fourth attention window, and employ a fourth attention mechanism to extract features from the third stage feature map using the fourth attention window to obtain a fourth stage feature map focused on the landslide stripping traces; The semantic segmentation module is used to combine the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on the remote sensing image and obtain the location of the landslide.

[0012] An embodiment of the present invention also provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned semantic segmentation method for remote sensing of landslides.

[0013] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned semantic segmentation method for remote sensing of landslides.

[0014] The semantic segmentation method for landslide remote sensing provided by the present invention has at least the following beneficial effects: For remote sensing images of landslides, the present invention adopts a heterogeneous attention mechanism composed of multiple attention mechanisms to process them. The multiple attention mechanisms use their own attention windows to extract features of remote sensing images in stages. Each stage focuses on different aspects of the landslide, including the main texture of the landslide, the edge extension of the landslide, the slope direction of the landslide, and the landslide peeling traces. The attention window used by each attention mechanism can be adaptively deformed according to the feature map of the current stage to adapt it to the currently processed feature map, so that it can pay attention to the edge objects of the feature map and expand the perception range of the attention window. Combining this window adaptive deformation mechanism and the heterogeneous attention mechanism, when processing remote sensing images, the receptive field of the traditional attention mechanism can be expanded to the global without increasing the computational complexity, effectively broadening the scope of feature information interaction, thereby improving the accuracy of remote sensing semantic segmentation of landslides. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 A flowchart of a semantic segmentation method for landslide remote sensing provided by the present invention; Figure 2 A schematic diagram of the architecture of a dynamic deformation window and heterogeneous local attention model provided by the present invention; Figure 3 A schematic diagram of a dynamic deformation window provided by the present invention; Figure 4 A schematic diagram of a dynamic deformation window intervention heterogeneous local attention module provided by the present invention; Figure 5 A schematic diagram of a heterogeneous local attention structure module provided by the present invention; Figure 6 A schematic diagram of a new progressive landslide feature aggregation decoder structure module provided by the present invention; Figure 7 A schematic diagram of the comparison effect of a model provided by the present invention Figure 1 ; Figure 8 A schematic diagram of the comparison effect of a model provided by the present invention Figure 2 ; Figure 9 A schematic diagram of the comparison effect of a model provided by the present invention Figure 3 . DETAILED DESCRIPTION

[0016] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0017] The technical solutions provided by various embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0018] One embodiment of the present invention relates to a semantic segmentation method for remote sensing of landslides. The specific process of the semantic segmentation method for remote sensing of landslides in this embodiment can be as follows: Figure 1 Shown, including: Step 101: Acquire a remote sensing image of a landslide and extract a feature map of the remote sensing image.

[0019] In step 102, an attention window consisting of a plurality of preset windows is obtained, each preset window in the attention window is deformed according to the feature map to obtain a first attention window, and a first attention mechanism is used to extract features from the feature map using the first attention window to obtain a first-stage feature map focused on the main texture of the landslide.

[0020] In step 103, each preset window in the first attention window is deformed according to the first-stage feature map to obtain a second attention window, and a second attention mechanism is used to extract features from the first-stage feature map with the second attention window to obtain a second-stage feature map focused on the extension of the landslide edge.

[0021] In step 104, each preset window in the second attention window is deformed according to the second stage feature map to obtain a third attention window, and a third attention mechanism is used to extract features from the second stage feature map with the third attention window to obtain a third stage feature map focused on the direction of the landslide slope.

[0022] In step 105, each preset window in the third attention window is deformed according to the third stage feature map to obtain a fourth attention window, and a fourth attention mechanism is used to extract features from the third stage feature map with the fourth attention window to obtain a fourth stage feature map focused on the landslide peeling traces.

[0023] Step 106 , combining the first stage feature map, the second stage feature map, the third stage feature map, and the fourth stage feature map, performs semantic segmentation on the remote sensing image to obtain the location of the landslide.

[0024] The following is a detailed description of the implementation details of the semantic segmentation method for landslide remote sensing in this embodiment. The following content is only provided for ease of understanding and is not necessary for implementing this solution.

[0025] First, an initial feature map of the remote sensing image is obtained. This initial feature map is then processed in stages using a heterogeneous (hybrid) local attention mechanism. Regarding the attention window of the heterogeneous local attention mechanism, this embodiment designs a dynamic deformation window mechanism. This mechanism establishes landslide spatial dependencies, implements adaptive window deformation through a learnable deformation parameter prediction network, and then applies the four local attention mechanisms of the heterogeneous local attention mechanism.

[0026] In the specific implementation, the window is firstly performed according to the heterogeneous local attention mechanism, that is, the division of the local attention calculation unit, and then the self-attention window is dynamically adapted and deformed through the dynamic deformation window mechanism module. After the remote sensing image of the landslide is input into the semantic segmentation model, it first enters the dynamic window deformation module. In the module, the window is deformed and displaced according to the landslide characteristics, and then the local attention calculation is performed according to the deformed window. The four local attention calculation results are channel-spliced ​​to obtain a feature map that integrates local and global features. This process is as follows Figure 2 As shown, the calculation reference formula is as follows: ; ; Where, It is the output of the feature map after dynamic window deformation and heterogeneous local attention; Calculate heterogeneous local attention; is the layer normalization function; is the output of the feedforward neural network; It is a feedforward neural network; It is a dynamic window deformation module.

[0027] The dynamic deformation window mechanism module of this embodiment expands the window perception range through a learnable deformation process, so that the window can include the window edge objects before performing the attention calculation. This method allows the window to focus on distant areas through learning and flexibly determine the size and shape of the window according to the characteristics of the landslide image. The size and position of each window on the feature map are adjusted by the corresponding learning offset. and After deformation, the window will focus more on the main features related to the landslide and ignore other irrelevant information.

[0028] The dynamic deformation window mechanism can be found in Figure 3 , mainly includes the following steps: (1) Determine the prototype window. The feature map is divided into different windows according to each attention mechanism. The four local attention mechanisms include four window division methods. It should be noted that the size and position of the prototype window are fixed and non-overlapping. Among them, the division process of the dynamic deformation window module can be expressed as: ; Where: is the number of windows; The width of the predefined window; is the height of the predefined window; These windows are assembled according to the shape of the original feature map after partitioning, thus maintaining the original layout after assembly.

[0029] (2) Determine the deformation scale. After obtaining the prototype windows, these windows will obtain the offset and deformation scale through the learnable deformation prediction module, thereby completing the deformation of the window. This embodiment sets two learnable quantities to control the degree of window deformation. One is the ratio of window deformation and the displacement of window deviation. The ratio of window deformation and the displacement of window deviation are obtained through the deformation prediction module. The deformation prediction module consists of an average pooling layer, The activation layer consists of a 1×1 convolution layer with a stride of 1. The kernel size and stride of the pooling layer follow the prototype window size, so that each window will learn the specific value of the deformation scale during training.

[0030] The deformation scale is calculated as follows: ; Where, is the deformation scale; is the offset; is the convolutional layer; is the activation function; is the average pooling layer; ; is the number of feature map channels at different stages. Because both involve horizontal and vertical directions, and , the window can be calculated in the coordinate axis of the feature map.

[0031] (3) Window sampling. Since the shape of each window after deformation is different and cannot be predicted before training, it is almost impossible to perform self-attention calculations on these different shapes simultaneously. If forced to do so, it is necessary to cyclically obtain the degree of deformation and offset of these windows and then perform separate calculations. The calculation module will become very complicated and the time complexity will increase dramatically. Therefore, the window sampling technology is proposed. Specifically, before dividing the window, the upper left corner of the feature map is taken as the origin, and the right and downward directions are respectively Axis direction and Axis direction, record the two-dimensional coordinates of each pixel point, after dividing the window, each window also obtains its corresponding coordinate window. It is equivalent to generating a coordinate map of the same height and width for each prototype window. , After calculation, the coordinate points in the window are deformed and offset. The calculation process is shown in the following formula: ; Where, , Respectively The deformation scale and offset of each window; is the original horizontal coordinate; is the original vertical coordinate. and ,These windows are assembled according to the shape of the original feature map after ,partitioning, thus keeping the original layout after assembly.

[0032] This effectively maps the original window to a deformed window while maintaining the same number of pixels. Then, using sampling, the deformed pixels in the coordinate map are assigned to the original window, replacing the pixel values ​​in the original window. This allows for window deformation and displacement without changing the window size.

[0033] Based on the above process, it can be seen that the first attention window, the second attention window, the third attention window and the fourth attention window are obtained according to the following steps: through the average pooling layer, The deformation prediction module composed of the activation layer and the convolution layer determines the deformation ratio and offset of each preset window in the attention window according to the feature map, the first-stage feature map, the second-stage feature map, and the third-stage feature map; and deforms each preset window in the attention window according to the corresponding deformation ratio and offset to obtain the first attention window, the second attention window, the third attention window, and the fourth attention window, respectively.

[0034] After the dynamic window deformation module, Figure 4 As shown in FIG, heterogeneous local attention is introduced. The heterogeneous local attention calculation module is the core module of the landslide segmentation algorithm. Its main purpose is to enable the model to better extract landslide features through four different local attention mechanisms. For details of the heterogeneous local attention calculation module, please refer to the above description.

[0035] In the specific implementation, the heterogeneous local attention mechanism contains four modules, such as Figure 5 As shown in the figure, the first part is basic static local attention (i.e., the first attention mechanism), which means that the windows are stacked together in a non-overlapping manner like brick tiles, focusing on the texture consistency of the main landslide area. The second part is cross-domain expansion attention (i.e., the second attention mechanism), which requires all pixels at the same position in the window block to be grouped together for self-attention calculation to track the radial extension of the edge cracks. The next quarter is axial sparse divide-and-conquer attention (i.e., the third attention mechanism), in which column pixels arranged at the same interval form a set for local attention calculation. Similarly, the last quarter, vertical sparse divide-and-conquer attention (i.e., the fourth attention mechanism), will use windows combined at the same interval by row. The latter two are used to extract the directional characteristics of slope slip and layered peeling traces, respectively.

[0036] That is, the first attention mechanism performs attention calculations on all pixels in the first attention window to extract features from the feature map; the second attention mechanism performs attention calculations on pixels in the same position in multiple preset windows corresponding to the second attention window to extract features from the first-stage feature map; the third attention mechanism performs attention calculations on pixels in preset windows in fixed-arranged rows corresponding to the third attention window to extract features from the second-stage feature map; the fourth attention mechanism performs attention calculations on pixels in preset windows in fixed-arranged columns corresponding to the fourth attention window to extract features from the second-stage feature map.

[0037] Among them, the basic static local attention needs to decompose the feature map along the spatial dimension and divide it into non-overlapping windows. These windows fit tightly together like bricks to form a separate computational unit. The size of the tile is set to × The feature map is decomposed into The shape of self-attention is in a × Executed on the tile.

[0038] While basic static local attention eliminates the huge computational overhead of global attention and achieves linear complexity, its divided windows do not overlap and lack communication between windows. To supplement the global interaction capability of the attention mechanism, we propose cross-domain expanded local attention as a global enhancement based on basic static local attention.

[0039] Expanding local attention across domains allows the size of the computational unit to remain constant. × Cross-domain expanded local attention does not select pixels from continuous regions of the feature map, but rather selects pixels from the entire feature map, which performs attention in the spatial dimension. This embodiment specifies the matrix for attention × It should be noted that this matrix is ​​different from the block of the basic static local attention: although the number of pixels and the window shape in the matrix are the same, the pixels of the matrix are selected from the entire feature map. The computation unit of the basic static local attention is composed of a window composed of spatially adjacent pixels, while the pixels within a single window matrix in the cross-domain dilated local attention mechanism are from the same position in different windows under the basic static local attention mechanism. Divide the feature map into This approach effortlessly blends global labels within the matrix, where pixels in the blended window come from the same positions in the original window. Compared to the shifted window-based SWin, this attention mechanism is more intuitive and easier to implement without increasing the number of parameters or FLOPS.

[0040] Next, this embodiment introduces a direct and effective method to expand the scope of the local window globally, called axial sparse divide-and-conquer attention, which refers to selecting a group of rows arranged at a fixed distance to form the required window. Defined as the height of a window in the axial sparse divide-and-conquer attention. In order to obtain a set of axial pixels respectively, a row or column of pixels is selected at a fixed interval. The feature map is divided into Group, N Refers to the number of windows, H Refers to the height of the feature map. This model uses the window height Set to 1, 2, 7, 7 in four stages. The window size is At each stage, the feature map is decomposed into shape. At each stage, pixels within each window are selected on different rows of the feature map. Considering that the image shape has a divisor relationship with the number of window rows, in order to ensure that an integer number of windows is obtained, this embodiment uses image padding to meet the divisibility requirement. Compared with traditional local attention, the receptive field of each self-attention calculation unit is expanded to the global, effectively broadening the scope of information interaction.

[0041] Similar to the axial sparse divide-and-conquer attention window selection mechanism, Defined as the width of a window in vertical sparse divide-and-conquer attention. In vertical sparse divide-and-conquer attention, the feature map is divided into Group, M Refers to the number of windows, W Refers to the width of the feature map. Window width Set to 1, 2, 7, 7 in four stages. The window size is At each stage, the feature map is decomposed into shape. At each stage, pixels within each window are selected from different columns of the feature map. This approach effectively expands the scope of the local window without increasing computational complexity. Compared to traditional local attention, the receptive field of each self-attention computation unit is expanded to the global field, effectively broadening the scope of information interaction.

[0042] In one example, before using the heterogeneous local attention mechanism for the above processing, the initial feature map of the remote sensing image is first divided according to the channel dimension of the initial feature map to obtain multiple feature sub-maps; for each feature sub-map, the first attention mechanism, the second attention mechanism, the third attention mechanism, and the fourth attention mechanism are used to extract features (equivalent to self-attention in horizontal, vertical, tiled, and expanded windows).

[0043] In some embodiments, after using a heterogeneous local attention mechanism to perform the above-mentioned processing on the remote sensing image to obtain a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map, a spatial weight map for indicating the distribution of different landforms of landslides is generated through the fourth-stage feature map, and the spatial weight map is taken as a Haddam product with the initial feature map to obtain an enhanced feature map, and then the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map are concatenated in a cascade manner and fused with the enhanced feature map to obtain a fused feature map, so as to perform semantic segmentation on the remote sensing image based on the fused feature map.

[0044] In the specific implementation, after the encoder, which includes dynamic deformation windows and heterogeneous local attention, completes the work of the encoder, the output feature map of each stage encoder (an attention mechanism) first passes through the decoder's landslide feature extraction module (SFEM), which implements dynamic focusing on the landslide area by building a landslide weight extraction mechanism. The landslide feature extraction module takes high-level features as input and uses their clear semantics to combine with low-level features to recover landslide details, such as Figure 6 shown.

[0045] Specifically, a 1×1 convolution is applied to the feature map to compress the channel dimension to 1. A sigmoid function is then used to generate a spatial weight map with values ​​between [0, 1]. This weight map is essentially a probabilistic heat map, with high-response areas corresponding to pixel locations of key landforms such as landslide boundaries and cracks. Shallow features, which contain rich details such as edges and textures, can effectively identify the transition area between the landslide and the background. The nonlinear mapping of the sigmoid function converts fuzzy primary features into discriminative spatial weights. This weight map is then subjected to a Haddam product with the downsampled low-level features. This achieves targeted enhancement of deep semantic features, strengthening feature responses in areas with high landslide probability and suppressing redundant noise in background areas. Multi-scale features are then fused, and through cross-level fusion and segmentation, a progressive interaction of multi-level feature pyramids is achieved, completing the multi-scale modeling of landslide morphology.

[0046] The landslide feature extraction module cascades features from different stages: first, features from stages 4 and 3 are fed into the first-level landslide feature extraction module to generate mid-level features that incorporate landslide weights. This feature then interacts with features from stage 2 through the secondary landslide feature extraction module to extract enhanced features that balance semantic consistency and spatial accuracy. Finally, it is fused with the low-level detail features from stage 1 in the final landslide feature extraction module to form a multi-level joint representation covering the entire resolution. Finally, the enhanced features from all levels are channel-wise concatenated and fed into the segmentation head, where they are gradually restored to their original resolution through 3×3 convolutions and bilinear upsampling. Skip connections are used to preserve the discriminative information of each stage.

[0047] In some embodiments, a landslide dataset is first obtained and divided into a training set, a validation set, and a test set in proportion; the training set is used to train the model, and the test set is used to test the trained model. The above-mentioned dataset division can be random or divided according to a preset ratio, which is not limited in this embodiment. Then, a dynamic deformation window mechanism module is used to dynamically adapt and deform the self-attention window, and a heterogeneous local attention mechanism is used to calculate the feature map. Please refer to the above-mentioned processing process, which will not be repeated here. Then, through the landslide feature extraction module, dynamic focusing of the landslide area is achieved by constructing a landslide weight extraction mechanism. In addition, this embodiment uses the above-mentioned training set to train the landslide semantic segmentation model according to the batch-based stochastic gradient descent method. Finally, the trained model is used to perform semantic segmentation on the landslide test set to obtain the landslide area recognition result.

[0048] In summary, the semantic segmentation method for remote sensing of landslides of the present invention proposes a landslide segmentation algorithm based on dynamic deformation window and heterogeneous local attention, which can more effectively adapt to different regions and different types of landslides and capture the unified characteristics of landslides. After the network inputs the image, it first passes through the window deformation module to perform window deformation and division. The window deformation and division enable the local attention in the network to establish a complete dependence on the landslide target. Four different local attention mechanisms are used in the hybrid local attention module to extract different landslide features, which are then cascaded into the same feature map. The feature map then passes through a new progressive landslide feature aggregation decoder, and the landslide features are enhanced by the landslide feature extraction module, so that the model has a strong global modeling capability and is more adaptable to landslide segmentation in different scenes and different landforms.

[0049] The dynamic window deformation mechanism expands the window's perceptual range so that it can include objects at its edge before performing attention calculations. This allows the window to learn to focus on distant areas and flexibly determines the window's size and shape based on landslide image features.

[0050] The heterogeneous local attention module effectively expands the scope of the local window without increasing computational complexity. Compared with traditional local attention, the receptive field of each self-attention calculation unit is expanded to the global field, effectively broadening the scope of information interaction; enabling the model to simultaneously learn the spatial and channel features of the input image, thereby better segmenting the landslide region and accurately identifying the location of landslides in complex geographical environments.

[0051] The progressive landslide feature aggregation decoder convolves the output of the encoder at each stage as fine-grained features and fuses them into the coarse-grained features of the next stage. Before inputting the decoder, an upsampling module is used to assign the coarse-grained features to the fine-grained features of the next stage. Ultimately, the features of each stage have a global semantic perspective. Not only does the decoder input of each stage of the model have small-scale, coarse-grained semantic information, but it also obtains large-scale, fine-grained semantic information. The model has a larger semantic perspective that focuses on long-range dependencies, has better generalization to different terrains, and achieves the best effect on landslide segmentation.

[0052] Ultimately, after the feature pyramid decoder, the final feature map contains not only coarse-grained and fine-grained semantic information but also multi-scale global information, which makes this method more effective in landslide segmentation tasks.

[0053] The following tests are conducted on three landslide data sets, and the experimental results show that the present invention significantly improves the landslide area recognition capability of the model.

[0054] As shown in Table 1, a numerical comparison of precision, recall, F1-score, mean average accuracy (mACC), and mean intersection over union (mIoU) on the first landslide dataset, the proposed model is compared with other baseline methods in terms of precision, recall, F1-score, mACC, and mIoU on the landslide dataset. The benchmarks used for comparison include the publicly published ViT-CoMer, RMT, Segformer, SWin-Transformer, SeaFormer, and CSWin-Transformer. The "Small," "Base," "Tiny," "Large," "B4," and "B5" in the table below represent the specific specifications of the selected models. The quantitative results in Table 1 show that the proposed model significantly outperforms the other methods in all five metrics. The dataset contains a total of 1025 annotated frames. There are 820 training samples, accounting for about 80% of the total, and 96 and 109 samples are retained in the validation set and test set respectively, accounting for 10% each.

[0055] Table 1

[0056] The second landslide dataset contains 275 samples, 84% of which are used for training, and the remaining 16% for validation and testing. The remote sensing images primarily depict landslides caused by earthquakes in a specific region. The experimental results for this dataset are as follows: Table 2

[0057] The third landslide dataset manually segmented a total of 770 landslide samples from satellite image data, including rockfalls, landslides, and a small amount of rock slides. The samples were randomly divided into 8:1:1 ratios, with 616 cases retained in the training set, 77 cases in the validation set, and 77 cases in the test set. The experimental results are as follows: Table 3

[0058] Figure 7 The landslide areas of the first dataset identified by the above-mentioned baseline method and the landslide semantic segmentation model proposed in this paper are shown. Figure 7 It can be seen that the landslide semantic segmentation model proposed in the present invention has very good segmentation effect on landslides of different scales in various geographical environments, with clear edges and accurate recognition.

[0059] Figure 8 The segmentation results of different comparison methods and the method of the present invention on a second dataset are presented. The second landslide dataset is characterized by low image resolution, a very small proportion of landslide pixels in the image, and a small color difference between the landslide area and the background, which poses a significant challenge to the model segmentation. The method of the present invention is closest to the label in terms of shape. Visual comparison results show that the present method still performs well under conditions such as low image resolution, a very small proportion of landslide pixels in the image, and interference from external cloud and fog conditions.

[0060] Figure 9 The following figure shows the segmentation results of different comparison methods and the method of the present invention on a third landslide dataset. This third landslide dataset also has low image resolution. While the landforms are similar, the landslides have a wide variety of shapes, posing a significant challenge to the segmentation model. The two figures show the segmentation results of strip and square landslides, demonstrating the strong ability of the model to capture landslide shapes.

[0061] The segmentation results show that the proposed method provides the best control over the landslide area shape. Other compared methods misidentify landslides, resulting in overly large segmented areas. Similarly, adhesion is quite severe, with the proposed method being the closest to the label in terms of shape. Visual comparison results show that the proposed method exhibits lower image resolution, with landslide pixels dominating the image.

[0062] Visual comparison results demonstrate that the proposed method demonstrates robust segmentation capabilities across all three datasets, outperforming existing comparable methods overall. The datasets include diverse landslide scenarios, including deserts, Gobi deserts, mountain forests, farmland, and reservoirs, as well as various landslide types, including strips, bands, and blocks. The model maintains robust segmentation results even in complex environments, cloud and fog interference, and low-resolution conditions, demonstrating strong generalization and robustness.

[0063] The steps of the various methods above are divided only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are within the scope of protection of the present invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of the invention.

[0064] Another embodiment of the present invention relates to a semantic segmentation system for remote sensing of landslides. The implementation details of the semantic segmentation system for remote sensing of landslides in this embodiment are described in detail below. The following content is only provided for ease of understanding and is not required for the implementation of this solution. The semantic segmentation system for remote sensing of landslides in this embodiment includes: An image acquisition module is used to acquire remote sensing images of landslides and extract feature maps of the remote sensing images; A first feature extraction module is used to obtain an attention window composed of multiple preset windows, deform each preset window in the attention window according to the feature map to obtain a first attention window, and use the first attention mechanism to extract features from the feature map with the first attention window to obtain a first-stage feature map focused on the main texture of the landslide; A second feature extraction module is configured to deform each preset window in the first attention window according to the first-stage feature map to obtain a second attention window, and adopt a second attention mechanism to extract features from the first-stage feature map with the second attention window to obtain a second-stage feature map focused on the extension of the landslide edge; A third feature extraction module is used to deform each preset window in the second attention window according to the second stage feature map to obtain a third attention window, and adopt a third attention mechanism to extract features from the second stage feature map with the third attention window to obtain a third stage feature map focused on the direction of the landslide slope; a fourth feature extraction module, configured to deform each preset window in the third attention window according to the third stage feature map to obtain a fourth attention window, and employ a fourth attention mechanism to extract features from the third stage feature map using the fourth attention window to obtain a fourth stage feature map focused on the landslide stripping traces; The semantic segmentation module is used to combine the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on the remote sensing image and obtain the location of the landslide.

[0065] It is not difficult to find that this embodiment is a system embodiment corresponding to the above-mentioned method embodiment, and this embodiment can be implemented in conjunction with the above-mentioned method embodiment. The relevant technical details and technical effects mentioned in the above-mentioned embodiment are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above-mentioned embodiment.

[0066] It is worth noting that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovations of the present invention, this embodiment does not include units that are not closely related to solving the technical problems proposed by the present invention. However, this does not mean that other units do not exist in this embodiment.

[0067] Another embodiment of the present invention relates to a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the semantic segmentation method for remote sensing of landslides in the above-mentioned embodiments.

[0068] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0069] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0070] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0071] That is, those skilled in the art will understand that all or part of the steps in the above-described method embodiments can be implemented by instructing the relevant hardware through a program. The program is stored in a storage medium and includes a number of instructions for causing a device (such as a microcontroller or chip) or a processor to execute all or part of the steps in the method embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0072] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present invention, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A semantic segmentation method for landslide remote sensing, characterized in that: The method comprises: Obtain remote sensing images of landslides and extract feature maps of remote sensing images; An attention window consisting of multiple preset windows is obtained, each preset window in the attention window is deformed according to the feature map to obtain a first attention window, and a first attention mechanism is used to extract features from the feature map using the first attention window to obtain a first-stage feature map focused on the main texture of the landslide. According to the first-stage feature map, each preset window in the first attention window is deformed to obtain a second attention window, and a second attention mechanism is used to extract features from the first-stage feature map with the second attention window to obtain a second-stage feature map focused on the extension of the landslide edge. According to the second-stage feature map, each preset window in the second attention window is deformed to obtain a third attention window, and a third attention mechanism is used to extract features from the second-stage feature map with the third attention window to obtain a third-stage feature map focused on the direction of the landslide slope. According to the third stage feature map, each preset window in the third attention window is deformed to obtain a fourth attention window, and the fourth attention mechanism is used to extract features from the third stage feature map with the fourth attention window to obtain a fourth stage feature map focusing on the landslide peeling traces; Combining the first stage feature map, the second stage feature map, the third stage feature map and the fourth stage feature map, the remote sensing image is semantically segmented to obtain the location of the landslide.

2. The semantic segmentation method for landslide remote sensing according to claim 1, characterized in that: The first attention mechanism performs attention calculation on all pixels in the first attention window to extract features from the feature map; The second attention mechanism performs attention calculation on pixels at the same position in multiple preset windows corresponding to the second attention window to extract features from the feature map of the first stage; The third attention mechanism performs attention calculation on pixels of a preset window in a fixedly arranged row corresponding to the third attention window to extract features from the second stage feature map; The fourth attention mechanism performs attention calculation on the pixels of a preset window in a fixedly arranged column corresponding to the fourth attention window to extract features from the second stage feature map.

3. The semantic segmentation method for landslide remote sensing according to claim 1, characterized in that: The first attention window, the second attention window, the third attention window and the fourth attention window are obtained according to the following steps: Through the average pooling layer, The deformation prediction module, which consists of activation layers and convolutional layers, determines the deformation ratio and offset of each preset window in the attention window based on the feature map, the first-stage feature map, the second-stage feature map, and the third-stage feature map. Each preset window in the attention window is deformed by a corresponding deformation ratio and offset to obtain a first attention window, a second attention window, a third attention window, and a fourth attention window, respectively.

4. The semantic segmentation method for landslide remote sensing according to claim 1, characterized in that: The method of combining the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on the remote sensing image includes: The fourth stage feature map is used to generate a spatial weight map indicating the distribution of different landforms of landslides, and the spatial weight map is used to perform a Haddam product with the feature map to obtain an enhanced feature map. The first stage feature map, the second stage feature map, the third stage feature map and the fourth stage feature map are connected in series in a cascade manner and then fused with the enhanced feature map to obtain a fused feature map; Based on the fused feature map, the remote sensing image is semantically segmented.

5. The semantic segmentation method for landslide remote sensing according to claim 1, characterized in that: After extracting the feature map of the remote sensing image, the method further includes: According to the channel dimension of the feature map, the feature map is divided into multiple feature sub-maps; For each feature subgraph, the first attention mechanism, the second attention mechanism, the third attention mechanism and the fourth attention mechanism are used for feature extraction.

6. A semantic segmentation system for landslide remote sensing, characterized by: The system comprises: An image acquisition module is used to acquire remote sensing images of landslides and extract feature maps of the remote sensing images; A first feature extraction module is used to obtain an attention window composed of multiple preset windows, deform each preset window in the attention window according to the feature map to obtain a first attention window, and use the first attention mechanism to extract features from the feature map with the first attention window to obtain a first-stage feature map focused on the main texture of the landslide; A second feature extraction module is configured to deform each preset window in the first attention window according to the first-stage feature map to obtain a second attention window, and adopt a second attention mechanism to extract features from the first-stage feature map with the second attention window to obtain a second-stage feature map focused on the extension of the landslide edge; A third feature extraction module is used to deform each preset window in the second attention window according to the second stage feature map to obtain a third attention window, and adopt a third attention mechanism to extract features from the second stage feature map with the third attention window to obtain a third stage feature map focused on the direction of the landslide slope; a fourth feature extraction module, configured to deform each preset window in the third attention window according to the third stage feature map to obtain a fourth attention window, and employ a fourth attention mechanism to extract features from the third stage feature map using the fourth attention window to obtain a fourth stage feature map focused on the landslide stripping traces; The semantic segmentation module is used to combine the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map to perform semantic segmentation on the remote sensing image and obtain the location of the landslide.

7. A computer device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the semantic segmentation method for remote sensing of landslides as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the semantic segmentation method for landslide remote sensing according to any one of claims 1 to 5 is implemented.

Images