Frozen soil hot melt collapse identification method, device and system, and storage medium

Abstract

The invention discloses a frozen soil hot melt collapse identification method, device and system, and a storage medium. The method comprises the following steps: S1, acquiring permanent frozen soil image information of a preset area; s2, training a dual-encoder RTS extraction network according to the permanent frozen soil image information of the preset area to obtain an RTS recognition model; and S3, inputting the permanent frozen soil image information of the target area into the RTS identification model to carry out frozen soil hot melt collapse identification. By adopting the technical scheme of the invention, the RTS and the small RTS in a complex environment can be identified more accurately.

Description

Technical Field

[0001] The present invention belongs to the field of disaster monitoring technology, and in particular to a method and device, system, and storage medium for identifying frozen soil thermal thaw landslides. Background Art

[0002] In the permafrost area under the background of climate warming, thermal thaw landslide (RTS) disasters frequently occur, which will have a series of disaster effects on the ecological environment, infrastructure, etc. It can be seen that rapid and accurate identification of RTS is particularly important. However, the existing RTS deep learning identification methods mostly use the convolutional neural network (CNN) architecture method, which is difficult to accurately extract RTS in complex geographical and geological environments, and there are many problems of missed detection and false detection of small RTS. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and device, a system and a storage medium for identifying thermal thaw landslides in frozen soil.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A method for identifying frozen soil thermal thaw landslides, comprising:

[0006] Step S1, obtaining permafrost image information of a preset area;

[0007] Step S2, training a dual encoder RTS extraction network HUTNet according to the permafrost image information of a preset area to obtain an RTS recognition model; the dual encoder RTS extraction network HUTNet comprises: a CNN-based U-Net encoder for extracting local spatial features, a Swin Transformer-based Transformer encoder for extracting global context information, and a feature fusion module for cross-enhancing features from two encoders; the Transformer encoder uses a channel attention module CIM to replace the fully connected layer of the Swin Transformer block;

[0008] Step S3: input the permafrost image information of the target area into the RTS recognition model to identify permafrost thermal thaw landslides.

[0009] The present invention also provides a frozen soil thermal thaw landslide identification device, comprising:

[0010] An acquisition module is used to obtain permafrost image information of a preset area;

[0011] A training module is used to train a dual-encoder RTS extraction network HUTNet according to the permafrost image information of a preset area to obtain an RTS recognition model; the dual-encoder RTS extraction network HUTNet includes: a CNN-based U-Net encoder for extracting local spatial features, a Swin Transformer-based Transformer encoder for extracting global context information, and a feature fusion module for cross-enhancing features from two encoders; the Transformer encoder uses a channel attention module CIM to replace the fully connected layer of the Swin Transformer block;

[0012] The recognition module is used to input the permafrost image information of the target area into the RTS recognition model for permafrost thermal thaw landslide recognition.

[0013] An embodiment of the present invention further provides a system for identifying thermal thaw landslides caused by frozen soil, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a method for identifying thermal thaw landslides caused by frozen soil when executed by the processor.

[0014] An embodiment of the present invention further provides a storage medium having a computer program stored thereon, and the computer program executes the method for identifying thermal thaw landslides caused by frozen soil when the computer program is run.

[0015] The present invention adopts HUTNet, a complementary network of CNN and Transformer for RTS segmentation. In HUTNet, a dual encoder structure is designed: one is a U-Net encoding branch based on CNN, which is used to extract local spatial features; the other is a Transformer encoding branch based on Swin Transformer, which is used to extract global context information. In addition, the present invention designs a channel attention module to replace the MLP layer of the Swin transformer block to enhance its sensitivity to important channels. At the same time, the present invention designs a feature fusion module to cross-enhance the features from the two branch encoders. The technical effects of the present invention are as follows:

[0016] 1. Adopt an end-to-end dual-encoder RTS extraction network to fuse complementary features from different encoders;

[0017] 2. Construct a channel interaction module (CIM) to enhance Swin Transformer's ability to explicitly model channels and improve the model's ability to select the importance of features, making the model more robust when dealing with complex scenarios;

[0018] 3. In order to better integrate features from different architectures, a multi-strategy feature fusion module (MSFM) is designed to explore the potential correlation of cross-domain features from different angles and enhance the complementarity between features. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0020] Figure 1 This is a flow chart of a method for identifying thermal thaw landslides in frozen soil according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of the dual encoder RTS extraction network HUTNet;

[0022] Figure 3 It is a schematic diagram of the structure of the Swin Transformer encoder;

[0023] Figure 4 It is a schematic diagram of the structure of the channel interaction module (CIM);

[0024] Figure 5 It is a structural diagram of the feature fusion module MSFM. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Embodiment 1:

[0028] like Figure 1 As shown, the present invention provides a method for identifying thermal thaw landslides in frozen soil, comprising:

[0029] Step S1, obtaining permafrost image information of a preset area;

[0030] Step S2, training a dual encoder RTS extraction network HUTNet according to the permafrost image information of the preset area to obtain an RTS recognition model;

[0031] Step S3: input the permafrost image information of the target area into the RTS recognition model to identify permafrost thermal thaw landslides.

[0032] As an implementation method of the embodiment of the present invention, Figure 2 As shown in the figure, the dual encoder RTS extraction network HUTNet includes: a CNN-based U-Net encoder for extracting local spatial features, a Swin Transformer-based Transformer encoder for extracting global context information, and a feature fusion module MSFM for cross-enhancing features from the two encoders; the Swin Transformer encoder is formed by replacing the MLP layer of the Swin Transformer block with a channel attention module CIM; four U-Net encoding blocks are stacked to construct a CNN network, and each U-Net encoding block consists of two consecutive convolutions (Conv), batch normalization (BN), and rectified linear units (Relu). Two times the downsampling connection is used between blocks. Given an input image x∈(h,w,c), three feature maps U are obtained through the Unet encoder. i , whose size is (h / i,w / i,c*2 (i-1) ),i∈[1,4]. In this process, the feature map becomes smaller and deeper, and the information it contains is richer and more abstract.

[0033] The Swin Transformer encoder builds three feature extraction stages, each of which consists of two consecutive Swin Transformer blocks. The standard Swin Transformer is Figure 3 As shown, it consists of two consecutive sub-blocks. The first sub-block includes layer normalization (LN), window-based multi-head self-attention (W-MSA), fully connected layer (MLP) and residual connection. The structure of the second sub-block is basically the same, except that the W-MSA is replaced by the shifted window-based MSA (SW-MSA). In order to further improve the performance of Swin Transformer, the present invention uses CIM to replace the MLP layer to enhance its channel modeling ability. In order to adapt to the input requirements of Transformer, the input X is first flattened into sequence data through the patch embed operation, and then the feature map T is obtained through the Transformer block and patch merge operation. j , its size is (h / j,w / j,c*2 (j -1)), j∈[1,3]. Patch merge is a downsampling operation used by the Transformer architecture, which merges adjacent 2×2 patches and adjusts the number of channels by splicing and linear projection. Then, the feature map of each stage will be sent to MSFM and fused with the U-Net feature map of the corresponding stage. After the above encoding process, the feature map U 4 The fused feature map passed by the input decoder and the skip connection is concatenated in the channel direction, and the number of channels is reduced by a 3×3 convolution layer. Here, each convolution layer has a batch normalization layer and a ReLU layer. After the above process is performed three times, the feature U 4 The image is restored to its original resolution (128, 128), and finally the image is binarized through sigmoid to obtain the predicted mask.

[0034] Swin Transformer uses the MLP layer to capture richer feature information by expanding and compressing the feature dimension, thereby enhancing the model's ability to express complex features. However, the feature processing method of the MLP layer relies on static weights, lacks the ability to explicitly model the relationship between channels, and cannot effectively distinguish the importance of features. It may introduce redundant information, thereby affecting the final performance of the model and the focus on specific features. Therefore, the present invention uses a channel interaction module (CIM) to replace the MLP layer to enhance the Swin Transformer's explicit modeling ability for inter-channel features. Since each channel of the feature map is regarded as a feature detector, the goal of CIM is to dynamically determine "which" channels are most important in the current task in the context of a given input image. In order to effectively capture the relationship between channels, maximum pooling (MaxPool) and average pooling (AvgPool) are used in CIM to compress the feature map. The maximum pooling features and average pooling features can greatly improve the representation ability of the network. The CIM structure is as follows Figure 4 shown.

[0035] Specifically, given a feature map T of size h×w×c i, first mapped to the new feature space through a linear layer (Linear), and then after maximum pooling and average pooling operations, two weight vectors of size 1×1×c are obtained. These two weight vectors represent the importance of the channel at different feature scales, and they form a mixed weight vector by adding element by element. Use one-dimensional convolution (1D Conv) to further adjust the mixed weight value to generate a feature vector f of size 1×1×c, which represents the final weight estimate for all channels. Then, f will be multiplied by Ti. This process strengthens the important channel features and suppresses the unimportant channels. Finally, the residual connection transfers the original features to obtain the final result. The CIM processing flow is as follows:

[0036] w i 1 =1Dconv(MaxPool(Linear(T i ))+AvgPool(Linear(T i )))

[0037] T i 2 =Linear(Linear(T i )×w i 1 )+T i

[0038] Among them, 1Dconv represents one-dimensional convolution, MaxPool represents maximum pooling, AvgPool represents average pooling, and Linear represents the linear layer of increasing and decreasing dimensions.

[0039] Transformer was originally proposed to solve NLP tasks. Compared with the CNN method for visual tasks, the two have completely different feature extraction methods. The features of Transformer encoder and U-Net encoder are generated in different domains, and the semantic information obtained is also different. Therefore, a simple fusion strategy may not be able to fully tap their potential. In order to obtain more comprehensive information, the present invention designs a multi-strategy feature fusion module to complete the semantic interaction of different domains, such as Figure 5 As shown in Figure 2, this module consists of three parallel fusion methods with different fusion angles. Suppose the two-dimensional feature map and three-dimensional feature map from Transformer and U-Net are T i , U i , whose sizes are (h×w)×c and h×w×c respectively.

[0040] In the fusion stage 1, a cross-domain attention mechanism module is designed to enable each feature map to focus on the most relevant area from another domain. Through this cross-domain attention mechanism, effective interaction between features is promoted. First, the feature map U from the CNN encoder is i Reshape it to make it consistent with the feature map T of the Transformer encoder i Keep the two-dimensional size consistent, that is, (h×w)×c. Then, U i The query matrix Q is generated through the linear layer, and Ti generates the query matrix Q and the key matrix K through the linear layer respectively. The Q, K, and V matrices are the mappings of the corresponding feature maps in the new vector space, with the size of (h×w)×c, and these parameters are trainable. Subsequently, the query matrix Q is multiplied by the transpose of the key matrix K and divided by the scaling factor Then, the similarity matrix S is obtained through the softmax function, with a size of (h×w)×c. This matrix reflects the importance or relevance of different positions in the feature map. Next, the similarity matrix S is multiplied by the value matrix V to generate a cross-domain fusion feature matrix with a size of (h×w)×c, which represents the relevant information extracted by the query matrix Q in the CNN feature map from the Transformer feature map. Finally, the result is reshaped into a three-dimensional matrix of h×w×c to complete the entire cross-domain feature fusion process. The specific process is as follows:

[0041] Q=Linear(reshape(U i ))

[0042] [K,V]=Linear(T i )

[0043]

[0044] In the fusion stage 2, the feature map T i will be reshaped into h×w×c and then added to U i Do a dot multiplication operation to get the fusion feature Figure 2 . Doing dot multiplication between feature maps can be seen as a process of feature enhancement and information filtering. For certain specific pixel locations, if the features between feature maps are consistent (for example, at the edge or in areas with obvious texture), the dot multiplication operation will amplify the response of these features, thereby highlighting their importance; while for certain locations where the features are inconsistent, the dot multiplication operation will weaken the impact of these irrelevant or noise information. This mechanism can effectively improve the recognition and representation capabilities of features.

[0045] Swin Transformer establishes the relationship between patch tokens within a limited window, effectively reducing memory overhead. However, the strategy of alternating between regular windows and shifted windows will also cause a certain degree of loss in global modeling capabilities. In addition, the edges of thermal melt collapse are more complex, which requires some local spatial information to help identify. Therefore, in the fusion stage 3, consider replacing the feature map U i The spatial semantic information of T is given i , in order to further enhance information exchange while encoding more accurate spatial information. Specifically, for the feature map U i Apply the global average pooling operation to obtain its statistical information in the spatial direction (vertical and horizontal), and obtain two tensors h and w. h and w aggregate the pixel-level weights of the feature map in space, so the two are multiplied to obtain the position weight matrix E. Then the feature map Ti is reshaped into (h, w, c), and the dot multiplication operation is performed with the matrix E to obtain the Transformer feature map F that integrates the spatial encoding information of Unet 3 The specific process is as follows:

[0046] V h =AvgPool_h(U i )

[0047] V W =AvgPool_w(U i )

[0048]

[0049] Finally, the feature maps F1, F2, and F3 fused from the three stages are stacked in the channel dimension and reduced in dimension using 1*1 convolution to obtain the final fused feature map. The process is as follows:

[0050] F=1*1Conv(cat(F 1 +F 2 +F 3 ))

[0051] Embodiment 2:

[0052] The embodiment of the present invention further provides a frozen soil thermal thaw landslide identification device, comprising:

[0053] An acquisition module is used to obtain permafrost image information of a preset area;

[0054] A training module is used to train a dual-encoder RTS extraction network HUTNet according to the permafrost image information of a preset area to obtain an RTS recognition model; the dual-encoder RTS extraction network HUTNet includes: a CNN-based U-Net encoder for extracting local spatial features, a Transformer-based Swin Transformer encoder for extracting global context information, and a feature fusion module for cross-enhancing features from two encoders; the Swin Transformer encoder uses a channel attention module CIM to replace the MLP layer of the Swin Transformer block;

[0055] The recognition module is used to input the permafrost image information of the target area into the RTS recognition model for permafrost thermal thaw landslide recognition.

[0056] Embodiment 3:

[0057] An embodiment of the present invention further provides a system for identifying thermal thaw landslides caused by frozen soil, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a method for identifying thermal thaw landslides caused by frozen soil when executed by the processor.

[0058] Embodiment 4:

[0059] An embodiment of the present invention further provides a storage medium having a computer program stored thereon, and the computer program executes the method for identifying thermal thaw landslides caused by frozen soil when the computer program is run.

[0060] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for identifying frozen soil thermal thaw landslide, characterized in that: include: Step S1, obtaining permafrost image information of a preset area; Step S2, training a dual encoder RTS extraction network HUTNet according to the permafrost image information of a preset area to obtain an RTS recognition model; the dual encoder RTS extraction network HUTNet comprises: a CNN-based U-Net encoder for extracting local spatial features, a Swin Transformer-based Transformer encoder for extracting global context information, and a feature fusion module for cross-enhancing features from two encoders; The Transformer encoder uses a channel attention module CIM to replace the fully connected layer of the Swin Transformer block; Step S3: input the permafrost image information of the target area into the RTS recognition model to identify permafrost thermal thaw landslides.

2. A frozen soil thermal thaw landslide identification device, characterized in that: include: An acquisition module is used to obtain permafrost image information of a preset area; A training module is used to train a dual-encoder RTS extraction network HUTNet according to the permafrost image information of a preset area to obtain an RTS recognition model; the dual-encoder RTS extraction network HUTNet includes: a CNN-based U-Net encoder for extracting local spatial features, a Transformer-based Swin Transformer encoder for extracting global context information, and a feature fusion module for cross-enhancing features from two encoders; the Swin Transformer encoder uses a channel attention module CIM to replace the fully connected layer of the Swin Transformer block; The recognition module is used to input the permafrost image information of the target area into the RTS recognition model for permafrost thermal thaw landslide recognition.

3. A frozen soil thermal thaw landslide identification system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the method for identifying thermal thawing landslides of frozen soil as claimed in claim 1 is executed.

4. A storage medium, characterized in that: The storage medium stores a computer program, which executes the method for identifying thermal thaw landslides of frozen soil as claimed in claim 1 when running.

Images