Double-flow coding landslide automatic identification method based on improved U-Net

Abstract

The invention discloses a double-flow coding landslide automatic identification method based on improved U-Net, and the method achieves the precise extraction of a landslide through the multi-dimensional features of an optical wave band, a spectral index, a terrain index, a texture index and the like. In order to make full use of sequence features after multi-dimensional feature superposition, the encoder of the invention designs two branches, one branch constructs a residual encoding module to extract spatial dimension features, and the other branch constructs a multi-scale ResNet Module (MSRNM) and a long short-term memory (LSTM) to extract sequence dimension features. Designing a fusion module to effectively fuse the spatial dimension features and the sequence dimension features to obtain deep features; and the decoder is provided with an attention gating decoding module (AGDM), so that the expression of deep layer features and shallow layer features is enhanced, and the sensitivity of the model to the landslide is remarkably improved. The constructed double-flow coding landslide identification model has good space and sequence feature learning ability, and can finely identify various landslides.

Description

Technical Field

[0001] The present invention relates to the field of landslide detection, and in particular to, but not limited to, a dual-stream coding landslide automatic identification method based on improved U-Net. Background Art

[0002] Accurate identification of landslides is of vital importance for disaster prevention and mitigation. As a sudden geological disaster, landslides are often accompanied by natural phenomena such as rainstorms and earthquakes, causing huge losses of life and property. Accurate identification of landslides can not only provide early warning and reduce casualties when disasters occur, but also provide a scientific basis for the government and relevant departments to guide the construction of protective projects and emergency response. Especially in complex geological environments, landslides are of various types and sizes, and traditional monitoring methods are often difficult to cover. Therefore, through high-precision landslide identification technology, the monitoring efficiency and response capabilities of landslide disasters can be effectively improved, thereby reducing potential socioeconomic losses and protecting people's lives.

[0003] At present, landslide identification methods are mainly divided into traditional manual field surveys, semi-automatic identification based on machine learning, and automated landslide identification based on deep learning. The traditional method is mainly based on field manual surveys to update the landslide inventory list. The landslide data obtained in this way is relatively accurate, but it requires a lot of manpower and material resources and is inefficient. For complex landslides, the traditional method of quickly obtaining landslide information has certain limitations.

[0004] Although the semi-automatic landslide identification method based on machine learning has a high degree of automation, it requires a lot of feature selection and parameter debugging. This method is prone to misidentification of landslides in mountainous areas where the spectral features, geometric features, and texture features are significantly different from those of the background objects.

[0005] Existing landslide detection methods based on deep learning usually only consider the spatial dimension characteristics of landslide multi-source data, but ignore the sequence dimension characteristics after the superposition of multi-source landslide data. It is difficult to comprehensively summarize the diverse characteristics of landslides in complex scenes, resulting in insufficient identification accuracy of landslides of different types and scales, and many missed detections and false detections. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a dual-stream coding landslide automatic identification method based on improved U-Net, which at least solves the problem that the existing method has insufficient identification accuracy for landslides of different types and sizes, and has many missed detections and false detections.

[0007] The technical solutions of the embodiments of the present invention are as follows: The embodiment of the present invention provides a dual-stream coding landslide automatic identification method based on improved U-Net, the method comprising: Based on multi-source data, a multi-dimensional feature data set is constructed; the multi-source data includes landslide vector data, optical band data, spectral index data, terrain factor data and texture index data; a dual-stream coded landslide recognition model is built based on the U-Net framework, including an encoder and a decoder, the encoder includes two branches and a fusion module, wherein the first branch is used to extract spatial dimension features, and the second branch is used to extract sequence dimension features; the fusion module is used to fuse the spatial dimension features and the sequence dimension features to obtain deep features; the decoder includes an AGDM module, which is used to reconstruct the prediction results in the feature space from the high-dimensional features obtained from the encoding stage; the dual-stream coded landslide recognition model is trained using the feature data set in combination with a weighted loss function; the weighted loss function is a weighted sum of BCE loss and Dice loss; the trained dual-stream coded landslide recognition model is evaluated using a variety of evaluation indicators.

[0008] Furthermore, the first branch designs two convolutional layers and four residual coding layers; the input data is first projected into a high-dimensional feature space by the two convolutional layers, and then the features are further projected into a higher-dimensional space through the four residual coding layers to extract rich deep feature information; the kernel size of the two convolutional layers is 3×3, the step size is 1, and each convolutional layer is followed by a batch normalization layer and a ReLU activation layer; the four residual coding layers are composed of 3, 4, 6 and 3 residual blocks respectively. In the process of spatial dimension feature extraction, the number of feature map channels doubles and the size is halved each time a residual coding layer is passed.

[0009] Furthermore, each of the residual blocks includes three convolutional layers and three BN layers, a ReLU activation layer is respectively arranged after the first BN layer and the second BN layer, a residual connection is realized between the first convolutional layer and the third convolutional layer of the residual block trunk, and on the residual connection branch, the input vector and the output vector are identically mapped through 1×1 convolution.

[0010] Furthermore, the second branch includes a constructed MSRNM module and an LSTM module; wherein the MSRNM module includes 4 downsampling steps, each downsampling step includes a residual learning unit, and the residual learning unit is composed of two convolutional layers. In the process of sequence dimension feature extraction, the size of the feature map is halved after each residual learning unit; the LSTM module includes three layers of LSTM units to capture the sequence dependency between images in different bands.

[0011] Furthermore, in the fusion module, the spatial dimension feature and the sequence dimension feature are cascaded through a vector concatenation method to form a new feature variable; the new feature variable participates in the processing process of the decoder module through an upsampling operation to generate a final output.

[0012] Furthermore, the AGDM module includes four layers of convolution blocks. In the jump connection between the encoder and the decoder, an attention gating module is introduced to generate a gating signal to highlight the importance of features at different spatial positions. Finally, a 1×1 convolution layer with a Sigmoid classifier is used to infer the probabilities of landslide targets and backgrounds, and finally a landslide prediction result with the same size as the input data is obtained.

[0013] Furthermore, the weighted loss function is expressed as follows: ; L BCE =− 1 N ∑ i = 1 n [ y i log a p i + ( 1 − y i ) log a ( 1 − p i ) ] ; ; Where L is the weighted loss function, For BCE loss, is the Dice loss, is the loss weight coefficient, is the number of categories, n is the number of pixels, For the The label value of each pixel, For the The predicted probability value of pixels; represent The coefficients, TP is true positive, indicating that the model correctly predicts the landslide target, FN is false negative, indicating that the landslide target is incorrectly predicted as non-landslide, and FP is false positive, indicating that the non-landslide target is incorrectly predicted as landslide.

[0014] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least: In an embodiment of the present invention, multidimensional features such as optical bands, spectral indices, terrain indices, and texture indices are used to accurately extract landslides. In order to make full use of the sequence features after the superposition of multidimensional features, the present invention builds a dual-stream coding landslide recognition model based on the U-Net framework. The model encoder is designed with two branches, which are used to extract spatial dimension features and sequence dimension features respectively. Then, a fusion module is designed to effectively fuse spatial dimension features and sequence dimension features to obtain deep features. Finally, the high-dimensional features obtained from the encoding stage are reconstructed into prediction results in the feature space through the decoder module. The model has good spatial and sequence feature learning capabilities and can accurately identify various landslides. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 A schematic flow chart of a method for automatic landslide identification based on dual-stream coding using an improved U-Net provided in an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a dual-stream encoding landslide identification model provided in an embodiment of the present invention; wherein part (a) is the input multi-source landslide data; part (b) is the spatial feature extraction module; part (c) is the sequence feature extraction module; and part (d) is the deep feature decoding module; Figure 3 A schematic diagram of the encoding process of the spatial feature extraction branch in the dual-stream encoding landslide recognition model provided in an embodiment of the present invention; wherein part (a) is a schematic diagram of a two-layer ordinary convolution and a four-layer residual convolution structure and a processing flow diagram thereof, and part (b) is a schematic diagram of a residual block structure and a processing flow diagram thereof; Figure 4 A schematic diagram of the encoding process of the sequence feature extraction branch in the dual-stream encoding landslide recognition model provided in an embodiment of the present invention; wherein part (a) is a schematic diagram of sequence feature extraction, part (b) is a schematic diagram of residual block processing with a step size equal to 2, and part (c) is a LSTM network structure diagram; Figure 5 A schematic diagram of the structure of an attention gated decoding module provided in an embodiment of the present invention; Figure 6 A schematic diagram of the identification result of a long and narrow landslide provided in an embodiment of the present invention; Figure 7 A schematic diagram of equidistant landslide identification results provided by an embodiment of the present invention; Figure 8 A schematic diagram of the identification result of a wide-spread landslide provided in an embodiment of the present invention; Fig. 9 A schematic diagram of a large-scale landslide identification result provided by an embodiment of the present invention; Fig.10 A schematic diagram of mesoscale landslide identification results provided by an embodiment of the present invention; Fig.11 A schematic diagram of a small-scale landslide identification result provided by an embodiment of the present invention; Fig.12 Visualization results of ablation of different types of landslide data provided by the embodiments of the present invention; Fig.13 The ablation visualization results of landslide data of different scales provided by the embodiment of the present invention; Fig.14 A schematic diagram of the model ablation implementation result provided by an embodiment of the present invention; Fig.15 This is a graph of loss weight analysis experimental results provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0017] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0018] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described here can be implemented in an order other than that illustrated or described here.

[0019] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the field to which the embodiments of the present invention belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0020] Figure 1 A flowchart of a dual-stream coding landslide automatic identification method based on improved U-Net provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method comprises at least the following steps: Step S110, constructing a multidimensional feature data set based on multi-source data; the multi-source data includes landslide vector data, optical band data, spectral index data, terrain factor data and texture index data.

[0021] Here, the landslide vector data, after being processed as label data, provides the definition of the true category of each pixel in the sample image, providing a reference for the subsequent training and testing of the dual-stream encoded landslide recognition model. Optical band data, spectral index data, terrain factor data, and texture index data are used as image data for landslide recognition, providing the visual information required by the model, enabling the model to understand the semantic content in the image and classify the image at the pixel level.

[0022] Step S120, building a dual-stream encoding landslide recognition model based on the U-Net framework, including an encoder and a decoder, the encoder including two branches and a fusion module, wherein the first branch is used to extract spatial dimension features, and the second branch is used to extract sequence dimension features; the fusion module is used to fuse the spatial dimension features and the sequence dimension features to obtain deep features; the decoder includes an AGDM module, which is used to reconstruct the prediction results in the feature space from the high-dimensional features obtained in the encoding stage.

[0023] Here, the architecture of the dual-stream coding landslide identification model is as follows Figure 2 As shown in the figure. The model follows the U-Net process and consists of two parts: encoder and decoder. The encoder is designed with two branches. The first branch constructs a residual coding module to extract spatial dimension features. The second branch constructs a multi-scale residual network (Multi-Scale ResNet Module, MSRNM) and a long short-term memory network (Long Short-Term Memory, LSTM) module to extract sequence dimension features. Then, a fusion module is designed to effectively fuse spatial dimension features and sequence dimension features to obtain deep features. The decoder designs an attention gate decoder module (Attention Gate Decoder Module, AGDM) to enhance the expression of deep and shallow features, significantly improving the model's sensitivity to landslides. Finally, a convolutional layer with a Sigmoid classifier is used to infer landslide targets and non-landslide targets to obtain the final landslide prediction results.

[0024] The first branch, namely the spatial dimension encoding branch, needs to fully extract the spatial correlation information of multi-source landslide data to enhance the proposed model's ability to understand and analyze the input data in space. The second branch, namely the pixel sequence dimension feature extraction branch, focuses on learning the mutual constraints between different landslide factor sequences and learning the relationship between different landslide factors in the pixel dimension. Then, a fusion module is designed to effectively fuse the spatial dimension features and sequence dimension features to obtain deep features. Thus, a dual-stream encoding landslide fine automatic recognition framework that takes into account the spatial dimension and sequence dimension features of multi-source remote sensing data is constructed, which can comprehensively summarize the diverse features of landslides in complex scenes and is suitable for landslide recognition scenes of different types and scales.

[0025] Step S130, using the feature data set in combination with a weighted loss function to train the dual-stream encoding landslide recognition model; the weighted loss function is a weighted sum of the BCE loss and the Dice loss.

[0026] Here, the loss function is a function used to measure the difference between the predicted output and the true label in the process of deep learning model optimization. Its main function is to guide the parameter update of the model and gradually approach the best prediction result by minimizing the loss function value. Depending on the specific task, there are a variety of loss functions to choose from. Commonly used loss functions in landslide identification tasks include Binary Cross-Entropy (BCE), Dice loss, Focal loss and Tversky loss. Among them, BCE loss gives the same weight to all samples and has high stability, but when dealing with highly unbalanced classification tasks (such as landslide and non-landslide areas in landslide identification), it will cause the model to pay more attention to the background class with a larger proportion, ignore the landslide area, and reduce the overall performance of the model. Although Dice loss can handle unbalanced data, for complex boundaries or small target categories, the model may find it difficult to accurately predict their morphology. Therefore, the application of a single loss function is difficult to meet the requirements for accuracy and small target detection in landslide identification tasks. To this end, during the training process, the embodiment of the present invention adopts a weighted loss function L, combined with BCE loss and Dice loss for full supervised learning.

[0027] Step S140, performing model evaluation on the trained dual-stream encoding landslide recognition model by using a variety of evaluation indicators.

[0028] Here, in order to compare and evaluate the performance of the proposed dual-stream encoding landslide recognition model in the landslide extraction task, widely used evaluation indicators such as precision, recall, F1-score and mean intersection over union (MIoU) are used for performance evaluation.

[0029] In an embodiment of the present invention, multidimensional features such as optical bands, spectral indices, terrain indices, and texture indices are used to accurately extract landslides. In order to make full use of the sequence features after the superposition of multidimensional features, the present invention builds a dual-stream coding landslide recognition model based on the U-Net framework. The model encoder is designed with two branches, which are used to extract spatial dimension features and sequence dimension features respectively. Then, a fusion module is designed to effectively fuse spatial dimension features and sequence dimension features to obtain deep features. Finally, the high-dimensional features obtained from the encoding stage are reconstructed into prediction results in the feature space through the decoder module. The model has good spatial and sequence feature learning capabilities and can accurately identify various landslides.

[0030] In some embodiments, the above step S110 "constructing a multidimensional feature data set based on multi-source data" is further implemented through the following process: the acquired optical band data, spectral index data, terrain factor data, texture index data and landslide vector data are respectively cropped into 256×256 sub-images with a step size of 128 pixels, and the remaining part of the cropped image is filled to obtain a sub-image data set; the sub-image data set is expanded using a data enhancement strategy to obtain a feature data set as a training sample and its corresponding real landslide label.

[0031] Here, considering the scale of the landslide in the experimental scene and the experimental hardware limitations, the embodiment of the present invention sets the size of each image in the multidimensional feature data set of the input model to 256×256. During the training process, in order to increase the number and diversity of samples and improve the generalization ability of the model, the cropped images and the real landslide sample images can be mirrored, flipped and rotated (90° counterclockwise). Subsequently, the multidimensional feature data set obtained above is randomly divided into a training set, a validation set, and a test set in a ratio of 7:2:1, where the training set is used for model training, the validation set is used to adjust and select the optimal parameters of the model, and the test set is used for quantitative performance evaluation of the model.

[0032] In some embodiments, the first branch designs two convolutional layers and four residual coding layers; the input data is first projected into a high-dimensional feature space by the two convolutional layers, and then the features are further projected into a higher-dimensional space through the four residual coding layers to extract rich deep feature information; the convolution kernel size of the two convolutional layers is 3×3, the step size is 1, and each convolutional layer is followed by a batch normalization layer and a ReLU activation layer; the four residual coding layers are composed of 3, 4, 6 and 3 residual blocks respectively. In the process of spatial dimension feature extraction, the number of feature map channels doubles and the size is halved each time a residual coding layer is passed.

[0033] It is worth noting that the encoder of the original U-Net is composed of a series of stacked convolutional layers and pooling layers, and the structural design is relatively simple. When the input data set is large or the input data contains complex features, the U-Net encoder may be underfitting and unable to fully capture the complex features in the input data. However, obtaining more comprehensive image detail features is crucial for landslide extraction. Therefore, in order to improve the model's ability to extract the entire context detail features during the feature extraction process and prevent the problem of training degradation as the depth of the network layer increases. Based on the U-Net framework, the present invention designs a new encoder backbone to extract spatial dimensional features. The specific encoding structure is as follows: Figure 3 As shown in part (a), the two convolutional layers are collectively represented as Layer0, and the four residual coding layers are Layer1, Layer2, Layer3, and Layer4 respectively.

[0034] In some embodiments, each of the residual blocks includes three convolutional layers and three batch normalization (BatchNormalization, BN) layers, and a ReLU activation layer is set after the first BN layer and the second BN layer respectively, and a residual connection is realized between the first convolutional layer and the third convolutional layer of the residual block trunk, and on the residual connection branch, the input vector and the output vector are identically mapped through 1×1 convolution.

[0035] Here, the j+1th residual block (j=1,2,...,5) optimizes the training progress by considering the residual vector of the jth input. Figure 3 As shown in part (b) of the figure. The mathematical expression of the residual module is: ; in, and are the input and output vectors of the jth residual block, respectively, and the function represents the residual mapping to be learned.

[0036] In some embodiments, the second branch includes a constructed MSRNM module and an LSTM module; wherein the MSRNM module includes 4 downsampling steps, each downsampling step includes a residual learning unit, and the residual learning unit is composed of two convolutional layers. During the sequence dimension feature extraction process, the feature map size is halved after each residual learning unit; the LSTM module includes three layers of LSTM units to capture the sequence dependency between images in different bands.

[0037] Here, the second branch network structure is as follows Figure 4 As shown in the figure, for the 16 landslide image datasets obtained, the MSRNM module is first used to extract features and reduce spatial dimensionality, and a multidimensional feature map is output. Then the obtained multidimensional feature map is flattened, and the feature dimension is reduced to one dimension to adapt to the feature sequence input by the LSTM network module, which is in pixels. Finally, the constructed LSTM network module is used to extract sequence features between different data sources. Through this branch, the sequence features of relatively key images can be extracted, which can continuously affect the model's judgment of the target area, and thus obtain more significant features that are more conducive to landslide identification.

[0038] The number of neurons in each LSTM layer is set to 32, and other parameters remain unchanged. In addition, during the model training process, the batch size will affect the optimization degree and training speed of the model. After multiple experimental verifications, as well as the hardware conditions and data set size of this experiment, the batch size of each LSTM is set to 8.

[0039] In some embodiments, the spatial dimension feature and the sequence dimension feature are cascaded in the fusion module through a vector concatenation method to form a new feature variable; the new feature variable participates in the processing process of the decoder module through an upsampling operation to generate a final output.

[0040] Here, for the spatial dimension features and sequence dimension features extracted from the two branches, the feature extraction results of different branches are fused from the perspective of one dimension and two dimensions. The embodiment of the present invention uses a vector splicing method to achieve this. The specific expression is as follows: E C oncat = [ E Resnet , E LSTM ] ; in, Represents the new feature vector after concatenation, and Respectively represent the feature vectors extracted by the Resnet network and the LSTM network.

[0041] In some embodiments, the AGDM module includes four layers of convolution blocks. An attention gating module is introduced in the jump connection between the encoder and the decoder to generate a gating signal to highlight the importance of features at different spatial positions. Finally, a 1×1 convolution layer with a Sigmoid classifier is used to infer the probabilities of landslide targets and backgrounds, and finally a landslide prediction result with the same size as the input data is obtained.

[0042] Here, the decoder module aims to reconstruct the prediction results in the feature space from the high-dimensional features obtained in the encoding stage. The model proposed in the present invention designs four layers of convolution blocks. In the landslide dataset, the non-landslide background occupies most of the image. These background information will have an adverse effect on the training process of the model, resulting in false detection and missed detection, thereby reducing the accuracy of landslide identification. Therefore, in order to solve the impact of the non-landslide background on the model learning process, the embodiment of the present invention introduces an attention gate module (Attention Gate Module, AGM) in the jump connection between the encoder and the decoder. The AGM module generates a gating signal to highlight the importance of features at different spatial positions, so that the model can automatically focus on landslide targets of different shapes and sizes. At the same time, the AGM module can suppress areas in the feature map that are not related to landslides, highlight significant features that are helpful for landslide identification, enhance the fusion ability of deep and shallow features, and improve the overall recognition performance. The detailed process is as follows. Figure 5 shown.

[0043] In the AGM module structure, each pixel The attention values ​​are calculated separately, and the gated signal determines the area of ​​the model's attention pixel by pixel. To achieve higher accuracy, the structure uses an additive attention mechanism to obtain the attention coefficient. The additive formula can be expressed as: ; ; In the formula, represents the additive formula symbol, and is the weight, and is the bias, Indicates that the input feature map is operated through a 1×1 convolution kernel. , is the activation function, , They represent the deep features from the decoder and the shallow features from the encoder respectively, and α represents the attention coefficient matrix obtained by upsampling operation. The sampling method of the attention coefficient is cubic linear interpolation. In addition, the update of AGM parameters is obtained according to back-propagation training.

[0044] In some embodiments, the weighted loss function is expressed as follows: ; L BCE =− 1 N ∑ i = 1 n [ y i log a p i + ( 1 − y i ) log a ( 1 − p i ) ] ; ; Where L is the weighted loss function, For BCE loss, is the Dice loss, is the loss weight coefficient, is the number of categories, n is the number of pixels, For the The label value of each pixel, For the The predicted probability value of pixels; represent The coefficients, TP is true positive, indicating that the model correctly predicts the landslide target, FN is false negative, indicating that the landslide target is incorrectly predicted as non-landslide, and FP is false positive, indicating that the non-landslide target is incorrectly predicted as landslide.

[0045] This weighted loss function combines the stability of BCE loss and the advantages of Dice loss in solving sample imbalance. By giving higher weights to landslide areas, this weighted loss function can guide the model to pay more attention to landslide targets during training, avoid them being overwhelmed by background classes, and ensure accurate segmentation of landslide areas.

[0046] The above-mentioned dual-stream coding landslide automatic identification method based on improved U-Net is described below in conjunction with a specific embodiment. However, it is worth noting that this specific embodiment is only for better illustrating the present invention and does not constitute an improper limitation to the present invention.

[0047] Firstly, the Bailong River Basin was selected as the area for model training and testing, and its diverse landslide types and complex geographical and geological conditions were used to verify the applicability and extraction accuracy of the model.

[0048] The data used in the present invention include landslide vector data, optical band data, spectral index data, terrain factor data and texture index data. The landslide vector data is processed as label data, providing a definition of the true category of each pixel in the image, and providing a reference for the training and testing of the subsequent deep learning model. Band, spectrum, terrain and texture data are used as image data for landslide identification, providing the visual information required by the model, enabling the model to understand the semantic content in the image and classify the image at the pixel level. The specific description of the multidimensional feature data set is as follows.

[0049] (1) Landslide vector data: Based on historical landslide data and field investigation, the present invention visually interpreted 773 landslide vector boundaries on Google Earth. The interpretation results were then converted into raster data and exported as binary images in PNG format (1 represents landslide area, 0 represents non-landslide area), thereby obtaining real landslide samples as label data for training the dual-stream encoded landslide recognition model.

[0050] (2) Sentinel-2A data: The optical remote sensing data of the embodiment of the present invention comes from the cloud-free Sentinel-2A data of a specific date from ESA. Among the 12 bands of optical data obtained after data preprocessing, bands 2, 3, 4, and 8 with a spatial resolution of 10 m (meters) are resampled, cropped according to the experimental scene, and finally exported to PNG format.

[0051] (3) Spectral index data: By analyzing the changes in spectral index in optical remote sensing images before and after the landslide occurs, it can help identify and extract the landslide boundary. When a landslide occurs, the soil exposure, vegetation loss, and soil moisture changes around it will cause the landslide location to differ from the surrounding objects, thereby causing changes in the spectral index in the optical remote sensing image. The spectral indexes selected in the embodiment of the present invention are the Normalized Red Band (NR), the Normalized Green Band (NG), the Normalized Blue Band (NB), and the Normalized Green-Red Difference Index (NGRDI). Then, the calculation results are resampled and exported in PNG format.

[0052] (4) Terrain factor data: Terrain factors have an important influence on the formation and distribution of landslides and can be used as auxiliary data for landslide identification and extraction. In the embodiment of the present invention, digital elevation model (DEM), slope and aspect data are selected as terrain data. Among them, the ALOS 12.5m DEM data is resampled to the same resolution as the slope and aspect and then exported as PNG; the slope and aspect are calculated from the DEM and resampled and then exported as PNG. Through the above steps, the DEM, slope and aspect image data in the experimental scene are obtained.

[0053] (5) Texture index data: The destruction and movement of the surface caused by landslides will cause significant changes in texture characteristics. For example, landslides cause the soil, rocks and other materials on the surface to move and mix, resulting in different texture characteristics in originally similar surface areas. These changes can be used as one of the important indicators for identifying landslides. The texture indices selected in the embodiment of the present invention are correlation, entropy, homogeneity, mean and angular second moment. Sentinel-2A data is used to calculate and generate texture index data based on the Grey Level Co-occurrence Matrix (GLCM) in ENVI 5.6 software, and then it is resampled and exported in PNG format. Through the above steps, the texture index image data in the experimental scene is obtained.

[0054] In complex geographical and geological environment scenes, landslide features are diverse and difficult to identify. Existing methods usually only consider the spatial dimension characteristics of multi-source landslide data, but ignore the sequence dimension semantic characteristics after the superposition of multi-source landslide data. It is difficult to comprehensively summarize the diverse characteristics of landslides in complex scenes, resulting in insufficient recognition accuracy of landslides of different types and sizes, and many missed detections and false detections.

[0055] The present invention designs a dual-stream coded landslide precise automatic recognition framework that takes into account the spatial dimension and sequence dimension characteristics of multi-source remote sensing data. The architecture of the constructed dual-stream coded landslide recognition model is as follows: Figure 2 As shown. The model is trained and evaluated on the multidimensional feature dataset of the Bailong River Basin obtained above. Taking into account the scale of the landslide in the experimental scene and the limitations of the experimental hardware, the size of the input image is set to 256×256 in the present invention. The acquired optical image, spectral index image, terrain factor image, texture index image and real landslide label image are cropped into 256×256 sub-images with a step size of 128 pixels, and the remaining part of the cropped image is filled. During the training process, in order to increase the number and diversity of samples and improve the generalization ability of the model. The present invention uses a data enhancement strategy to expand the cropped sub-image dataset, and mirrors, flips and rotates (90° counterclockwise) the cropped images and real landslide sample images. Through these operations, 3271 training samples and their corresponding real landslide labels are finally obtained. Subsequently, the training samples obtained above were randomly divided into training set: validation set: test set in a ratio of 7:2:1, that is, 2290 images were used for model training, 654 images were used to adjust and select the optimal parameters of the model, and 327 images were used for quantitative performance evaluation of the model.

[0056] Then, the training parameters are set; the proposed dual-stream encoding landslide recognition model is implemented on a single NVIDIA 3090 graphics processor (GPU) using the PyTorch deep learning framework with 24GB of GPU memory. During the training process, the batch size is set to 8 and the initial learning rate is λ0= , and the Adam optimizer was selected to adaptively adjust the learning rate, and the maximum number of training rounds (epochs) was set to 200.

[0057] Finally, the experimental results and analysis are carried out, including the model performance analysis and the analysis of the identification results of different types of landslides.

[0058] In the experiment, some of the current mainstream landslide identification methods were compared. The main deep learning models compared include U-Net, ResU-Net, Attention-Unet, TransUNet, DeeplabV3+ and SegNet. By comparing with the above network models, the accuracy advantage of the improved algorithm in landslide identification is proved. The experimental results are shown in Table 1, where the model proposed in the present invention is the above dual-stream encoding landslide identification model.

[0059] Table 1 Evaluation index table of landslide identification results of different models ; The results show that the proposed dual-stream encoding landslide recognition model is the best in four evaluation indicators, including precision, recall, F1-score and mean intersection over union (MIoU), and is 2.93%, 2.88%, 2.92% and 4.62% higher than other models respectively. Among them, the MIoU indicator is significantly higher than other models, proving that the proposed model has excellent segmentation ability and can accurately identify landslide and non-landslide areas in images.

[0060] At the same time, the model proposed in this invention and the above-mentioned comparison model are used to compare and analyze the identification results of landslides of different types and sizes, and the identification results of three different types of landslides, namely, narrow and long, equidistant and wide, are obtained. Figures 6 to 8 As shown in the figure, the identification results of landslides of different scales, large scale, medium scale and small scale, are shown in the figure. Figures 9 to 11 As shown in the figure, GT (Ground Truth) represents the true value label.

[0061] Finally, in order to systematically verify the effectiveness of the model proposed in the present invention for multi-source data input, the present invention uses four groups of ablation experiments, taking the optical band data as the benchmark, and gradually introduces spectral index data, terrain factor data and texture index data to conduct ablation experiments on the dual-stream coded landslide recognition model. At the same time, the performance of the dual-stream coded landslide recognition model on different data combinations is quantitatively evaluated using four indicators: accuracy, recall rate, F1 value and average intersection-over-union ratio, in order to evaluate the robustness of the proposed dual-stream coded landslide recognition model for multi-source data input. Different types of landslides and different scales of landslides were selected for analysis, and the experimental results are shown as follows. Fig.12 and Fig.13As shown, T1 represents the use of only optical image data, T2 represents the addition of spectral index data on the basis of T1, T3 represents the addition of terrain factor data on the basis of T2, and T4 represents the addition of texture index data on the basis of T3. It can be seen that the proposed model uses different data sources and can better identify landslides of different types and sizes, indicating that the model proposed in the present invention has strong generalization and good robustness for different data sources. However, in complex scenarios, adding data sources can supplement landslide features, especially the recognition ability of small landslides and marginal areas is significantly improved, and the recognition accuracy of landslide boundaries and small changes is improved. Overall, the gradual introduction of spectral index, terrain factor and texture index effectively enhances the extraction ability of the model and meets the needs of complex terrain.

[0062] The four sets of experiments were designed as follows: Group 1: using only optical images (T1); Group 2: Adding spectral index (T2) to the first group of experiments; The third group: adding terrain factors to the second group of experiments (T3); Group 4: Adding texture index (T4) to the third group of experiments; The comprehensive analysis of quantitative indicators of the four groups of experimental results shows that compared with the baseline model using only optical images, the model has significantly improved in evaluation indicators such as precision, recall rate, F1 value and average intersection-over-union ratio after gradually adding spectral index, terrain factor and texture index, as shown in Table 2. Especially in the fourth group of experiments, the precision and average intersection-over-union ratio of the dual-stream coded landslide recognition model exceeded 80%, indicating that multi-source data fusion significantly enhanced the model's perception of landslide areas. The spectral index improves the response ability of the dual-stream coded landslide recognition model to surface changes, the terrain factor strengthens the capture of the relationship between landslide and terrain features, and the texture index further enhances the recognition ability of small landslide areas and complex boundaries. In summary, the present invention has systematically verified through ablation experiments that the dual-stream coded landslide recognition model proposed in the present invention has good generalization and robustness for different data sources. At the same time, the gradual introduction of multi-source data improves the performance of the proposed model. It further illustrates that the proposed dual-stream coded landslide recognition model can extract richer features with the increase of input data, and the model has strong adaptability.

[0063] Table 2 Landslide identification indexes of different data combinations ; In order to verify the role of each module of the proposed model design in improving the accuracy of landslide identification, the present invention conducted four groups of ablation experiments on the landslide extraction task to analyze the contribution of each structure in landslide extraction.

[0064] Group 1: Using the U-Net baseline model.

[0065] The second group: Use the designed residual module to replace the U-Net encoder part.

[0066] Group 3: Add the attention gating module based on the second group of experiments.

[0067] Group 4: Based on the third group of experiments, a sequence feature extraction module branch is added to the encoding part (equivalent to the second branch mentioned above, including a multi-scale residual network and an LSTM network).

[0068] Fig.14 The landslide extraction results of two scenes are shown, and the quantitative evaluation refers to the selection of the average intersection-union ratio for evaluation. From the visualization results, the U-Net basic model has problems such as inaccurate edge detection and missed detection when extracting landslide edges in complex terrain areas or landslides with smaller areas. After the introduction of the residual module, the overall performance has been improved compared to the U-Net model, which has improved the model in capturing the overall contour and boundary details of the landslide area. Because the residual module improves the compactness of feature expression by reducing the number of channels of the feature map, it reduces the amount of calculation of the model and enhances the model's ability to capture important features. However, despite the improved performance, the model still has certain limitations when dealing with highly complex landslide scenes, especially in the accurate extraction of small areas, such as Fig.14 As shown in the red circle in the middle. On this basis, the attention gating module is further introduced. By dynamically adjusting the model's attention to the characteristics of different regions, the model's sensitivity to key areas is significantly improved when dealing with landslide boundaries and local features. The experimental results show that the introduction of the attention gating module makes the model particularly outstanding in the identification of landslide boundary areas. Compared with the first two groups of models, the attention gating module enhances the model's ability to capture landslide details, reduces false detection and missed detection, and the model also responds better to subtle landslide features in complex scenes. Finally, a sequence feature extraction module branch is added to the encoding part, that is, the dual-stream encoding landslide recognition model proposed in the present invention. The experimental results show that after the introduction of the sequence feature extraction module branch, the model's sensitivity to the sequence relationship and local details of multi-source data input is enhanced, so that the dual-stream encoding landslide recognition model can not only better capture the spatial landslide characteristics, but also better mine the sequence features between different data sources, thereby Fig.14 It can be seen that in the first three groups of experiments, where it is difficult to accurately identify the boundary areas and small landslides, the dual-stream coding landslide identification model has obtained relatively reliable identification results ( Fig.14Green circles in the middle). Quantitative indicators are shown in Table 3. With the gradual introduction of different modules, the average intersection-and-union ratio gradually increases to 81.74%. It can be seen that each module in the dual-stream coding landslide identification model proposed in the present invention has a gain effect on the model. It further illustrates that the framework design of the dual-stream coding landslide identification model proposed in the present invention is reasonable, the performance is good, and it can be used for the rapid extraction of landslides in complex environments.

[0069] Table 3 Model ablation experiment results ; In order to meet the requirements of accuracy and small target detection in landslide identification tasks, this paper designs a weighted loss function L based on BCE loss and Dice loss, and further analyzes the loss weight coefficient in the weighted loss function L. , loss weight coefficient Represents the weight of BCE loss. When it is set to 0.1 ~ 0.9, different loss weight coefficients are selected for experiments with a step size of 0.2, that is, = 0.1, 0.3, 0.5, 0.7, 0.9 were used for experimental comparison on the datasets used, and the average intersection-over-union ratio was used for evaluation. Fig.15 As shown. Fig.15 It can be seen that =0.5, the average intersection ratio is the highest. Therefore, during the training process =0.5 is considered to be an optimal weight coefficient.

[0070] The above experimental results show that the proposed method can accurately identify different types and sizes of landslides, reducing missed detection and false detection. Combining sequence features is more effective than focusing only on spatial features, and the average intersection-to-union ratio increased by 4.30%. Compared with existing deep learning models, the proposed model shows superior performance. Overall, the model has good spatial and sequence feature learning capabilities and can accurately identify various landslides.

[0071] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0072] It should be noted that, in the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0073] In the several embodiments provided by the present invention, it should be understood that the disclosed methods can be implemented in other ways. The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0074] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A dual-stream coding landslide automatic identification method based on improved U-Net, characterized in that: include: Based on multi-source data, a multi-dimensional feature data set is constructed; the multi-source data includes landslide vector data, optical band data, spectral index data, terrain factor data and texture index data; A dual-stream coding landslide recognition model is built based on the U-Net framework, including an encoder and a decoder. The encoder includes two branches and a fusion module, wherein the first branch is used to extract spatial dimension features, and the second branch is used to extract sequence dimension features; the fusion module is used to fuse the spatial dimension features and the sequence dimension features to obtain deep features; the decoder includes an AGDM module, which is used to reconstruct the prediction results in the feature space from the high-dimensional features obtained in the encoding stage; the first branch designs two convolutional layers and four residual coding layers; the input data is first projected to the high-dimensional feature space by the two convolutional layers, and then the features are further projected to a higher-dimensional space through the four residual coding layers to extract rich deep feature information; the second branch includes a constructed MSRNM module and an LSTM module; wherein the MSRNM module includes 4 downsampling steps, each downsampling step includes a residual learning unit, and the residual learning unit is composed of two convolutional layers. In the process of extracting sequence dimension features, the feature map size is halved after each residual learning unit; the LSTM module includes three layers of LSTM units, which are used to capture the sequence dependency between images of different bands; The dual-stream encoding landslide recognition model is trained by using the feature data set in combination with a weighted loss function; the weighted loss function is a weighted sum of a binary cross entropy BCE loss and a Dice loss; The trained dual-stream encoding landslide recognition model is evaluated using a variety of evaluation indicators.

2. The method according to claim 1, characterized in that: The multi-dimensional feature data set is constructed based on multi-source data, including: The acquired optical band data, spectral index data, terrain factor data, texture index data and landslide vector data are cropped into 256×256 sub-images with a step size of 128 pixels, and the remaining part of the cropped image is filled to obtain a sub-image data set; The sub-image dataset is expanded using a data enhancement strategy to obtain a multi-dimensional feature dataset as a training sample and its corresponding real landslide label.

3. The method according to claim 1, characterized in that: The convolution kernel size of the two convolution layers in the first branch is 3×3, the step size is 1, and each convolution layer is followed by a batch normalization layer and a ReLU activation layer; the four residual coding layers are composed of 3, 4, 6 and 3 residual blocks respectively. In the process of spatial dimension feature extraction, the number of feature map channels doubles and the size is halved each time it passes through a residual coding layer.

4. The method according to claim 3, characterized in that Each of the residual blocks includes three convolutional layers and three batch normalization (BN) layers. A ReLU activation layer is respectively arranged after the first BN layer and the second BN layer. A residual connection is realized between the first convolutional layer and the third convolutional layer of the residual block trunk. On the residual connection branch, the input vector and the output vector are identically mapped through a 1×1 convolution.

5. The method according to claim 1, characterized in that In the fusion module, the spatial dimension feature and the sequence dimension feature are cascaded through a vector concatenation method to form a new feature variable; the new feature variable participates in the processing process of the decoder module through an upsampling operation to generate a final output.

6. The method according to claim 1, characterized in that The AGDM module includes four layers of convolutional blocks. In the jump connection between the encoder and the decoder, an attention gating module is introduced to generate a gating signal to highlight the importance of features at different spatial positions. Finally, a 1×1 convolutional layer with a Sigmoid classifier is used to infer the probabilities of the landslide target and background, and finally a landslide prediction result with the same size as the input data is obtained.

7. The method according to any one of claims 1 to 6, characterized in that: The weighted loss function is expressed as follows: ； ； ； Where L is the weighted loss function, For BCE loss, is the Dice loss, is the loss weight coefficient, is the number of categories, n is the number of pixels, For the The label value of each pixel, For the The predicted probability value of pixels; represent The coefficients, TP is true positive, indicating that the model correctly predicts the landslide target, FN is false negative, indicating that the landslide target is incorrectly predicted as non-landslide, and FP is false positive, indicating that the non-landslide target is incorrectly predicted as landslide.

Images