A method for landslide susceptibility evaluation based on a Transformer model

Abstract

The application discloses a method for landslide susceptibility evaluation based on a Transformer model, and the method uses the Transformer model to establish a model for landslide susceptibility evaluation and to make a landslide susceptibility map. In the method, a disaster area of the Wenchuan earthquake in 2008 is taken as a research area, 18 landslide influence factors are selected, and a Transformer deep learning network is used to carry out landslide susceptibility evaluation. The performance of the Transformer deep learning network is verified by using accuracy, root mean square error and an ROC curve as evaluation indexes. The result shows that the accuracy of the algorithm based on the Transformer model is up to 88.53 %, and the landslide susceptibility evaluation model based on the Transformer model has a good application prospect.

Description

Technical Field

[0001] This invention relates to the field of geological hazard assessment technology, and in particular to a method for landslide susceptibility assessment based on the Transformer model. Background Technology

[0002] Landslides are a common type of geological hazard. According to the 2021 National Geological Disaster Bulletin, a total of 4,772 geological disasters occurred nationwide, of which 2,335 were landslides, accounting for nearly 50% of all incidents. These landslides caused significant socio-economic losses and casualties. Therefore, conducting scientific and accurate landslide susceptibility assessments is fundamental to disaster reduction and prevention efforts.

[0003] The quality of landslide susceptibility assessment depends primarily on the model's fit and the quality of the assessment data. Many researchers utilize machine learning methods for landslide susceptibility assessment, including logistic regression (LR), radial basis function (RBF), artificial neural networks (ANN), random forests (RF), and support vector machines (SVM). While these methods have achieved good results, they directly classify the input data and fail to discover more representative features from this data to further improve classification accuracy.

[0004] To address this problem, deep learning methods have attracted considerable attention from researchers. In recent years, the Transformer has shone brightly in Natural Language Processing (NLP), revolutionizing the field upon its introduction and being adopted in numerous subsequent studies. Similarly, the Transformer has achieved significant success in computer vision, with Transformer-based networks outperforming previous networks on many datasets. Meanwhile, the quality and quantity of geological disaster data have greatly improved in recent years. This data is characterized by its multi-source nature, spatiotemporal complexity, and nonlinearity, making the accurate and reliable processing of massive disaster data particularly crucial. Due to its inherent characteristics, the Transformer can adapt well to complex data, resulting in superior pattern alignment and better feature extraction. For landslides, with their complex formation mechanisms and numerous influencing factors, deep learning models can theoretically better simulate landslide formation and accurately predict potential landslides. Therefore, Transformer-based deep learning has broad application prospects in landslide susceptibility analysis methods.

[0005] Although the Transformer has a wide range of applications, there is limited research on its application in landslide susceptibility assessment. Therefore, this invention proposes a Transformer-based landslide susceptibility assessment model, using the Wenchuan earthquake-stricken area as the research region to explore its application in landslide susceptibility assessment. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a method for landslide susceptibility assessment based on the Transformer model, and explores its application in landslide susceptibility assessment.

[0007] The present invention adopts the following technical solution:

[0008] A method for landslide susceptibility assessment based on the Transformer model, characterized by the following steps:

[0009] Step S1: Obtain the historical landslide list of the study area, import the historical landslide list into ArcGIS software for processing, and form a landslide spatial dataset;

[0010] Step S2: Obtain landslide impact factor data for the study area, import the landslide impact factor data into ArcGIS software for processing, process all landslide impact factor data, sample the processed landslide impact factor data to the same resolution size, process continuous landslide impact factors into discrete factors, and finally export them for later use.

[0011] Step S3: Perform collinearity analysis on the landslide impact factor data in step S2, remove factors with high linearity, and determine the final landslide impact factor data used for landslide susceptibility assessment.

[0012] Step S4: Process the landslide spatial dataset from step S1 and the landslide impact factor data filtered in step S3 in ArcGIS software, extract the relevant landslide impact factor data of the landslide spatial data location, and export it as the training dataset for the network.

[0013] Step S5: Construct a landslide susceptibility assessment model based on the Transformer model;

[0014] Step S6: Input the training dataset extracted in step S4 into the landslide susceptibility evaluation model constructed in step S5 for training.

[0015] Step S7: Use the landslide susceptibility assessment model trained in step S6 to predict landslide susceptibility in other areas of the study area. A landslide susceptibility index will be obtained for each study area, and the landslide susceptibility index is between 0 and 1.

[0016] Step S8: Import all landslide susceptibility indices of the study area into ArcGIS software to generate a landslide sensitivity map of the study area.

[0017] In step S1, the locations where landslides occurred are used as positive samples for model training, and negative samples with the same number of positive samples are randomly selected from areas where no landslides occurred.

[0018] The landslide influencing factor data in step S2 include distance from fault, lithology, slope, landform type, distance from river, distance from road, land use type, aspect, elevation, plane curvature, profile curvature, rainfall, normalized difference vegetation index (NDVI), topographic wetness index (TWI), and stream power index (SPI).

[0019] In step S3, the variance inflation factor (VIF) is used to address the multicollinearity problem: the closer the VIF value is to 1, the less severe the multicollinearity, and vice versa. The variance inflation factor (VIF) and tolerance (TOL) are used to test the multicollinearity of landslide influencing factors. A VIF value less than 10 and a TOL value greater than 0.1 indicate that these variables are independent of each other, and factors with high correlation are removed.

[0020] In step S4, the training samples are divided into a training set and a validation set in a 7:3 ratio, and their performance is verified using accuracy, root mean square error, and ROC curve evaluation metrics.

[0021] In step S5, eight Transformer modules are used to construct a landslide susceptibility evaluation model.

[0022] In step S6, a Dropout layer is added to prevent the model from overfitting.

[0023] This invention offers the following advantages: The Transformer model demonstrates superior accuracy in landslide susceptibility assessment, with its ACC, AUC, MAE, and RSME scores being 5.73%, 4.8%, 0.0893, and 0.0621 higher than the CNN-1D model, respectively. Furthermore, the Transformer-based landslide susceptibility assessment model predicts that 73.84% of landslide points occur in extremely high-risk areas, higher than the 56.39% predicted by the CNN-1D model, thus better reflecting the spatial distribution of landslides in the study area. This research indicates that the Transformer model has significant application potential in landslide susceptibility assessment. Attached Figure Description

[0024] Figure 1 This is a flowchart of the present invention;

[0025] Figure 2 This is a diagram of the Transformer network model of the present invention;

[0026] Figure 3 This refers to the geographical location and landslide distribution of the study area in this invention;

[0027] Figure 4 This is the ROC curve of the landslide susceptibility assessment method based on the Transformer model in this invention;

[0028] Figure 5 This is a comparison diagram of the susceptibility partitions of different methods in this invention. Detailed Implementation

[0029] The embodiments of the present invention will be described in detail below with reference to examples. However, those skilled in the art will understand that the following examples are only for illustrating the present invention and should not be regarded as limiting the scope of the present invention.

[0030] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0031] refer to Figure 1 The present invention provides an automatic identification method based on edge-guided attention neural network, comprising the following steps: Step S1, obtaining a historical list of landslides in the study area, importing it into ArcGIS software for processing, and forming a landslide spatial dataset.

[0032] Step S2: Obtain landslide impact factor data for the study area, import the landslide impact factor data into ArcGIS software for processing, process all landslide impact factor data, sample the landslide impact factor data to the same resolution size, process continuous landslide impact factors into discrete factors, and finally export them for later use.

[0033] Step S3: Perform collinearity analysis on the landslide influencing factor data in S2, remove factors with high linearity, and determine the final factor data that can be used for landslide susceptibility assessment.

[0034] Step S4: Process the landslide spatial dataset from Step S1 and the landslide impact factor data filtered in Step S3 in ArcGIS software, extract the relevant landslide impact factor data of the landslide spatial data location, and export it as the training dataset for the network.

[0035] Step S5: Construct a landslide susceptibility assessment model based on the Transformer model; the constructed model is as follows: Figure 2 As shown, the input data for landslide susceptibility assessment can be viewed as an image, with each channel representing a landslide assessment factor. Therefore, each grid cell of the input data is represented by a column vector, the length of which is determined by the landslide assessment factor. Each element in this vector represents a landslide assessment factor. This invention uses column vectors as the data format for input to the Transformer model.

[0036] The Transformer-based landslide susceptibility assessment model consists of an embedding input layer, a Transformer, and a Multilayer Perceptron (MLP). The embedding input layer encodes the input data and feeds it into the Transformer for feature extraction. The Transformer comprises multiple Transformer modules. After processing the data, the Transformer inputs it into the MLP for classification, ultimately outputting two classes: 0 and 1 (0 representing no landslide and 1 representing a landslide).

[0037] The Transformer consists of a multi-head attention layer and a feedforward layer. Pixel summation and layer normalization are performed after both layers to improve model training performance. The attention layer includes three matrices W that participate in network training. q W q W k W k and W v W v Each input vector is multiplied by these three matrices to obtain the query matrix, key matrix, and value matrix. The attention mechanism is as follows:

[0038]

[0039] In the formula, Q, K, and V are the query matrix, key matrix, and value matrix, respectively, and d k dk Let W be the dimension of the input. The multi-head attention mechanism uses matrix W. q W q W k W k and W v W v Multiple query matrices, key matrices, and value matrices are generated. Each query matrix, key matrix, and value matrix undergoes an attention mechanism to calculate corresponding output feature values. These multiple output features are then concatenated and finally linearly output to the feedforward layer via a fully connected layer. The feedforward layer in this invention uses two fully connected layers as the feedforward neural network. The final output consists of two layers: 0 represents non-slope and 1 represents slope.

[0040] Step S6: Input the training samples extracted in step S4 into the model constructed in step S5 for training.

[0041] Step S7: Use the model trained in step S6 to predict landslide susceptibility in other areas of the study area. A landslide susceptibility index (between 0 and 1) will be obtained for each study area.

[0042] Step S8: Import all landslide susceptibility indices of the study area into ArcGIS software to generate a landslide sensitivity map of the study area.

[0043] Simulation experiment:

[0044] The effects of the present invention can be further illustrated by the following specific example:

[0045] 1. The study area is located in the disaster-stricken area of ​​the 2008 Wenchuan earthquake, such as... Figure 3 As shown, it includes 13 districts and counties: Dujiangyan City, Pengzhou City, Chongzhou City, Shifang City, Mianzhu City, Anzhou District, Beichuan Qiang Autonomous County, Pingwu County, Jiangyou City, Qingchuan County, Wenchuan County, Li County, and Maoxian County, with a total area of ​​34,402.28 km². 2 The study area has a large elevation range, with the lowest elevation in the east being 402 meters and the highest elevation in the west being 5912 meters. The main fault zone consists of three major faults, which are, from west to east: the Wenchuan-Maoxian-Pingwu-Qingchuan fault zone in the back mountain of Longmenshan; the Yingxiu-Beichuan fault zone in the central part of Longmenshan; and the Dujiangyan-Hanwang-Anxian fault zone at the front boundary of the main mountain of Longmenshan.

[0046] 2. Evaluation Indicators

[0047] To evaluate the performance of the proposed model, this invention uses accuracy (ACC), ROC curve measurements, root mean square error (RMSE), and mean absolute error (MAE) as evaluation metrics. Accuracy is calculated as follows:

[0048]

[0049] In this context, True Positive (TP) represents the number of correctly classified landslide samples, False Positive (FP) represents the number of misclassified non-landslide samples, True Negative (TN) represents the number of correctly classified non-landslide samples, and False Negative (FN) represents the number of misclassified landslide samples. A higher ACC value indicates better classification accuracy. The ROC curve is a standard technique for evaluating the performance of landslide prediction methods. It is generated by plotting the true positive (TP) rate against the false positive (FP) rate at various thresholds. The TP rate and FP rate are also statistically referred to as "sensitivity" and "100-specificity," respectively. Furthermore, the Area Under the ROC Curve (AUC) has been widely used to quantitatively evaluate the performance of LSM methods. Specifically, if the AUC value is close to 1, the prediction method is considered good. The smaller the RMSE and MSE values, the better the model's performance. RMSE and MSE are calculated as follows:

[0050]

[0051]

[0052] Where n is the number of samples. For the ii-th sample observation, y i y i The predicted value for the ii-th sample.

[0053] 3. Model parameter settings

[0054] All models in this invention were built using Tensorflow 2.2. First, the models were initially constructed, and then a trial-and-error method was used to adjust the model parameters. The optimal parameters were then used for landslide susceptibility evaluation. The susceptibility evaluation model based on Transformer will be referred to as Transformer in the following text. To verify the effectiveness of the proposed method, the CNN-1D model proposed by Wang et al. was used for accuracy comparison. The parameter settings for the Transformer and CNN-1D models are shown in Table 1.

[0055] Table 1 Transformer model parameter settings

[0056]

[0057] 4. Collinearity test of landslide influencing factors

[0058] Before assessing landslide susceptibility, it is necessary to test for multicollinearity among the landslide assessment factors, as correlations between factors can lead to incorrect modeling results. This invention uses the variance inflation factor (VIF) and tolerance level (TOL) to test for multicollinearity. According to some studies, a VIF value less than 10 and a TOL value greater than 0.1 indicate that these variables are independent. The multicollinearity test results for the landslide assessment factors of this invention are shown in Table 2. All factor values ​​meet the conditions, and there is no multicollinearity relationship between the factors.

[0059] Table 2 Results of the Collinearity Test for Evaluation Factors

[0060]

[0061]

[0062] 5. Experimental Results

[0063] Table 3 shows the accuracy results of the landslide susceptibility assessment based on Transformer and compared with the CNN-1D model. The table shows that Transformer has the highest ACC, 5.73% higher than CNN-1D, and the lowest MAE and RMSE, 0.0893 and 0.0621 lower than CNN-1D, respectively. This indicates that Transformer performs better and can better fit the nonlinear relationship between landslide development and its evaluation factors. Transformer, through its transformer module, effectively considers the importance of each evaluation factor for landslide development and better fits the development conditions of landslides. The ROC curves of the two models are shown below. Figure 4 As shown, the Transformer model has the highest AUC value of 0.959, indicating that it can pay different attention to different landslide evaluation factors, focusing its attention on the evaluation factors that contribute significantly to landslide development. The accuracy comparison demonstrates the superior performance of the Transformer model.

[0064] Table 3 Accuracy Evaluation of Transformer and CNN-1D Models

[0065]

[0066] After training both models, the Transformer model and the CNN-1D model were used to predict the landslide index of landslide grid cells. The landslide index ranges from 0 to 1, with a higher value indicating a higher likelihood of landslides and vice versa. The model predictions were used to categorize the resulting susceptibility map of the study area into five levels: extremely low susceptibility, low susceptibility, moderate susceptibility, high susceptibility, and extremely high susceptibility. The susceptibility assessment mapping results of the two models are shown below. Figure 5 As shown.

[0067] from Figure 5 It can be seen that the susceptibility partitioning of the two models is similar, with extremely low susceptibility areas located in the east and west, and extremely high susceptibility areas located in the middle (near the Minjiang River and Yingxiu-Beichuan fault zones). However, there are also many differences between the two. The susceptibility partitioning map of the Transformer model has more extremely high susceptibility areas in the middle, which is consistent with the spatial distribution of landslide points in this invention. In contrast, the susceptibility partitioning map of the CNN-1D model has many low susceptibility areas in the east, while the susceptibility partitioning map of the Transformer model has fewer such areas and more extremely low susceptibility areas.

[0068] Table 4 shows the distribution of each susceptibility level in the susceptibility zoning maps predicted by the two models. In the Transformer model's predictions, the proportions of extremely high susceptibility areas, high susceptibility areas, medium susceptibility areas, low susceptibility areas, and extremely low susceptibility areas are 48.66%, 13.79%, 10.88%, 10.38%, and 16.29%, respectively. In the CNN-1D model's predictions, the proportions of extremely high susceptibility areas, high susceptibility areas, medium susceptibility areas, low susceptibility areas, and extremely low susceptibility areas are 38.76%, 17.71%, 15.47%, 14.55%, and 13.51%, respectively. It can be seen from the table that the Transformer model predicts larger extremely low and extremely high susceptibility areas.

[0069] Table 4 Statistical Analysis of Landslide Susceptibility Zones

[0070]

[0071]

[0072] Table 5 shows the number of landslide points in each zone of the landslide susceptibility zoning for both models. It can be seen that the number of landslide points increases with the landslide severity level in both models. The highest proportion of landslide points is found in the extremely high susceptibility zone, while the lowest proportion is found in the extremely low susceptibility zone, indicating that both models can accurately reflect the spatial distribution trend of landslides. Specifically, the Transformer model shows a higher proportion of landslide points in the extremely high susceptibility zone than the CNN-1D model, and a lower proportion in the extremely low susceptibility zone than the CNN-1D model. This suggests that the Transformer model's results better match the spatial distribution characteristics of landslides, making its assessment of landslide susceptibility more reasonable.

[0073] Table 5. Analysis of the percentage of landslide points in landslide-prone zones.

[0074]

[0075] Based on the comprehensive visualization results of landslide susceptibility zoning and the statistical analysis of accuracy, the landslide susceptibility assessment model based on Transformer has better performance. This is due to the self-attention mechanism in the Transformer module, which allows the Transformer model to focus on the assessment factors that have a greater impact on landslide development. Moreover, the Transformer model can better handle complex data, better align data patterns, and better learn the characteristics of the data, thereby fitting the nonlinear relationship between assessment factors and better predicting the occurrence of landslides.

[0076] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for landslide susceptibility evaluation based on a Transformer model, characterized in that, The method comprises the following steps: Step S1, obtaining a landslide history list of a study area, importing the landslide history list into ArcGIS software for processing to form a landslide spatial data set; Step S2, obtaining landslide influencing factor data of the study area, importing the landslide influencing factor data into ArcGIS software for processing, processing all the landslide influencing factor data, sampling the processed landslide influencing factor data to the same resolution size, and processing continuous landslide influencing factors into discrete factors, and finally exporting the data for standby; Step S3, performing collinearity analysis on the landslide influencing factor data in step S2, removing factors with high linearity, and determining the final landslide influencing factor data used for landslide susceptibility evaluation; Step S4, processing the landslide spatial data set in step S1 and the screened landslide influencing factor data in step S3 in ArcGIS software, extracting relevant landslide influencing factor data of the landslide spatial data position, and exporting the data as a training data set of the network; Step S5, constructing a landslide susceptibility evaluation model based on a Transformer model; Step S6, inputting the training data set extracted in step S4 into the landslide susceptibility evaluation model constructed in step S5 for training; Step S7, using the trained landslide susceptibility evaluation model in step S6 to predict landslide susceptibility of other areas in the study area, obtaining a landslide susceptibility index in each study area, and the landslide susceptibility index being between 0 and 1; Step S8, importing all the landslide susceptibility indexes of the study area into ArcGIS software to generate a landslide susceptibility map of the study area. 2.The method of landslide susceptibility evaluation based on the Transformer model according to claim 1, characterized in that, In step S1, the positions where landslides occur are taken as positive samples for model training, and negative samples with the same number of positive samples are randomly selected from the areas where landslides do not occur.

3. The method for landslide susceptibility evaluation based on the Transformer model according to claim 1, characterized in that, The landslide influencing factor data in step S2 includes distance from fault, lithology, slope, landform type, distance from river, distance from road, land use type, slope direction, elevation, plan curvature, profile curvature, rainfall, normalized difference vegetation index (NDVI), topographic wetness index (TWI), and stream power index (SPI).

4. The method for landslide susceptibility evaluation based on a Transformer model according to claim 1, characterized in that, In step S3, the variance inflation coefficient VIF index is used to solve the problem of multicollinearity: the closer the VIF value is to 1, the lighter the multicollinearity, and vice versa; the variance inflation factor (VIF) and tolerance (TOL) are used to test the multicollinearity of the landslide influencing factors, and when the value of VIF is less than 10 and the value of TOL is greater than 0.1, it indicates that these variables are independent, and the factors with high correlation are removed. 5.The method of landslide susceptibility evaluation based on the Transformer model according to claim 1, characterized in that, In step S4, the training samples are divided into a training set and a validation set in a ratio of 7:3, and the accuracy, root mean square error, and ROC curve evaluation index are used to verify the performance.

6. The method for landslide susceptibility evaluation based on a Transformer model according to claim 1, characterized in that, In the step S5, 8 pieces of the Transformer module are used to construct the landslide susceptibility evaluation model.

7. The method for landslide susceptibility evaluation based on a Transformer model according to claim 1, characterized in that, In the step S6, the model overfitting is prevented by adding the Dropout layer.

Images