Intelligent early warning method for short-term and temporary landslide
By setting a monitoring sensor on the landslide and using the XGBOOST algorithm for prediction, the problem of long landslide deformed intelligent prediction time and poor predictive accuracy in existing technologies have been solved, and a faster and accurate landslide disaster warning was achieved.
Patent Information
- Application Number
- CN202411423466.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-09
- Publication Date
- 2025-01-21
AI Technical Summary
The existing machine learning model has a long training time and poor prediction accuracy in landslide deformation intelligent prediction.
Adopt the intelligent warning method of landslides based on the XGBOOST algorithm, to set up data to collect data by monitoring sensors on the landslide, build a characteristic vector of the slope body, and use the pre -trained XGBOOST model to predict. Essence
The speed and accuracy of the prediction of landslide deformation can be improved, and the landslide warning can be performed faster and accurately, reducing casualties and property losses.
Smart Images
Figure CN119338059A_ABST
Abstract
Description
[0001] This case is a divisional application of the invention patent application with application date of July 9, 2020, application number 202010659730.8, and invention name "A short-term intelligent early warning method for landslides based on XGBoost algorithm". Technical Field
[0002] The invention discloses a method for short-term intelligent early warning of landslides, and relates to the fields of landslide monitoring and early warning and machine learning. Background Art
[0003] As an important geological disaster, landslides are widely present in my country and occur frequently, causing great casualties and property losses. The current disaster situation is still severe. With the rapid development of science and technology, monitoring and early warning have become the most direct and effective means to actively prevent landslide geological disasters. Landslide is a complex multidimensional nonlinear dynamic system affected by its own geological conditions and external factors. Landslide deformation is the result of the joint action of multiple factors in this complex multidimensional nonlinear dynamic system. It is a comprehensive variable that can directly reflect the evolution of landslide deformation. Landslide deformation corresponds to the movement characteristics of landslides. The prediction of landslide deformation is the basis and prerequisite for accurately grasping the evolution stage of landslide deformation and realizing accurate landslide early warning. Based on deformation prediction, short-term intelligent early warning of landslides can leave sufficient time for emergency response to carry out disposal work and reduce casualties and property losses.
[0004] A lot of research has been done on landslide deformation prediction at home and abroad. Deformation prediction models are mainly divided into statistical prediction models and machine learning models. Statistical prediction models include gray prediction models, regression analysis models, time series models, exponential smoothing methods, etc. In recent years, machine learning models have been widely used in intelligent prediction and early warning of single landslides, mainly using intelligent algorithms such as RBF (Radial Basis Function), BP (Back Propagation), k-means, SVR (Support Vactor Regression), etc. However, there are some problems with the above models. For example, the statistical prediction model is relatively simple and cannot fit the highly complex nonlinear changes of landslides well, and it is difficult to improve the prediction accuracy. Machine learning models such as the SVR model have the problems of long time consumption and the need to improve the prediction accuracy; BP neural network can solve nonlinear problems, but it is easy to fall into local minima during training, converges slowly, and is dependent on training samples; k-means is prone to local optimality and clustering stagnation.
[0005] In view of the above problems, the present invention proposes a deformation prediction method based on XGBoost. XGBoost (Extreme Gradient Boosting) is an algorithm based on weak learner integration proposed by Chen Tianqi in 2016. Weak learner refers to a learner with a generalization performance higher than random guessing. It is an improved algorithm based on gradient boosting tree (GBDT). XGBoost has the following theoretical advantages: (1) The regularization term is introduced into the loss function to control the model complexity and prevent overfitting; (2) The second-order Taylor expansion of the loss function is performed to improve the convergence speed and convergence accuracy; (3) Column sampling is introduced to further improve the calculation speed and prevent overfitting. (4) XGBoost allows the selection of multiple weak learners for integrated prediction, and the prediction results of multiple weak learners are integrated as the final result to improve the accuracy. (5) XGBoost adopts many strategies to avoid overfitting, supports early stopping of tree construction to avoid invalid calculations, has fast training speed and high model accuracy, is widely praised by the industry, and has been widely used in time series prediction. Summary of the invention
[0006] The present invention proposes a method for short-term intelligent early warning of landslides, which can solve the problems of long training time and poor prediction accuracy of existing machine learning models in intelligent prediction of landslide deformation.
[0007] According to one aspect of the present invention, there is provided a method for short-term intelligent early warning of landslides, comprising:
[0008] Monitoring sensors are deployed on the landslide body to collect the characteristic data of the landslide body in real time, thereby constructing the characteristic vector of the landslide body;
[0009] The rainfall forecast data provided by the meteorological department and the landslide feature vector constructed from the historical time series before the forecast date are input into the pre-trained XGBoost model to obtain the landslide surface deformation prediction results;
[0010] The XGBoost model predicts the surface deformation characteristics of the landslide body on the prediction day based on the landslide body feature vector constructed from the historical time series in a period of time before the prediction day and the rainfall predicted by the weather forecast on the prediction day.
[0011] When the predicted result of the landslide surface deformation is greater than the threshold, a landslide disaster warning is issued;
[0012] in,
[0013] The landslide body characteristic vector includes three dimensions: the characteristics of the external inducing factors of the landslide body, the internal characteristics of the landslide body, and the surface deformation characteristics of the landslide body;
[0014] The characteristics of the external inducing factors of the landslide body are characterized by rainfall;
[0015] The internal characteristics of the landslide body are characterized by soil moisture content;
[0016] The surface deformation characteristics of the landslide body are GNSS surface absolute displacement, or surface inclination change monitored by an inclinometer, or surface acceleration monitored by an accelerometer, or crack meter deformation time series data, or a combination of these data.
[0017] According to some embodiments, the step further includes constructing a training set and a test set:
[0018] Data preprocessing is performed on the monitored landslide feature vectors, including removing outliers and supplementing missing values;
[0019] In the data preprocessing, the time series analysis model ARIMA model is used to fit the monitoring data when supplementing the missing values;
[0020] The regularized data after supplementing the missing values are firstly subjected to first-order difference, then normalized and divided into training set and test set.
[0021] According to some embodiments, the formula for normalization is as follows:
[0022]
[0023] Among them, x is the sample monitoring data before normalization, x* is the sample monitoring data after normalization, max is the maximum value of the sample data, and min is the minimum value of the sample data.
[0024] According to some embodiments, the parameters for tuning the XGBoost model include two major parts of parameters, tree ensemble model parameters and regularization parameters.
[0025] According to some embodiments, the tree ensemble model parameters are responsible for fitting the samples;
[0026] The tree ensemble model parameters include 'learning_rate' learning rate, 'n_estimators' maximum number of decision trees, 'max_depth' maximum depth of the tree, 'min_child_weight' minimum sample weight sum in the leaf node, if the sample weight sum of a leaf node is less than min_child_weight, the splitting process ends.
[0027] According to some embodiments, the regularization parameter helps control the complexity of the model and avoid overfitting;
[0028] 'subsample' is the ratio of the subsamples used to train the model to the entire sample set to prevent overfitting, 'colsample_bytree' is the ratio of feature sampling when building a tree, 'gamma' is the minimum loss function drop value required for node splitting, 'reg_alpha' is the L1 regularization coefficient of the weight, and 'reg_lambda' is the L2 regularization coefficient of the weight.
[0029] According to some embodiments, the specific tuning steps of the XGBoost model tuning are:
[0030] Step 1: Set the initial value of the parameter;
[0031] Step 2: According to the greedy algorithm, determine the parameters {'n_estimators', ('max_depth', 'min_child_weight'), 'gamma', ('subsample', 'colsample_bytre'), ('reg-lambda', 'reg_alpha')} in turn when other parameters are determined; Step 3: Adjust the learning rate to obtain a better model effect;
[0032] Repeat the second step until the model reaches the local optimal parameters and the model training is completed.
[0033] According to another aspect of the present invention, a computer program product is provided, characterized in that it comprises a computer program, and when the computer program is executed by a processor, it implements any of the above methods.
[0034] The present invention can make intelligent predictions of the deformation of the landslide body in the future based on the XGBoost algorithm according to the real-time monitoring data of monitoring instruments such as rain gauges, crack meters, GNSS, inclinometers, accelerometers, soil moisture meters, and soil thermometers, and make intelligent warnings of possible landslide deformation in the future according to the early warning criterion threshold. This method can make one-dimensional predictions or multi-dimensional predictions, with fast prediction speed and high prediction accuracy. The specific technical scheme is as follows:
[0035] Step 1, deploy monitoring sensors on the landslide body, collect landslide body characteristic data in real time, and construct the landslide body characteristic vector; the surface deformation of the landslide body is affected by multiple factors such as historical surface deformation, rainfall, soil moisture content, etc. In order to characterize the characteristics of the landslide body, the present invention constructs the landslide body characteristic vector from three dimensions: external inducing factors of the landslide body, internal characteristics of the landslide body, and surface deformation characteristics of the landslide body. Among them, the characteristics of the external inducing factors of the landslide body are characterized by rainfall, the internal characteristics of the landslide body are characterized by soil moisture content, and the surface deformation characteristics of the landslide body are GNSS surface absolute displacement, or surface inclination change monitored by inclinometer, or surface acceleration monitored by accelerometer, or crack meter deformation time series data, or a combination of these data.
[0036] Step 2: Data preprocessing is performed on the monitored landslide feature vector, including removing outliers and supplementing missing values. When supplementing missing values, the classic time series analysis model ARIMA (Autoregressive Integrated Moving Average model: differential integrated moving average autoregression) model is used to fit the monitoring data to obtain the deformation data at the missing time. After data filling, the first-order difference sequence of the landslide deformation monitoring data and the first-order difference sequence of the missing value filling conform to the same distribution and pass the distribution test. This step uses the differential integrated moving average autoregression ARIMA model to fill in the missing data. Compared with the commonly used linear difference, the filled data can better reflect the deformation trend of the landslide.
[0037] Step 3: Firstly, perform first-order difference on the filled regularized data, and then perform normalization. The normalization formula is as follows: Among them, x is the sample monitoring data before normalization, x* is the sample monitoring data after normalization, max is the maximum value of the sample data, and min is the minimum value of the sample data.
[0038] Step 4. Use the training set to train the XGBoost model and determine the model parameters: The XGBoost model predicts the surface deformation characteristics of the landslide body on the prediction day based on the landslide feature vector constructed by the historical time series before the prediction day and the rainfall predicted by the weather forecast on the prediction day. The core idea of the XGBoost algorithm is to add a tree each iteration to fit the residual of the predicted value and the true value of the last iteration process, that is, each tree is learning the residual of the previous N-1 trees, so as to continuously approach the actual value. The score of each sample is the sum of the scores of the sample in each tree. Training the model is to find the best model parameters for the given training data. It is necessary to define the objective function to measure the model performance for a given set of parameters. The objective function includes two parts: training loss function and regularization term, as shown in formula (1.1). Among them, L(θ) is the training loss function, Ω(θ) is the regularization term, the training loss measures the prediction performance of the model for the training data, and the role of the regularization term is to ensure that the model is not too complex;
[0039] Obj(θ)=L(θ)+Ω(θ)(1.1)
[0040] The training loss function is the mean square error, see formula (1.2), y i represents the real monitoring value of the surface deformation characteristics of the landslide body of the i-th sample, represents the predicted value of the landslide surface deformation characteristics of the i-th sample. A sample includes a landslide surface deformation characteristic and the corresponding rainfall and soil moisture content;
[0041]
[0042] The regularization term is shown in formula (1.3), where T is the number of leaf nodes in each tree, and w j 2 is the square of the weight of the leaf node on the tree, γ and λ are model parameters that control the complexity of the model;
[0043] From the objective function, we can see that the parameters that need to be tuned in the XGBoost model include two major parameters: one is the tree ensemble model parameters, which are responsible for fitting the samples as much as possible; the other is the regularization parameters, which help control the complexity of the model and avoid overfitting;
[0044] The tree ensemble model parameters include 'learning_rate' learning rate, 'n_estimators' maximum number of decision trees, 'max_depth' maximum depth of the tree, 'min_child_weight' minimum sample weight sum in the leaf node, if the sample weight sum of a leaf node is less than min_child_weight, the split process ends, 'subsample' the proportion of subsamples used to train the model to the entire sample set to prevent overfitting, 'colsample_bytree' the proportion of feature sampling when building a tree, 'gamma' the minimum loss function drop value required for node splitting, 'reg_alpha' L1 regularization coefficient of weight, 'reg_lambda' L2 regularization coefficient of weight;
[0045] Specific tuning steps:
[0046] Step 1: Set the initial value of the parameter;
[0047] Step 2: According to the greedy algorithm, determine the parameters one by one when other parameters are determined.
[0048] {'n_estimators', ('max_depth', 'min_child_weight'), 'gamma', ('subsample',
[0049] 'colsample_bytre'),('reg-lambda','reg_alpha')} and other parameters;
[0050] Step 3: Adjust the learning rate to obtain better model results.
[0051] Repeat the second step until the model reaches the local optimal parameters and the model training is completed;
[0052] Step 5: Input the rainfall forecast data provided by the meteorological department and the landslide feature vector constructed by the historical time series before the forecast date into the trained XGBoost model to obtain the landslide surface deformation prediction result;
[0053] Step 6: When the predicted result of the landslide surface deformation is greater than the judgment threshold, a landslide disaster warning is issued.
[0054] Beneficial Effects
[0055] The intelligent short-term warning method for landslides based on the XGBoost algorithm provided by the present invention has the following advantages: 1. The present invention applies the XGBoost algorithm to the field of intelligent short-term warning for landslides for the first time. The XGBoost algorithm implements an approximate algorithm for split leaf node search, which can reduce memory consumption, and significantly improves calculation speed and prediction accuracy. The algorithm can be used for landslide prediction to improve the prediction speed and prediction accuracy of landslide deformation; 2. Considering the influence of rainfall on landslide deformation, the selection of variables is more typical, and intelligent short-term prediction of landslide deformation rate can be carried out based on future rainfall forecast data; 3. This method can be extrapolated to short-term prediction of other measurement items such as landslide body accelerometers and inclinometers, and can perform one-dimensional prediction or multi-dimensional prediction. 4. Improved traditional prediction methods. The prediction speed and prediction accuracy are significantly improved. 5. It can provide improved ideas for single variable model modeling for rainfall-type landslide deformation prediction, and perform accurate, effective and fast prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 , a flowchart of the method of the present invention. DETAILED DESCRIPTION
[0057] This paper takes the monitoring data of the crack meter installed on the landslide body as an example. The specific implementation includes the following steps. This method can be extrapolated to the prediction of the data collected by other sensors such as GNSS surface absolute displacement, inclinometer, accelerometer, etc. installed on the landslide body to monitor the surface deformation of the landslide body.
[0058] Step 1: Deploy monitoring sensors on the landslide body to collect real-time data on the amount of cracks, rainfall, and soil moisture content of the landslide body.
[0059] Step 2: Preprocess the monitored crack volume, rainfall and soil moisture content, including removing outliers and supplementing missing values. When supplementing missing values, the classic time series analysis model ARIMA is used to fit the monitoring data to obtain the deformation data at the missing time. After data filling, the first-order difference sequence of the landslide deformation monitoring data and the first-order difference sequence of the missing value filling conform to the same distribution and pass the distribution test.
[0060] Step 3: Firstly, perform first-order difference on the filled regularized data, and then perform normalization. The normalization formula is as follows: Among them, x is the sample monitoring data before normalization, x* is the sample monitoring data after normalization, max is the maximum value of the sample data, and min is the minimum value of the sample data.
[0061] Step 4, train the XGBoost model using the training set and determine the model parameters: the XGBoost model predicts the surface deformation characteristics of the landslide body on the prediction day based on the landslide body feature vector constructed by the historical time series before the prediction day and the rainfall predicted by the weather forecast on the prediction day. For example, taking January 8 as the prediction date, the crack volume, rainfall and soil moisture content obtained from January 1 to January 7, as well as the rainfall predicted by the weather forecast on January 8, can be used to predict the crack value of the landslide body on January 8. During training, the crack value collected on January 8 is used as the true value to calculate the objective function; the selection of the time span of historical data can be adjusted according to actual conditions.
[0062] The tree ensemble model parameters in step 4 include 'learning_rate' (learning rate), 'n_estimators' (maximum number of decision trees), 'max_depth' (maximum depth of the tree), 'min_child_weight' (the minimum sample weight sum in the leaf node. If the sample weight sum of a leaf node is less than min_child_weight, the splitting process ends.), 'subsample' (the proportion of subsamples used to train the model to the entire sample set to prevent overfitting), 'colsample_bytree' (the proportion of feature sampling when building a tree), 'gamma' (the minimum loss function drop value required for node splitting), 'reg_alpha' (L1 regularization coefficient of weight), 'reg_lambda' (L2 regularization coefficient of weight).
[0063] Specific tuning steps in step 4:
[0064] Step 1: Set the initial values of the parameters, such as using the default parameter settings of XGBoost. You can choose a relatively high learning rate. The value range is: [0,1]. Generally speaking, the learning rate is set to 0.1.
[0065] For different problems, the ideal learning rate sometimes fluctuates between 0.05 and 0.3.
[0066] Step 2: According to the greedy algorithm, determine the parameters one by one when other parameters are determined.
[0067] {'n_estimators', ('max_depth', 'min_child_weight'), 'gamma', ('subsample',
[0068] 'colsample_bytre'),('reg-lambda','alpha')} and other parameters. Specifically, 1. The cv function in xgboost can be used to find the optimal number of decision trees that meet the learning rate. 2. For a given learning rate and number of decision trees, the decision tree specific parameters max_depth (value range: [1,∞], generally between 3-10) and min_child_weight (value range: [0,∞]) are tuned; 3. Gamma parameter tuning, value range: [0,∞], the starting value can be selected between 0-0.2; 4. 'subsample'
[0069] and 'colsample_bytre' parameter tuning. subsample (value range: (0,1]), the initial value can be selected between 0.5-0.9. colsample_bytree (value range: (0,1]) parameter tuning, the initial value can be selected between 0.5-0.9. 5. Tuning of xgboost regularization parameters, for example: reg_lambda defaults to 1, reg_alpha defaults to 0. These parameters can reduce model complexity and increase model running speed, and appropriately reduce overfitting.
[0070] Step 3: Adjust the learning rate to obtain better model results.
[0071] Repeat the second step to check whether the local optimal parameters are reached. If the model effect does not improve after 100 consecutive trainings (this value can be set), the model training is terminated.
[0072] Step 5. Input the rainfall forecast data provided by the meteorological department and the landslide feature vector constructed from the historical time series before the forecast date into the trained XGBoost model to obtain the surface deformation prediction result of the landslide body. Taking the prediction of the amount of cracks on January 10 as an example, the amount of cracks, soil moisture content, rainfall collected from January 3 to January 9, and the rainfall predicted by the weather forecast on January 10 can be input into the trained model to obtain the predicted value of the amount of cracks on January 10.
[0073] Step 6: When the predicted value of the crack volume is greater than the threshold, a landslide disaster warning is automatically issued.
[0074] The threshold is set according to the actual situation.
Claims
1. A method for short-term intelligent early warning of landslides, characterized in that: include: Monitoring sensors are deployed on the landslide body to collect the landslide body characteristic data in real time, thereby constructing the landslide body characteristic vector; The rainfall forecast data provided by the meteorological department and the landslide feature vector constructed by the historical time series before the forecast date are input into the pre-trained XGBoost model to obtain the landslide surface deformation prediction result, wherein the XGBoost model predicts the landslide surface deformation characteristics on the forecast date based on the landslide feature vector constructed by the historical time series within a period of time before the forecast date and the rainfall predicted by the weather forecast on the forecast date; When the predicted result of the landslide surface deformation is greater than the threshold, a landslide disaster warning is issued; in, The landslide body characteristic vector includes three dimensions: the characteristics of the external inducing factors of the landslide body, the internal characteristics of the landslide body, and the surface deformation characteristics of the landslide body; The characteristics of the external inducing factors of the landslide body are characterized by rainfall; The internal characteristics of the landslide body are characterized by soil moisture content; The surface deformation characteristics of the landslide body are GNSS surface absolute displacement, or surface inclination change monitored by an inclinometer, or surface acceleration monitored by an accelerometer, or crack meter deformation time series data, or a combination of these data.
2. The method according to claim 1, characterized in that It also includes constructing training and test sets: Data preprocessing is performed on the monitored landslide feature vectors, including removing outliers and supplementing missing values; In the data preprocessing, the time series analysis model ARIMA model is used to fit the monitoring data when supplementing the missing values; The regularized data after supplementing the missing values are firstly subjected to first-order difference, then normalized and divided into training set and test set.
3. The method according to claim 2, characterized in that The formula for normalization is as follows: Among them, x is the sample monitoring data before normalization, x* is the sample monitoring data after normalization, max is the maximum value of the sample data, and min is the minimum value of the sample data.
4. The method according to claim 1, characterized in that: The parameters of the XGBoost model tuning include two major parameters, tree ensemble model parameters and regularization parameters.
5. The method according to claim 4, characterized in that The tree ensemble model parameters are responsible for fitting the samples; The tree ensemble model parameters include 'learning_rate' learning rate, 'n_estimators' maximum number of decision trees, 'max_depth' maximum depth of the tree, 'min_child_weight' minimum sample weight sum in the leaf node, if the sample weight sum of a leaf node is less than min_child_weight, the splitting process ends.
6. The method according to claim 4, characterized in that The regularization parameter helps control the complexity of the model and avoid overfitting; 'subsample' is the ratio of the subsamples used to train the model to the entire sample set to prevent overfitting, 'colsample_bytree' is the ratio of feature sampling when building a tree, 'gamma' is the minimum loss function drop value required for node splitting, 'reg_alpha' is the L1 regularization coefficient of the weight, and 'reg_lambda' is the L2 regularization coefficient of the weight.
7. The method according to claim 4, characterized in that The specific tuning steps of the XGBoost model tuning are: Step 1: Set the initial value of the parameter; Step 2: According to the greedy algorithm, determine the parameters {'n_estimators', ('max_depth', 'min_child_weight'), 'gamma', ('subsample', 'colsample_bytre'), ('reg-lambda', 'reg_alpha')} in turn when other parameters are determined; Step 3: Adjust the learning rate to obtain a better model effect; Repeat the second step until the model reaches the local optimal parameters and the model training is completed.
8. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.