Highway geological disaster prediction method and system

Abstract

The invention relates to the technical field of geological disaster detection, in particular to a highway geological disaster prediction method and system, and the method comprises the steps: data preprocessing: carrying out the preprocessing of collected data; model training: an Informer model is adopted, the model structure comprises an input layer, an encoder, a decoder and an output layer, and a sparse self-attention mechanism and a position encoding technology are utilized to enhance the processing capability of long-time sequence data and the accuracy of geological disaster prediction; the model training comprises pre-training and additional training for specific case data, and the pre-training is performed by using global data of NASA so as to learn wide soil humidity and meteorological data modes; the additional training is carried out on specific data on the basis of pre-training, and a data set is divided according to a 60% training set, a 20% verification set and a 20% test set; rMSE is used as a loss function, and k-fold cross validation is adopted to ensure the stability and generalization ability of the model; and model evaluation: evaluating the performance of the model by using a plurality of performance indexes, including accuracy, accuracy, recall rate, F1-score, ROC curve and AUC value.

Description

Technical Field

[0001] The present invention relates to the technical field of geological disaster detection, and in particular to a highway geological disaster prediction method and system thereof. Background Art

[0002] In the construction and maintenance of highway infrastructure, geological disasters such as landslides and mud-rock flows pose a serious threat to the safety and reliability of highways. Traditional geological disaster prediction methods mainly rely on empirical models and statistical analysis. These methods are usually based on historical data and simple meteorological factors for prediction, and fail to fully consider the complexity and diversity of geological conditions, resulting in insufficient prediction accuracy in the face of dynamically changing environments. The geological disaster prediction methods in the prior art generally include the following steps:

[0003] 1. Data collection: It mainly relies on manual observation and simple meteorological data collection, lacking a comprehensive capture of geological characteristics and dynamic changes in the environment.

[0004] 2. Data analysis: Analyzing limited data sets through statistical regression models, such as linear regression or logistic regression, cannot effectively handle nonlinear relationships in the data.

[0005] 3. Prediction model: It uses a simple mathematical model for prediction, lacks in-depth exploration of time series and spatial characteristics, and the generalization ability of the model is weak.

[0006] With the development of deep learning technology, prediction models based on time series data have shown superior performance in processing complex time-varying systems. However, their application in the field of geological disaster prediction is still in its infancy. Therefore, there is an urgent need for a new technical method that can improve prediction accuracy while adapting to dynamically changing environments and provide reliable technical support for highway safety.

[0007] The information disclosed in this background technology section is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art already known to those skilled in the art. Summary of the invention

[0008] The purpose of the present invention is to provide a highway geological disaster prediction method to solve the technical problems existing in the prior art.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] In a first aspect, the present invention provides a method for predicting highway geological disasters, comprising:

[0011] Data preprocessing: preprocess the collected data;

[0012] Model training: Informer model is used, and the model structure includes: input layer, encoder, decoder and output layer. Sparse self-attention mechanism and position encoding technology are used to enhance the processing ability of long time series data and the accuracy of geological disaster prediction. Model training includes pre-training and additional training for specific case data. The pre-training uses NASA's global data to learn a wide range of soil moisture and meteorological data patterns. The additional training is based on pre-training and performs additional training on specific data. The data set is divided into 60% training set, 20% validation set and 20% test set. RMSE is used as the loss function, and k-fold cross validation is used to ensure the stability and generalization ability of the model.

[0013] Model evaluation: Use a variety of performance indicators to evaluate the performance of the model, including accuracy, precision, recall, F1-score, ROC curve and AUC value.

[0014] Preferably, in the step of data preprocessing, the following are included: missing value filling, outlier processing and data standardization.

[0015] Preferably, in the step model training, the input layer is used to receive multidimensional time series data, including: soil moisture, temperature, precipitation; the encoder includes: a sparse self-attention module, a feedforward neural network layer, and a residual connection and layer normalization layer; the decoder includes: a masked self-attention module, a feedforward neural network layer, and a residual connection and layer normalization layer; the output layer uses ReLU as a nonlinear activation function.

[0016] Preferably, in the step model training, the pre-training uses NASA's surface soil moisture data set and the global geological disaster event database to provide geological and meteorological information, and integrates the soil moisture data with the GLC data through matching of timestamps and geographic locations.

[0017] Preferably, data set integration includes the following steps:

[0018] Time matching: Match the time of disaster events in GLC with soil moisture observation data according to the time of event occurrence;

[0019] Location matching: Based on the geographic coordinates, the longitude and latitude of the GLC event are matched with the location of the soil moisture data to ensure that each event can be associated with the corresponding soil moisture data.

[0020] Preferably, in the step model training, k-fold cross validation includes the following steps:

[0021] Dataset partitioning: The entire data set is randomly divided into k subsets of equal size, each of which contains a portion of training samples;

[0022] Training and validation cycle: loop k times: for each subset, perform the following steps: use k-1 subsets as training sets to train the geological disaster prediction model; use the remaining 1 subset as the validation set to evaluate the performance of the model to calculate the prediction error of the model on the validation set;

[0023] Calculate the average performance: After completing k training and validation cycles, calculate the average of all validation results as the overall performance indicator of the model.

[0024] In a second aspect, the present invention provides a highway geological disaster prediction system, which adopts the highway geological disaster prediction method, including: a data preprocessing module, a model training module, and a model evaluation module; the data preprocessing module is connected to the model training module, and is used to preprocess the collected data; the model training module is connected to the model evaluation module, and the model training module adopts an Informer model, including pre-training and additional training for specific case data, uses RMSE as a loss function, and adopts k-fold cross validation to ensure the stability and generalization ability of the model; the model evaluation module uses a variety of performance indicators to evaluate the performance of the model.

[0025] Preferably, the model structure of the model training module includes: an input layer, an encoder, a decoder and an output layer connected in sequence; the input layer is used to receive multidimensional time series data; the encoder includes: a sparse self-attention module, a feedforward neural network layer, a residual connection and a layer normalization layer; the decoder includes: a masked self-attention module, a feedforward neural network layer, a residual connection and a layer normalization layer; the output layer uses ReLU as a nonlinear activation function.

[0026] By adopting the above technical solution, the present invention has the following beneficial effects:

[0027] The present invention analyzes the geological and meteorological data along the highway to achieve early warning of potential geological disasters to ensure the safe operation of highway infrastructure.

[0028] 1. Technical Effects:

[0029] 1. Combining the innovation of the Informer model:

[0030] The present invention adopts the Informer model, which can effectively process long time series data through sparse self-attention mechanism and position encoding technology, and improve the accuracy of geological disaster risk prediction.

[0031] The sparse self-attention mechanism adopted in the present invention reduces the computational complexity, maintains the ability to capture global information, and is suitable for processing large-scale, high-dimensional geological and meteorological data.

[0032] 2. Fusion of multiple data sources:

[0033] By integrating NASA's surface soil moisture data and the Global Geohazard Event Database (GLC), the present invention can more comprehensively capture the precursors and influencing factors of geological disasters and enhance the prediction ability of the model.

[0034] The present invention adopts data fusion technology to transform multi-source heterogeneous data into a unified data format, thereby improving data utilization and model generalization ability.

[0035] 3. Efficient model training and evaluation methods:

[0036] The present invention utilizes a pre-trained model to accelerate the training process and combines k-fold cross-validation to improve the stability and generalization ability of the model.

[0037] The present invention comprehensively evaluates the model through multiple indicators (accuracy, precision, recall, F1-score and AUC value) to ensure the reliability of model performance.

[0038] 2. Economic Effects:

[0039] 1. Reduce economic losses caused by geological disasters:

[0040] The present invention can accurately predict the geological disaster risks along the highway, so as to take preventive measures in advance and reduce the damage of disasters to transportation and infrastructure.

[0041] The present invention improves the timeliness and accuracy of early warning, helps traffic management departments optimize resource allocation, and reduces disaster response costs.

[0042] 2. Improve the efficiency of highway construction and maintenance:

[0043] The present invention provides reliable geological disaster prediction data, assists highway planning and construction decision-making, and optimizes highway design and layout.

[0044] During highway operation, the present invention improves maintenance efficiency and reduces the impact of emergencies on traffic flow through real-time prediction and monitoring.

[0045] 3. Social Effects:

[0046] 1. Enhance public safety and security:

[0047] The present invention improves the accuracy of geological disaster prediction, provides more reliable safety protection for residents and drivers along the route, and reduces the risk of casualties.

[0048] The present invention improves society's ability to cope with natural disasters, and enhances social security and the service quality of the highway system.

[0049] 2. Promote scientific and technological progress and technology application:

[0050] This invention promotes the application of deep learning models in the field of geological disaster prediction and promotes the deep integration of artificial intelligence technology in the fields of geology and transportation.

[0051] The present invention improves the application level of related technologies and promotes the innovative development of geological disaster prediction technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 An overall architecture diagram of a highway geological disaster prediction system provided by an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of a model structure provided by an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of the structure of an encoder and a decoder provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0058] Embodiment 1

[0059] Combination Figures 1 to 3 As shown, this embodiment 1 provides a highway geological disaster prediction method, which includes:

[0060] Data preprocessing: preprocess the collected data;

[0061] Model training: Informer model is used, and the model structure includes: input layer, encoder, decoder and output layer. Sparse self-attention mechanism and position encoding technology are used to enhance the processing ability of long time series data and the accuracy of geological disaster prediction. Model training includes pre-training and additional training for specific case data. The pre-training uses NASA's global data to learn a wide range of soil moisture and meteorological data patterns. The additional training is based on pre-training and performs additional training on specific data. The data set is divided into 60% training set, 20% validation set and 20% test set. RMSE is used as the loss function, and k-fold cross validation is used to ensure the stability and generalization ability of the model.

[0062] Model evaluation: Use a variety of performance indicators to evaluate the performance of the model, including accuracy, precision, recall, F1-score, ROC curve and AUC value.

[0063] In this embodiment, the specific prediction method is as follows:

[0064] 1. Data preprocessing

[0065] Data preprocessing is an important step to ensure the quality of model input data. It can ensure the integrity and consistency of the data, enable the model to learn the inherent patterns of the data under uniform input conditions, maximize the use of data feature information, and improve the generalization ability and prediction performance of the model. The specific steps are as follows:

[0066] 1.1 Missing value filling:

[0067] In order to ensure the integrity of the data, linear interpolation is used to fill in the missing values:

[0068]

[0069] The missing value filling method fills the missing data by calculating the average of the adjacent observations before and after, maintaining the continuity and smoothness of the data, which helps to improve the model's sensitivity to time series changes.

[0070] 1.2 Outlier processing:

[0071] Use the interquartile range (IQR) method to identify outliers to ensure data accuracy and consistency:

[0072]

[0073] Q1 (first quartile): represents the value of the lower quarter of the data set, reflecting the lower position of the data.

[0074] Q3 (third quartile): represents the value of the upper quarter of the data set, reflecting the upper position of the data.

[0075] IQR (interquartile range): the difference between the third quartile and the first quartile, representing the range of the middle 50% of the data.

[0076] Outliers are identified using the following rules:

[0077] If the data point is lower than , it is considered as an outlier.

[0078] If the data point is higher than , it is considered as an outlier.

[0079] That is, outliers are defined as values ​​below or higher The removal of outliers can prevent them from having adverse effects on model training, ensure the quality and consistency of input data, enable the model to learn normal data patterns more accurately, and avoid interference with model training caused by outliers.

[0080] 1.3 Data Standardization:

[0081] All data are standardized to a standard normal distribution with a mean of 0 and a standard deviation of 1 to eliminate the impact of data scale differences on model training.

[0082]

[0083] in, is the standardized data, is the original data, is the sample mean, is the sample standard deviation. Standardization can improve the stability and convergence speed of model training, so that the impact of data with different characteristics on the model is at the same scale.

[0084] 2. Model structure and principle

[0085] The Informer model is a deep learning model designed for long time series data. It combines sparse self-attention mechanism and efficient time series capture capability, making it suitable for geological disaster prediction.

[0086] 2.1 Model Principle:

[0087] 1. Sparse self-attention mechanism: By selectively focusing on important information in the input sequence, the sparse self-attention mechanism reduces the computational complexity while maintaining the ability to capture global information in the sequence. This mechanism is suitable for processing long time series because it can adaptively select key nodes and reduce time complexity.

[0088] 2. Position encoding: By adding position information to the input sequence, position encoding enables the model to capture the temporal features and spatial relationships in the sequence, ensuring the model's sensitivity to the input order. Through position encoding, the model can learn the temporal and spatial relationships of the sequence without explicit position information, enhancing the model's performance when processing time series data.

[0089] Uniqueness: The positional encoding generated by the sine and cosine functions is periodic and unique, and can generate a unique code for each position without relying on the length of the sequence.

[0090] Smoothness: Smooth changes in positional encodings can help the model capture the relative distance and order information between different positions in the sequence.

[0091] Frequency encoding: Using sine and cosine functions of different frequencies allows the model to focus on patterns in the sequence at different granularities.

[0092]

[0093]

[0094] and The positions are No. and Dimensional position encoding.

[0095] is a position index, indicating a specific position in the sequence.

[0096] is the dimension index, indicating the specific dimension in the model input vector.

[0097] is the dimensionality of the model, i.e. the dimension of the feature vector at each position in the input sequence.

[0098] and are sine and cosine functions used to generate position information in different dimensions.

[0099] 2.2 Model structure:

[0100] 2.2.1 Input layer: Receives multidimensional time series data, including soil moisture, temperature, precipitation, etc.

[0101] 2.2.2 Encoder-Decoder Structure:

[0102] 1. Encoder:

[0103] The encoder in Informer is composed of multiple stacked sparse self-attention modules, feedforward neural network layers, and residual connection layers.

[0104] Sparse Self-Attention Module:

[0105] The sparse self-attention mechanism is used to selectively focus on important information in the input sequence, thereby improving the model's ability to process long time series data.

[0106] Global information capture: Through the self-attention mechanism, the model can effectively capture the correlation between all elements in the input sequence, thereby capturing global information.

[0107] Dynamic weight assignment: The self-attention mechanism can dynamically assign different attention weights to each input element to highlight important information.

[0108] Sparsity optimization: The sparse self-attention mechanism optimizes the efficiency of processing long time series data by reducing unnecessary calculations.

[0109] Self-Attention Mechanism

[0110] The self-attention mechanism captures global dependencies by calculating the correlation between each element in the input sequence and other elements. The commonly used self-attention mechanism formula is as follows:

[0111] (1) Calculate the attention score:

[0112]

[0113] in:

[0114] (Query): Query matrix, used to query the input. Each input element generates its query vector through linear transformation.

[0115] (Key): Key matrix, used to store the key of input information. Each input element generates its key vector through linear transformation.

[0116] (Value): Value matrix, used to store the value of input information. Each input element generates its value vector through linear transformation.

[0117] : Dimension size of the key vector. Used to scale dot products to avoid vanishing or exploding gradients.

[0118] : The activation function applied to the attention scores to normalize them into a probability distribution.

[0119] : represents the dot product between the query matrix and the transpose of the key matrix, and calculates the attention score matrix.

[0120] (2) Sparsity:

[0121] In the sparse self-attention mechanism, sparsity is achieved by pruning the attention matrix and threshold filtering, retaining only important attention weights, thereby reducing computational complexity. The specific implementation steps are as follows:

[0122] Pruning: By setting a threshold, only elements with attention scores higher than the threshold are retained.

[0123] Threshold filtering: By adjusting the threshold, the computational efficiency is optimized while maintaining the ability to capture global information in the sequence.

[0124] (3) Output generation:

[0125]

[0126] in:

[0127] : is the attention weight of the i-th element, calculated by the softmax function.

[0128] : The value vector of the i-th input element.

[0129] : is the output vector after weighted summation, representing the input sequence representation after the attention mechanism.

[0130] 2. Feedforward Neural Network (FFN):

[0131] Feedforward Neural Network (FFN) is an important component of encoder and decoder for nonlinear transformation of output at each time step. Feedforward neural network performs nonlinear transformation on the output of each time step through two layers of fully connected networks to improve the expressiveness of the model.

[0132] Increased expressive power: By introducing nonlinear activation functions, FFN can capture the nonlinear relationship of input features and improve the model's ability to learn complex patterns.

[0133] Feature Transformation: Through linear transformation, FFN is able to rescale and combine input features to provide richer feature representations for subsequent attention calculations.

[0134] Inter-layer information transfer: In the encoder and decoder structures, FFN provides a mechanism that enables each layer to transform the input independently, thereby promoting the fusion and sharing of features at different levels.

[0135]

[0136] in:

[0137] x: Input vector. Represents the feature representation passed down from the previous layer (or through the self-attention module).

[0138] : The weight matrix of the first layer linear transformation, size is ,in is the dimension of the input features, is the hidden dimension of the feed-forward layer.

[0139] : The bias vector of the first layer linear transformation, dimension is .

[0140] : Activation function, expressed as a Rectified Linear Unit, defined as . It introduces nonlinear characteristics, allowing the model to better fit complex data patterns.

[0141] : The weight matrix of the second layer linear transformation, size is , transform the hidden layer features back to the original feature dimension.

[0142] : The bias vector of the second layer linear transformation, dimension is .

[0143] Workflow:

[0144] Linear transformation and activation: The input vector x first passes through the first layer of linear transformation (i.e. matrix multiplication ), and then a nonlinear transformation is performed through the ReLU activation function. This step is used to increase the nonlinear expression ability of the model, enabling it to learn the complex relationship between input features.

[0145] Second linear transformation: The output of the activation function is then transformed by the second layer of linear transformation (i.e. matrix multiplication). , to restore the dimension of the original input. This step remaps the activated features to the output space.

[0146] Output generation: The vector output by the FFN is passed to the self-attention module in the next encoder or decoder layer or subsequent neural network layer.

[0147] 3. Residual connection and layer normalization:

[0148] Residual connection and layer normalization are key components in the encoder and decoder structures for stabilizing the training process and improving model performance. Through the combination of residual connection and layer normalization, the model can more effectively capture global features and temporal relationships when processing long sequence data, while improving the stability of the model and training efficiency.

[0149] Residual Connection

[0150] Residual connection is a network structure design that alleviates the gradient vanishing problem of deep networks by adding skip connections between layers of neural networks.

[0151] Alleviating the vanishing gradient problem: By providing a direct gradient propagation path, residual connections make deep networks easier to train.

[0152] Enhanced model stability: Residual connections help the model maintain stable performance as the depth increases, avoiding degradation problems caused by increased model complexity.

[0153] The specific formula is as follows:

[0154]

[0155] in:

[0156] : Represents the calculation result of the current network layer, usually the output after the self-attention mechanism or feedforward neural network (FFN).

[0157] : Input vector. Represents the input data of this layer.

[0158] : Represents the output result after residual connection, that is, the result after adding the output of the current layer to the input.

[0159] Working principle:

[0160] Directly connect input and output: Residual connection preserves the original input information by adding the input directly to the output. This design enables the network to better propagate gradients during training and alleviate the gradient vanishing problem.

[0161] Feature reuse: Through skip connections, the model can reuse the features of the previous layer in each layer, which helps to integrate information at multiple levels and improve the feature expression ability of the model.

[0162] ② Layer Normalization

[0163] Layer normalization is a regularization technique for neural networks that improves training speed and model stability by normalizing the activations of each layer.

[0164] Improve training speed: By stabilizing the input distribution of each layer, layer normalization accelerates the convergence of the model and improves training efficiency.

[0165] Enhance the robustness of the model: Layer normalization reduces the impact of input data perturbations on model training and improves the generalization ability of the model.

[0166] The specific formula is as follows:

[0167]

[0168]

[0169] in:

[0170] : Represents the input feature vector of the current layer.

[0171] : represents the mean of the input features of the current layer, and the calculation formula is ,in is the dimension of the input features.

[0172] : represents the variance of the input feature of the current layer, and the calculation formula is .

[0173] : is a very small constant used to prevent division by zero errors, usually with a value of 10−610^{-6}10−6.

[0174] : It is the normalized feature.

[0175] and : are learnable parameters used to scale and translate the normalized output to restore the representation capability of the network.

[0176] Working principle:

[0177] Normalized features: Normalize the mean and variance of the input features so that the input features of each layer have zero mean and unit variance. This helps eliminate scale differences between features.

[0178] Scaling and translation: Through learnable parameters γ and β, layer normalization not only standardizes the input, but also maintains the expressiveness of the network, and enables the model to adapt to different data distributions by adjusting the scale and offset of the output.

[0179] 2.Decoder:

[0180] It also consists of a sparse self-attention module, a feedforward neural network layer, and a residual connection layer, and additionally includes a masking mechanism to ensure that only known sequence information is predicted.

[0181] 1. Masked Self-Attention Module:

[0182] The Masked Self-Attention Module is an important component in the decoder structure. It is used to ensure that only known information is used to generate outputs when making predictions, thereby avoiding information leakage (i.e., not using future time step information when predicting the current time step of the sequence). The core of the Masked Self-Attention mechanism is to apply a mask to future time steps, allowing the model to only focus on known information.

[0183] Preventing information leakage: Through the masking mechanism, it is ensured that the decoder does not access future information when generating sequences, thereby achieving causal modeling.

[0184] Enhanced sequence modeling capabilities: The masked self-attention mechanism enables the model to better capture temporal relationships, especially in generation and translation tasks, by gradually generating sequences to ensure the rationality of the prediction.

[0185] The specific formula is as follows:

[0186]

[0187]

[0188]

[0189] in:

[0190] Q: Query Matrix, which represents the matrix after the decoder input has been linearly transformed.

[0191] K: Key Matrix, which represents the matrix after the decoder input has been linearly transformed.

[0192] : The dimension of the key vector, used to scale the dot product to prevent the value from being too large.

[0193] : Mask Matrix, used to perform mask operations on future time steps. If the future time step is not visible, the corresponding position is set to negative infinity, otherwise it is 0.

[0194] : Represents the attention weight of position i to position j, and the attention score is normalized to between 0 and 1 through the softmax function.

[0195] : Value Matrix, which represents the matrix of the decoder input after linear transformation.

[0196] : represents the final attention output, which is the weighted sum of the value vectors at each position.

[0197] Working principle:

[0198] Input Transformation: The input sequence is first transformed into query, key, and value vectors, denoted as Q, K, V respectively.

[0199] Mask application: The attention scores of future time steps are masked through the mask matrix M, so that the model can only use the information of previous time steps when predicting a certain time step.

[0200] Attention calculation: Calculate the attention score by scaling the dot product , and calculate the attention weights through the softmax function after applying the mask .

[0201] Output generation: The value vector is weighted and summed using the attention weights to generate the final attention output. This output reflects the model's comprehensive consideration of the known information at the current time step.

[0202] 2. Sparse Self-Attention Module:

[0203] Function: Same as the encoder's sparse self-attention module, capturing sequence dependencies through self-attention across time steps.

[0204] 3. Feed-Forward Neural Network (FFN):

[0205] Function: Similar to the feedforward neural network module of the encoder, it performs nonlinear transformation on the decoded output of each time step to improve the feature expression ability of the decoder.

[0206] 4. Residual connection and layer normalization:

[0207] Function: Same as the encoder, enhancing model stability and training efficiency.

[0208] 2.2.3 Output layer:

[0209] The function of the output layer is to map the high-dimensional feature vector generated by the decoder into specific geological disaster prediction values, which can be directly used to assess and warn of the risk of geological disasters. The output layer uses a fully connected layer combined with a ReLU activation function to complete this conversion. ReLU is a commonly used activation function because it is simple to calculate and can effectively alleviate the gradient vanishing problem. The output layer in the Informer model used in the present invention uses ReLU (Rectified Linear Unit) as a nonlinear activation function. Its purpose is to introduce nonlinearity in the output stage, which can not only achieve effective conversion from linear transformation to nonlinear output, but also enhance the performance of the model in processing complex geological disaster prediction tasks. Through the use of ReLU, the model can capture potential data patterns in changing disaster scenarios and improve overall prediction accuracy and practicality.

[0210] Introducing nonlinearity: The ReLU activation function introduces nonlinearity by setting negative values ​​to zero, enabling the model to better fit complex feature-output relationships, especially in geological disaster prediction, and to capture complex patterns of data.

[0211] Improve computational efficiency: ReLU is simple to calculate and only requires comparison and maximum value operations, which improves the training speed and efficiency of the model.

[0212] Alleviate the vanishing gradient problem: The gradient of ReLU in the positive interval is always 1, which can effectively avoid the vanishing gradient and accelerate the convergence of deep networks.

[0213] Enhance model sparsity: Some outputs of ReLU are 0, which makes the model produce sparse activation and improves the generalization ability of the model.

[0214] Linear transformation:

[0215] First, the feature vector output by the decoder is linearly transformed through the fully connected layer:

[0216]

[0217] in:

[0218] z is the output of the linear transformation.

[0219] It is the weight matrix of the output layer, which is responsible for projecting the high-dimensional feature vector hhh output by the decoder into the target output space.

[0220] h is the output feature vector of the decoder.

[0221] is a bias vector used to adjust the result of the linear transformation, providing additional flexibility.

[0222] ReLU activation function:

[0223] After the linear transformation, the ReLU activation function is used to perform nonlinear mapping on the result:

[0224]

[0225] It is the output of the activation function, which represents the predicted value after nonlinear transformation.

[0226] Ensure that the output is non-negative and introduce non-linearity by setting negative values ​​to zero.

[0227] Final output:

[0228] The output of the ReLU activation function is used as the final prediction value for geological hazard risk assessment:

[0229]

[0230] in:

[0231] It is the predicted output of the model, representing the geological hazard risk score at a future moment.

[0232] The result of linear transformation is converted into prediction output with nonlinear characteristics.

[0233] Combining meaning and function

[0234] Introducing nonlinearity: The ReLU activation function introduces nonlinearity by setting negative values ​​to zero, enabling the model to better fit complex feature-output relationships, especially in geological disaster prediction, and to capture complex patterns of data.

[0235] Improve computational efficiency: ReLU is simple to calculate and only requires comparison and maximum value operations, which improves the training speed and efficiency of the model.

[0236] Alleviate the vanishing gradient problem: The gradient of ReLU in the positive interval is always 1, which can effectively avoid the vanishing gradient and accelerate the convergence of deep networks.

[0237] Enhance model sparsity: Some outputs of ReLU are 0, which makes the model produce sparse activation and improves the generalization ability of the model.

[0238] 3. Model Training

[0239] Model training includes pre-training and additional training for specific case data to ensure the effectiveness of the model in different data sets and application scenarios. Model training adopts supervised learning method, and the detailed process is as follows:

[0240] 3.1 Model Pre-training

[0241] Purpose: Through pre-training, the model can learn generalized feature representations on large-scale and diverse datasets.

[0242] process:

[0243] Pre-training data collection:

[0244] The present invention uses NASA's "surface soil moisture" data set and the Global Landslide Catalog (GLC) to provide geological and meteorological information. The geological disaster event information provided by GLC includes time, location, disaster type, etc. By integrating multi-source data such as soil moisture, temperature, precipitation, etc., a comprehensive assessment of geological disasters can be achieved.

[0245] Dataset integration:

[0246] The soil moisture data is integrated with the GLC data by matching the timestamp and geographic location. The specific steps include:

[0247] 1. Time matching: According to the time of event occurrence, the time of disaster events in GLC is matched with soil moisture observation data.

[0248] 2. Location matching: Based on the geographic coordinates, the longitude and latitude of the GLC event are matched with the location of the soil moisture data to ensure that each event can be associated with the corresponding soil moisture data.

[0249] Data format: The integrated pre-training dataset example is as follows:

[0250] Data format: stored in JSON format, including the following fields:

[0251] {

[0252] "time": "2023-08-04T00:00:00Z",

[0253] "latitude": 35.6895,

[0254] "longitude": 139.6917,

[0255] "soil_moisture": 0.23,

[0256] "temperature": 298.15,

[0257] "precipitation": 0.5,

[0258] "landslide_event": 1

[0259] }

[0260] Timestamp (time): observation time.

[0261] Latitude and longitude: Geographical location.

[0262] Soil moisture (soil_moisture): cubic meter of water / cubic meter of soil.

[0263] Temperature: Kelvin (K).

[0264] Precipitation: mm / hour.

[0265] Landslide event (landslide_event): 1 means a landslide has occurred, 0 means no landslide has occurred.

[0266] The entire pre-training dataset contains about 10,000 records, of which the occurrence rate of landslide events is about 5%.

[0267] 3.2 Loss Function:

[0268] The loss function is used to measure the deviation between the model prediction value and the actual value, thereby guiding the optimization of model parameters. The present invention uses the root mean square error (RMSE) as the loss function to evaluate the accuracy of the model in the geological disaster prediction task.

[0269] Evaluate prediction accuracy: RMSE directly reflects the average deviation between the model's predicted value and the actual value. A lower RMSE value means that the model's prediction is closer to the actual situation, and vice versa, it means that the prediction is inaccurate.

[0270] Guide model optimization: During the training process, RMSE is used as an objective function to guide the model to adjust parameters through optimization algorithms (such as gradient descent) to minimize the prediction error.

[0271] Unified metric: Since RMSE maintains the same units and scale as the original data, it facilitates direct comparison of the predictive performance of different models or algorithms.

[0272] Sensitivity to outliers: RMSE is more sensitive to large errors (outliers), so when there are large errors in the data, the RMSE value will increase significantly, indicating that the model may need further optimization to handle outliers.

[0273] The loss function formula is:

[0274]

[0275] in:

[0276] : Represents the value of the root mean square error, which is a key indicator of the model's prediction performance.

[0277] N: represents the total number of data samples. In the geological disaster prediction task, NNN represents the number of data points used to train or evaluate the model.

[0278] : represents the true value of the i-th sample. In this patent, Provide actual geological hazard risk scores or event occurrences.

[0279] : represents the predicted value of the i-th sample, that is, the geological hazard risk score output by the model.

[0280] : Represents the prediction error of the iiith sample, that is, the difference between the model prediction value and the true value.

[0281] : represents the square of the error of the iiith sample. The square operation eliminates the difference in the positive and negative signs of the errors and unifies all errors into non-negative numbers.

[0282] : Represents the average of the squared errors of all samples, that is, the mean square error (MSE).

[0283] ⋅: Square root operation, used to convert the mean square error to the root mean square error to maintain consistency with the original data scale.

[0284] By adopting RMSE as the loss function, the geological disaster prediction model in this patent can effectively adjust parameters during the training process, improve prediction accuracy, and provide reliable data support and technical guarantee for highway geological disaster warning.

[0285] 3.3 Training process:

[0286] Dataset division: The dataset is divided into 60% training set, 20% validation set and 20% test set to ensure the independence of model training, validation and testing.

[0287] Cross Validation:

[0288] In order to evaluate the stability and generalization ability of the geological disaster prediction model, the k-fold cross-validation method was used. K-fold cross-validation is a commonly used model validation technique that can effectively prevent the overfitting problem of the model and provide a more accurate estimate of the performance of the model on unseen data. The basic idea of ​​k-fold cross-validation is to divide the data set into k non-overlapping subsets (or "folds"), and then perform k training and validation. Each time training, k-1 subsets are used for training, and the remaining subset is used for validation. In this way, the model can be trained and evaluated under different data partitions, and finally the average performance of the model is obtained.

[0289] Improve model stability: By performing multiple training and validation on different data partitions, k-fold cross-validation reduces the bias that may be introduced by a single data partition and provides more reliable performance estimates.

[0290] Prevent overfitting: Through multiple validations, we can better observe the performance of the model on different data and prevent the model from overfitting to specific training data.

[0291] Optimize model parameters: Through cross-validation, you can more accurately adjust the model's hyperparameters (such as learning rate, regularization coefficient, etc.) and select the model configuration with the best performance.

[0292] Comprehensively evaluate model performance: k-fold cross validation provides performance indicators of the model on multiple validation sets, which can comprehensively evaluate the performance of the model on different data and provide more reliable guarantees for the practical application of the model.

[0293] The specific steps of k-fold cross validation

[0294] Dataset division:

[0295] The entire data set is randomly divided into k equal-sized subsets (folds), each of which contains a portion of training samples. In this patent, 5-fold cross validation (k=5) is used, that is, the data set is divided into 5 subsets.

[0296] Training and validation loop:

[0297] Loop k times: For each subset, perform the following steps:

[0298] Training: Use k-1 subsets as training sets to train the geological disaster prediction model. Use these data to update model parameters and learn patterns and laws in the data.

[0299] Validation: Use the remaining subset as the validation set to evaluate the performance of the model. Calculate the prediction error of the model on the validation set (using evaluation indicators such as RMSE and accuracy).

[0300] Calculate the average performance:

[0301] After completing k training and validation cycles, the average of all validation results is calculated as the overall performance indicator of the model.

[0302] This approach ensures that each data point is used once as validation data and k-1 times as training data.

[0303] 3. Optimizer: Use the Adam optimizer, set the initial learning rate to 0.001, and set the batch size to 64. The Adam optimizer can accelerate the convergence process of the model through an adaptive learning rate adjustment mechanism.

[0304] 4. Training iteration: Perform multiple rounds of training iterations, monitor the loss value on the validation set, and use an early stopping strategy to prevent overfitting. When the loss on the validation set no longer decreases significantly, stop training.

[0305] 4. Model Evaluation

[0306] In the geological disaster prediction model of this patent, the model evaluation uses a variety of performance indicators to evaluate the performance of the model, including accuracy, precision, recall, F1-score, ROC curve and AUC value. Accuracy provides an evaluation of the overall classification ability, precision and recall reflect the reliability and coverage of the model on positive samples respectively, F1-score balances the two, and ROC curve and AUC value further provide classification ability analysis of the model under different decision thresholds. These evaluation indicators can comprehensively measure the prediction performance of the model under complex geological conditions, help optimize the model structure and training strategy, and ultimately achieve efficient early warning of highway geological disasters.

[0307] 4.1 Accuracy:

[0308]

[0309] Explanation of symbols:

[0310] : True positives, the number of samples correctly predicted as positive by the model.

[0311] : True negative examples, the number of samples correctly predicted as negative by the model.

[0312] : False positive examples, the number of samples that the model incorrectly predicts as positive.

[0313] : False negative examples, the number of samples that the model incorrectly predicts as negative.

[0314] Accuracy measures the proportion of samples that are correctly classified by the model among all samples. It is an important indicator for evaluating the overall predictive ability of the model and is used to evaluate the overall classification ability of the model when the samples are balanced.

[0315] 4.2 Precision:

[0316]

[0317] The precision rate indicates the proportion of samples that are actually positive among all samples predicted to be positive, reflecting the reliability of the model in predicting positive samples.

[0318] 4.3 Recall:

[0319]

[0320] The recall rate indicates the proportion of samples that are correctly predicted as positive among all samples that are actually positive, reflecting the model's ability to cover positive samples. Precision and recall are particularly important when evaluating the model's prediction of a specific class (such as disaster occurrence). A high precision rate indicates that the model is more reliable when predicting positive, while a high recall rate indicates that the model can effectively capture most of the actual positive samples.

[0321] F1-score:

[0322]

[0323] F1-score is the harmonic average of precision and recall, which takes into account the accuracy and coverage of the model. It is a comprehensive indicator used to balance model performance. When there is a trade-off between precision and recall, F1-score provides a balanced evaluation.

[0324] 4.5 ROC curve and AUC value:

[0325] The ROC curve and AUC value are used to evaluate the classification ability of the model at different decision thresholds and help select the optimal threshold to maximize the model performance.

[0326] ROC curve: The ROC (Receiver Operating Characteristic) curve plots the relationship between the true positive rate (TPR) and the false positive rate (FPR). Each point on the curve corresponds to a different classification threshold.

[0327]

[0328]

[0329] TPR (True Positive Rate), also known as sensitivity or recall, indicates the proportion of samples that are correctly predicted as positive among all actual positive samples. FPR (False Positive Rate) indicates the proportion of samples that are incorrectly predicted as positive among all actual negative samples. The ROC curve evaluates the performance of the classifier by plotting the true positive rate (TPR) against the false positive rate (FPR). The larger the area under the curve, the better the performance of the classifier.

[0330] AUC value (Area Under Curve): AUC is the area under the ROC curve, ranging from 0 to 1. The closer the AUC value is to 1, the better the classification performance of the model. The AUC value reflects the average performance of the model at different thresholds. A higher AUC value indicates that the model has better performance in distinguishing positive and negative samples.

[0331] The formula is:

[0332]

[0333] in:

[0334] is the true positive rate under the ith threshold.

[0335] is the false positive rate under the ith threshold.

[0336] n is the number of thresholds.

[0337] Trapezoidal rule: Used to approximate integrals by calculating the areas between adjacent points (viewed as trapezoids) to approximate the total area under the curve.

[0338] AUC evaluation criteria

[0339] AUC = 1: indicates a perfect classification model that can perfectly distinguish between positive and negative samples at any threshold.

[0340] AUC = 0.5: indicates that the model has no discriminatory ability and is equivalent to random guessing.

[0341] AUC < 0.5: indicates that the model performs extremely poorly and it is recommended to reverse the prediction (usually it is impossible to become a good model after reversal).

[0342] By adopting the above technical solution, the present invention has the following beneficial effects:

[0343] 1. Technical Effects:

[0344] 1. Combining the innovation of the Informer model:

[0345] The present invention adopts the Informer model, which can effectively process long time series data through sparse self-attention mechanism and position encoding technology, and improve the accuracy of geological disaster risk prediction.

[0346] The sparse self-attention mechanism adopted in the present invention reduces the computational complexity, maintains the ability to capture global information, and is suitable for processing large-scale, high-dimensional geological and meteorological data.

[0347] 2. Fusion of multiple data sources:

[0348] By integrating NASA's surface soil moisture data and the Global Geohazard Event Database (GLC), the present invention can more comprehensively capture the precursors and influencing factors of geological disasters and enhance the prediction ability of the model.

[0349] The present invention adopts data fusion technology to transform multi-source heterogeneous data into a unified data format, thereby improving data utilization and model generalization ability.

[0350] 3. Efficient model training and evaluation methods:

[0351] The present invention utilizes a pre-trained model to accelerate the training process and combines k-fold cross-validation to improve the stability and generalization ability of the model.

[0352] The present invention comprehensively evaluates the model through multiple indicators (accuracy, precision, recall, F1-score and AUC value) to ensure the reliability of model performance.

[0353] 2. Economic Effects:

[0354] 1. Reduce economic losses caused by geological disasters:

[0355] The present invention can accurately predict the geological disaster risks along the highway, so as to take preventive measures in advance and reduce the damage of disasters to transportation and infrastructure.

[0356] The present invention improves the timeliness and accuracy of early warning, helps traffic management departments optimize resource allocation, and reduces disaster response costs.

[0357] 2. Improve the efficiency of highway construction and maintenance:

[0358] The present invention provides reliable geological disaster prediction data, assists highway planning and construction decision-making, and optimizes highway design and layout.

[0359] During highway operation, the present invention improves maintenance efficiency and reduces the impact of emergencies on traffic flow through real-time prediction and monitoring.

[0360] 3. Social Effects:

[0361] 1. Enhance public safety and security:

[0362] The present invention improves the accuracy of geological disaster prediction, provides more reliable safety protection for residents and drivers along the route, and reduces the risk of casualties.

[0363] The present invention improves society's ability to cope with natural disasters, and enhances social security and the service quality of the highway system.

[0364] 2. Promote scientific and technological progress and technology application:

[0365] This invention promotes the application of deep learning models in the field of geological disaster prediction and promotes the deep integration of artificial intelligence technology in the fields of geology and transportation.

[0366] The present invention improves the application level of related technologies and promotes the innovative development of geological disaster prediction technology.

[0367] Embodiment 2

[0368] Combination Figures 1 to 3 As shown, this embodiment 2 provides a highway geological disaster prediction system based on the embodiment 1. This embodiment 2 includes the technical content disclosed in the embodiment 1, and the technical content of this embodiment 2 that is the same as that of the embodiment 1 is not repeated. The differences between this embodiment 2 and the embodiment 1 are described below.

[0369] This embodiment provides a highway geological disaster prediction system, which adopts the highway geological disaster prediction method described in Example 1, including: a data preprocessing module, a model training module, and a model evaluation module; the data preprocessing module is connected to the model training module to preprocess the collected data; the model training module is connected to the model evaluation module, the model training module adopts the Informer model, including pre-training and additional training for specific case data, using RMSE as the loss function, and using k-fold cross validation to ensure the stability and generalization ability of the model; the model evaluation module uses a variety of performance indicators to evaluate the performance of the model. Among them, the model structure of the model training module includes: an input layer, an encoder, a decoder, and an output layer connected in sequence; the input layer is used to receive multi-dimensional time series data; the encoder includes: a sparse self-attention module, a feedforward neural network layer, and a residual connection and a layer normalization layer; the decoder includes: a masked self-attention module, a feedforward neural network layer, a residual connection and a layer normalization layer; the output layer uses ReLU as a nonlinear activation function.

[0370] Embodiment 3

[0371] Combination Figures 1 to 3As shown, this embodiment 3 provides a specific application example based on the embodiments 1 and 2. For example, during the rainy season on mountain highway sections, landslides are common and high-risk geological disasters. In the past few years, many highways in a certain area have been affected by landslides, resulting in traffic interruptions and significant economic losses.

[0372] 1. Data collection and integration

[0373] Data source: Data in the same format as NASA's "Surface Soil Moisture" dataset are obtained from local geological monitoring stations and meteorological stations, including soil moisture, temperature, precipitation, etc.

[0374] Dataset integration: Combined with landslide event records, the locally collected data are matched with the landslide events in time and space to form training, validation, and test data sets.

[0375] Data Format:

[0376] {

[0377] "time": "2023-07-01T00:00:00Z",

[0378] "latitude": 34.0522,

[0379] "longitude": 117.9436,

[0380] "soil_moisture": 0.29,

[0381] "temperature": 293.15,

[0382] "precipitation": 2.3,

[0383] "landslide_event": 1

[0384] }

[0385] The dataset includes about 5,000 records, with an occurrence rate of 8% for landslide events.

[0386] 2. Model training

[0387] Pre-training: Pre-trained using NASA’s global data to learn a wide range of soil moisture and meteorological data patterns.

[0388] Additional training: Based on pre-training, additional training is performed on the specific data of the province. The dataset is divided into 60% training set, 20% validation set and 20% test set.

[0389] Evaluation method: RMSE is used as the loss function, and k-fold cross validation is adopted to ensure the stability and generalization ability of the model.

[0390] 3. Performance Evaluation

[0391] The model evaluation results on the test dataset are as follows:

[0392] Accuracy: 86%

[0393] Precision: 84%

[0394] Recall: 82%

[0395] F1-score: 83%

[0396] AUC value: 0.88

[0397] The evaluation indicators show that the present invention performs well in actual scene applications. Through this geological disaster prediction method, the traffic management department can effectively warn of landslide risks along the highway, significantly improve road safety, and reduce potential losses caused by landslides. This system provides important technical support for highway management and has significant practical application value.

[0398] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A highway geological disaster prediction method, characterized in that: include: Data preprocessing: preprocess the collected data; Model training: Informer model is used, and the model structure includes: input layer, encoder, decoder and output layer. Sparse self-attention mechanism and position encoding technology are used to enhance the processing ability of long time series data and the accuracy of geological disaster prediction. Model training includes pre-training and additional training for specific case data. The pre-training uses NASA's global data to learn a wide range of soil moisture and meteorological data patterns. The additional training is based on pre-training and performs additional training on specific data. The data set is divided into 60% training set, 20% validation set and 20% test set. RMSE is used as the loss function, and k-fold cross validation is used to ensure the stability and generalization ability of the model. Model evaluation: Use a variety of performance indicators to evaluate the performance of the model, including accuracy, precision, recall, F1-score, ROC curve and AUC value.

2. The highway geological disaster prediction method according to claim 1, characterized in that: In the step data preprocessing, it includes the following: missing value filling, outlier processing and data standardization.

3. The highway geological disaster prediction method according to claim 1, characterized in that: In the step model training, the input layer is used to receive multidimensional time series data, including: soil moisture, temperature, precipitation; the encoder includes: a sparse self-attention module, a feedforward neural network layer, and a residual connection and layer normalization layer; the decoder includes: a masked self-attention module, a feedforward neural network layer, and a residual connection and layer normalization layer; the output layer uses ReLU as a nonlinear activation function.

4. The highway geological disaster prediction method according to claim 1, characterized in that: In the step model training, the pre-training uses NASA's surface soil moisture dataset and the global geological disaster event database to provide geological and meteorological information, and integrates the soil moisture data with the GLC data through matching timestamps and geographic locations.

5. The highway geological disaster prediction method according to claim 4, characterized in that: Dataset integration includes the following steps: Time matching: Match the time of disaster events in GLC with soil moisture observation data according to the time of event occurrence; Location matching: Based on the geographic coordinates, the longitude and latitude of the GLC event are matched with the location of the soil moisture data to ensure that each event can be associated with the corresponding soil moisture data.

6. The highway geological disaster prediction method according to claim 1, characterized in that: In step model training, k-fold cross validation includes the following steps: Dataset partitioning: The entire data set is randomly divided into k subsets of equal size, each of which contains a portion of training samples; Training and validation cycle: loop k times: for each subset, perform the following steps: use k-1 subsets as training sets to train the geological disaster prediction model; use the remaining 1 subset as the validation set to evaluate the performance of the model to calculate the prediction error of the model on the validation set; Calculate the average performance: After completing k training and validation cycles, calculate the average of all validation results as the overall performance indicator of the model.

7. A highway geological disaster prediction system, characterized in that: The highway geological disaster prediction method according to any one of claims 1 to 6 comprises: a data preprocessing module, a model training module, and a model evaluation module; the data preprocessing module is connected to the model training module and is used to preprocess the collected data; the model training module is connected to the model evaluation module, the model training module adopts an Informer model, including pre-training and additional training for specific case data, uses RMSE as a loss function, and adopts k-fold cross validation to ensure the stability and generalization ability of the model; the model evaluation module uses a variety of performance indicators to evaluate the performance of the model.

8. The highway geological disaster prediction system according to claim 7, characterized in that: The model structure of the model training module includes: an input layer, an encoder, a decoder and an output layer connected in sequence; the input layer is used to receive multidimensional time series data; the encoder includes: a sparse self-attention module, a feedforward neural network layer, a residual connection and a layer normalization layer; the decoder includes: a masked self-attention module, a feedforward neural network layer, a residual connection and a layer normalization layer; the output layer uses ReLU as a nonlinear activation function.

Images