High and steep slope deformation monitoring method combining xLSTM-Attention time sequence prediction model and large language model

Abstract

The invention discloses a high and steep slope deformation monitoring method combining an xLSTM-Attention time sequence prediction model and a large language model. The method comprises the following steps: 1) constructing a time sequence prediction model based on xLSTM and an attention mechanism; 2) setting a plurality of side slope monitoring points on the side slope to be detected, acquiring three-dimensional deformation time sequence data of the side slope monitoring points in the T time period, and performing feature extraction to obtain statistical features; inputting the three-dimensional deformation time sequence data in the T time period into a slope deformation monitoring model to obtain a deformation prediction result in a future T'time period; 3) extracting an important hysteresis feature Top-KLags of the slope deformation monitoring model to form a standardized natural language prompt template; 4) calling a pre-training language model with engineering understanding ability, and performing field fine tuning; and 5) inputting the standardized natural language prompt template into the pre-training language model after field fine tuning to obtain a slope deformation safety risk assessment report. According to the invention, efficient deformation prediction and risk analysis can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of slope monitoring, and particularly relates to a high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model. BACKGROUND

[0002] With the increasing emphasis of the country on efficient allocation and sustainable use of water resources, large-scale water resource allocation projects continue to be promoted nationwide. Such projects often face challenges such as complex geological conditions and large-scale operations, especially in high and steep slope areas widely distributed in mountainous areas, which have significant geological disaster hazards. The stability of the slope not only relates to the safety of engineering construction and operation, but also directly affects the protection of the surrounding environment. Therefore, carrying out deformation monitoring and risk warning of high and steep slopes is of great significance to ensure the safety of engineering.

[0003] In recent years, with the full completion and popularization of the Beidou satellite navigation system (BDS) in China, high-precision positioning technology based on Beidou has been widely used in deformation monitoring of large-scale engineering structures. This technology has the advantages of all-weather, high frequency, and automation, and is particularly suitable for slope monitoring scenarios with wide distribution of points and large spatial span. However, Beidou monitoring data usually has strong nonlinearity, high time variability, and is easily affected by environmental interference, which brings great challenges to the accurate modeling of slope deformation trends and disaster warning.

[0004] To improve the processing and analysis capabilities of monitoring data, more and more researches introduce machine learning and deep learning methods into the field of engineering monitoring. Among them, the long short-term memory network (LSTM) is widely used in structure deformation prediction due to its advantages in time series modeling. However, LSTM still has problems such as information decay and unstable prediction when facing ultra-long time span data, and its sequence-dependent calculation method limits the parallel efficiency of the model. To break through the above bottlenecks, the Transformer architecture and its derivative models such as Informer, Autoformer and FEDformer are introduced into the time series prediction task. Transformer realizes the modeling of long-distance time dependence through multi-head self-attention mechanism, and has high parallel computing capacity, which improves the training efficiency and prediction accuracy of the model. However, such models usually have complex structure and large parameter size, requiring high computing power and memory resources, which are not suitable for deployment on edge computing terminals required by slope monitoring. At the same time, in actual engineering scenarios where the amount of monitoring data is limited or unevenly distributed, Transformer models also face problems such as weak generalization ability and large fluctuations in prediction effect.

[0005] In addition, most current slope monitoring systems can only provide numerical output of structural deformation, and it is difficult to automatically generate safety analysis reports with readability and interpretability, and still need to rely on manual interpretation of results and risk judgment. This not only increases the workload, but also may cause response lag and judgment error, which is not conducive to the realization of the intelligent monitoring system of "monitoring-early warning-decision".

[0006] Therefore, there is an urgent need for a new technical system that can balance high-precision time series prediction and intelligent analysis interpretation to realize efficient modeling, automatic analysis and intelligent early warning of slope deformation data, and promote the development of water resources engineering to a higher level of safety and intelligence. SUMMARY

[0007] The purpose of the present application is to provide a high-steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, comprising the following steps:

[0008] 1) Set multiple slope monitoring points and collect historical three-dimensional deformation time series data of the slope monitoring points to construct a deformation monitoring sequence;

[0009] 2) Preprocess the deformation monitoring sequence;

[0010] 3) Convert the preprocessed deformation monitoring sequence into a standardized sequence sample for model training;

[0011] 4) Construct a time series prediction model based on xLSTM and attention mechanism, and train the time series prediction model using the standardized sequence sample to obtain a slope deformation monitoring model;

[0012] 5) Set multiple slope monitoring points on the slope to be detected, collect three-dimensional deformation time series data of these slope monitoring points within a T time period, and perform feature extraction to obtain statistical features;

[0013] Input the three-dimensional deformation time series data within the T time period into the slope deformation monitoring model to obtain the deformation prediction result within the future T' time period;

[0014] 6) Extract the statistical features of the input sequence, and perform Fourier transform on the input time series to calculate the power spectrum and find the most relevant time delay lags as lag features Top-KLags, thereby forming a standardized natural language prompt template including statistical features, important lag features and deformation prediction results;

[0015] 7) Call a pre-trained language model with engineering understanding ability and perform domain fine-tuning;

[0016] 8) input the standardized natural language prompt template into the pre-trained language model fine-tuned in the field to obtain a slope deformation safety risk assessment report.

[0017] Further, the three-dimensional deformation time series data includes east-west displacement E_shift, north-south displacement N_shift, and vertical elevation change U_shift in the surface displacement reference coordinate system.

[0018] Further, in step 2), the method for preprocessing the deformation monitoring sequence includes standardization processing, outlier rejection, missing value repair, and time alignment.

[0019] Further, in step 2), the step of preprocessing the deformation monitoring sequence includes:

[0020] 2.1) resampling operation on the deformation monitoring sequence to reduce the sampling frequency;

[0021] 2.2) using K-nearest neighbor interpolation method to fill in the missing data;

[0022] 2.3) obtaining a slope safety assessment report, extracting descriptions related to three-dimensional deformation time series data and trends, and performing standardized format conversion to obtain structured language model data; the standardized format conversion includes removing irrelevant information, sentence processing, and structure annotation;

[0023] 2.4) arranging the structured language model data into a standardized question and answer pair format to obtain slope monitoring field question and answer data, and arranging the question and answer data to obtain risk level determination, cause analysis and recommended measures as training corpus for fine-tuning large language model;

[0024] Wherein, the question of the question and answer pair data includes monitoring background, E / N / U direction deformation trend and predicted value.

[0025] Further, in step 7), the slope monitoring field question and answer data, the semantic trend label extracted from the input sequence, and the predicted value output by the time series prediction model are used as prediction level information to fine-tune the pre-trained language model in the field;

[0026] The semantic trend label includes rising, falling and no change;

[0027] The step of fine-tuning the pre-trained language model in the field includes adjusting the query and numerical weight matrix in the attention layer of the pre-trained language model using LoRA technology.

[0028] Further, in step 3), the step of converting the preprocessed deformation monitoring sequence into a standardized sequence sample for model training includes:

[0029] 3.1) Denote the deformation monitoring sequence as X = {x1, x2,..., xT}, where xTrepresents the multivariate observation at the tth time point; d is the dimension; T},

[0030] denotes the multivariate observation at the tth time point; d is the dimension;

[0031] 3.2) Perform a sliding window operation on the deformation monitoring sequence to construct the input sequence and the target sequence, thereby generating a T-L pair of normalized sequence samples (X i , Y i );

[0032] wherein the input sequence is denoted as X i = {x i , x i+1 ,..., x i+L-1}, the target sequence is denoted as Y i =, x i+L ,

[0033] Further, before performing the sliding window operation on the deformation monitoring sequence in step 3.2), the deformation monitoring sequence is also normalized, that is:

[0034]

[0035] wherein and are the minimum value and the maximum value of feature j in the entire data set, respectively. is the normalized deformation monitoring quantity.

[0036] Further, in step 4), the time series prediction model based on xLSTM and attention mechanism adopts an xLSTM variant network to model the time series dependence, and captures the importance weight of deformation data in different directions through a dimension attention module.

[0037] The xLSTM variant network includes a plurality of sLSTM units and mLSTM units; the sLSTM performs embedding and enhancement on the input through a residual structure block, and captures the local context dynamics in the time series through causal convolution;

[0038] wherein the input gate i t of the sLSTM unit is as follows:

[0039] i t = exp(W i · [h t-1 , x t ] + b i ) (2)

[0040] wherein, E, N, U are three-dimensional displacement of the current time step, h t-1 is the hidden state of the previous moment; W i , b i are weights and biases;

[0041] The forget gate f of the sLSTM unit t As follows:

[0042] f t = σ (W f · [h t-1 , x t ] + b f ) OR exp (W f · [h t-1 , x t ] + b f ) (3)

[0043] In the formula, W f , b f are weights and biases;

[0044] The normalized state update and hidden state calculation mode of the sLSTM unit are as follows:

[0045] n t = f t · n t-1 + i t (4)

[0046]

[0047] Where o t is the output gate, c t is the cell state, and n t is the normalization factor;

[0048] At each time step t, the mLSTM unit stores the key vector k t and the value vector v t , and at future time step t+τ, the mLSTM unit retrieves the relevant value v t by querying the vector q t+τ , that is:

[0049] q t = W q x t +b q (6)

[0050]

[0051] In the formula, x t is the input; W q , W kis a weight; b q , b k is a bias; d represents a dimension.

[0052] Further, in step 4), when training the time series prediction model using the standardized sequence samples, mean absolute error MAE and root mean square error RMSE are selected as evaluation indexes;

[0053] The mean absolute error MAE and the root mean square error RMSE are as follows:

[0054]

[0055] Where n represents the total number of samples, y i represents the actual value of the high and steep slope deformation data, represents the predicted value of the high and steep slope deformation data.

[0056] Further, in step 8), the slope deformation safety risk assessment report includes monitoring background, E / N / U direction deformation trend analysis, risk level determination, cause analysis and recommended measures.

[0057] The technical effect of the present application is self-evident. The present application proposes an innovative method combining an xLSTM-Attention time series prediction model and a large language model. This method can realize efficient deformation prediction and risk analysis by extracting features from slope displacement data and performing time series prediction, and generating a landslide risk assessment report by combining a large language model.

[0058] The model proposed in the present application performs excellently in time series prediction, can more accurately capture deformation trends, and can automatically generate detailed assessment reports through fine-tuning of a large language model, significantly improving analysis efficiency and accuracy. This method provides a more intelligent and efficient solution for slope deformation monitoring and landslide warning. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is the overall architecture diagram of the BDT-LLM model;

[0060] Figure 2 is the sLSTM network structure;

[0061] Figure 3 is the mLSTM network structure;

[0062] Figure 4 is the xLSTM-Attention network structure.

[0063] Figure 5 is the LoRA-based large language model fine-tuning process. DETAILED DESCRIPTION

[0064] The application will be further described in connection with the following examples, which should not be construed as limiting the above-mentioned subject matter of the application to the following examples. Various replacements and modifications can be made according to ordinary technical knowledge and conventional means in the art without departing from the above-mentioned technical ideas of the application, and all of them should be included in the protection scope of the application.

[0065] Example 1:

[0066] Referring to Figures 1 to 5 A high-steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, comprising the following steps:

[0067] 1) Setting a plurality of slope monitoring points and collecting historical three-dimensional deformation time series data of the slope monitoring points, thereby constructing a deformation monitoring sequence;

[0068] 2) Preprocessing the deformation monitoring sequence;

[0069] 3) Converting the preprocessed deformation monitoring sequence into a standardized sequence sample for model training;

[0070] 4) Constructing a time series prediction model based on xLSTM and attention mechanism, and training the time series prediction model using the standardized sequence sample to obtain a slope deformation monitoring model;

[0071] 5) Setting a plurality of slope monitoring points on the slope to be detected, collecting three-dimensional deformation time series data of the slope monitoring points within a T time period, and performing feature extraction to obtain statistical features;

[0072] Inputting the three-dimensional deformation time series data within the T time period into the slope deformation monitoring model to obtain a deformation prediction result within a T' time period in the future;

[0073] 6) Extracting the statistical features (minimum value, maximum value, median, trend label) of the input sequence, and performing Fourier transform on the input time series to calculate the power spectrum and find the most relevant time delay lags as lag features Top-K Lags, thereby forming a standardized natural language prompt template including statistical features, important lag features and deformation prediction results;

[0074] 7) Calling a pre-trained language model with engineering understanding ability and performing domain fine-tuning;

[0075] 8) Inputting the standardized natural language prompt template into the pre-trained language model after domain fine-tuning to obtain a slope deformation safety risk assessment report.

[0076] The three-dimensional deformation time series data includes eastward displacement E_shift, northward displacement N_shift, and vertical elevation change U_shift in a surface displacement reference coordinate system.

[0077] In step 2), the method for preprocessing the deformation monitoring sequence includes normalization processing, outlier rejection, missing value repair, and time alignment.

[0078] In step 2), the step of preprocessing the deformation monitoring sequence includes:

[0079] 2.1) Resampling operation is performed on the deformation monitoring sequence to reduce the sampling frequency;

[0080] 2.2) K-Nearest Neighbor interpolation method is used to fill in the missing data;

[0081] 2.3) Obtain a slope safety evaluation report, extract the description related to three-dimensional deformation time series data and trend, and perform standardized format conversion to obtain structured language model data; the standardized format conversion includes removing irrelevant information, sentence processing, and structure annotation;

[0082] 2.4) The structured language model data is arranged into a standardized question and answer pair format to obtain slope monitoring field question and answer data, and the question and answer data is arranged to obtain risk level determination, cause analysis and suggestion measures as training corpus for fine-tuning of large language model;

[0083] The question of the question and answer pair data includes monitoring background, E / N / U direction deformation trend and predicted value.

[0084] In step 7), the slope monitoring field question and answer data, the semantic trend label extracted from the input sequence, and the predicted value output by the time series prediction model are used as prediction level information to fine-tune the pre-trained language model in the field;

[0085] The semantic trend label includes rising, falling and no change.

[0086] The step of fine-tuning the pre-trained language model in the field includes adjusting the query and numerical weight matrix in the attention layer of the pre-trained language model using LoRA technology.

[0087] In step 3), the step of converting the preprocessed deformation monitoring sequence into a standardized sequence sample for model training includes:

[0088] 3.1) Let the deformation monitoring sequence be X={x1,x2,…,x T},

[0089] Xt∈Rd represents the multivariate observation at the tth time point; d is the dimension;

[0090] 3.2) A sliding window operation is performed on the deformation monitoring sequence to construct the input sequence and the target sequence, thereby generating a T-L pair of normalized sequence samples (X i ,Y i );

[0091] wherein the input sequence is denoted as X i = {x i ,x i+1 ,…,x i+L-1}, and the target sequence is denoted as Y i =,x i+L ,

[0092] In step 3.2), before the sliding window operation is performed on the deformation monitoring sequence, the deformation monitoring sequence is also normalized, that is:

[0093]

[0094] wherein and are the minimum value and the maximum value of the feature j in the entire data set, respectively. is the normalized deformation monitoring quantity.

[0095] In step 4), the time series prediction model based on xLSTM and attention mechanism adopts an xLSTM variant network to model the timing dependence, and captures the importance weight of deformation data in different directions through a dimensional attention module.

[0096] The xLSTM variant network includes a plurality of sLSTM units and mLSTM units; the sLSTM performs embedding and enhancement on the input through a residual structure block, and captures the local context dynamics in the time series through causal convolution;

[0097] wherein the input gate i t of the sLSTM unit is as follows:

[0098] i t = exp(W i ·[h t-1 ,x t ]+b i ) (2)

[0099] wherein is the E, N, and U three-dimensional displacement at the current time step, h t-1 is the hidden state at the previous moment; W i and b i are the weights and the bias;

[0100] The forget gate of the sLSTM unit is as follows:

[0101] f t = σ(W f ·[h t-1 ,x t ]+b f ) OR exp(W f ·[h t-1 ,x t ]+b f ) (3)

[0102] where W f , b f are weights and biases;

[0103] The normalized state update and hidden state calculation of the sLSTM unit are as follows:

[0104] n t = f t ·n t-1 +i t (4)

[0105]

[0106] where o t is the output gate, c t is the cell state, and n t is the normalization factor;

[0107] At each time step t, the mLSTM unit stores a key vector k t and a value vector v t , and at a future time step t+τ, the mLSTM unit retrieves the relevant value v t by querying the vector q t+τ , i.e.:

[0108] q t = W q x t +b q (6)

[0109]

[0110] where x t is the input; W q , W k are weights; b q , b k are biases; and d denotes the dimension of the key vector.

[0111] In step 4), when training the time series prediction model with standardized sequence samples, mean absolute error (MAE) and root mean square error (RMSE) are selected as evaluation indexes.

[0112] The mean absolute error (MAE) and the root mean square error (RMSE) are as follows:

[0113]

[0114] Where n represents the total number of samples, y i represents the actual value of high and steep slope deformation data, represents the predicted value of high and steep slope deformation data.

[0115] In step 8), the slope deformation safety risk assessment report includes monitoring background, E / N / U direction deformation trend analysis, risk level determination, cause analysis and recommended measures.

[0116] Embodiment 2:

[0117] A high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, comprising the following steps:

[0118] 1) Set multiple slope monitoring points and collect historical three-dimensional deformation time series data of these slope monitoring points, thereby constructing deformation monitoring sequences;

[0119] 2) Preprocess the deformation monitoring sequences;

[0120] 3) Convert the preprocessed deformation monitoring sequences into standardized sequence samples for model training;

[0121] 4) Construct a time series prediction model based on xLSTM and attention mechanism, and train the time series prediction model with standardized sequence samples to obtain a slope deformation monitoring model;

[0122] 5) Set multiple slope monitoring points on the slope to be detected, collect three-dimensional deformation time series data of these slope monitoring points within a T time period, and perform feature extraction to obtain statistical features;

[0123] Input the three-dimensional deformation time series data within the T time period into the slope deformation monitoring model to obtain the deformation prediction results within a T' time period;

[0124] 6) Extract the statistical features of the input sequence, perform Fourier transform on the input time series, calculate the power spectrum, find the most relevant time delay lags as lag features Top-KLags, and form a standardized natural language prompt template including statistical features, important lag features, and deformation prediction results;

[0125] 7) Call the pre-trained language model with engineering understanding ability and perform domain fine-tuning;

[0126] 8) Input the standardized natural language prompt template into the pre-trained language model fine-tuned in the domain to obtain a slope deformation safety risk assessment report.

[0127] Embodiment 3:

[0128] A high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, the technical content is the same as embodiment 2, further, the three-dimensional deformation time series data includes east-west displacement E_shift, north-south displacement N_shift, and vertical elevation change U_shift in the surface displacement reference coordinate system.

[0129] Embodiment 4:

[0130] A high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, the technical content is the same as any one of embodiments 2-3, further, in step 2), the method for preprocessing the deformation monitoring sequence includes: standardization processing, outlier rejection, missing value repair, and time alignment.

[0131] Embodiment 5:

[0132] A high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, the technical content is the same as any one of embodiments 2-4, further, in step 2), the preprocessing step for the deformation monitoring sequence includes:

[0133] 2.1) Resampling operation is performed on the deformation monitoring sequence to reduce the sampling frequency;

[0134] 2.2) K-Nearest Neighbor interpolation method is used to fill in the missing data;

[0135] 2.3) Obtain a slope safety assessment report, extract the description of the three-dimensional deformation time series data and trend, and perform standard format conversion to obtain structured language model data; the standard format conversion includes removing irrelevant information, sentence processing, and structure annotation;

[0136] 2.4) The structured language model data is arranged into a standardized question and answer pair format to obtain slope monitoring domain question and answer data, and the question and answer data is arranged to obtain risk level determination, cause analysis and suggestion measures, which are used as training corpus for fine-tuning the large language model;

[0137] Wherein, the question of the question and answer pair data includes monitoring background, E / N / U direction deformation trend and predicted value.

[0138] Example 6:

[0139] A method for monitoring deformation of steep slopes that combines an xLSTM-Attention time series prediction model with a large language model, with the same technical content as any one of Examples 2-5. Furthermore, in step 7), the pre-trained language model is fine-tuned in the field using the slope monitoring domain question-and-answer data, the semantic trend labels extracted from the input sequence, and the predicted values ​​output by the time series prediction model as prediction magnitude information;

[0140] The semantic trend labels include rising, falling and no change;

[0141] The steps for domain fine-tuning the pre-trained language model include: using LoRA technology to adjust the parameters of the query and numerical weight matrices in the attention layer of the pre-trained language model.

[0142] Example 7:

[0143] A method for monitoring deformation of a steep slope by combining an xLSTM-Attention time series prediction model with a large language model, the technical content of which is the same as any one of Examples 2-6. Furthermore, in step 3), the step of converting the preprocessed deformation monitoring sequence into a standardized sequence sample for model training includes:

[0144] 3.1) Let the deformation monitoring sequence be X = {x1, x2, ..., x T},

[0145] represents the multivariate observation value at the t-th time point; d is the dimension;

[0146] 3.2) Perform sliding window operation on the deformation monitoring sequence to construct the input sequence and target sequence, thereby generating TL pair standardized sequence samples (X i ,Y i );

[0147] The input sequence is denoted as X i ={x i ,x i+1 ,…,x i+L-1}, The target sequence is denoted as Y i =,x i+L ,

[0148] Example 8:

[0149] A high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, the technical content of any one of embodiments 2-7, further, in step 3.2), before the sliding window operation on the deformation monitoring sequence, the deformation monitoring sequence is also normalized, that is:

[0150]

[0151] wherein, and are the minimum and maximum values of feature j in the entire data set, respectively.

[0152] Embodiment 9:

[0153] A high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, the technical content of any one of embodiments 2-8, further, in step 4), the time series prediction model based on xLSTM and attention mechanism uses xLSTM variant network to model time series dependence, and captures the importance weight of deformation data in different directions through dimension attention module.

[0154] The xLSTM variant network includes a plurality of sLSTM units and mLSTM units; sLSTM embeds and enhances the input through a residual structure block, and captures local context dynamics in the time series through causal convolution;

[0155] wherein, the input gate of the sLSTM unit is as follows:

[0156] i t = exp(W i ·[h t-1 ,x t ]+b i ) (2)

[0157] wherein, is the E, N, U three-dimensional displacement of the current time step, h t-1 is the hidden state of the previous moment;

[0158] The forget gate of the sLSTM unit is as follows:

[0159] f t = σ(W f ·[h t-1 ,x t ]+b f ) OR exp(W f ·[h t-1 ,x t ]+b f ) (3)

[0160] The normalized state update and hidden state calculation of the sLSTM unit are as follows:

[0161] n t =f t ·n t-1 +i t (4)

[0162]

[0163] where o t is the output gate, c t is the cell state, and n t is the normalization factor;

[0164] At each time step t, the mLSTM unit stores a key vector k t and a value vector v t At a future time step t+tau, the mLSTM unit retrieves the relevant value v t by querying the vector q t+τ , i.e.:

[0165] q t =W q x t +b q (6)

[0166]

[0167] where x t is the input.

[0168] Embodiment 10:

[0169] A high-steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, the technical content is the same as any one of embodiments 2-9, further, in step 4), when training the time series prediction model using standardized sequence samples, mean absolute error MAE and root mean square error RMSE are selected as evaluation indexes;

[0170] The mean absolute error MAE and the root mean square error RMSE are as follows:

[0171]

[0172] where n represents the total number of samples, y i represents the actual value of the high-steep slope deformation data, represents the predicted value of the high-steep slope deformation data.

[0173] Embodiment 11:

[0174] A high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, the technical content of any one of embodiments 2-10, further, in step 8), the slope deformation safety risk assessment report includes monitoring background, E / N / U direction deformation trend analysis, risk level determination, cause analysis and recommended measures.

[0175] Embodiment 12:

[0176] A high and steep slope deformation monitoring method combining an xLSTM-Attention time series prediction model and a large language model, the content is as follows,

[0177] The present application takes the surface displacement data of the high and steep slope in Chongqing Yuxi large water resources allocation project monitored by Beidou satellite navigation (BDS) measurement system as an example for empirical analysis. Its deformation prediction based on xLSTM-Attention algorithm and fine-tuning language model are automatically generated to generate a safety analysis report. Specifically, the steps include:

[0178] S100: Prediction target definition. Prediction task and input definition. Select multiple slope monitoring points in Chongqing Yuxi water resources allocation project, collect three types of high-precision time series data based on ground displacement reference coordinate system: E_shift (east-west displacement), N_shift (north-south displacement) and U_shift (vertical elevation change) as input data of the model, construct a multivariate, high-dimensional deformation monitoring sequence, and predict the slope deformation in the next week.

[0179] S200: Data preprocessing. Standardize and remove outliers of the original monitoring data, repair missing values and complete unified time alignment. Normalize the displacement data in E / N / U three directions respectively according to the dimension, and retain its physical quantity level characteristics.

[0180] S300: Sequence generation. Construct a sliding window input sequence for time series prediction. Based on the set history window length L and prediction step T, the normalized displacement data of each monitoring point is divided into input sample pairs.

[0181] S400: xLSTM-Attention model construction. Construct an xLSTM and attention mechanism (Attention) time series prediction model.

[0182] The model body adopts a multilayer LSTM variant network to model the time series dependence, and introduces a dimensional attention module to capture the importance weight of the deformation data in different directions. The network output end is connected with a linear prediction layer to complete multi-step and multi-dimensional displacement prediction.

[0183] S500: Feature extraction and Prompt construction. Extract the statistical features (minimum, maximum, median, trend label) of the input sequence and the important lag features (Top-K Lags) identified by the trained model, and organize them into a standardized natural language prompt template (Prompt). At the same time, combine the model prediction results to generate a 7-day deformation prediction text description, forming an "input data features + future prediction results" combined Prompt to provide structured context conditions for large language models.

[0184] S600: Large language model fine-tuning. Select pre-trained language models with engineering understanding capabilities such as DeepSeek-LLM-7B-base, and fine-tune the model in the slope monitoring field using question and answer data. Use LoRA (Low-Rank Adaptation) technology to adjust the query (Wq) and value (Wv) weight matrices in the attention layer, achieving knowledge transfer at low resource cost. Training data includes structured safety assessment report question and answer pairs, semantic trend labels, and prediction level information, enhancing the model's language generation capabilities in the geological disaster field.

[0185] S700: Safety assessment report generation. Concatenate the constructed input feature Prompt and future prediction result Prompt, and input it into the fine-tuned large language model. The model generates a complete slope deformation safety risk assessment report based on the prompt content. The report includes five parts: monitoring background, E / N / U direction deformation trend analysis, risk level determination, cause analysis, and recommended measures. The generated results are in standard format, professional language, and can be directly used for engineering applications and safety warning systems.

[0186] S201: For original slope deformation monitoring data with minute-level time granularity, first perform resampling operation on the time series data to convert the time unit to hour-level, and further adjust the sampling frequency to record every 6 hours according to experimental requirements, to reduce data dimension and improve model training efficiency and stability.

[0187] S202: During the resampling process, there may be missing values. To improve data continuity and completeness, K-Nearest Neighbors (KNN) interpolation method is used to fill in the missing data. The specific operation is as follows: identify the missing data points, select several similar samples within their adjacent time window, calculate the weighted value based on their feature values, and generate the filling value accordingly to restore the consistency of the data structure.

[0188] S203: For the language data set required for large language model training, extract the three-dimensional deformation data involving E_shift, N_shift, and U_shift and its trend description from the slope safety evaluation report of Chongqing West Water Resources Allocation Project, expand the corpus source combined with public landslide analysis data. Perform standardized format conversion on the report content, including removing irrelevant information, sentence processing, structure labeling, etc., to ensure that the data has good consistency and analyzability.

[0189] S204: Organize the structured language model data into a standardized question and answer pair format. Each set of questions and answers revolves around specific slope deformation monitoring data, ensuring that the questions have an engineering background and the answers provide specific numerical values and judgment basis. Finally, all question and answer data is saved in a unified JSON format as training corpus for subsequent fine-tuning of large language model to enhance the model's application capability in slope safety analysis scenarios.

[0190] S301: For the structural characteristics of the slope deformation monitoring data with multiple features (such as E, N, and U direction deformation values), convert the preprocessed time series data into standardized sequence samples suitable for model training. Let the original time series be:

[0191] X = {x1, x2, …, x T},

[0192] where, represents the multivariate observation value at the t-th time point, d = 3 represents the number of features, and represents the E, N, and U direction monitoring data.

[0193] To train the time series prediction model, perform sliding window operation on the above time series data to construct input sequence and target sequence. Given a fixed length time window L, for each time step i, generate input-output pairs in the following form:

[0194] Input sequence (prediction sample):

[0195] X i = {x i ,x i+1 ,…,x i+L-1},

[0196] Target output (next time step prediction value):

[0197] Y i =,x i+L ,

[0198] Generate T-L pairs of training samples (X i , Yi ), to achieve the modeling task of predicting future values based on multivariate historical sequences.

[0199] To improve the model training effect and numerical stability, the original data set X is compressed to the interval [-1, 1] using the Min-Max Scaling method, and the specific transformation method is as follows:

[0200]

[0201] wherein, and are the minimum and maximum values of feature j in the entire data set, respectively.

[0202] Through the above method, training sets, validation sets and test sets can be constructed in the form of and , wherein N = T-L represents the number of samples.

[0203] S401: The improved long short-term memory unit (sLSTM) is designed to better capture the local dynamic information in the time series and enhance the sensitivity of the model to the slope changes. The sLSTM retains the basic structure of the LSTM, but redesigns the calculation method of the input gate, the forgetting gate and the hidden state.

[0204] The input gate uses an exponential activation method to enhance the response ability to input information:

[0205] i t = exp(W i ·[h t-1 ,x t ]+b i )

[0206] wherein, is the three-dimensional displacement of E, N and U at the current time step, h t-1 is the hidden state at the previous time.

[0207] The calculation method of the forgetting gate supports two forms: Sigmoid or exponential activation function:

[0208] f t = σ(W f ·[h t-1 ,x t ]+b f ) OR exp(W f ·[h t-1 ,x t ]+b f )

[0209] The introduction of the exponential activation function enhances the flexibility of dynamic information filtering, but also can cause rapid growth of the numerical range, thereby causing numerical instability problems. The sLSTM introduces a normalized state, and the normalized state update and hidden state calculation are as follows:

[0210] n t =f t ·n t-1 +i t

[0211]

[0212] Where o t is the output gate, c t is the cell state, and n t is the normalization factor.

[0213] The sLSTM does not directly stack, but uses a residual structure block to embed and enhance the input. The input first enters a Pre-LayerNorm residual structure, which is normalized before nonlinear transformation, stabilizing the training process. Then it passes through a causal convolution with a kernel size of 4 to capture the local context dynamics in the time series. The causal convolution only uses current and past information to ensure the time sequence of the model. Then the input gate and the forget gate use the Swish activation function Swish(x) = x·σ(x) to improve the nonlinearity and liquidity of information extraction. The input gate i, the forget gate f, the output gate o, and the state update item z are all processed through a projection layer with four diagonal blocks (NH = 4): each head learns independent patterns, improving the parallelism and expressiveness of the model. Each head is independently processed by GroupNorm to maintain numerical stability when parallel computing. The sLSTM output is projected (expanded dimension) and projected (restored dimension) through a two-layer MLP, using the GeLU activation function, and the projection factor is PF = 4 / 3.

[0214] S402: For the input x t , at each time step t, the mLSTM first stores a pair of vectors: the key vector k t and the value vector v t , at the future time step t+τ, the model retrieves the relevant value v t by querying the vector q t+τ

[0215] q t =W q x t +b q

[0216]

[0217] The timing prediction model selects three columns, E_shift, N_shift, and U_shift, as input features, representing displacement changes in the E, N, and U directions, respectively. The dataset is divided into training, validation, and test sets in the ratio of 8:1:1. The input dimension of the model is 3, corresponding to the three features mentioned above. The batch size is set to 256, the training process lasts for 300 rounds, the sequence length is 96, the initial learning rate is set to 0.001, and the early stopping strategy (patience = 30 rounds) is used to avoid overfitting during the training process. The mean absolute error (MAE) and root mean square error (RMSE) are used as evaluation indicators

[14] .

[0218] The LoRA fine-tuning experiment uses the DeepSeek-LLM-7B model for supervised fine-tuning, with an initial learning rate of 5e-5 and an AdamW optimizer. To avoid gradient explosion, the maximum gradient norm is set to 1.0, and a gradient accumulation strategy is used, updating the gradient every 8 steps. 10% of the training dataset is extracted as the validation set to monitor the performance changes during the training process. In terms of learning rate scheduling, we use the cosine scheduler to help the model converge stably during training. The BLEU and ROUGE indicators are used to evaluate the generation effect.

[0219] To evaluate the prediction performance of various evaluation indicators, we use the mean absolute error (MAE) and mean square error (MSE) as evaluation indicators, which are calculated as follows:

[0220]

[0221] where n represents the total number of samples, y i represents the actual value of the high and steep slope deformation data, represents the predicted value of the high and steep slope deformation data.

[0222] Example 13:

[0223] A validation of a high and steep slope deformation monitoring method combining an xLSTM-Attention timing prediction model and a large language model, with the following content:

[0224] This example compares linear and Transformer-based methods, including Autoformer and Informer. At the same time, we also consider the improved traditional model LSTM of xLSTM. After training, the prediction results are compared with the comparative models on the test set to evaluate the performance of xLSTM-Attention.

[0225] As shown in Table 1, the MAE of xLSTM-Attention on the Beidou high and steep slope deformation dataset is 0.163, and the RMSE is 0.238, which is overall better than other models, highlighting its enhanced generalization ability and transferability. Compared with the best-performing Autoformer, the MAE of xLSTM-Attention decreased by 0.016; the RMSE decreased by 0.021. These results show that xLSTM-Attention can effectively provide accurate results for time series prediction.

[0226] Table 1 Comparison of prediction errors of all models

[0227]

[0228]

[0229] In the ablation experiment, we compared the performance of xLSTM-Attention and xLSTM models. The experimental results show that xLSTM-Attention is better than xLSTM in both evaluation indicators. Specifically, the MAE of xLSTM-Attention is 0.163, which is 0.008 lower than that of xLSTM, which is 0.171, with a performance improvement of 4.67%. In terms of RMSE, the value of xLSTM-Attention is 0.238, while that of xLSTM is 0.243, which is 0.005 lower, with an improvement of 2.05%. These results show that the xLSTM-Attention model with attention mechanism can better capture the complex patterns in time series data, thereby improving the prediction accuracy of the model.

[0230] Table 2 Comparison of prediction errors of xLSTM-Attention and xLSTM

[0231]

[0232] At the same time, the performance of the fine-tuned large language model in the slope deformation safety risk assessment task is evaluated, mainly using text generation quality-related indicators, including BLEU-4 and ROUGE (ROUGE-1, ROUGE-2, ROUGE-L), and model inference efficiency indicators.

[0233] As shown in the following table, the BLEU-4 score is 14.33, indicating that the model has some ability to capture key expressions and contextual structures, and can generate report s text with semantic rationality and readability. The ROUGE-1 score is 33.72%, the ROUGE-2 score is 11.54%, and the ROUGE-L score is 24.60, which indicates that the model performs relatively stably in overall lexical coverage, key phrase extraction, and semantic coherence, and can generate evaluation reports with rich content and clear structure. In terms of reasoning efficiency, the prediction speed is 0.087 samples / second, and the reasoning speed per step is 0.044 steps / second. Since this task requires the model to generate a structured and relatively long professional evaluation text based on multivariate time series prediction data, it belongs to the long text generation task, so the reasoning speed is within an acceptable range and still has certain practical application potential.

[0234] In summary, the fine-tuned large language model exhibits good text generation capability in the slope deformation risk assessment task, and can automatically generate professional report text with engineering significance based on prediction data, providing an efficient and intelligent auxiliary tool for slope safety analysis.

[0235] Table 3 Evaluation indicators of fine-tuned language model

[0236]

[0237]

[0238] In view of the shortcomings of existing high and steep slope deformation safety analysis methods in prediction accuracy and analysis ability, an innovative method combining xLSTM-Attention time series prediction model and large language model is proposed. Through feature extraction and time series prediction of slope displacement data, and combining with the large language model to generate landslide risk assessment report, efficient deformation prediction and risk analysis can be realized. The experimental results show that the proposed model performs excellently in time series prediction, and can more accurately capture the deformation trend, and through the fine-tuned large language model, it can automatically generate detailed evaluation reports, significantly improving the analysis efficiency and accuracy. This method provides a more intelligent and efficient solution for slope deformation monitoring and landslide warning. This method can be applied in the field of more extensive geological disaster prediction and engineering safety.

Claims

1. A high and steep slope deformation monitoring method combining the xLSTM-Attention time series prediction model and the large language model, characterized in that: The following steps are involved: 1) Set up multiple slope monitoring points and collect historical 3D deformation time series data of these slope monitoring points to construct a deformation monitoring sequence; 2) Preprocessing the deformation monitoring sequence; 3) Convert the preprocessed deformation monitoring sequence into a standardized sequence sample for model training; 4) Construct a time series prediction model based on xLSTM and attention mechanism, and use standardized sequence samples to train the time series prediction model to obtain a slope deformation monitoring model; 5) Set up multiple slope monitoring points on the slope to be tested, collect the three-dimensional deformation time series data of these slope monitoring points within the time period T, and perform feature extraction to obtain statistical features; Input the three-dimensional deformation time series data within the T time period into the slope deformation monitoring model to obtain the deformation prediction results within the future T' time period; 6) Extract the statistical features of the input sequence, perform Fourier transform on the input time series, calculate the power spectrum, and find the most relevant time delay lags as the lag features Top-KLags, thereby forming a standardized natural language prompt template including statistical features, important lag features, and deformation prediction results; 7) Calling a pre-trained language model with engineering understanding capabilities and performing domain fine-tuning; 8) The standardized natural language prompt template is input into the pre-trained language model after domain fine-tuning to obtain a slope deformation safety risk assessment report.

2. A high and steep slope deformation monitoring method combining the xLSTM-Attention time series prediction model and the large language model according to claim 1, characterized in that: The three-dimensional deformation time series data includes the east-west displacement E_shift, north-south displacement N_shift, and vertical elevation change U_shift in the surface displacement reference coordinate system.

3. The method for monitoring deformation of steep slopes by combining the xLSTM-Attention time series prediction model and the large language model according to claim 1 is characterized in that: In step 2), the method of preprocessing the deformation monitoring sequence includes: standardization, outlier removal, missing value repair, and time alignment.

4. The method for monitoring deformation of steep slopes by combining the xLSTM-Attention time series prediction model and the large language model according to claim 1 is characterized in that: In step 2), the step of preprocessing the deformation monitoring sequence includes: 2.1) Resample the deformation monitoring sequence to reduce the sampling frequency; 2.2) Use K-nearest neighbor interpolation method to fill missing data; 2.3) Obtaining the slope safety assessment report, extracting descriptions of the three-dimensional deformation time series data and trends, and converting them into a standardized format to obtain structured language model data; the standardized format conversion includes removing irrelevant information, sentence processing, and structural annotation; 2.4) Organize the structured language model data into a standardized question-answer pair format to obtain slope monitoring question-answer data. This data is then organized to obtain risk level determinations, cause analysis, and recommended measures, which serve as training data for fine-tuning the large language model. Among them, the questions and answers about the data include monitoring background, deformation trends in the E / N / U directions and predicted values.

5. A method for monitoring deformation of steep slopes combining an xLSTM-Attention time series prediction model and a large language model according to claim 4, characterized in that: In step 7), the pre-trained language model is fine-tuned in the field using the slope monitoring field question-answering data, the semantic trend labels extracted from the input sequence, and the predicted values ​​output by the time series prediction model as prediction magnitude information; The semantic trend labels include rising, falling and no change; The steps for domain fine-tuning the pre-trained language model include: using LoRA technology to adjust the parameters of the query and numerical weight matrices in the attention layer of the pre-trained language model.

6. The method for monitoring deformation of steep slopes combining the xLSTM-Attention time series prediction model and the large language model according to claim 1 is characterized in that: In step 3), the step of converting the preprocessed deformation monitoring sequence into a standardized sequence sample for model training includes: 3.1) Let the deformation monitoring sequence be X = {x1, x2, ..., x T }, represents the multivariate observation value at the t-th time point; d is the dimension; 3.2) Perform sliding window operation on the deformation monitoring sequence to construct the input sequence and target sequence, thereby generating TL pair standardized sequence samples (X i ,Y i ); The input sequence is denoted as X i ={x i ,x i+1 ,…,x i+L-1 }, The target sequence is denoted as Y i =,x i+L , 7. A method for monitoring deformation of steep slopes combining an xLSTM-Attention time series prediction model and a large language model according to claim 6, characterized in that: In step 3.2), before performing the sliding window operation on the deformation monitoring sequence, the deformation monitoring sequence is also normalized, that is: in, and are the minimum and maximum values ​​of feature j in the entire data set; is the normalized deformation monitoring value.

8. The method for monitoring deformation of steep slopes by combining the xLSTM-Attention time series prediction model and the large language model according to claim 1 is characterized in that: In step 4), the time series prediction model based on xLSTM and attention mechanism uses the xLSTM variant network to model temporal dependencies, and captures the importance weights of deformation data in different directions through the dimensional attention module. The xLSTM variant network includes multiple sLSTM units and mLSTM units; the sLSTM embeds and enhances the input through the residual structure block and captures the local context dynamics in the time series through causal convolution; Among them, the input gate i of the sLSTM unit t As shown below: i t =exp(W i ·[h t-1 ,x t ]+b i ) (2) in, is the E, N, and U three-dimensional displacement of the current time step, h t-1 is the hidden state at the previous moment; W i 、b i are weights and biases; The forget gate f of the sLSTM unit t As shown below: f t =σ(W f ·[h t-1 ,x t ]+b f )ORexp(W f ·[h t-1 ,x t ]+b f ) (3) Where W f 、b f are weights and biases; The normalized state update and hidden state calculation of the sLSTM unit are as follows: n t =f t ·n t-1 +i t (4) Among them, t is the output gate, c t is the cell state, n t is the normalization factor; At each time step t, the mLSTM unit stores the key vector k t Sum value vector v t , at the next time step t+τ, the mLSTM unit passes the query vector q t+τ Retrieve the associated value v t ,Right now: q t =W q x t +b q (6) Where x t is the input; W q 、W k is the weight; b q 、b k is the bias; d represents the dimension.

9. The method for monitoring deformation of steep slopes by combining the xLSTM-Attention time series prediction model and the large language model according to claim 1 is characterized in that: In step 4), when using standardized sequence samples to train the time series prediction model, the mean absolute error (MAE) and the root mean square error (RMSE) are selected as evaluation indicators; The mean absolute error MAE and root mean square error RMSE are as follows: Where n represents the total sample size, y i Indicates the actual value of the high and steep slope deformation data, Represents the predicted value of high and steep slope deformation data.

10. The method for monitoring deformation of steep slopes combining the xLSTM-Attention time series prediction model and the large language model according to claim 1, characterized in that: In step 8), the slope deformation safety risk assessment report includes monitoring background, E / N / U direction deformation trend analysis, risk level determination, cause analysis and recommended measures.

Images