A dynamic prediction method for slope stability based on heterogeneous transformer integration and grey clustering association

Abstract

This invention relates to the field of dynamic prediction technology, specifically to a dynamic prediction method for slope stability based on heterogeneous Transformer integration and grey clustering association. This invention solves the technical problems of single-model feature expression, gradient vanishing in traditional recurrent networks, and insufficient capture of long-range dependence and causal relationship of slope parameters by integrating the global coupling feature capture capability of BERT, the causal relationship mining capability of GPT2, and the basic nonlinear fitting capability of MLP.

Description

Technical Field

[0001] This invention relates to the field of dynamic prediction technology, specifically to a dynamic prediction method for slope stability based on heterogeneous Transformer integration and grey clustering association. Background Technology

[0002] Accurate slope stability prediction is a core aspect of geotechnical engineering in fields such as open-pit mining, water conservancy project construction, and roadbed maintenance, directly determining the timeliness of geological disaster early warning and the scientific validity of prevention and control plans. Traditional prediction methods often employ a unified modeling and training approach, making it difficult to dynamically match similar cases in historical databases for targeted analysis based on the specific geological conditions and structural characteristics of a slope. Existing slope analysis techniques typically include qualitative methods (such as stereographic projection analysis) and quantitative methods (such as limit equilibrium methods and numerical calculation methods). Among these, the limit equilibrium slice method based on an assumed sliding surface is widely used in practical engineering due to its clear calculation process and strong engineering applicability. However, soil and rock masses inherently possess significant heterogeneity, anisotropy, and complex nonlinear mechanical behavior. Traditional methods have limitations in characterizing the high-order coupling relationships between key parameters such as cohesion, internal friction angle, and unit weight. Furthermore, their calculation processes are relatively cumbersome and inefficient, making it difficult to meet the needs of real-time monitoring and rapid early warning. Summary of the Invention

[0003] To address the technical problems mentioned above, this invention provides a dynamic prediction method for slope stability based on heterogeneous Transformer integration and grey clustering association.

[0004] In a first aspect, the present invention provides a dynamic prediction method for slope stability based on heterogeneous Transformer ensemble and grey clustering association, comprising: Step 1: Construct a dataset of slope data samples and preprocess the slope data samples; Step 2: Extract gray structure features from the preprocessed slope data samples using K-Means clustering; Step 3: Construct a heterogeneous Transformer expert model pool and predict the slope instability probability or stability probability through the heterogeneous Transformer expert model pool; Step 4: Construct a multi-objective joint loss function and train the heterogeneous Transformer expert model pool model; Step 5: Through structure-driven dynamic weighted integration decision-making, the integration weights are dynamically calculated based on the prediction errors of the heterogeneous Transformer expert model pool and the structural fit, and the final slope state prediction results are obtained. Step 6: Visualize the final slope condition prediction results.

[0005] Further, step one includes: A dataset of slope data samples was constructed by collecting multiple slope data samples from historical cases of slope engineering. The Z-Score standardization method was used to process the slope data samples in the dataset to eliminate differences in feature dimensions, resulting in a standardized feature matrix that includes standardized slope data samples.

[0006] Furthermore, step two includes: K-Means clustering was performed on the slope data samples in the dataset; Calculate the global weight for each cluster; For each slope data sample, calculate its grey correlation coefficient with each cluster center; Calculate the gray structure weight vector for each slope data sample.

[0007] Furthermore, step three includes: The standardized feature vectors are linearly projected using a linear projection model to obtain the projected vectors. The gray structure weight vector is structurally embedded using a structural embedding model to obtain the structural embedding vector. The structural embedding vector and the projection vector are fused using a feature fusion model to obtain a fused vector; The feature gating model uses the sigmoid function to filter the fused vectors to obtain the filtered fused feature vectors; The backbone network predicts and outputs the probability of slope instability or stability based on the selected fused feature vectors.

[0008] Furthermore, the backbone network includes a BERT bidirectional coding expert model, a GPT2 one-way coding expert model, an MLP nonlinear compensation expert model, and a classification head model; BERT bidirectional coding expert model: It adopts a 2-layer TransformerEncoder structure, sets 4 attention heads, and models the bidirectional interaction between all slope data samples simultaneously through a multi-head self-attention mechanism to extract global spatial coupling features; GPT2 One-Way Decoding Expert Model: It adopts a 2-layer TransformerDecoder structure and sets up 4 attention heads. It uses a one-way mask attention mechanism to mine the potential causal progression logic between slope data samples. MLP Nonlinear Compensation Expert Model: It adopts a 2-layer fully connected structure of "linear layer + ReLU activation + Dropout regularization" to capture the basic linear and simple nonlinear relationships between slope parameters, as a residual supplement to the first two Transformer models. The classification head model outputs binary logits.

[0009] Furthermore, step five includes: Calculate the prediction error rate of the heterogeneous Transformer expert model pool; Calculate the structural fit score of the heterogeneous Transformer expert model pool; The ensemble weights of the heterogeneous Transformer expert model pool models are calculated based on the prediction error rate and structural fit score. Calculate the final integrated prediction probability and the final slope state prediction results.

[0010] Secondly, the present invention provides a computer-readable storage medium including a stored program that, when the program is running, controls the power equipment where the computer-readable storage medium is located to execute the above-described dynamic prediction method for slope stability based on heterogeneous Transformer integration and grey clustering association.

[0011] The beneficial effects of this invention are as follows: (1) For the first time, a heterogeneous Transformer expert model pool with bidirectional encoding, unidirectional decoding and nonlinear compensation was proposed. It integrates the global coupling feature capture capability of BERT, the causal relationship mining capability of GPT2 and the basic nonlinear fitting capability of MLP, and solves the technical problems of single model feature expression, gradient vanishing of traditional recurrent network and insufficient capture of long-term dependence and causal relationship of slope parameters.

[0012] (2) This invention proposes a gray structure embedding module based on K-Means clustering, which combines the cluster group structure features with gray relational analysis to generate a global structure weight vector of the sample and integrate it into the end-to-end encoding process of the deep learning model. This realizes the deep integration of the sample's local physical features with the prior geological structure of the group, and solves the problem that traditional models ignore geological structure information and lack physical constraints in prediction.

[0013] (3) The present invention designs a multi-objective joint loss function, which introduces gray consistency loss and trend prediction loss on the basis of classification loss, realizes the synergistic optimization of classification accuracy, physical consistency and mutation capture ability, significantly improves the convergence speed and prediction stability of the model, and avoids overfitting and abnormal prediction.

[0014] (4) The present invention proposes a structure-driven dynamic weighted ensemble strategy (SDLMS), which dynamically allocates ensemble weights based on the prediction accuracy of a single model and the fit of the sample structure, replacing the traditional fixed-weight voting, effectively offsetting the discrimination bias of a single model in the critical instability state of a slope, and significantly improving the generalization ability and robustness of the model. Attached Figure Description

[0015] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0016] Figure 1 The flowchart shows the slope stability dynamic prediction method based on heterogeneous Transformer integration and grey clustering association of the present invention. Figure 2 A schematic diagram illustrating the process of extracting gray structure features for K-Means clustering in this invention; Figure 3 This is a comparison diagram of the actual slope state and the model prediction state in the final output of this invention. Detailed Implementation

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, each technical and scientific term used in these embodiments has the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0020] In this invention, terms such as "upper," "lower," "left," "right," "front," "back," "vertical," "horizontal," "side," and "bottom" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used only to facilitate the description of the structural relationships of the various components or elements of this invention and do not specifically refer to any component or element in this invention. They should not be construed as limiting the invention.

[0021] In this invention, terms such as "fixed connection," "connected," and "linked" should be interpreted broadly, indicating a fixed connection, an integral connection, or a detachable connection; a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can determine the specific meaning of these terms in this invention based on the specific circumstances, and they should not be construed as limitations on the invention.

[0022] Example 1: This invention provides a dynamic prediction method for slope stability based on heterogeneous Transformer ensemble and grey clustering association, comprising the following steps: S1: Construct a dataset of slope data samples and preprocess the slope data samples; Specifically, it includes the following steps: S1-1: Collect multiple slope data samples through historical cases of slope engineering and construct a dataset of slope data samples.

[0023] Each slope data sample includes multidimensional physical characteristics and slope condition labels.

[0024] The six physical characteristics include: slope height (m), slope angle (°), cohesion (kPa), internal friction angle (°), soil and rock mass unit weight (kg / m³), and pore water pressure ratio; the slope state label is defined as a binary classification: 0 = slope instability, 1 = slope stability; S1-2: The Z-Score standardization method is used to process the slope data samples in the dataset to eliminate the differences in feature dimensions, and a standardized feature matrix including standardized slope data samples is obtained. The Z-Score standardization formula is as follows: ; For the first The first sample The original values ​​of the dimensional features; For the first The mean of the dimensional feature across the entire dataset; For the first The standard deviation of the dimensional feature across the entire dataset; The standardized eigenvalues ​​are used to obtain the standardized feature matrix. ,in The total number of samples, This represents the total number of feature dimensions.

[0025] S2: Extract gray structure features from the preprocessed slope data samples using K-Means clustering; Specifically, such as Figure 2 As shown, it includes the following steps: This step uses clustering to divide the slope data samples in the dataset into group structures, and combines grey relational analysis to calculate the global structural features of each slope data sample, providing prior geological structural information for subsequent models. Specifically, it includes the following steps: S2-1 performs K-Means clustering on the slope data samples in the dataset; Specifically, K-Means clustering is performed on the obtained standardized feature matrix to divide the samples into groups. Clusters with similar geological structures are selected, and the clustering optimization objective is: ; In the formula, For the number of clusters (e.g., The value is 3). For the first A cluster, It is a set of clusters; For the first The standardized feature vector of each slope data sample (which is in the standardized feature matrix) middle); For the first The center vector of each cluster is calculated from the mean of the features of all samples within the cluster.

[0026] S2-2: Calculate the global weight of each cluster, using the following formula: ; In the formula, For the first The number of samples within each cluster For the first The sample proportion weight of each cluster represents the structural proportion of that cluster in the entire dataset.

[0027] S2-3: For each slope data sample, calculate its grey relational coefficient with each cluster center. The calculation formula is as follows: ; In the formula, For the first The slope data sample and the first The cluster centers at the in The absolute difference of the features; It is the minimum value of the absolute difference across all dimensions; It represents the maximum value of the absolute differences across all dimensions; The resolution coefficient has a range of values. The default value is 0.5, which is used to adjust the resolution of the correlation coefficient.

[0028] The similarity between a sample and each cluster structure is quantified by calculating the grey relational coefficient.

[0029] S2-4: Calculate the gray structure weight vector for each slope data sample.

[0030] Specifically, it includes the following steps: A: Based on the grey relational coefficient between each slope data sample and each cluster center, the grey relational degree of each sample cluster center is calculated. The calculation formula is as follows: ; In the formula For the first The sample and the first The grey relational degree of a cluster center indicates that the higher the relational degree, the stronger the similarity between the sample and the geological structure of that cluster.

[0031] B: Based on the calculated grey relational degree of the cluster center of each slope data sample, calculate the grey structure weight vector of each slope data sample (i.e., the global structural features of each slope data sample).

[0032] S3: Physical features and gray structure features are uniformly expressed through linear projection and structural embedding, and effective information filtering is achieved through feature fusion and gating mechanisms to obtain fused feature vectors; Specifically, the structure-aware input encoding module performs unified encoding on the standardized feature vector and the gray structure weight vector. The structure-aware input encoding module includes: a linear projection model, a structure embedding model, a feature fusion model, a feature gate sigmoid weight selection model, and a backbone model. The linear projection module maps the original physical features to the latent space, and the structural embedding module vectorizes the gray structural features. Then, a fused feature vector is obtained through feature fusion and gating mechanisms. This fused feature vector is fed as a shared input to each expert model in the heterogeneous Transformer expert model pool, including the following steps: S3-1: The standardized feature vectors are linearly projected using a linear projection model to obtain the projected vectors. ; In the formula, , For the weights and biases of the linear projection layer, It is the first The standardized feature vector of each slope data sample will 3D physical feature mapping to Implicit space, default ; This is the projection vector.

[0033] S3-2: The gray structure weight vector is structurally embedded using a structural embedding model to obtain the structural embedding vector. The calculation formula is as follows: ; In the formula, , For the weights and biases of the structure embedding layer, The gray structure weight vector is mapped to the same dimension as the feature projection. For the first Grey structure weight vector of each slope data sample; This is the structure embedding vector.

[0034] S3-3: The structural embedding vector and the projection vector are fused using a feature fusion model to obtain a fused vector. The calculation formula is shown below: ; S3-4: The feature gating model uses the sigmoid function to filter the fused feature vector. The calculation formula for the fused feature vector after the German-Austrian filtering is as follows: ; ; In the formula The weighting coefficients are calculated using the sigmoid function. It is the sigmoid activation function. It is a gated weight matrix. It is the offset parameter of the gated layer. It is the filtered fused feature vector. For the Hadamard volume, the gating mechanism enables adaptive filtering of fused features, strengthens effective features related to slope stability, and can effectively suppress redundant noise.

[0035] S4: Construct a heterogeneous Transformer expert model pool and predict the slope instability probability or stability probability through the heterogeneous Transformer expert model pool; The heterogeneous Transformer expert model pool includes: BERT bidirectional coding expert model, GPT2 unidirectional coding expert model, MLP nonlinear compensation expert model and classification head model. Each model independently completes the slope stability prediction task and outputs the corresponding prediction results.

[0036] The heterogeneous Transformer expert model pool predicts and outputs the probability of slope instability or stability based on fused feature vectors. Among them, the BERT bidirectional coding expert model, the GPT2 unidirectional coding expert model, and the MLP nonlinear compensation expert model all integrate a gray structure embedding module and a feature gating layer to achieve deep fusion of sample features and structural priors. More specifically: B1. BERT Bidirectional Encoding Expert Model: It adopts a 2-layer TransformerEncoder structure and sets 4 attention heads. Through the multi-head self-attention mechanism, it simultaneously models the bidirectional mutual influence between all slope data samples, extracts global spatial coupling features, and solves the problem of insufficient long-range dependency capture in traditional recurrent networks.

[0037] B2, GPT2 one-way decoding expert model: adopts a 2-layer TransformerDecoder structure, sets 4 attention heads, and mines the potential causal progression logic between slope data samples through a one-way mask attention mechanism (such as the induced effect of the ratio of bulk density to pore pressure on cohesion). At the same time, a trend prediction head is added to achieve auxiliary fitting of slope state trend.

[0038] B3, MLP Nonlinear Compensation Expert Model: It adopts a 2-layer fully connected structure of "linear layer + ReLU activation + Dropout regularization" to capture the basic linear and simple nonlinear relationships between slope parameters, as a residual supplement to the first two Transformer models, and improves the convergence stability of the model on small sample datasets.

[0039] D: Classification head model, which outputs binary classification logits, with the formula: ; ; In the formula, The indexes are for the expert models, corresponding to the three heterogeneous expert models constructed in this paper: BERT, GPT2, and MLP. The feature vector output by the backbone network. For the model The predicted probability of slope instability or stability.

[0040] The BERT bidirectional coding expert model, the GPT2 unidirectional coding expert model, and the MLP nonlinear compensation expert model share a unified feature fusion and gating pre-structure. Their forward propagation includes the following steps: S5: Construct a multi-objective joint loss function and train a heterogeneous Transformer expert model pool; Specifically, the following steps are included: S5-1: Define the total loss function, its formula is as follows: ; The detailed definitions and explanations of each loss term are as follows: Cross-entropy classification loss: This is the core classification task loss, used to constrain the class prediction accuracy of the backbone network. Its calculation formula is as follows: ; In the formula, For the sample One-hot encoding of the real label, For the model Predicted Samples Category The probability of.

[0041] Grey consistency loss: Physical consistency constraint loss, which aligns the backbone network prediction results with the grey structure features of the samples to avoid abnormal predictions that do not conform to geological laws. The formula is: ; In the formula, For the sample The mean of the gray structure weights characterizes the global structural stability tendency of the samples, and this loss ensures that the model predictions are consistent with the prior geological structures.

[0042] Trend prediction loss: This is an auxiliary loss applied only to the GPT2 model to enhance the backbone network's ability to capture abrupt changes in slope conditions. The formula is as follows: ; In the formula These are the predicted values ​​output by the trend head of the GPT2 model. For the sample The true label (0 / 1) is used to achieve regression fitting of the slope state trend, which helps improve the robustness of the model in the classification task.

[0043] Hyperparameter settings: This is the weight for the gray consistency loss, with a default value of 0.5. The trend loss weight is set, with a default value of 0.3. This is an indicator function; it takes the value 1 when the model is GPT2, and 0 otherwise.

[0044] The model was trained using the Adam optimizer with a default learning rate of 0.001 and 500 training epochs. Full-batch gradient descent was used to update the parameters of all expert models.

[0045] S6: Construct a structure-driven dynamic decision-making mechanism, use the trained heterogeneous Transformer expert model pool for inference, calculate the integration weight in real time and perform decision fusion based on the historical prediction error of each expert model and the structural fit of the current sample, so as to obtain the final slope state prediction result. Specifically, the following steps are included: S6-1: Calculate the prediction error rate of the heterogeneous Transformer expert model pool. The calculation formula is as follows: ; In the formula For the model The classification error rate, To prevent division by zero and minima, the lower the error rate, the higher the prediction accuracy of the model.

[0046] S6-2: Calculate the structure fit score of the heterogeneous Transformer expert model pool. The calculation formula is as follows: ; In the formula, For the model The average prediction probability of the stable state of the slope data sample represents the overall adaptability of the model to the global structural features of the sample.

[0047] S6-3: Calculate the ensemble weights of the heterogeneous Transformer expert model pool based on the prediction error rate and structural fit score, using the following formula: ; In the formula, For the model The final integrated weights are determined by the model's accuracy and structural fit; the higher the accuracy and structural fit, the greater the weights it receives.

[0048] S6-4: Calculate the final integrated prediction probability and the final slope state prediction result, using the following formula: ; ; In the formula Slope data sample The final ensemble prediction probability, Slope data sample The final slope condition prediction results.

[0049] S7: Visualize the final slope condition prediction results; Specifically, the final prediction results are backfilled into the original dataset to generate a result file containing the original features, the actual slope state, the model's predicted state, and the prediction accuracy, which is then exported as an Excel file. Simultaneously, a comparison curve between the predicted state and the actual state is plotted (e.g., ...). Figure 3 As shown in the figure, a classification report and accuracy index are generated to complete the entire process of slope stability prediction.

[0050] Specific implementation steps of Example 1: (a) Implementation Environment Hardware environment: CPU is Intel Core i7-12700H, GPU is NVIDIA RTX 3060 12G, memory is 16G; Software environment: Python 3.9, PyTorch 2.0, scikit-learn 1.2, pandas 2.0, transformers 4.30.

[0051] (II) Implementation Steps 1. Dataset preparation: Collect 200 sets of slope engineering case data as slope data samples. Each set of data contains multi-dimensional feature data (this experiment uses six-dimensional feature data, specifically: slope height, slope angle, cohesion, internal friction angle, unit weight, and pore pressure ratio) and slope state labels (0=instability, 1=stable), and saves them as data.csv files.

[0052] 2. Data preprocessing: StandardScaler is used to perform Z-Score standardization on the six-dimensional feature data to eliminate dimensional differences and obtain a standardized feature matrix.

[0053] 3. Clustering and Grey Structure Feature Extraction: Set the number of clusters K=3, perform K-Means clustering on the standardized feature matrix to obtain 3 cluster centers and cluster weights; for each sample, calculate its grey correlation coefficient and correlation degree with the 3 cluster centers, and finally generate a 3D grey structure weight vector for each sample.

[0054] 4. Model Pool Construction: Initialize three heterogeneous Transformer expert model pools: BERT model: 32 hidden layer dimensions, 2 TransformerEncoder layers, 4 attention heads; GPT2 model: 32 embedding dimensions, 2 TransformerDecoder layers, 4 attention heads, with a newly added trend prediction head; MLP model: 2 fully connected layers, 32 hidden layer dimensions, ReLU activation, Dropout ratio of 0.2. All of the above models are deployed to GPU devices.

[0055] 4. Model Training: Set the Adam optimizer learning rate to 0.001, λgra=0.5, λtrend=0.3, and train for 500 epochs; print the total loss value every 50 epochs to complete the end-to-end joint training of all expert models.

[0056] 5. Dynamic weight calculation: Based on the trained model, calculate the prediction error rate and structure fit score of each model, and generate the ensemble weights of the three models through Softmax.

[0057] 6. Integrated Prediction and Result Output: Perform weighted integrated prediction on all samples to obtain the final slope state prediction results (as shown in Table 1); backfill the prediction results into the original dataset, add a "Model Prediction State" column, and export it as an Excel prediction comparison file; print the final prediction accuracy and classification report to complete the entire process.

[0058] Table 1: Slope Condition Prediction Results

[0059] (III) Implementation Results In this embodiment, the model achieves a final prediction accuracy of 98%, with precision, recall, and F1-score all greater than 0.97. Compared to the traditional LSTM model, the accuracy is improved by 6 percentage points, and the training and inference efficiency is improved by more than 85%, fully realizing the technical effects described in this invention.

[0060] Example 2: This embodiment provides a dynamic slope stability prediction system based on heterogeneous Transformer ensemble and grey clustering association, including: The data preprocessing module is used to construct a dataset of slope data samples and preprocess the slope data samples. The gray structure feature extraction module is used to extract gray structure features from preprocessed slope data samples using K-Means clustering. The structure-aware input encoding module is used to uniformly represent physical features and gray structure features through linear projection and structure embedding, and to achieve effective information filtering through feature fusion and gating mechanisms to obtain fused feature vectors. The structure-aware input encoding layer unifies the representation of physical features and gray structure features through linear projection and structure embedding, and achieves effective information filtering through feature fusion and gating mechanisms, feeding it as shared input into various expert models. Heterogeneous Transformer expert model pool is used to construct a heterogeneous Transformer expert model pool and predict the probability of slope instability or stability through the heterogeneous Transformer expert model pool. The training module is used to construct a multi-objective joint loss function and train a heterogeneous pool of Transformer expert models. The structure-driven dynamic integrated decision module is used to construct a structure-driven dynamic decision mechanism. It uses trained heterogeneous Transformer expert models for inference, calculates the integration weights in real time and performs decision fusion based on the historical prediction errors of each expert model and the structural fit of the current sample, so as to obtain the final slope state prediction result. The display module is used to visualize the final slope state prediction results.

[0061] Example 3: The present invention provides a computer-readable storage medium, which includes a stored program. When the program runs, it controls the power equipment where the computer-readable storage medium is located to execute the slope stability dynamic prediction method based on heterogeneous Transformer integration and grey clustering association described in Example 1.

[0062] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0063] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0064] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] Additionally, it should be noted that the flowcharts in the accompanying drawings illustrate methods according to embodiments of this disclosure. In the descriptions corresponding to the flowcharts or block diagrams in the drawings, the operations or steps corresponding to different blocks may occur in a different order than disclosed in the description; sometimes, there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the function involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic prediction method for slope stability based on heterogeneous Transformer ensemble and grey clustering association, characterized in that, include: Step 1: Construct a dataset of slope data samples and preprocess the slope data samples; Step 2: Extract gray structure features from the preprocessed slope data samples using K-Means clustering; Step 3: Physical features and gray structure features are uniformly represented by linear projection and structural embedding, and effective information filtering is achieved through feature fusion and gating mechanisms to obtain fused feature vectors; Step 4: Construct a heterogeneous Transformer expert model pool and predict the slope instability probability or stability probability through the heterogeneous Transformer expert model pool; Step 5: Construct a multi-objective joint loss function and train a heterogeneous Transformer expert model pool; Step 6: Construct a structure-driven dynamic decision-making mechanism, use the trained heterogeneous Transformer expert models for inference, calculate the integration weights in real time and perform decision fusion based on the historical prediction errors of each expert model and the structural fit of the current sample, so as to obtain the final slope state prediction result. Step 7: Visualize the final slope condition prediction results.

2. The slope stability dynamic prediction method based on heterogeneous Transformer ensemble and grey clustering association as described in claim 1, characterized in that, Step one includes: A dataset of slope data samples was constructed by collecting multiple slope data samples from historical cases of slope engineering. The Z-Score standardization method was used to process the slope data samples in the dataset to eliminate differences in feature dimensions, resulting in a standardized feature matrix that includes standardized slope data samples.

3. The slope stability dynamic prediction method based on heterogeneous Transformer ensemble and grey clustering association as described in claim 1, characterized in that, Step two includes: K-Means clustering was performed on the slope data samples in the dataset; Calculate the global weight for each cluster; For each slope data sample, calculate its grey correlation coefficient with each cluster center; Calculate the gray structure weight vector for each slope data sample.

4. The slope stability dynamic prediction method based on heterogeneous Transformer ensemble and grey clustering association as described in claim 1, characterized in that, Step three includes: The standardized feature vectors are linearly projected using a linear projection model to obtain the projected vectors. The gray structure weight vector is structurally embedded using a structural embedding model to obtain the structural embedding vector. The structural embedding vector and the projection vector are fused using a feature fusion model to obtain a fused vector; The feature gating model uses the sigmoid function to filter the fused vectors, resulting in the fused feature vectors selected by the German and Austrian authorities.

5. The slope stability dynamic prediction method based on heterogeneous Transformer ensemble and grey clustering association as described in claim 1, characterized in that, The heterogeneous Transformer expert model pool includes BERT bidirectional coding expert model, GPT2 unidirectional coding expert model, MLP nonlinear compensation expert model, and classification head model; BERT bidirectional coding expert model: It adopts a 2-layer TransformerEncoder structure, sets 4 attention heads, and models the bidirectional interaction between all slope data samples simultaneously through a multi-head self-attention mechanism to extract global spatial coupling features; GPT2 One-Way Decoding Expert Model: It adopts a 2-layer TransformerDecoder structure and sets up 4 attention heads. It uses a one-way mask attention mechanism to mine the potential causal progression logic between slope data samples. MLP Nonlinear Compensation Expert Model: It adopts a 2-layer fully connected structure of "linear layer + ReLU activation + Dropout regularization" to capture the basic linear and simple nonlinear relationships between slope parameters, as a residual supplement to the first two Transformer models. The classification head model outputs binary logits.

6. The slope stability dynamic prediction method based on heterogeneous Transformer ensemble and grey clustering association as described in claim 1, characterized in that, Step six includes: Calculate the prediction error rate of the heterogeneous Transformer expert model pool; Calculate the structural fit score of the heterogeneous Transformer expert model pool; The ensemble weights of the heterogeneous Transformer expert model pool models are calculated based on the prediction error rate and structural fit score. Calculate the final integrated prediction probability and the final slope state prediction results.

7. A dynamic prediction system for slope stability based on heterogeneous Transformer ensemble and grey clustering association, characterized in that, include: The data preprocessing module is used to construct a dataset of slope data samples and preprocess the slope data samples. The gray structure feature extraction module is used to extract gray structure features from preprocessed slope data samples using K-Means clustering. The structure-aware input encoding module is used to uniformly represent physical features and gray structure features through linear projection and structure embedding, and to achieve effective information filtering through feature fusion and gating mechanisms to obtain fused feature vectors. Heterogeneous Transformer expert model pool is used to construct a heterogeneous Transformer expert model pool and predict the probability of slope instability or stability through the heterogeneous Transformer expert model pool. The training module is used to construct a multi-objective joint loss function and train a heterogeneous pool of Transformer expert models. The structure-driven dynamic integrated decision module is used to construct a structure-driven dynamic decision mechanism. It uses trained heterogeneous Transformer expert models for inference, calculates the integration weights in real time and performs decision fusion based on the historical prediction errors of each expert model and the structural fit of the current sample, so as to obtain the final slope state prediction result. The display module is used to visualize the final slope state prediction results.

8. A computer-readable storage medium comprising a stored program, characterized in that, During program execution, the power equipment containing the computer-readable storage medium is controlled to perform the slope stability dynamic prediction method based on heterogeneous Transformer integration and grey clustering association as described in any one of claims 1 to 6.

Images