Reservoir dam disease risk feature mining method based on multi-model consensus and risk mapping

Abstract

This invention discloses a method for mining the risk features of reservoir dams based on multi-model consensus and risk mapping, belonging to the field of water conservancy engineering safety assessment technology. The method includes: acquiring multi-dimensional risk feature data of the reservoir dam; obtaining the predicted probability distributions of multiple base models using a pre-constructed consensus attribution model pool; calculating consensus weights based on the performance evaluation indicators of the base models; determining the risk weight vector corresponding to each risk level based on the inter-class distribution differences in risk levels or the engineering risk cost characteristics; weighting and fusing the feature marginal contributions calculated by the base models based on the consensus weights and risk weight vectors to generate a consensus risk-driven contribution value; and generating interpretable mining results characterizing the risk evolution mechanism based on this contribution value. This invention eliminates single-model attribution bias through a multi-model consensus mechanism and utilizes adaptive risk mapping to calculate the nonlinear transitions of the risk gradient, achieving robust, accurate, and in-depth mining of the dam risk mechanism.

Description

Technical Field

[0001] This invention relates to the field of water conservancy project safety assessment technology, and in particular to a method for mining the risk features of reservoir dams based on multi-model consensus and risk mapping. Background Technology

[0002] As complex large-scale hydraulic structures, reservoir dams are subject to nonlinear coupling effects from multiple dimensions of factors, including hydrology, meteorology, engineering geology, and structural aging. Utilizing machine learning techniques to uncover hidden patterns of dam deterioration within massive monitoring data is of significant engineering value for real-time diagnosis of dam health. The key technical challenge lies not only in constructing a high-precision state mapping model but also in endowing this black-box model with physical interpretability, mapping data features into understandable disaster-causing factors, and providing logically supported computational basis for dam reinforcement and mitigation.

[0003] Currently, the field of dam safety monitoring mainly employs single machine learning models such as random forests, support vector machines, or deep neural networks to perform regression predictions of effect quantities such as deformation and seepage, or to classify safety levels. Regarding model interpretability research, existing techniques typically apply post-processing algorithms such as Shapley Additive Explanations (SHAP) or Local Interpretable Model-agnostic Explanations (LIME) to the single optimal model selected through training, calculating the marginal contribution of features to the model's predicted probability. When handling multi-classification tasks, different risk levels (e.g., Class I, Class II, and Class III dams) are usually treated as independent parallel categories, and the importance ranking of each feature is calculated based on the background distribution of the global samples.

[0004] However, when applied to complex scenarios of dam failure, the aforementioned existing technologies mainly suffer from the following three technical problems: First, the robustness of the attribution results is insufficient. The explanation results of a single model are highly sensitive to the random seed and inductive bias of the algorithm. Different models may conflict in their explanations of the same physical process, and there is a lack of effective consensus mechanisms to eliminate structural biases in the models. Second, the physical mapping of risk gradients is distorted. Existing methods ignore the strict ordinal relationship between failure levels (healthy < general < severe) and the nonlinear transition of risk costs, and cannot adaptively calculate the risk gain from general failure to severe failure based on data distribution distance or engineering consequences. Third, the mining of deep coupling mechanisms is limited. Only the first-order main effect of features is considered, making it difficult to quantitatively decouple and construct high-order interaction networks between features (such as the synergistic amplification effect of reservoir water level rise on seepage gradient), and it is impossible to accurately identify the interactive hub features that dominate the evolution of failure. Summary of the Invention

[0005] Purpose of the invention: To provide a method for mining the risk features of reservoir dams based on multi-model consensus and risk mapping, in order to solve the above-mentioned problems in the existing technology.

[0006] Technical solution: A method for mining the risk features of reservoir dams based on multi-model consensus and risk mapping, including:

[0007] Obtain a dataset of disease risk features covering the multidimensional attributes of reservoir dams. The dataset includes disease risk level labels representing different safety states.

[0008] The disease risk feature dataset is processed using a pre-built consensus attribution model pool to obtain the predicted probability distribution of the base model output in the consensus attribution model pool, and the performance evaluation index of the base model on the validation set is obtained.

[0009] Calculate the consensus weight of the base model in attribution fusion based on the performance evaluation index of the base model.

[0010] Based on the inter-class distribution differences or engineering risk cost characteristics of each disease risk level corresponding to the predicted probability distribution, the risk weight vector corresponding to each disease risk level is determined.

[0011] Using a pre-defined interpretable attribution algorithm, the feature marginal contribution of the base model to the central samples of the disease risk feature dataset is calculated;

[0012] Based on consensus weight and risk weight vectors, the feature marginal contributions calculated by the base model are weighted and fused to generate consensus risk-driven contribution values ​​of multi-dimensional features;

[0013] Based on consensus risk-driven contribution values, interpretable mining results representing the evolution mechanism of disease risks are generated.

[0014] Beneficial effects: This invention eliminates single-model attribution bias through a multi-model consensus mechanism and uses adaptive risk mapping to calculate the nonlinear transition of the risk gradient, thereby achieving robust, accurate and in-depth mining of the dam risk mechanism. Attached Figure Description

[0015] Figure 1 This is a general framework diagram of a method for mining the risk features of reservoir dams based on multi-model consensus and risk mapping in an embodiment of this application.

[0016] Figure 2 This is a flowchart illustrating the steps for identifying the time window characteristics of disease outbreaks in an embodiment of this application.

[0017] Figure 3 This is a flowchart illustrating the steps involved in calculating the consensus weights of the base model using a linear weighting mechanism based on excess performance, as described in this application.

[0018] Figure 4This is a flowchart illustrating the steps involved in calculating the feature marginal contribution of the base model for the sample to be explained in this embodiment of the application.

[0019] Figure 5 This is a technical flowchart of the method for mining the risk features of reservoir dams based on multi-model consensus and risk mapping in the embodiments of this application.

[0020] Figure 6 This is a comparison chart of the accuracy of multi-model competitive selection in the embodiments of this application. Detailed Implementation

[0021] Example 1 describes the overall framework of a method for mining the risk features of reservoir dams based on multi-model consensus and risk mapping, such as... Figure 1 As shown in the figure, this embodiment details the complete signal flow process from multidimensional data input to the final interpretability mechanism output, laying the foundation for subsequent embodiments on consensus weights, adaptive risk mapping, and deep interaction mining.

[0022] Step 101: Obtain a dataset of risk features covering the multidimensional attributes of reservoir dams. The dataset of risk features includes risk level labels representing different safety states.

[0023] In this embodiment, the "disease risk feature dataset" refers to a comprehensive dataset used to describe the physical entity of a reservoir dam and its surrounding environment. Specifically, the acquisition process can collect data from multiple heterogeneous data sources through data interfaces. These data sources include, but are not limited to, dam safety monitoring systems, water conservancy survey databases, and historical safety assessment report databases. The acquired data covers multidimensional attributes that comprehensively reflect the dam's characteristics. These multidimensional attributes include at least project scale attributes, geographical environment attributes, and operation and management attributes. For example, project scale attributes may involve physical parameters reflecting the project's size, such as total reservoir capacity, maximum dam height, and dam crest length; geographical environment attributes may involve parameters reflecting external environmental constraints, such as the watershed, dam site elevation, and geological structure type; and operation and management attributes may involve time-series data reflecting the project's entire life cycle status, such as dam construction time, reinforcement history, and daily inspection records. The acquired raw data typically needs to be cleaned, standardized, and structured to construct a feature matrix X∈R suitable for input to machine learning models. N×M Where N represents the number of samples and M represents the feature dimension. Furthermore, the disease risk feature dataset also includes the corresponding disease risk level label Y∈R. N Used as the target variable for supervised learning, the risk level label is usually classified into Class I, Class II and Class III dams according to the current dam safety assessment standards.

[0024] Step 102: Process the disease risk feature dataset using the pre-built consensus attribution model pool, obtain the predicted probability distribution of the base model output in the consensus attribution model pool, and obtain the performance evaluation index of the base model on the validation set.

[0025] The consensus attribution model pool is built on a competitive prediction framework: the competitive prediction framework integrates extreme random tree model, random forest model, gradient boosting decision tree model and lightweight gradient boosting machine model; the consensus attribution model pool consists of multiple base models in the competitive prediction framework whose performance evaluation metrics on the validation set meet the preset admission threshold.

[0026] In this embodiment, the consensus attribution model pool refers to a set of machine learning models with differentiated inductive biases that have been pre-trained. Specifically, in this embodiment, the feature matrix X is input into each base model m in the model pool. j In this process, forward inference is performed using the nonlinear mapping relationships learned within the base model. For each sample x... i Each base model m j It will output for all possible disease insurance levels. Predicted probability distribution vector , where p j,k (x i ) represents the probability that the j-th model predicts that sample i belongs to the k-th disease risk level, and satisfies Σ k=0 K-1 ， p j,k (x i =1. By utilizing the prediction results of multiple base models, we can fully explore the advantages of different algorithms in specific feature subspaces and avoid the structural biases that may exist in a single model.

[0027] Step 103: Calculate the consensus weight of the base model in attribution fusion based on the performance evaluation index of the base model.

[0028] In this embodiment, the consensus weight Ω jConsensus weights are scalar coefficients used to measure the importance of each base model in the final attribution result. Specifically, this embodiment dynamically allocates weights based on the performance of the base models on the validation or test set. Models with better performance will be assigned higher consensus weights, ensuring that the final attribution result is mainly dominated by high-quality models. The performance evaluation metric can preferably be the area under the receiver operating characteristic curve (AUC), or it can comprehensively consider stability metrics such as model prediction accuracy, F1 score, or cross-validation variance. Based on this, there are multiple implementation paths for calculating consensus weights. For example, in one implementation, linear weights can be calculated based on the excess performance of the base models, i.e., the portion of the AUC value exceeding a preset baseline; in another implementation, weights including stability penalties can be calculated using an exponential function based on a combination of the AUC mean and variance. Regardless of the method used, the final calculated consensus weights usually need to be normalized to ensure that Σ j=1 J Ω j =1.

[0029] Step 104: Based on the inter-class distribution difference characteristics or engineering risk cost characteristics of each disease risk level corresponding to the predicted probability distribution, determine the risk weight vector corresponding to each disease risk level.

[0030] In this embodiment, the risk weight vector This refers to a numerical sequence used to calculate the degree of threat to dam safety posed by different risk levels. Specifically, this embodiment aims to address the problem that traditional classification models only output category probabilities and cannot reflect the differences in risk gradients between categories. For example, a Class I dam is in a healthy state, and its risk contribution should be extremely low or zero; while a Class III dam is in a dangerous state, and its risk contribution should be higher than that of a Class II dam. To objectively and reasonably determine the specific values ​​of the class weights, this embodiment provides two optional determination mechanisms. In one implementation, an adaptive nonlinear weight mapping relationship can be constructed based on a data-driven perspective, utilizing the inter-class distribution differences of samples at each risk level in the feature space, such as the inter-class Mahalanobis distance, so that the risk weights can reflect the inherent distribution structure of the data. In another implementation, a risk weight can be set and calibrated based on an engineering consequence-driven perspective, according to the actual engineering risk cost characteristics corresponding to each risk level, such as the probability of dam failure or expected economic losses, so that the risk weights have clear physical and economic meanings.

[0031] Step 105: Using a pre-defined interpretable attribution algorithm, calculate the feature marginal contribution of the base model to the central samples of the disease risk feature dataset; based on the consensus weight and risk weight vector, perform weighted fusion on the feature marginal contribution calculated by the base model to generate the consensus risk-driven contribution value of multi-dimensional features.

[0032] In this embodiment, the feature marginal contribution typically refers to the contribution value of a feature to the model prediction result calculated based on Shapley value theory, i.e., the SHAP value. Specifically, this embodiment applies this value to each base model m. j Calculate the original SHAP value φ for each feature f with respect to each disease risk level k. j,k (f). Perform a double-weighted fusion operation: The first weighted application risk weight vector W will be used to calculate the multidimensional SHAP values ​​φ for different levels. j,k (f) Aggregate into a single risk contribution value under this model. This achieves a semantic transformation from classification probability contribution to risk level contribution; the second weighted average consensus weight Ω is used. j The risk contribution values ​​of multiple base models are fused to obtain the final consensus risk-driven contribution value. Through this two-layer fusion mechanism, the generated consensus risk-driven contribution value not only integrates the judgment advantages of multiple models, but also incorporates the calculation consideration of the risk gradient, which can more accurately reflect the actual driving effect of features on the dam's safety status.

[0033] Step 106: Based on the consensus risk-driven contribution value, generate interpretable mining results that characterize the disease risk evolution mechanism.

[0034] In this embodiment, interpretable mining results refer to a set of information that can intuitively reveal the causes and evolution patterns of dam defects. Specifically, this embodiment performs in-depth analysis and visualization of consensus risk-driven contribution values. The mining methods can be diverse. For example, a partial dependency graph of key features can be constructed to show the nonlinear scale effect between reservoir capacity, dam height, and defect risk; the risk evolution trajectory of dam age features can be fitted to identify the time window for defect outbreaks; further calculations of interaction effects between features can be performed to construct an interaction network graph and identify pivotal features in defect evolution; and structured risk assessment rules can be automatically synthesized through statistical analysis. These mining results can provide white-box decision support for dam safety managers, assisting in the formulation of differentiated reinforcement and mitigation strategies.

[0035] Example 2 describes the specific construction process of the disease risk feature dataset, the multi-label encoding strategy, and the offline training and screening process of the consensus attribution model pool, providing a solid data foundation and model carrier for the entire mining method.

[0036] Step 201: The disease risk feature dataset is constructed through the following steps: extracting engineering scale features representing the physical volume of the project, geographical environment features representing regional background constraints, and time-varying evolution features representing the service life of the project from multi-source heterogeneous data.

[0037] In this embodiment, data construction is the foundation of the entire method. Specifically, the engineering scale characteristics are continuous numerical variables extracted from dam design and construction data, mainly including total reservoir capacity V (unit: 10,000 cubic meters), maximum dam height H (unit: meters), dam crest length L (unit: meters), and catchment area A (unit: square kilometers). These characteristics determine the dam's engineering scale and potential disaster-causing energy. Geographical environment characteristics are categorical or numerical variables extracted from geographic information system and administrative division data, mainly including the province where the reservoir is located, the river basin region it belongs to (e.g., the Yangtze River basin, the Yellow River basin), the latitude and longitude coordinates (lon, lat) of the dam site, and the dam site elevation E (unit: meters). These characteristics reflect the external natural environment and regional management background of the dam, implicitly including hydrological and meteorological conditions and geological structural constraints. Time-varying evolution characteristics are variables constructed based on the time dimension, with the dam age T being the most crucial. age This is the difference between the current time and the time the dam was built (in years), and may also include the number of reinforcement and safety improvement projects (N) that have been carried out historically. fix By extracting these three types of features, a multi-dimensional feature space was constructed that can comprehensively characterize the dam's physical condition, environment, and history.

[0038] Step 202: Perform multi-label classification encoding on the disease risk level labels used to represent the safety status in the disease risk feature dataset, and map different safety evaluation dimensions into numerical category labels.

[0039] In this embodiment, to achieve a refined assessment of the dam's condition, a single, general evaluation of the dam is not performed. Instead, prediction tasks are established for different safety dimensions. Specifically, based on the evaluation conclusions in the dam safety assessment report, evaluation levels for multiple dimensions, including flood control safety, seepage safety, structural safety, and metal structure safety, are extracted. For each dimension, the traditional qualitative evaluation results (e.g., categories A, B, and C) are mapped to computer-processable numerical labels. For example, category A (safe / healthy) is mapped to label 0, category B (basic safety / general risk) to label 1, and category C (unsafe / serious risk) to label 2. Thus, for each sample i, the target variable is no longer a scalar but a multi-dimensional label vector Y. i =[y i,flood ,y i,seepage ,y i,struct This supports the model in exploring disease risk-driving mechanisms from different perspectives.

[0040] Step 203: Encode the categorical features in the disease risk feature dataset with integer sequences and preserve the original physical scale of the numerical features.

[0041] In this embodiment, differentiated preprocessing strategies are employed for different types of input features. Specifically, for numerical features such as reservoir capacity, dam height, and dam age, normalization or standardization is not performed to preserve their physical meaning and interpretability; instead, their original physical scale is directly retained. This is because the subsequent tree model is invariant to monotonic transformations of features, and retaining the original scale facilitates the direct generation of intuitive physical rules, such as increased risk when the dam height exceeds 30 meters. For categorical features such as province, watershed, and dam type, label encoding is used to convert them into integer sequences. For example, earth-rock dams are encoded as 1, and concrete dams as 2. Compared to one-hot encoding, label encoding avoids the problems of excessively sparse feature matrices and dimensionality explosion when processing high-cardinality categorical features (such as 30 provinces), making it more suitable for tree model processing.

[0042] Step 204: The consensus attribution model pool is pre-built through the following steps: training multiple candidate base models with different inductive biases using historical sample sets.

[0043] In this embodiment, to build a high-quality model pool, diverse candidate models need to be trained. Specifically, based on the historical sample set, various decision tree-based ensemble learning algorithms are selected as candidate base models. These algorithms include, but are not limited to: extreme random tree models, random forest models, gradient boosting decision tree models (such as XGBoost, eXtremeGradient Boosting), and lightweight gradient boosting machine models (such as LightGBM, Light Gradient BoostingMachine). Although these models are all based on tree structures, they have different inductive biases: random forests reduce variance through double random sampling of samples and features; extreme random trees further introduce randomness in the split threshold; while XGBoost and LightGBM are based on boosting strategies, reducing bias by progressively fitting residuals. Introducing this diversity at the algorithmic level helps the model pool improve its overall generalization ability and robustness when facing complex and ever-changing disease patterns through complementary effects.

[0044] Step 205: Use hierarchical cross-validation to evaluate the performance metrics of each candidate base model on the test set.

[0045] In this embodiment, to objectively evaluate the model's predictive ability and prevent overfitting, a K-fold hierarchical cross-validation strategy is adopted. Specifically, the historical sample set is divided into K mutually exclusive subsets (e.g., K=5). During the partitioning, it is ensured that the proportion of samples for each disease risk level (0 / 1 / 2) in each subset remains consistent with the overall distribution, addressing the class imbalance problem where disease risk samples are typically few. In each round of validation, K-1 subsets are used for training, and the remaining subset is used for testing. For each candidate base model, its performance evaluation metric on the test set is recorded. Preferably, the performance evaluation metric is the area under the receiver operating characteristic curve (AUC), as it comprehensively measures the model's ability to rank positive and negative samples and is insensitive to threshold selection. The AUC value for each round is calculated, and the mean AUC of each model across K rounds of validation is statistically analyzed. mean and variance Var AUC .

[0046] Step 206: Set a performance admission threshold and select candidate base models whose performance evaluation indicators are higher than the performance admission threshold into the consensus attribution model pool.

[0047] In this embodiment, not all trained models can enter the final consensus pool; they must undergo rigorous screening. Specifically, a performance admission threshold τ is set. The value of this threshold τ is typically set to [0.75, 0.85], and in this embodiment, it is preferably set to τ = 0.80. The average AUC value of the test set is compared with this threshold; only models that meet the threshold are considered acceptable. j The average AUC value satisfies AUC(m) j Only when the value of τ is greater than or equal to 0 is the model deemed a qualified model and included in the consensus attribution model pool M. * This screening mechanism eliminates models with poor predictive performance, ensuring that subsequent attribution analysis is based on high-precision and high-reliability predictions, and preventing inferior models from introducing noise and misleading interpretations.

[0048] Example 3 describes the specific paths for calculating the consensus weights of two base models: one is a linear weighting mechanism based on excess performance, which focuses on amplifying the contribution of high-performance models, such as... Figure 3 As shown; another is the exponential weighting mechanism based on stability penalty, which focuses on balancing the model's prediction accuracy and robustness.

[0049] Step 301: Obtain the area under the receiver operating characteristic curve of the base model on the test set obtained from the disease risk feature dataset.

[0050] This embodiment forms the basis for weight calculation. Specifically, for the consensus attribution model pool M... * ={m1,m2,…,m j Each base model m in}j The average AUC value of the test set obtained through stratified cross-validation during the offline construction phase is directly read and denoted as AUC(m). j This indicator objectively reflects the ability of each model to distinguish between dangerous and critical conditions.

[0051] Step 302: Set the performance admission threshold and calculate the difference between the area under the receiver operating characteristic curve of the base model and the performance admission threshold to obtain the excess performance value.

[0052] In this embodiment, excess performance, rather than raw performance, is used to calculate weights, aiming to widen the weight differences between models. Specifically, the performance admission threshold τ (e.g., τ=0.80) is used for each model m. j Calculate its excess performance value ΔAUC j =AUC(mj)-τ; where ΔAUC j For the mth j The excess performance value of each base model, AUC(mj) is the m-th... j The area under the receiver operating characteristic (AUC) curve of each base model on the test set. For example, if model A has an AUC of 0.90 and model B has an AUC of 0.82, the difference between the two is not significant when directly using AUC to calculate weights; however, when calculating excess performance, the excess value of model A is 0.90 - 0.80 = 0.10, and the excess value of model B is 0.82 - 0.80 = 0.02, meaning that the contribution of model A is amplified to 5 times that of model B. This mechanism ensures that models that barely meet the criteria account for only a small proportion in attribution fusion, while high-performance models will dominate the attribution results.

[0053] Step 303: Normalize the excess performance values ​​of all base models in the consensus attribution model pool to obtain the consensus weights of the base models.

[0054] In this embodiment, a normalization operation is required to ensure that the sum of the weights is 1. Specifically, the excess performance values ​​ΔAUC of each model calculated above are normalized. j Sum the results and calculate the percentage of total excess performance for each model. Base model m j consensus weight Ω j The calculation formula is as follows:

[0055] ;

[0056] Where Ωj is the consensus weight of the j-th base model, and ΔAUCj is the excess performance value of the j-th model. For the first The excess performance value of each model, J is the total number of base models in the consensus attribution model pool.

[0057] Using this formula, a set of conditions satisfying ΣΩ can be obtained. j =1 and Ω j A consensus weight vector greater than 0 is used for subsequent SHAP value weighted fusion.

[0058] Step 304: Calculate the consensus weight of the base model in attribution fusion, specifically using the following formula: .

[0059] Among them, Ω j Let AUC be the consensus weight of the j-th base model. j Var represents the area under the receiver operating characteristic curve (AUC) of the j-th base model obtained through hierarchical cross-validation. j Let denot γ be the performance variance of the j-th base model obtained through hierarchical cross-validation, J be the total number of base models, γ be the preset accuracy sensitivity coefficient, and η be the preset stability penalty coefficient. For the first The average area under the receiver operating characteristic curve (AUC) of each base model obtained through hierarchical cross-validation. For the first The performance variance of each base model obtained through hierarchical cross-validation.

[0060] The formula for calculating the variance of model performance is: ;

[0061] Among them, Var j The variance of the performance of the j-th base model obtained through K-fold cross-validation is AUC, where K is the number of folds in the cross-validation. j,k Let AUC be the area under the receiver operating characteristic curve of the j-th base model on the k-th fold validation set. j Let be the area under the receiver operating characteristic curve (AUC) of the j-th base model in K-fold cross-validation.

[0062] In some alternative implementations, particularly for dam safety assessment scenarios with extremely high requirements for model stability, the stability-penalized exponential weighting mechanism of this embodiment can be used as an alternative. Specifically, in addition to focusing on the model's average accuracy (AUC), j It also explicitly introduces the variance Var of the model in cross-validation. j As a negative penalty, γ is a preset precision sensitivity coefficient used to control the sensitivity of the weights to the AUC value, usually taken as a positive value (e.g., γ=10); η is a preset stability penalty coefficient used to control the penalty strength for variance, usually also taken as a positive value (e.g., η=5). This formula maps the scores to non-negative weights using the exponential function exp(×) and then normalizes them.

[0063] By introducing -η×Var jFor two models with the same average AUC, the model with smaller variance (i.e., more stable performance) will receive greater weight. This mechanism reduces the impact of models with drastic performance fluctuations due to the randomness of data partitioning on the final attribution results, thus improving the robustness of the mining conclusions.

[0064] To illustrate the calculation process of consensus weights more clearly, a specific numerical example is given below.

[0065] Assume the consensus attribution model pool M* contains 4 base models, whose average AUC values ​​obtained from 5-fold cross-validation on the test set are: AUC(m1)=0.92, AUC(m2)=0.88, AUC(m3)=0.85, and AUC(m4)=0.82. Set the performance admission threshold τ=0.80.

[0066] Consensus weights are calculated using the excess performance method:

[0067] Calculate the excess performance values ​​for each model:

[0068] ΔAUC1 = 0.92 - 0.80 = 0.12;

[0069] ΔAUC² = 0.88 - 0.80 = 0.08;

[0070] ΔAUC3 = 0.85 - 0.80 = 0.05;

[0071] ΔAUC4 = 0.82 - 0.80 = 0.02;

[0072] The total excess performance value is: 0.12 + 0.08 + 0.05 + 0.02 = 0.27.

[0073] After normalization, the consensus weights of each model are obtained:

[0074] Ω1 = 0.12 / 0.27 ≈ 0.444;

[0075] Ω² = 0.08 / 0.27 ≈ 0.296;

[0076] Ω3 = 0.05 / 0.27 ≈ 0.185;

[0077] Ω4 = 0.02 / 0.27 ≈ 0.074.

[0078] This demonstrates that the best-performing model, m1, received the highest consensus weight (approximately 44.4%), and its contribution to the final attribution result will be far greater than that of the barely satisfactory model, m4 (approximately 7.4%). The differentiated weight allocation based on excess performance effectively achieves the design goal of having high-quality models dominate attribution.

[0079] Example 4 describes how to construct a risk weight vector that reflects the differences in risk gradients from two distinct perspectives: the intrinsic distribution of data and the extrinsic constraints of engineering. This invention provides two specific implementation paths: one is a data-driven mapping based on inter-class Mahalanobis distance, suitable for scenarios lacking prior expert knowledge but with abundant data accumulation; the other is a rule-driven mapping based on engineering consequences and costs, suitable for engineering scenarios with clearly defined risk economics.

[0080] Step 401: Calculate the centroid vector of each disease risk level sample in the feature space in the disease risk feature dataset, and calculate the global covariance matrix.

[0081] In a preferred embodiment of this example, a data-driven approach is used to calculate the risk gradient. Specifically, assume that the disease risk feature dataset contains N samples, each sample has F features, and is labeled with K ordered disease risk levels (e.g., C0 represents healthy, C1 represents moderate disease risk, and C2 represents severe disease risk). For each disease risk level k, the set of all samples belonging to that level is extracted, and its geometric center in the feature space, i.e., the centroid vector μ, is calculated. k The centroid vector μ k The calculation formula is: ;where x i Let n be the feature vector of a sample belonging to level k. k This represents the number of samples at this level. Simultaneously, to eliminate the impact of dimensional differences between different features and the correlation between features on distance calculation, it is necessary to calculate the global covariance matrix σ of the entire dataset. This covariance matrix reflects the dispersion and correlation structure of the data in various directions and is the basis for subsequent Mahalanobis distance calculation.

[0082] Step 402: Calculate the inter-class Mahalanobis distance between adjacent disease risk levels based on the centroid vector and the global covariance matrix.

[0083] In this embodiment, Mahalanobis distance is used to accurately measure the degree of separation between different disease risk levels in the feature space. Specifically, the distance d between healthy individuals and those with general disease risk is calculated separately. 01 And the distance d between general medical insurance and critical medical insurance. 12 The Mahalanobis distance d between adjacent categories i and j ij Specifically, it is calculated using the following formula: Among them, μ i and μ j The centroid vectors of categories i and j are respectively, σ -1Let be the inverse of the global covariance matrix, and T denote the transpose operation. Compared to traditional Euclidean distance, Mahalanobis distance considers the covariance between features, and can more realistically reflect the statistical distance between different risk states in high-dimensional manifold space. For example, if the distributions of general risks and severe risks overlap less in certain key feature directions (such as displacement rate), then the calculated Mahalanobis distance d... 12 This will increase, indicating a large risk gap between the two levels.

[0084] Step 403: Construct a nonlinear weight mapping function based on the inter-class Mahalanobis distance to calculate the risk weight vector corresponding to each disease risk level.

[0085] In this embodiment, inter-class distances are mapped to risk weights, ensuring that the increase in weights matches the actual separation degree of the feature space. Specifically, the weight for health status is typically set to w0=0, and the weight for general illness risk is used as a benchmark (e.g., setting w1 and d...). 01 (Relative values ​​after correlation or normalization). For the weight w2 of severe illness risk, the following nonlinear mapping formula is constructed: Where w2 is the risk weight corresponding to the severity of illness risk level (set w0=0, w1=1); 1 is the baseline constant; β is the risk amplification coefficient, used to control the overall gain of the high-risk category; d 12 The inter-class Mahalanobis distance between the general illness risk level and the severe illness risk level; d 01 α represents the inter-class Mahalanobis distance between the health level and the general disease risk level; α is the exponentiation operation, and the exponent is the gradient shape parameter α, which is used to control the shape of the weight growth curve.

[0086] This formula utilizes the ratio d of two distances. 12 / d 01 To calculate the relative span of risk. If d 12 Much greater than d 01 This indicates that the range from general deterioration to severe is large, and the weight w2 will increase.

[0087] Step 404, wherein the nonlinear weight mapping function includes a gradient shape parameter for controlling the shape of the growth curve, and a risk amplification coefficient for controlling the gain of the high-risk category.

[0088] In this embodiment, two key hyperparameters are introduced to give the mapping function stronger engineering adaptability. The gradient shape parameter α controls the shape of the weight growth curve. Specifically, the gradient shape parameter α controls the shape of the weight growth curve. When α > 1, the weight growth is accelerated, suitable for sudden-onset risks, i.e., once the condition crosses a critical point, the risk increases exponentially; when α < 1, the weight growth is slower, suitable for gradual-onset risks. In an optional implementation, α is set to 1.5. Those skilled in the art can select an appropriate value of α based on the risk characteristics of risk deterioration in actual engineering scenarios, so that the shape of the weight growth curve matches the actual risk transition law. The risk amplification coefficient β controls the overall gain amplitude of the high-risk category. In an optional implementation, β is set to 1.0. Those skilled in the art can adaptively adjust β according to the engineering management needs of high-risk events to determine an appropriate value.

[0089] Step 405: Obtain the pre-stored preset engineering consequence costs corresponding to each disease risk level, normalize the preset engineering consequence costs, and obtain the risk weight vector corresponding to each disease risk level.

[0090] In some alternative implementations, particularly when comprehensive risk economics data is available, a mapping mechanism based on engineering consequence costs can be adopted. Specifically, an engineering consequence cost L can be set directly for each risk level k based on water conservancy industry standards or expert experience. k For example, the cost of a healthy state is set to L0=0 (no loss), the cost of general health insurance is L1=10 (maintenance cost), and the cost of severe health insurance is L2=100 (dam failure loss). Then, these costs are normalized to obtain the risk weight vector w. k =L k / ∑L i Alternatively, keeping the relative proportions constant and setting w1=1, then w2=L2 / L1=10. This approach links the model's feature contributions to the actual potential economic or social losses, giving the interpretation results a clear economic meaning.

[0091] Step 406, wherein the method further includes performing probability calibration on the predicted probability distribution of the consensus attribution model pool output before calculating the marginal contribution of the feature.

[0092] In this embodiment, when using risk calculation based on engineering consequences, it is essential to ensure that the probability p output by the model is accurate. j,k (x) represents the true and reliable data, not just the scores used for ranking. Therefore, a probabilistic calibration step is needed. Specifically, the original output of the base model is corrected using calibration parameters fitted on the validation set.

[0093] Step 407, probability calibration specifically involves using calibration parameters fitted on the validation set to map the original output of the base model to the calibrated predicted probability distribution through the Platt calibration method.

[0094] In this embodiment, the Platt Scaling method is preferably used for calibration. Specifically, for each base model j and each class k, a calibrator in the form of a sigmoid is trained. Assume that the original uncalibrated score output by the base model is u. j,k (x) (e.g., the leaf node score or Logit value of the decision tree), then the calibrated probability p' j,k The formula for calculating (x) is: ; where p' jk (x) is the calibrated predicted probability of base model j for sample x belonging to class k; exp is the natural exponential function; u jk (x) represents the raw output score (e.g., uncalibrated Logit value) of base model j for sample x belonging to class k; α cal and β cal The calibration parameters are obtained by fitting the model through minimizing the negative log-likelihood loss (NLL) on the validation set. After calibration, the probability values ​​output by the model will approximate the true empirical probabilities. For example, an output probability of 0.8 indicates that 80% of the samples with that score do indeed belong to that category, ensuring the mathematical rigor of the subsequent calculation of expected risk = probability × consequence cost.

[0095] Example 5 describes how to eliminate the confounding bias of macro data by constructing a hierarchical background set when calculating SHAP values, and how to screen out truly credible driving features through consistency evaluation after obtaining multi-model attribution results.

[0096] Step 501: Construct a hierarchical identification function based on the regional and engineering attributes in the disease risk feature dataset, and use the hierarchical identification function to divide the disease risk feature dataset into different hierarchical subsets.

[0097] In this embodiment, considering the wide range and high heterogeneity of reservoir dam data sources, a simple global sample cannot be used as the background set for SHAP calculation. Specifically, a hierarchical identification function s(x) is constructed, which maps samples to specific hierarchical IDs. For example, s(x) = Hash(Province,Basin,DamType) is defined to group earth-rock dams in the Yangtze River basin into one group and concrete dams in the Yellow River basin into another. This function is then used to traverse the entire dataset, dividing all samples into non-overlapping hierarchical subsets D. s This division ensures that, during attribution analysis, the sample to be explained is compared with groups with similar geographical and structural backgrounds, eliminating background noise caused by differences in region or dam type.

[0098] Step 502: For each sample to be explained, extract samples from the same hierarchical subset to construct a hierarchical background set based on its corresponding hierarchical identifier.

[0099] In this embodiment, for each target sample x that needs to be explained... target Calculate its hierarchical identifier s target =s(x target From the corresponding hierarchical subset D_s target A certain number (e.g., K) are randomly selected from the data. bg Samples with a value of 100 constitute the hierarchical background set B_s specific to that sample. target It should be noted that, to ensure the stability of the calculation, a minimum sample size threshold B can also be set. min If the sample size of a certain stratified subset is |D_s target | min If this happens, a rollback strategy is triggered, which involves relaxing the stratification conditions (e.g., removing the province constraint and stratifying only by watershed + dam type) and extracting background samples from the previous stratification level to ensure the statistical validity of the background distribution.

[0100] Step 503: Initialize the base models in the consensus attribution model pool using the hierarchical background set, and calculate the feature marginal contribution of the base models to the sample to be explained, such as... Figure 4 As shown.

[0101] In this embodiment, based on the constructed hierarchical background set B_s target The interpreter (e.g., TreeExplainer) for each base model (e.g., XGBoost or Random Forest) in the model pool is initialized. Specifically, the SHAP value calculation depends on the expected output when features are missing, and the expected output is based on the hierarchical background set B_s. target Estimated. The characteristic marginal contribution φ calculated in this way is... j,k (f) represents the risk contribution of this feature deviating from the average level of similar dams in a specific engineering context, and its physical meaning is more accurate and fair than attribution based on the global context.

[0102] Step 504: Calculate the global importance score of each feature for the base models in the consensus attribution model pool.

[0103] In this embodiment, to evaluate the credibility of the attribution results, it is necessary to calculate the dependence of each model on each feature. Specifically, for any feature f and model m... j Its global importance score I j ​(f) is defined as the average of the absolute values ​​of the SHAP values ​​calculated by the model across all samples. The calculation formula can be expressed as: Among them, I j (f) represents the global importance score of the j-th base model for feature f; N is the total number of samples; i is the sample index; k is the disease risk level category index; K is the total number of disease risk level categories; |...| represents the absolute value operation; φ j，i，k (f) represents the SHAP value of feature f calculated by the j-th base model for sample i and category k. This score reflects the performance of model m. j The average contribution strength of feature f to the prediction result is considered.

[0104] Step 505: Based on the global importance score of the same feature by the base model, calculate the coefficient of variation of the judgment of the degree of dispersion between the models.

[0105] In this embodiment, the coefficient of variation is used to measure the degree of disagreement among different models regarding the importance of the same feature. Specifically, for feature f, a set of global importance scores {I1(f), I2(f), ..., I...} given by all base models is collected. j Calculate the mean μ(f) and standard deviation σ(f) of the set. The coefficient of variation CV(f) of feature f is calculated as: CV(f) = σ(f) / μ(f). The smaller the coefficient of variation, the more consistent the models' views on the importance of the feature; the larger the coefficient of variation, the more divergent the views among the models, and the more likely the feature is to only play a role in some models, making it a model-sensitive feature with a risk of spurious correlation.

[0106] Step 506: Construct the SHAP consistency index using the coefficient of variation, and select features with a SHAP consistency index higher than a preset consistency threshold as a subset of reliable features.

[0107] In this embodiment, to provide users with an intuitive credibility indicator, the SHAP Consistency Index is defined. The specific calculation formula for SCI is: SCI(f) = 1 / (1 + σ(f) / μ(f)); where SCI(f) is the SHAP consistency index of feature f; σ(f) is the standard deviation of the global importance scores of all base models for feature f; μ(f) is the average of the global importance scores of all base models for feature f; and σ(f) / μ(f) is the coefficient of variation of feature f, i.e., CV.

[0108] The index ranges from (0,1). When all models agree, CV(f) = 0, and SCI(f) = 1. Based on engineering experience, a consistency threshold θ can be set. SCI For example, let θ be set. SCI=0.7. Only features with an SCI(f) ≥ 0.7 are included in the reliable feature subset. For features with an SCI(f) < 0.4, the system will mark or filter them, indicating to decision-makers that the attribution conclusions for that feature may have model structure bias and should be accepted with caution. This screening constitutes the last line of defense in the multi-model consensus mechanism, ensuring that the output risk drivers are robust and reliable.

[0109] Example 6 describes how to advance from traditional single-feature independent attribution to second-order interactive attribution, revealing the hidden synergistic and antagonistic mechanisms in the evolution of disease risk through tensor decomposition and graph theory analysis.

[0110] Step 601: Calculate the second-order SHAP interaction tensor of the base model respectively, and perform weighted fusion of the second-order SHAP interaction tensor based on the consensus weight to obtain the consensus interaction tensor.

[0111] In this embodiment, a high-dimensional interaction matrix needs to be calculated to capture the nonlinear coupling relationships between features. Specifically, for each base model m in the model pool... j The second-order SHAP interaction value is calculated using the path interpretation algorithm of the tree model, and a four-dimensional tensor is constructed. Where N is the number of samples, F is the number of features, and K is the number of classes. Elements in the tensor Let represent the contribution of the interaction between features f1 and f2 in the j-th model to the prediction of sample i as class c. This is achieved using consensus weights Ω. j We then perform a weighted average on this tensor to obtain the consensus interaction tensor Ψ. cons This embodiment integrates the judgments of multiple models on interaction effects, smoothing out the structural biases that may exist in a single model.

[0112] Step 602: Perform diagonal decomposition on the consensus interaction tensor to separate the consensus main effect component, which represents the independent influence of features, and the consensus interaction effect component, which represents the synergistic or antagonistic influence between features; based on the consensus main effect component and the consensus interaction effect component, generate a hierarchical attribution result that includes the main effect ranking and the interaction effect ranking.

[0113] In this embodiment, it is necessary to separate independent actions and interactions from the total interaction tensor. Specifically, the consensus main effect components correspond to the diagonal elements of the tensor, i.e. This represents the independent contribution of feature f without considering interactions with other features. The consensus interaction effect components correspond to the off-diagonal elements. To ensure the symmetry of the interaction effects (i.e., the interaction of feature A to B equals the interaction of B to A), the symmetric pure interaction components are usually calculated: This decomposition clearly divides the driving forces of disease risk into intrinsic factors and coupling factors.

[0114] Step 603: Calculate the global main effect intensity of the feature based on the consensus main effect component, and calculate the sum of the global interaction intensity of the feature and other features based on the consensus interaction effect component; calculate the ratio of the sum of the global interaction intensity to the sum of the global main effect intensity and the sum of the global interaction intensity to obtain the interaction contribution ratio of the feature; identify the interaction-dominant feature in the disease risk evolution based on the interaction contribution ratio.

[0115] In this embodiment, in order to calculate the proportion of interaction effect in the total feature contribution, an interaction contribution ratio index R is defined. inter Specifically, the global main effect strength of feature f is calculated. And the sum of the global interaction strengths of this feature with all other features, I. total_inter (f)=Σ f'≠f Σ|φ inter [f,f']|.

[0116] The formula for calculating the percentage of interaction contribution of feature f is: R inter (f)=I total_inter (f) / (I main (f)+I total_inter (f)); where R inter (f) represents the percentage of interaction contribution of feature f; I total_inter (f) represents the sum of the global interaction strengths of feature f with all other features; I main (f) represents the global main effect intensity of feature f; I main (f)+I total_inter (f) represents the total effect intensity of characteristic f.

[0117] The value range of this indicator is [0,1]. When R inter When (f)>0.5, feature f is determined to be an interactive dominant feature. This means that the feature alone has little impact on dam safety, but it is very easy to couple with other factors to cause risks (for example, the reservoir water level itself may not pose a great risk, but when it is combined with the crack width, the risk increases sharply). Such features are often the key causes of hidden defects.

[0118] Step 604: Calculate the global interaction strength between feature pairs based on the consensus interaction effect component, and select interaction pairs that meet the preset strength threshold.

[0119] In this embodiment, strength filtering is required to construct a sparse yet critical interaction network. Specifically, the global interaction strength I of each pair of features (f1, f2) is calculated. inter (f1, f2) represents the average of the absolute values ​​of the interaction effects across all samples. To determine the screening threshold θ... inter An adaptive statistical threshold is preferred over a fixed value. The mean μ of the interaction strength for all feature pairs is calculated.inter and standard deviation σ inter Set threshold θ inter =μ inter +λ×σ inter ; where θ inter An adaptive strength threshold for screening significant interaction pairs; μ inter σ is the average global interaction strength among all feature pairs; λ is a preset screening strictness parameter (e.g., 1.5); inter This represents the standard deviation of the global interaction strength among all feature pairs.

[0120] In this embodiment, the screening strictness parameter λ is preferably set to 1.5, retaining those strong interaction pairs that are above average, forming the interaction pair set E. sig .

[0121] Step 605: Construct a feature interaction network graph based on interaction pairs, where nodes represent features, edges represent interaction relationships between features, and edge weights represent global interaction strength.

[0122] In this embodiment, graph theory is used to visualize the disease risk mechanism. Specifically, an undirected weighted graph G=(V,E,W) is constructed. The node set V contains all features involved in the interaction; the edge set E directly corresponds to the interaction pair set E. sig The weight w of each edge in the edge weight set W. uv Equal to the global interaction strength I between feature u and feature v inter (u,v). Through graph representation, complex numerical matrices are transformed into intuitive network topology structures, clearly demonstrating which features have strong coupling relationships. Furthermore, as a preferred implementation, community detection algorithms (such as the Louvain algorithm for Leuven community detection) can be applied to the network graph to identify several tightly connected clusters of interactive features, each cluster potentially corresponding to a specific risky sub-pattern (such as the seepage-structure coupling pattern).

[0123] Step 606: Calculate the weighted degree centrality of each node in the feature interaction network graph, and identify the interaction hub features that play a key transmission role in the evolution of disease risk based on the weighted degree centrality.

[0124] In this embodiment, the aim is to identify key nodes in the network. Specifically, the weighted degree centrality of each node (feature) in the graph is calculated. Among them, DC w (f) represents the weighted degree centrality of node feature f in the feature interaction network graph; Neighbors(f) is the set of neighbor nodes directly connected to feature f in the network graph;

[0125] w ff'is the edge weight between feature f and feature f', i.e., the global interaction strength between the two.

[0126] This indicator reflects the total energy of the interaction between feature f and other features. Features with high weighted centrality are defined as interaction hub features. These features are similar to transportation hubs; their changes can affect the safety status of the entire system through multiple paths. Identifying hub features is crucial for developing risk mitigation and reinforcement strategies; controlling these hub factors can block multiple risk transmission paths.

[0127] Example 7: This example focuses on transforming abstract attribution values ​​into structured rules and spatial distribution layers that can directly guide engineering practice, and addresses the robustness and applicability of the mining results.

[0128] Step 701: Based on the consensus risk-driven contribution value, construct a partial dependency curve for key continuous features. Identify the scale effect characteristics of disease risks based on the positive and negative distribution patterns of the partial dependency curve across different value ranges. Based on the consensus risk-driven contribution value, fit the risk evolution trajectory of dam age characteristics. Identify the time window characteristics of disease risk outbreaks based on the fluctuation patterns of the risk evolution trajectory, such as... Figure 2 As shown.

[0129] The partial dependency function is: ;

[0130] Among them, PD k (v) represents the partial dependency function value of the k-th feature when it takes the value v, N is the total number of samples in the disease risk feature dataset, f is the weighted fusion prediction function of the consensus attribution model pool, and X k =v means fixing the value of the k-th feature to v, X -k =x i,-k This indicates that all features except the k-th feature take the actual value of the i-th sample.

[0131] In this embodiment, basic univariate pattern mining is performed. Specifically, for key continuous features such as reservoir capacity and dam height, a partial dependency plot (PDP) is drawn. If the curve is positive in the low-value range (increasing risk) and negative in the high-value range (reducing risk), it is identified as a scale effect. For dam age features, a trajectory curve of its risk contribution changing over time is fitted. The system automatically detects the shape of the curve, such as identifying the inflection point where the curve slope changes from gentle to steep, or identifying the hump section where the risk value is consistently above zero. Such sections are identified as high-risk time windows for the outbreak of disease and danger (e.g., the dam age range of 30-40 years).

[0132] The formula for identifying the boundary value of scale effect is: ;

[0133] Where, θ kV represents the scale effect boundary value for the k-th project scale characteristic. k Let k be the set of candidate segmentation points for the k-th feature. The average consensus risk-driven contribution value for samples whose k-th feature value is greater than v. argmax is the average consensus risk-driven contribution value for samples whose k-th feature value is less than or equal to v, and argmax is the parameter value that maximizes the objective function.

[0134] Step 702: Binning is performed on the value range of continuous features, the mean of consensus risk-driven contribution value of samples in each bin is calculated, and the robust effect curve of the feature is constructed.

[0135] In this embodiment, to achieve automated rule synthesis, continuous features need to be discretized. Specifically, the range of values ​​for feature f is divided into B bins. For the b-th bin, all samples falling within this range are collected, and the average consensus risk-driven contribution value corresponding to this type of sample is calculated to form a discretized robust effect curve g. f (b) Compared to the original scatter distribution, binning can effectively smooth noise and extract the mainstream trend of feature influence.

[0136] Step 703: Based on the morphological change characteristics of the robust effect curve, automatically identify the risk threshold or risk window that causes a sudden change in the risk level.

[0137] The samples in each bin were resampled using the bootstrap sampling method, and the confidence interval of the robust effect curve was calculated. Only the segments with a lower bound greater than zero or an upper bound less than zero in the confidence interval were retained as candidate risk windows.

[0138] In this embodiment, to avoid misjudgments of patterns due to isolated extreme samples, a bootstrap statistical test is introduced. Specifically, T (e.g., 1000) bootstrap samplings with replacement are performed on the sample set within each bin. The mean is calculated for each sampling, yielding an empirical distribution of the mean. Based on this distribution, a 95% confidence interval [CI] is calculated. low CI high Only when CI low Only when the value is greater than 0 is the sub-container considered to have a role in increasing risk; only when CI... high A confidence interval of less than 0 is considered to have a risk-reducing effect. Sections containing zero values ​​in the confidence interval are considered areas of uncertainty and are therefore excluded. This embodiment improves the statistical confidence of the mining rules.

[0139] Step 704: Apply the piecewise constant change point detection algorithm to the robust effect curve. By minimizing the sum of the piecewise fitting error and the number of segments penalty term, locate the optimal segmentation point as the risk threshold.

[0140] In this embodiment, the aim is to automatically extract precise thresholds (e.g., reservoir water level > 125m) from the curve. Specifically, a change point detection algorithm based on dynamic programming (e.g., PELT, Pruned Exact Linear Time algorithm) is employed. The objective function is constructed. J=Σ q=1 Q Σ b∈seg_q (g f (b)-μ q ) 2 +β×Q; J is the objective function value that the change point detection algorithm needs to minimize; Q is the number of segments into which the robust effect curve is divided; seg_q is the set of bins contained in the q-th segment; b is the bin index; g f (b) represents the mean robust risk contribution of feature f on bin b; μ q β is the mean risk contribution value of all bins within the q-th segment; β is a preset penalty coefficient used to prevent overfitting (too many segments). The optimal split point is found by minimizing this objective function. The feature values ​​corresponding to these split points are the key risk thresholds, dividing the feature interval into several segments with distinctly different risk characteristics.

[0141] Step 705: Encapsulate the risk threshold or risk window into a structured rule object. The structured rule object includes the triggering conditions, the direction of risk impact, and the rule confidence level.

[0142] In this embodiment, the above-mentioned data mining results are transformed into standardized objects that are computer-readable and engineering-executable. Specifically, a structured rule object R is defined as a quintuple (f, type, cond, dir, conf). Here, f is the feature name; type identifies the rule type (e.g., threshold or window type); cond is the specific triggering condition (e.g., x∈[30,40]); dir is the direction of influence (+ indicates increased risk, - indicates decreased risk); and conf is the confidence level calculated based on Bootstrap (e.g., 1-α). This type of structured output can be directly input into the dam safety early warning system to achieve automated risk monitoring.

[0143] Step 706: Based on the spatial location information in the disease risk feature dataset, aggregate the samples into preset spatial units and calculate the average risk contribution value within each spatial unit; construct a spatial adjacency weight matrix and calculate local spatial statistics based on the average risk contribution value and the spatial adjacency weight matrix; detect regions where the local spatial statistics are higher than the global expected value and identify them as spatial hotspots that lead to high incidence of disease risks.

[0144] In this embodiment, the mining of the geospatial dimension is added.

[0145] Specifically, dam samples are aggregated according to administrative divisions or watershed grids (spatial units), and the average risk contribution value of dams within each unit is calculated. Construct the spatial adjacency weight matrix W. sp , where w ij =1 indicates that elements i and j are adjacent. Calculate local G. i *Statistic, formula is: Among them, G i * represents the local Getis-Ord statistic for spatial cell i; w ij The elements in the spatial weight matrix represent the adjacency weights between spatial units i and j; x j The average risk contribution value within spatial unit j; S is the global average of the risk contribution values ​​of all spatial units; S is the global standard deviation of the risk contribution values ​​of all spatial units; the statistic can identify high-value clusters, and n is the total number of spatial units.

[0146] When G i When the value is greater than 0 and passes the test (e.g., p < 0.05), the area is identified as a high-risk spatial hotspot. This helps to identify contiguous risk areas affected by specific geological structures or regional climates, providing a basis for coordinated regional governance.

[0147] To more clearly illustrate the practical application effect of the method of the present invention, a typical application scenario example is given below.

[0148] Scenario: A provincial water resources department needs to identify the defects and risks of approximately 500 small reservoir dams within its jurisdiction to guide the prioritization of reinforcement and repair. These dams are mostly earth-rock dams built between the 1960s and 1980s, with an average age of 40-60 years. Historical safety assessment data shows that approximately 15% are classified as Class III dams (severely defective), 35% as Class II dams (generally defective), and 50% as Class I dams (basically normal).

[0149] Data preparation: Collect feature data in 12 dimensions, including total reservoir capacity, maximum dam height, dam crest length, watershed, dam age, and number of historical reinforcements, as well as seepage safety and structural safety evaluation level labels for each dam.

[0150] Results of method application:

[0151] Consensus attribution model pool selection results: After evaluation by the competitive prediction framework, three models, Extreme Random Tree (AUC=0.91), Random Forest (AUC=0.89), and LightGBM (AUC=0.87), were selected for the consensus pool, while XGBoost (AUC=0.78) was removed because it was below the admission threshold.

[0152] Key Risk Characteristics Identification Results: Through consensus risk-driven contribution value analysis, dam age and total reservoir capacity were identified as the two most important risk drivers. The partial dependence curve of dam age shows a significant risk jump in the 35-45 year range, a finding highly consistent with the province's engineering background where many dams were constructed in batches during the same period, using similar materials and processes.

[0153] Interaction effect mining results: The feature interaction network diagram shows that dam age and total reservoir capacity form the strongest interaction pair (global interaction strength = 0.23), indicating a synergistic amplification effect on the risk of disease in aging large reservoirs. Based on weighted degree centrality analysis, dam age was identified as the interaction hub feature (DC). w =0.52).

[0154] Example of robust rule output: The structured rules automatically generated by the system include:

[0155] Rule R1: {Feature = Dam Age, Type = Window Type, Condition = ∈ [35, 45] years, Direction = +, Confidence = 0.93};

[0156] Rule R2: {Feature = Total storage capacity, Type = Threshold type, Condition => 5 million m³, Direction = +, Confidence = 0.89}.

[0157] Technical effect comparison: Compared with the traditional single-model attribution method, the SHAP consistency index (SCI) of the method of this invention is improved by an average of 32%, indicating that the robustness of the attribution conclusions is significantly enhanced; the key risk features identified have a 91% agreement with the experience judgment of water conservancy experts, verifying the engineering applicability of the method.

[0158] According to one aspect of this application, a method for mining the risk features of reservoir dams based on multi-model consensus and risk mapping is provided, such as... Figure 5 As shown, it includes the following steps:

[0159] Step 1: Construct a structured database that covers multi-dimensional information such as structural parameters, geographical environment, and operation management.

[0160] Step 2: Construct a multi-model competitive prediction framework for machine learning, such as Extreme Random Tree, Random Forest, and XGBoost.

[0161] Step 3 compares the predictive performance of six machine learning algorithms, including Extreme Random Tree, Random Forest, and XGBoost, on safety evaluation indicators such as flood control, seepage, and structure.

[0162] Step 4: Quantify the contribution of each feature variable to different safety evaluation indicators based on the SHAP method.

[0163] Step 5: Through global and local interpretation analysis, conduct in-depth analysis of the nonlinear relationship and driving mechanism between reservoir size, dam age evolution, geographical environment and reservoir risk characteristics.

[0164] Step 1: Construct a multi-dimensional structured database of dangerous reservoir features.

[0165] Step 1.1: Based on reservoir safety assessment reports, water conservancy survey data, and geographic information data as the basic data sources, collect multi-source heterogeneous data, including engineering design parameters, historical hazard records, and hydrogeological data, and perform cleaning operations to remove abnormal samples with excessively high missing values ​​or obvious logical errors, and construct a standardized dataset.

[0166] A multi-dimensional feature system is constructed to extract engineering scale features that characterize the physical volume and potential destructive capacity of the project, including continuous numerical variables such as reservoir capacity and dam height; geographical environmental features that characterize external environmental constraints and regional background, including key variables such as the province, the river basin, and the dam site elevation; features that characterize the structural attributes and functional positioning of the project, including categorical variables such as dam type, reservoir type, and primary function; and time-varying evolution features that characterize the service time and aging degree of the project.

[0167] Step 1.2 involves adaptively encoding and preprocessing the data to meet the input requirements of the subsequent ensemble model.

[0168] Step 1.2.1: Perform multi-label classification coding on the output labels. The output labels cover six core dimensions: flood control safety, operation management, seepage safety, structural safety, metal structure safety, and seismic safety. Map the A, B, and C evaluation levels in the safety assessment report to computer-processable numerical category labels to construct a multi-objective classification task.

[0169] Step 1.2.2: Differentiate the input features. For numerical features that have a natural segmentation advantage in tree models, maintain the original physical scale and do not perform normalization to preserve their physical meaning and interpretability. For categorical features, use label encoding to convert them into integer sequences to avoid the high-dimensional sparsity and dimensionality curse problems that may be caused by one-hot encoding when the feature cardinality is large.

[0170] Step 2: Construct a machine learning multi-model competitive prediction framework.

[0171] Step 2.1: Establish a model library containing multiple inductive bias algorithms. Through parallel training and competitive selection of multiple models, solve the underfitting or overfitting problems that may exist in a single model, and determine the best predictor for a specific disease risk type.

[0172] The model library integrates a variety of advanced ensemble learning algorithms based on decision trees, including an extreme random tree model that uses random feature subsets and random threshold partitioning, a random forest model that uses bootstrap sampling and random feature selection, an XGBoost model based on a gradient boosting framework, and a LightGBM model. It also integrates logistic regression and K-nearest neighbors as benchmark models to evaluate the performance gains of nonlinear models.

[0173] Accuracy comparison chart of multi-model competitive selection, such as Figure 6 As shown.

[0174] Step 2.2: Construct independent prediction tasks for safety evaluation indicators of multiple dimensions. Considering the differences in the disaster-causing mechanisms of different risks such as flood control, seepage, and structure, each task is independently trained and its parameters are optimized to capture the specific driving patterns of different risk types.

[0175] Step 2.3: The dataset is divided into K mutually exclusive subsets using hierarchical cross-validation to ensure that the proportion of samples of each disease risk level in the training set and validation set is consistent with the overall distribution, effectively solving the class imbalance problem. In addition, grid search and random search strategies are combined to globally optimize key hyperparameters such as the number of trees, maximum depth, learning rate, regularization coefficient and minimum number of samples in leaf nodes.

[0176] Step 3: Model performance evaluation and establishment of optimal benchmark.

[0177] Step 3.1: Based on the test set data, systematically evaluate the performance of each model in different disease risk prediction tasks from multiple dimensions such as accuracy, discrimination and generalization ability.

[0178] Accuracy is calculated to assess the overall correctness of predictions; precision and recall are calculated to assess the ability to identify risky samples; F1 score is calculated to comprehensively measure the model's performance under imbalanced samples; and the area under the receiver operating characteristic curve (AUC) is calculated to assess the model's ranking ability and robustness at different thresholds.

[0179] Step 3.2: Select the most suitable explanatory benchmark model for the current data distribution based on the evaluation results.

[0180] The model with the highest AUC and the smallest variance in cross-validation is selected as the carrier for subsequent SHAP interpretability analysis to ensure that the interpretation results are based on high-precision and high-confidence predictions, and to avoid erroneous attribution analysis due to model prediction bias.

[0181] Step 4: Construct an interpretability analysis module based on SHAP game theory and graph theory.

[0182] Step 4.1: Introduce the SHAP method to construct the attribution analysis mechanism, initialize the TreeExplainer interpreter, and use the TreeSHAP algorithm to calculate the original marginal contribution matrix of features at the sample level.

[0183] Step 4.2, execute the ordered classification SHAP value weighted fusion strategy: For reservoir risk levels with clear physical order (0=healthy, 1=average, 2=risky), construct a linear weighted fusion algorithm to map the SHAP values ​​of multi-dimensional categories to a single-dimensional risk-driven value.

[0184] Step 4.2.1, define the risk weight vector W=[w0,w1,w2], where w0=0 represents no risk contribution from a healthy state, w1=1 represents moderate risk, and w2=2 represents high risk.

[0185] Step 4.2.2: Calculate the fused SHAP value. The Shapley value measures the average contribution of a feature to the prediction result. Given a feature set N={1,2,…,n}, the Shapley value of any feature i is defined as: ;

[0186] Where: S represents any subset of features that does not contain feature i; f(S) is the expected output of the model when using only the subset of features S; f(S∪{i})–f(S) represents the change in the model output after adding feature i, i.e., the marginal contribution of feature i to the prediction. To be applicable to complex machine learning models, SHAP further constructs a linearly additive interpretable framework:

[0187] ;

[0188] Where: zi∈{0,1} indicates whether the feature is enabled; This is the baseline output; Let be the Shapley value of feature i.

[0189] Step 4.3: Calculate the global feature importance. Based on the fused SHAP value, calculate the average of the absolute values ​​of each feature as a quantitative indicator to measure the global influence of the feature on the dam's safety status.

[0190] Step 4.4: Construct a feature interaction network graph based on graph theory, and combine SHAP interaction values ​​with graph theory algorithms to visualize the coupling and collaboration mechanism between multi-dimensional features.

[0191] Step 4.4.1: Calculate the interaction strength matrix and the second-order interaction effect value between features; if the model does not support direct calculation, calculate the Pearson correlation coefficient matrix between the feature SHAP value vectors as an alternative interaction strength.

[0192] Step 4.4.2: Construct the network topology by introducing graph theory methods to build an undirected weighted graph.

[0193] Step 5: Automated mining and determination of multidimensional disease risk characteristics and patterns.

[0194] Step 5.1: Apply specific mining logic to identify the scale effect of risk. Construct a SHAP bias dependency graph for key features (reservoir capacity, dam height). If the curve is significantly positive in the low value range, it increases the risk; if it is significantly negative in the high value range, it decreases the risk.

[0195] Step 5.2: Apply mining logic to identify the time-varying nonlinear characteristics of dam risks. Fit the SHAP evolution trajectory of dam age characteristics. Detect curve changes and automatically identify the concentrated window period of dam risk outbreaks.

[0196] Step 5.3: Apply mining logic to identify the spatial heterogeneity of risk hazards. Map the SHAP mean of geographical features (provinces, river basins, etc.) to GIS space. Identify high SHAP value clusters and determine the constraints of topographic structure or hydro-meteorological conditions on risk hazards.

[0197] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A reservoir dam disease risk feature mining method based on multi-model consensus and risk mapping, characterized in that, include: Obtain a dataset of disease risk features covering the multidimensional attributes of reservoir dams. The dataset includes disease risk level labels representing different safety states. The disease risk feature dataset is processed using a pre-built consensus attribution model pool to obtain the predicted probability distribution of the base model output in the consensus attribution model pool, and the performance evaluation index of the base model on the validation set is obtained. Calculate the consensus weight of the base model in attribution fusion based on the performance evaluation index of the base model. Based on the inter-class distribution differences or engineering risk cost characteristics of each disease risk level corresponding to the predicted probability distribution, the risk weight vector corresponding to each disease risk level is determined. Using a pre-defined interpretable attribution algorithm, the feature marginal contribution of the base model to the central samples of the disease risk feature dataset is calculated; Based on consensus weight and risk weight vectors, the feature marginal contributions calculated by the base model are weighted and fused to generate consensus risk-driven contribution values ​​of multi-dimensional features; Based on consensus risk-driven contribution values, interpretable mining results representing the evolution mechanism of disease risk are generated; The consensus weight of the base model in attribution fusion is calculated using the following formula: ； Among them, Ω j Let AUC be the consensus weight of the j-th base model. j Var represents the area under the receiver operating characteristic curve (AUC) of the j-th base model obtained through hierarchical cross-validation. j Let denot γ be the performance variance of the j-th base model obtained through hierarchical cross-validation, J be the total number of base models, γ be the preset accuracy sensitivity coefficient, and η be the preset stability penalty coefficient. For the first The average area under the receiver operating characteristic curve (AUC) of each base model obtained through hierarchical cross-validation. For the first The performance variance of each base model obtained through hierarchical cross-validation; The feature marginal contributions calculated from the base model include: Based on the regional and engineering attributes in the disease risk feature dataset, a hierarchical identification function is constructed, and the hierarchical identification function is used to divide the disease risk feature dataset into different hierarchical subsets. For each sample to be explained, samples are extracted from the same hierarchical subset to construct a hierarchical background set based on its corresponding hierarchical identifier; The base models in the consensus attribution model pool are initialized using a hierarchical background set, and the feature marginal contribution of the base models to the samples to be explained is calculated.

2. The method of claim 1, wherein, The disease risk feature dataset is constructed through the following steps: Extract engineering scale features that characterize the physical volume of the project, geographical environmental features that characterize the regional background constraints, and time-varying evolution features that characterize the service life of the project from multi-source heterogeneous data. Multi-label classification encoding is performed on the disease risk level labels used to represent the safety status in the disease risk feature dataset, mapping different safety evaluation dimensions into numerical category labels; The categorical features in the disease risk feature dataset are encoded with integer sequences, while the original physical scale of the numerical features is preserved.

3. The method of claim 1, wherein, The consensus attribution model pool is built on a competing prediction framework: The competitive prediction framework integrates extreme random tree models, random forest models, gradient boosting decision tree models, and lightweight gradient boosting machine models; The consensus attribution model pool consists of multiple base models in the competitive prediction framework whose performance evaluation metrics on the validation set meet the preset admission threshold.

4. The method of claim 1, wherein, Generate interpretable mining results that characterize the evolutionary mechanisms of disease risks, including: Based on the consensus risk-driven contribution value, a partial dependency curve of key continuous features is constructed. According to the positive and negative distribution of the partial dependency curve in different value ranges, the scale effect characteristics of disease risk are identified. Based on the consensus risk-driven contribution value, the risk evolution trajectory of dam age characteristics is fitted, and the time window characteristics of disease outbreaks are identified according to the fluctuation pattern of the risk evolution trajectory.

5. The method of claim 1, wherein, The consensus weights of the base model in attribution fusion are calculated, including: Obtain the area under the receiver operating characteristic curve (AUC) of the base model on the test set partitioned from the disease risk feature dataset; Set a performance admission threshold and calculate the difference between the area under the receiver operating characteristic curve of the base model and the performance admission threshold to obtain the excess performance value; The excess performance values ​​of all base models in the consensus attribution model pool are normalized to obtain the consensus weights of the base models.

6. The method of claim 1, wherein, Determine the risk weight vector corresponding to each level of disease risk, specifically including: Calculate the centroid vector of each disease risk level sample in the feature space in the disease risk feature dataset, and calculate the global covariance matrix; Based on the centroid vector and the global covariance matrix, calculate the inter-class Mahalanobis distance between adjacent disease risk levels; A nonlinear weight mapping function is constructed based on the inter-class Mahalanobis distance to calculate the risk weight vector corresponding to each disease risk level. The nonlinear weight mapping function includes a gradient shape parameter to control the shape of the growth curve, and a risk amplification coefficient to control the gain of the high-risk category.

7. The method of claim 1, wherein, Determine the risk weight vector corresponding to each disease risk level, including: Obtain the pre-stored preset engineering consequence costs corresponding to each disease risk level, normalize the preset engineering consequence costs, and obtain the risk weight vector corresponding to each disease risk level. The method further includes performing probability calibration on the predicted probability distribution output by the consensus attribution model pool before calculating the marginal contribution of the feature. Probabilistic calibration specifically uses calibration parameters fitted on the validation set to map the original output of the base model to the calibrated predicted probability distribution using the Platt calibration method.

8. The method of claim 1, wherein, Generate interpretable mining results representing the evolutionary mechanism of disease risks, including performing hierarchical decomposition of interaction effects: The second-order SHAP interaction tensor of the base model is calculated separately, and the second-order SHAP interaction tensor is weighted and fused based on the consensus weight to obtain the consensus interaction tensor. Diagonal decomposition of the consensus interaction tensor separates the consensus main effect component, which represents the independent influence of features, and the consensus interaction effect component, which represents the synergistic or antagonistic influence between features. Based on the consensus main effect component and the consensus interaction effect component, a hierarchical attribution result containing the main effect ranking and the interaction effect ranking is generated.

Images