Earth-rock dam burst flood peak prediction method and system based on Copula data enhancement and XG-Boost algorithm

Abstract

The present invention belongs to the field of machine learning technology, and specifically discloses a method and system for predicting earth-rock dam burst peaks based on Copula data enhancement and XG-Boost algorithm; the method comprises: S1, obtaining historical dam body data and performing data preprocessing on the historical dam body data; S2, performing data enhancement on the dam body feature data based on the Student-T-Copula model, and generating an enhanced data set XG-Boost algorithm. aug ; S3, introduce the gradient perturbation algorithm based on model training feedback; S4, generate enhanced data X aug and perturbation-enhanced samples X grad After that, an inverse transformation is performed and label prediction is performed to obtain a final, complete enhanced dataset. S5: The enhanced dataset is combined with the dam body characteristic data and its corresponding label data to form a new dataset, and a regression model is constructed and trained. S6: The current dam body characteristic data is input to obtain the predicted current dam-break peak flow. By combining data enhancement and regression models, this invention effectively addresses the problems of insufficient accuracy and data in existing methods for predicting earth-rockfill dam burst peak flow.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and more specifically, to a method and system for predicting earth-rock dam burst flood peaks based on Copula data enhancement and XG-Boost algorithms. Background Art

[0002] Dam failure (dam damage) is a major disaster in water conservancy projects, often accompanied by casualties, property losses, and ecological damage. The prediction and prevention of dam failures have long been a research priority in water conservancy projects. Predicting dam failures through monitoring and data analysis of the dam and its surroundings, and providing early warning and emergency response time to relevant departments, is key to ensuring public safety and reducing disaster losses. However, dam failures are often the result of the interaction of multiple factors, including but not limited to the dam structure, environmental changes (such as precipitation and temperature fluctuations), geological conditions, and external pressures. The relationships between these factors are often complex and can exhibit significant temporal variability and spatial heterogeneity, making dam failure prediction extremely challenging.

[0003] Existing dam-break prediction technologies primarily include physical models, statistical methods, and machine learning. Physical models assess safety by simulating the mechanical behavior of dam bodies. While highly accurate, they require large amounts of input data and high-precision parameters, resulting in complex computations and difficulties in real-time prediction. Statistical methods analyze historical data to reveal the relationship between dam failure and related factors. While simple and easy to use, they struggle to capture complex nonlinear and interactive effects and exhibit low accuracy. Machine learning methods leverage big data for prediction and possess strong adaptability, but they rely on data quality and diversity and lack accuracy and generalization for complex problems. Furthermore, dam-break prediction faces data imbalance and high dimensionality. The relatively small number of dam-break events leads to biased training data, which can easily affect prediction accuracy. Furthermore, nonlinear relationships between multiple variables further complicate processing of high-dimensional data. Existing solutions often employ data augmentation methods to expand the data sample size. However, traditional data augmentation methods typically rely on simple transformations such as rotation, translation, and scaling. While these methods can expand the dataset to some extent, the generated data often lacks sufficient exploration of the data's inherent structure and complex patterns. Traditional methods have the following shortcomings: lack of targeted optimization, limited diversity of augmented data, risk of overfitting, and low generation efficiency.

[0004] Furthermore, the above method has certain limitations in data enhancement and prediction of dam break status, and it is difficult to provide accurate prediction results.

[0005] In view of this, the present invention proposes an earth-rock dam burst flood peak prediction method and system based on improved Copula data enhancement and XG-Boost algorithm to solve the above problems. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned objectives, the present invention provides the following technical solution: a method for predicting earth-rock dam burst peaks based on Copula data enhancement and XG-Boost algorithm, comprising the following steps:

[0007] S1. Obtain historical dam body data and perform data preprocessing on the historical dam body data, thereby obtaining dam body feature data and its corresponding label data; the specific steps include:

[0008] Preset X={X1,X2,X3,…,X n} represents the dam body characteristic data;

[0009] Preset y={y1,y2,y3,…,y n} represents the label data corresponding to the dam body feature data;

[0010] The dam characteristic data are processed using a standardized formula, which is:

[0011] ;

[0012] Where, represents the standardized dam characteristic data, represents the mean value of the dam characteristic data, is the standard deviation of the dam characteristic data, Indicates the characteristic data of the dam body;

[0013] The standardized dam characteristic data is converted into uniform distribution. The conversion formula is:

[0014] ;

[0015] Where, is the transformed dam characteristic data, is the cumulative distribution function CDF of the dam characteristic data x, It is the inverse function of the cumulative distribution function of the dam characteristic data;

[0016] S2. Based on the Student-T-Copula model, the dam feature data is enhanced and an enhanced dataset is generated:

[0017] The step of performing data enhancement on the dam body characteristic data based on the Student-T-Copula model to obtain enhanced data specifically includes:

[0018] The Student-T-Copula model is used to capture the relationship between variables in the dam characteristic data, and its expression is:

[0019] ;

[0020] Where, is a Student's t distribution with ν degrees of freedom, is the ith marginal distribution The inverse CDF of are the parameters of the Student-T-Copula model;

[0021] The Student-T-Copula model and random variables are sampled from the uniform distribution to generate new dam characteristic data, namely enhanced data, which is expressed as:

[0022] ;

[0023] Where, is the enhanced data, u=(u1,u2,…,u d ) is a random variable sampled from a uniform distribution, It is the inverse transformation operation of the Student-T-Copula model;

[0024] S3. To improve the diversity of augmented data and the quality of boundary samples, a gradient perturbation algorithm based on model training feedback is introduced based on the Student-T-Copula augmented samples. The steps are as follows:

[0025] First, use the base model f trained with the original training data 0， The model uses random forest to estimate pseudo labels for augmented samples:

[0026] ;

[0027] Where x i is the enhanced sample set The i-th enhanced sample in ;

[0028] Set the currently trained regression model f to effectively fit the nonlinear mapping relationship between the dam feature data and its labels. The model is based on support vector regression (SVR) and combined with particle swarm optimization (PSO) algorithm for hyperparameter tuning.

[0029] according to The sample prediction result obtained with f, defines the loss function as the mean square error between the sample and its estimated label:

[0030] ;

[0031] Where, is the loss function, Represents the basic regression model trained using the original training data Pseudo-label estimation for augmented samples; f is the regression model currently being trained, and the fitting target is the pseudo-label above;

[0032] According to the loss function, the gradient direction of the calculated sample in the input space is obtained:

[0033] ;

[0034] Where, is the loss function for sample x i The gradient vector of ; this gradient represents the sensitive direction of the loss function along the feature vector, which is used to guide the sample to be disturbed to a higher error area to improve the model's learning ability for boundary samples or easily confused samples; Indicates the loss function in sample x i The gradient on

[0035] Combined with the gradient vector, the perturbation sample is generated. The perturbation method is as follows:

[0036] ;

[0037] Where, is the i-th disturbance enhancement sample, η is the disturbance step length, Prevent division by zero for small constants; is the Euclidean norm (L2 norm) of the gradient, used to normalize the direction;

[0038] Add the perturbed samples to the enhanced data to form a perturbation-enhanced sample set:

[0039] ;

[0040] Where, Enhance the sample set for perturbation;

[0041] S4. Generate enhanced data X aug and perturbation-enhanced samples X grad After that, perform the inverse transformation and label prediction; specifically including:

[0042] The enhanced data is inversely transformed to make it conform to the original feature space; the specific inverse transformation formula is:

[0043] ;

[0044] Where, It is the value after matching the enhanced data back to the original distribution through CDF, that is, the result after inverse transformation; is the enhanced feature data X augOne of the sample components comes from the pseudo sample generated by the Student-T-Copula model; represents the current estimated CDF of the augmented data; It is the inverse function of the CDF of the original dam characteristic data;

[0045] The purpose of the inverse transform is to map the enhanced samples from the standardized or uniformly distributed space back to the feature space of the original data, so that the generated samples have the same physical meaning and distribution as the original data;

[0046] Use the trained regression model to predict the labels of the augmented data. The expression is:

[0047] ;

[0048] ;

[0049] Where f is the regression model trained by augmented samples and pseudo labels;

[0050] Construct the final enhanced dataset and merge the original enhanced samples with the gradient perturbation enhanced samples to form the final enhanced dataset:

[0051] ;

[0052] ;

[0053] Where, Original augmented samples generated for Student-T-Copula; Gradient perturbation enhancement samples generated after introducing the gradient perturbation algorithm based on model training feedback; It is the combination of the original enhanced sample and the gradient perturbation enhanced sample; is the set of predicted labels of the two parts of enhanced samples mentioned above (i.e., the set of predicted labels of the two parts of enhanced samples, the original enhanced samples and the gradient perturbation enhanced samples);

[0054] Finally, we get the enhanced dataset (X total ,y total ) for subsequent model training and testing;

[0055] S5. The enhanced dataset, the dam feature data, and their corresponding label data are combined to form a new dataset, and the XG-Boost regression model is constructed and trained using the new dataset to obtain a trained XG-Boost regression model. The specific steps include: combining the final enhanced dataset, the dam feature data, and their corresponding label data to form a new dataset, and dividing the new dataset into a training set and a test set in a ratio of 8:2.

[0056] Define an XG-Boost regression model, set a base learner, and then use the training set to train the XG-Boost regression model to capture nonlinear relationships in the data;

[0057] Hyperparameter tuning of the XG-Boost regression model was performed using grid search and cross-validation;

[0058] Re-predict the output data in the new dataset using the test set, and evaluate the XG-Boost regression model using RMSE and R² until the RMSE and R² of the XG-Boost regression model reach the preset thresholds.

[0059] S6. Obtain the current dam body characteristic data, preprocess the current dam body characteristic data and input it into the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

[0060] Preferably, the historical dam body data includes dam height, storage capacity, porosity and dam break peak flow, wherein the dam height, storage capacity and porosity are used as dam body characteristic data, and the dam break peak flow corresponding to the dam body characteristic data is used as label data.

[0061] Preferably, the step of obtaining current dam body characteristic data, preprocessing the current dam body characteristic data and inputting the data into a trained XG-Boost regression model, i.e., obtaining the predicted current dam break peak flow, includes:

[0062] Obtaining current dam body characteristic data and preprocessing the current dam body characteristic data;

[0063] Input the pre-processed current dam feature data into the trained XG-Boost regression model.

[0064] The XG-Boost regression model outputs the predicted current dam break peak flow.

[0065] The present invention also provides a system for predicting earth-rock dam burst flood peaks based on Copula data enhancement and XG-Boost algorithms, which is applied to any of the earth-rock dam burst flood peak prediction methods based on Copula data enhancement and XG-Boost algorithms, and includes:

[0066] A preprocessing module acquires historical dam body data and performs data preprocessing on the historical dam body data to obtain dam body feature data and its corresponding label data;

[0067] The data enhancement module performs data enhancement on the dam feature data based on the Student-T-Copula model to obtain the initial enhanced data samples. On this basis, the enhanced samples are further fine-tuned using a perturbation mechanism based on gradient optimization to achieve dual enhancement of sample feature diversity and boundary discriminability. The enhanced data is then subjected to inverse transformation operations and label data estimation to obtain the final enhanced dataset.

[0068] The model training module combines the enhanced dataset, dam feature data, and their corresponding label data into a new dataset, and uses the new dataset to build and train the XG-Boost regression model, thereby obtaining the trained XG-Boost regression model.

[0069] The prediction module obtains the current dam body characteristic data, preprocesses the current dam body characteristic data and inputs the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

[0070] The technical effects and advantages of the earth-rock dam burst flood peak prediction method and system based on the improved Copula data enhancement and XG-Boost algorithm of the present invention are as follows:

[0071] 1. By introducing the Student-T-Copula model to construct a multivariate joint distribution, a large number of enhanced samples with consistent distributions with the original samples are synthesized. On this basis, a perturbation mechanism based on gradient optimization is further adopted to fine-tune the enhanced samples, achieving a dual enhancement of sample feature diversity and boundary discriminability. It can effectively capture the correlation between variables in the dam characteristic data and generate a richer data set. The enhancement process can make up for the lack of actual historical dam data and significantly improve the generalization ability and prediction accuracy of the model. By using the XG-Boost algorithm to construct a regression model, taking advantage of its advantages in processing nonlinear relationships and high-dimensional data, it can accurately capture the complex mapping relationship between dam characteristic data and dam break peak flow, thereby achieving more accurate flood peak flow prediction.

[0072] 2. Through data preprocessing steps including standardization and distribution conversion, the impact of dimensional differences between feature variables on the training of the XG-Boost regression model can be effectively reduced. At the same time, by converting the data into a uniform distribution, the distribution uniformity of the data is improved, and the stability of the XG-Boost regression model is enhanced. By preprocessing the current dam body characteristic data and using the trained XG-Boost regression model for prediction, it is possible to quickly respond to changes in the current state of the earth-rock dam and realize real-time dam break peak flow prediction, providing an efficient and real-time decision-making basis for flood disaster prevention and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1Schematic diagram of the flow of the earth-rock dam burst flood peak prediction method based on Copula data enhancement and XG-Boost algorithm of the present invention;

[0074] Figure 2 This is an effect diagram of the peak flow prediction training set and test set in an embodiment of the present invention;

[0075] Figure 3 This is a structural diagram of the earth-rock dam burst flood peak prediction system based on Copula data enhancement and XG-Boost algorithm of the present invention. DETAILED DESCRIPTION

[0076] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0077] Example 1

[0078] See also Figure 1 and Figure 2 As shown, the earth-rock dam burst flood peak prediction method based on Copula data enhancement and XG-Boost algorithm described in this embodiment includes:

[0079] S1. Acquire historical dam body data and perform data preprocessing on the historical dam body data, thereby obtaining dam body feature data and its corresponding label data;

[0080] S2. Based on the Student-T-Copula model, data enhancement is performed on the dam feature data to obtain enhanced data, and the enhanced data is subjected to inverse transformation and label data prediction to obtain an enhanced data set;

[0081] S3. To improve the diversity of augmented data and the quality of boundary samples, a gradient perturbation algorithm based on model training feedback is introduced based on the Student-T-Copula augmented samples.

[0082] S4. Generate enhanced data X aug and perturbation-enhanced samples X grad After that, the inverse transformation operation is performed and the label prediction is performed, and finally the enhanced data set (X total ,y total ) for subsequent model training and testing;

[0083] S5. The final enhanced dataset, dam feature data, and their corresponding label data are combined to form a new dataset. The new dataset is used to build and train an XG-Boost regression model, thereby obtaining a trained XG-Boost regression model.

[0084] S6. Obtain the current dam body characteristic data, preprocess the current dam body characteristic data and input it into the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

[0085] Furthermore, the historical dam body data includes dam height (m), storage capacity (m³), porosity and dam break peak flow, wherein the dam height, storage capacity and porosity are used as dam body characteristic data, and the dam break peak flow corresponding to the dam body characteristic data is used as label data.

[0086] Furthermore, the step of acquiring historical dam body data and performing data preprocessing on the historical dam body data to obtain dam body characteristic data and its corresponding label data includes:

[0087] Preset X={X1,X2,X3,…,X n} represents the dam body characteristic data;

[0088] Preset y={y1,y2,y3,…,y n} represents the label data corresponding to the dam body feature data;

[0089] The dam characteristic data are processed using a standardized formula, which is:

[0090] ;

[0091] Where, represents the standardized dam characteristic data, represents the mean value of the dam characteristic data, is the standard deviation of the dam characteristic data, Indicates the characteristic data of the dam body;

[0092] The standardized dam characteristic data is converted into uniform distribution. The conversion formula is:

[0093] ;

[0094] Where, is the transformed dam characteristic data, is the cumulative distribution function (CDF) of the dam characteristic data x, It is the inverse function of the cumulative distribution function of the dam characteristic data.

[0095] Specifically, missing values ​​are processed (deleted or filled) for the dam characteristic data and the corresponding dam-break peak flow, and outliers are identified and processed to ensure the quality of the dam characteristic data and its corresponding label data.

[0096] It should be noted that the dam body characteristic data are processed through the standardized formula to make their mean 0 and variance 1 for subsequent model training; the dam body characteristic data are quantitatively transformed. The purpose of the quantitative transformation is to convert the characteristic data into a uniform distribution or normal distribution, which helps to reduce the skewness of the data and facilitates the use of Copula modeling.

[0097] Furthermore, the step of performing data enhancement on the dam body characteristic data based on the Student-T-Copula model to obtain enhanced data includes:

[0098] The Student-T-Copula model is used to capture the relationship between variables in the dam characteristic data (variables include dam height, reservoir capacity and porosity). The Student-T-Copula model expression is:

[0099] ;

[0100] Where, is a Student's t distribution with ν degrees of freedom, is the ith marginal distribution The inverse CDF of are the parameters of the Student-T-Copula model, which are usually related to marginal distributions and correlations;

[0101] The Student-T-Copula model and random variables are sampled from the uniform distribution to generate new dam characteristic data, namely enhanced data, which is expressed as:

[0102] ;

[0103] Where, is the enhanced data, u=(u1,u2,…,u d ) is a random variable sampled from a uniform distribution, It is the inverse transformation operation of the Student-T-Copula model.

[0104] Specifically, the code is expressed as: [def student_t-copula sample(n,dim,nu=5):

[0105] Customize the student-T copula sampling function

[0106] - n: number of samples

[0107] - dim: data dimension

[0108] - nu degrees of freedom parameter

[0109] # Generate t-distribution samples, use standard t distribution and perform cumulative distribution function (CDF) conversion u=t.cdf(np.random.standard_t(nu,size=(n,dim)),df=nu)

[0110] return u】;

[0111] By using t distribution to generate samples, it can have a heavier tail than normal distribution and can better describe extreme events. Here we use t distribution to generate a set of samples (a set of dam feature data and its corresponding label data). np.random.standard_t(nu,size=(n,dim)) will generate n samples, each sample has dim dimensions, which conforms to the t distribution with nu degrees of freedom. These samples themselves are still t-distributed values, and the range may not be between [0,1], and there is no clear marginal distribution. In order to map these t-distributed samples to the interval [0,1], we use the cumulative distribution function (CDF); Simply put, the role of CDF conversion is to map each t-distributed sample point to between [0,1], so that the generated samples conform to a uniform distribution;

[0112] Furthermore, the use of Student-T-Copula can effectively capture the tail dependence of the data and more accurately simulate the relationship between variables in extreme cases.

[0113] Furthermore, to improve the diversity of augmented data and the quality of boundary samples, a gradient perturbation algorithm based on model training feedback is introduced based on the Student-T-Copula augmented samples. The specific steps are as follows:

[0114] First, use the base model f trained with the original training data 0， The model uses random forest to estimate pseudo labels for augmented samples:

[0115] ;

[0116] Where x i is the enhanced sample set The i-th enhanced sample in ;

[0117] The currently trained regression model is set to f(⋅) to effectively fit the nonlinear mapping relationship between the dam feature data and its labels. The model is based on support vector regression (SVR) and combined with particle swarm optimization (PSO) algorithm for hyperparameter tuning.

[0118] according to The sample prediction result obtained with f, defines the loss function as the mean square error between the sample and its estimated label:

[0119] ;

[0120] Where, is the loss function, It represents the pseudo label estimation of the enhanced sample by the basic regression model f0 trained with the original training data; f is the regression model currently being trained, and the fitting target is the above pseudo label;

[0121] According to the loss function, the gradient direction of the calculated sample in the input space is obtained:

[0122] ;

[0123] Where, is the loss function for sample x i The gradient vector of ; this gradient represents the sensitive direction of the loss function along the feature vector, which is used to guide the sample to perturb the higher error area to improve the model's learning ability for "boundary samples" or "easy to confuse samples"; Indicates the loss function in sample x i The gradient on

[0124] Combined with the gradient vector, the perturbation sample is generated. The perturbation method is as follows:

[0125] ;

[0126] Where, is the i-th disturbance enhancement sample, η is the disturbance step length, Prevent division by zero for small constants; is the Euclidean norm (L2 norm) of the gradient, used to normalize the direction;

[0127] Add the perturbed samples to the enhanced data to form a perturbation-enhanced sample set:

[0128] ;

[0129] Where, Enhance the sample set for perturbation.

[0130] Furthermore, when generating enhanced data X aug and perturbation-enhanced samples X gradAfter that, perform the inverse transformation operation and perform label prediction; the specific steps include:

[0131] The enhanced data is inversely transformed to make it conform to the original feature space; the specific inverse transformation formula is:

[0132] ;

[0133] Where, It is the value after matching the enhanced data back to the original distribution through CDF, that is, the result after "inverse transformation"; is the enhanced feature data X aug One of the sample components comes from the pseudo sample generated by the Student-T-Copula model; is the cumulative distribution function (CDF) value of the transformed enhanced data; It is the inverse function of the CDF of the dam characteristic data.

[0134] The purpose of the inverse transform is to map the enhanced samples from the standardized or uniformly distributed space back to the feature space of the original data, so that the generated samples have the same physical meaning and distribution as the original data;

[0135] Use the trained regression model to predict the labels of the augmented data. The expression is:

[0136] ;

[0137] ;

[0138] Where f is the regression model trained by augmented samples and pseudo labels;

[0139] Construct the final enhanced dataset and merge the original enhanced samples with the gradient perturbation enhanced samples to form the final enhanced dataset:

[0140] ;

[0141] ;

[0142] Where, Original augmented samples generated for Student-T-Copula; Gradient perturbation enhancement samples generated after introducing the gradient perturbation algorithm based on model training feedback; It is the combination of the original enhanced sample and the gradient perturbation enhanced sample; is the set of predicted labels of the two parts of enhanced samples mentioned above (i.e., the set of predicted labels of the two parts of enhanced samples, the original enhanced samples and the gradient perturbation enhanced samples);

[0143] Finally, we get the enhanced dataset (X total ,y total ) for subsequent model training and testing;

[0144] All augmented data and their corresponding label data constitute an augmented dataset;

[0145] Among them, the training process of the regression model is: taking the dam body feature data as input feature data, taking the label data corresponding to the dam body feature data as output data, and using the input feature data and output data to train the regression model, that is, obtaining the trained regression model.

[0146] Specifically, the enhanced data is evenly distributed, so we need to restore it to the distribution of the dam feature data. The code is expressed as: [#Generate enhanced data of the same size as the dam feature data

[0147] generated_X_transformed=student_t_copula_sample(200,X.shape[1],nu=5)

[0148] #3Restore the generated data to the original distribution

[0149] generated_X=quantile_transformer.inverse_transform(generated_X_transformed)];

[0150] Through the above-mentioned enhanced data generation process, the required sample size can be generated, and the dimension of each sample is ensured to be consistent with the dam body feature data. At the same time, the degree of freedom parameter nu (or v) is set to 5; in the t distribution, the smaller the degrees of freedom, the heavier the tail, and the higher the probability of extreme values; the larger the degrees of freedom, the closer the distribution is to the standard normal distribution, and the tail becomes lighter. This parameter can be adjusted according to the generation effect until a satisfactory result is achieved; the generated data generated_X_transformed is uniformly distributed, while the distribution of the dam body feature data X is not uniform; therefore, we use the inverse_transform method to convert these uniformly distributed data back to a distribution form similar to the dam body feature data; the inverse transformation of the quantization transformation restores the uniformly distributed data to the original distribution, thereby ensuring that the generated enhanced data is similar to the dam body feature data in statistical characteristics, while retaining the dependency relationship of the Copula simulation. This step uses the QuantileTransformer trained in step S1 to implement the inverse transformation.

[0151] Furthermore, the step of forming a new dataset from the enhanced dataset, the dam feature data, and their corresponding label data, and using the new dataset to construct and train an XG-Boost regression model to obtain the trained XG-Boost regression model includes:

[0152] The enhanced dataset, dam feature data and their corresponding label data are combined to form a new dataset, which is then divided into a training set and a test set in a ratio of 8:2.

[0153] Define an XG-Boost regression model, set a base learner (such as a decision tree), and then use the training set to train the XG-Boost regression model to capture nonlinear relationships in the data;

[0154] Hyperparameter tuning of the XG-Boost regression model was performed using grid search and cross-validation;

[0155] Re-predict the output data in the new dataset using the test set, and evaluate the XG-Boost regression model using RMSE and R² until the RMSE and R² of the XG-Boost regression model reach the preset thresholds. (The performance of the XG-Boost regression model can be further evaluated by visualizing the prediction results (for example, comparing the predicted values ​​with the actual values). If the results are not satisfactory, try adjusting the parameter structure of the XG-Boost regression model until satisfactory results are achieved.)

[0156] ;

[0157] Where n is the number of samples, y i is the true value, is the predicted value;

[0158] ;

[0159] Where n is the number of samples, is the true value, is the predicted value, is the average of the true values.

[0160] Specifically, the code is as follows:

[0161] 【param_grid={

[0162] 'learning_rate':[0.01, 0.05, 0.1, 0.2],

[0163] 'max_depth': [3, 5, 7],

[0164] 'subsample':[0.7, 0.8, 0.9],

[0165] 'colsample_bytree':[0.7, 0.8, 0.9],

[0166] 'n_estimators':[100, 200, 300]

[0167] grid_search = GridsearchCV(estimator=xgb_model, param_grid=paramgrid,cv=5,scoring='neg_mean_squared_error',verbose=1)

[0168] grid_search.fit(generated_X_train,generated_y_train)

[0169] # Print best parameters from grid search

[0170] print(f"Best hyperparameters:{grid_search.best_params_}"].

[0171] It should be noted that (grid search cross validation) GridSearchCV is a hyperparameter tuning tool provided by scikit-learn. It searches for the best parameter combination by traversing a given parameter grid (param_grid). The search parameters include learning rate (learning_rate), the maximum depth of the decision tree (max_depth), the proportion of the data subset used to train each tree (subsample), etc. The setting of each parameter plays a certain role in the model performance. For example, max_depth determines the complexity of each tree. A larger depth can capture more complex data patterns, but may lead to overfitting; subsample, the proportion of the data subset used to train each tree, is usually set between 0.7 and 1. A lower subsample value can help reduce overfitting.

[0172] cv=5 means using 5-fold cross-validation for evaluation, that is, dividing the training data into 5 parts, using 4 parts of the data for training each time, and the remaining part of the data for validation, and repeating it 5 times. The purpose of cross-validation is to ensure that the hyperparameters we choose can improve the performance of the model on unknown data, rather than just adapting to the data of the training set. By combining grid search and cross-validation, we can find the best hyperparameters, and then use the negative mean squared error (MSE) as the scoring indicator (scoring='neg_mean_squared_error). In regression problems, the lower the MSE value, the better the model performance. Through the above setting method, the training set part of the enhanced dataset is input again for grid search and cross-validation to obtain a better parameter combination, and then use this group of parameters to refit the XG-Boost regression model.

[0173] Furthermore, the step of obtaining current dam body characteristic data, preprocessing the current dam body characteristic data and inputting the data into the trained XG-Boost regression model to obtain the predicted current dam break peak flow includes:

[0174] Acquire current dam body characteristic data, and preprocess the current dam body characteristic data, wherein the preprocessing includes standardization processing (the same as the preprocessing process in S1);

[0175] Input the pre-processed current dam feature data into the trained XG-Boost regression model.

[0176] The XG-Boost regression model outputs the predicted current dam break peak flow.

[0177] Specifically, in the present invention, simulated dam break experimental data is used for research. The dam body characteristic data (historical dam body data) includes 40 sets of experimental records, each set of data has the following characteristics:

[0178] Dam height: the height of the dam, in meters;

[0179] Reservoir capacity: the volume of water that a dam can store, in cubic meters;

[0180] Porosity: The percentage of the volume of voids in soil or rock to the overall volume;

[0181] Dam break peak flow: the maximum flow rate of water at the breach during the dam break process, measured in cubic meters per second.

[0182] Data preprocessing: addressing missing values ​​(deleting or filling); identifying and addressing outliers to ensure data quality. To increase the data volume, we used the Copula method for data augmentation, generating an additional 200 data sets, ultimately resulting in a total of 240 sets. The Copula method preserves correlations in the dam feature data while generating more samples to improve the model's generalization capabilities. After data augmentation, the dataset was divided into training and test sets. Typically, 80% of the data is used for training and 20% for testing.

[0183] Feature selection and model training:

[0184] Based on the characteristic variables in the dataset, dam height, reservoir capacity, and porosity were selected as model input features, and dam-break peak flow was used as the target variable (label data). A model was constructed to predict dam-break peak flow. During the training process, the regression model was first trained using the XGBoost algorithm. XGBoost is an ensemble learning method based on decision trees that is efficient, accurate, and scalable, and can effectively handle multiple feature types and nonlinear relationships.

[0185] During the training process, grid search and cross-validation methods were used to select the optimal model hyperparameters to ensure the model performs optimally on the training set. At the same time, evaluation metrics such as mean squared error (RMSE) and R² coefficient were used to assess the model's predictive performance.

[0186] Dam failure prediction:

[0187] The trained XGBoost regression model is applied to the prediction of the test set data. By inputting the feature data in the test set, the model outputs the peak flow rate that may occur during the dam break process. The results are as follows: Figure 2 As shown in the figure, the XG-Boost regression model combined with augmented data performs well on both the training set and the test set, with an RMSE of 0.0045 and an R² score of 0.993 on the training set, and an RMSE of 0.0063 and an R² score of 0.981 on the test set, demonstrating excellent prediction accuracy and good generalization ability.

[0188] Furthermore, the above-mentioned simulated dam break experiment effectively generated more dam break samples by introducing Copula data augmentation technology. In particular, when the frequency of dam break events is low, the generated synthetic samples are consistent with the dam body characteristic data by maintaining the dependency structure between samples, thereby enhancing the model's ability to identify dam break events. By combining data augmentation with the XG-Boost regression model, and using grid search and cross-validation parameter optimization methods, the prediction results are made more accurate and overfitting is reduced. The root mean square error (RMSE) of the XG-Boost regression model is 0.0594 when using dam body characteristic data for prediction, while the RMSE after using enhanced data is reduced to 0.0063, and the accuracy is improved by about 89.4%. The XG-Boost algorithm is computationally efficient and suitable for the rapid processing and prediction of real-time monitoring data.

[0189] In this embodiment, by introducing the Student-T-Copula model to enhance the dam body characteristic data, the correlation between the variables in the dam body characteristic data can be effectively captured, and a richer data set can be generated. The enhancement process can make up for the lack of actual historical dam body data and significantly improve the generalization ability and prediction accuracy of the model; by adopting the XG-Boost algorithm to construct a regression model, and taking advantage of its advantages in processing nonlinear relationships and high-dimensional data, the complex mapping relationship between the dam body characteristic data and the dam break peak flow can be accurately captured, thereby achieving a more accurate flood peak flow prediction; the data preprocessing steps include standardization and distribution conversion, which can effectively reduce the impact of the dimensional differences between the characteristic variables on the XG-Boost regression model training, and at the same time, by Converting the data into a uniform distribution improves the uniformity of data distribution and enhances the stability of the XG-Boost regression model. By constructing a new dataset, the applicability of dam-break flood peak prediction can be further improved, providing accurate prediction support for different types of earth-rock dams. Through grid search and cross-validation, the hyperparameters of the XG-Boost regression model are tuned to ensure that the model reaches the preset thresholds in performance indicators such as RMSE and R², avoiding irrational parameter settings and enhancing the reliability of the prediction results. By preprocessing the current dam body characteristic data and using the trained XG-Boost regression model for prediction, it is possible to quickly respond to changes in the current state of the earth-rock dam and achieve real-time dam-break flood peak flow prediction, providing an efficient and real-time decision-making basis for flood disaster prevention and control.

[0190] Example 2

[0191] See also Figure 3 As shown, the earth-rock dam burst flood peak prediction system based on Copula data enhancement and XG-Boost algorithm described in this embodiment includes:

[0192] A preprocessing module acquires historical dam body data and performs data preprocessing on the historical dam body data to obtain dam body feature data and its corresponding label data;

[0193] The data enhancement module performs data enhancement on the dam feature data based on the Student-T-Copula model to obtain the initial enhanced data samples. On this basis, the enhanced samples are further fine-tuned using a perturbation mechanism based on gradient optimization to achieve dual enhancement of sample feature diversity and boundary discriminability. The enhanced data is then subjected to inverse transformation operations and label data estimation to obtain the final enhanced dataset.

[0194] The model training module combines the enhanced dataset, dam feature data, and their corresponding label data into a new dataset, and uses the new dataset to build and train the XG-Boost regression model, thereby obtaining the trained XG-Boost regression model.

[0195] The prediction module obtains the current dam body characteristic data, preprocesses the current dam body characteristic data and inputs the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

[0196] In this embodiment, the combination of data augmentation and the XG-Boost regression model can effectively address the problems of insufficient accuracy and insufficient data in existing methods for predicting earth-rock dam burst peaks. Furthermore, based on the improved Copula data augmentation and XG-Boost algorithm, the present invention achieves the following technical effects and advantages:

[0197] 1. By introducing the Student-T-Copula model to construct a multivariate joint distribution, a large number of enhanced samples with consistent distributions with the original samples are synthesized. On this basis, a perturbation mechanism based on gradient optimization is further adopted to fine-tune the enhanced samples, achieving a dual enhancement of sample feature diversity and boundary discriminability. It can effectively capture the correlation between variables in the dam characteristic data and generate a richer data set. The enhancement process can make up for the lack of actual historical dam data and significantly improve the generalization ability and prediction accuracy of the model. By using the XG-Boost algorithm to construct a regression model, taking advantage of its advantages in processing nonlinear relationships and high-dimensional data, it can accurately capture the complex mapping relationship between dam characteristic data and dam break peak flow, thereby achieving more accurate flood peak flow prediction.

[0198] 2. Through data preprocessing steps including standardization and distribution conversion, the impact of dimensional differences between feature variables on the training of the XG-Boost regression model can be effectively reduced. At the same time, by converting the data into a uniform distribution, the distribution uniformity of the data is improved, and the stability of the XG-Boost regression model is enhanced. By preprocessing the current dam body characteristic data and using the trained XG-Boost regression model for prediction, it is possible to quickly respond to changes in the current state of the earth-rock dam and realize real-time dam break peak flow prediction, providing an efficient and real-time decision-making basis for flood disaster prevention and control.

[0199] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0200] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only one type. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0201] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

[0202] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The earth-rock dam burst peak prediction method based on Copula data enhancement and XG-Boost algorithm is characterized by: The following steps are involved: S1. Obtain historical dam body data and perform data preprocessing on the historical dam body data, thereby obtaining dam body feature data and its corresponding label data; the specific steps include: Preset X={ X1,X2,X3,…,X n } represents the dam body characteristic data; Preset y={y1,y2,y3,…,y n } represents the label data corresponding to the dam body feature data; The dam characteristic data are processed using a standardized formula, which is: ； Where, represents the standardized dam characteristic data, represents the mean value of the dam characteristic data, is the standard deviation of the dam characteristic data, Indicates the characteristic data of the dam body; The standardized dam characteristic data is converted into uniform distribution. The conversion formula is: ； Where, is the transformed dam characteristic data, is the cumulative distribution function CDF of the dam characteristic data x, It is the inverse function of the cumulative distribution function of the dam characteristic data; The historical dam body data includes dam height, storage capacity, porosity and dam-break peak flow, wherein the dam height, storage capacity and porosity are used as dam body characteristic data, and the dam-break peak flow corresponding to the dam body characteristic data is used as label data; S2. Based on the Student-T-Copula model, the dam feature data is enhanced and an enhanced dataset is generated: The step of performing data enhancement on the dam body characteristic data based on the Student-T-Copula model to obtain enhanced data specifically includes: The Student-T-Copula model is used to capture the relationship between variables in the dam characteristic data, and its expression is: ； Where, is a Student's t distribution with ν degrees of freedom, is the ith marginal distribution The inverse CDF of are the parameters of the Student-T-Copula model; The Student-T-Copula model and random variables are sampled from the uniform distribution to generate new dam characteristic data, namely enhanced data, which is expressed as: ； Where, is the augmented data, u=(u1,u2,…,u d ) is a random variable sampled from a uniform distribution, It is the inverse transformation operation of the Student-T-Copula model; Specifically, the code for customizing the student-T copula sampling function is expressed as: [def student_t_copula_sample(n,dim,nu=5): Customize the student-T copula sampling function - n: number of samples - dim: data dimension - nu: degree of freedom parameter # Generate t-distribution samples, use standard t-distribution and perform cumulative distribution function (CDF) conversion u=t.cdf(np.random.standard_t(nu,size=(n,dim)),df=nu) return u】; S3. To improve the diversity of augmented data and the quality of boundary samples, a gradient perturbation algorithm based on model training feedback is introduced based on the Student-T-Copula augmented samples. The steps are as follows: First, use the basic regression model trained with the original training data , the model uses random forest to estimate pseudo labels for augmented samples: ； Where, is used Pseudo labels predicted by the model; is the enhanced sample set The i-th enhanced sample in ; Set the currently trained regression model f to effectively fit the nonlinear mapping relationship between the dam feature data and its labels. The model is based on support vector regression (SVR) and combined with particle swarm optimization (PSO) algorithm for hyperparameter tuning. according to The sample prediction result obtained with f, defines the loss function as the mean square error between the sample and its estimated label: ； Where, is the loss function, Represents the basic regression model trained using the original training data Pseudo-label estimation for augmented samples; f is the regression model currently being trained, and the fitting target is the pseudo-label above; According to the loss function, the gradient direction of the calculated sample in the input space is obtained: ； Where, is the loss function for sample x i The gradient vector of ; this gradient represents the sensitive direction of the loss function along the feature vector, which is used to guide the sample to be disturbed to a higher error area to improve the model's learning ability for boundary samples or easily confused samples; Indicates the loss function in sample x i The gradient on Combined with the gradient vector, the perturbation sample is generated. The perturbation method is as follows: ； Where, is the i-th disturbance enhancement sample, η is the disturbance step length, Prevent division by zero for small constants; is the Euclidean norm of the gradient, used to normalize the direction; Add the perturbed samples to the enhanced data to form a perturbation-enhanced sample set: ； Where, Enhance the sample set for perturbation; S4. Generate enhanced data X aug and perturbation-enhanced samples X grad After that, perform the inverse transformation and label prediction; specifically including: The enhanced data is inversely transformed to make it conform to the original feature space; the specific inverse transformation formula is: ； Where, It is the value after matching the enhanced data back to the original distribution through CDF, that is, the result after inverse transformation; is the enhanced feature data X aug One of the sample components comes from the pseudo sample generated by the Student-T-Copula model; represents the current estimated CDF of the augmented data; It is the inverse function of the CDF of the original dam characteristic data; The purpose of the inverse transform is to map the enhanced samples from the standardized or uniformly distributed space back to the feature space of the original data, so that the generated samples have the same physical meaning and distribution as the original data; Use the trained regression model to predict the labels of the augmented data. The expression is: ； ； Where f is the regression model trained by augmented samples and pseudo labels; Construct the final enhanced dataset and merge the original enhanced samples with the gradient perturbation enhanced samples to form the final enhanced dataset: ； ； Where, Original augmented samples generated for Student-T-Copula; Gradient perturbation enhancement samples generated after introducing the gradient perturbation algorithm based on model training feedback; It is the combination of the original enhanced sample and the gradient perturbation enhanced sample; The collection of predicted labels for the two parts of enhanced samples: the original enhanced samples and the gradient perturbation enhanced samples; finally, the enhanced dataset (X total ,y total ) for subsequent model training and testing; The training process of the regression model is as follows: taking the dam body feature data as input feature data, taking the label data corresponding to the dam body feature data as output data, and using the input feature data and output data to train the regression model, thus obtaining a trained regression model; S5. The enhanced dataset, the dam feature data, and their corresponding label data are combined to form a new dataset, and the XG-Boost regression model is constructed and trained using the new dataset to obtain a trained XG-Boost regression model. The specific steps include: combining the final enhanced dataset, the dam feature data, and their corresponding label data to form a new dataset, and dividing the new dataset into a training set and a test set in a ratio of 8:

2. Define an XG-Boost regression model, set a base learner, and then use the training set to train the XG-Boost regression model to capture nonlinear relationships in the data; Hyperparameter tuning of the XG-Boost regression model was performed using grid search and cross-validation; Re-predict the output data in the new dataset using the test set, and evaluate the XG-Boost regression model using RMSE and R² until the RMSE and R² of the XG-Boost regression model reach the preset thresholds. S6. Obtain the current dam body characteristic data, preprocess the current dam body characteristic data and input it into the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

2. The earth-rock dam burst flood peak prediction method based on Copula data enhancement and XG-Boost algorithm according to claim 1 is characterized in that: The step of obtaining current dam body characteristic data, preprocessing the current dam body characteristic data and inputting the data into the trained XG-Boost regression model to obtain the predicted current dam break peak flow includes: Obtaining current dam body characteristic data and preprocessing the current dam body characteristic data; Input the pre-processed current dam feature data into the trained XG-Boost regression model. The XG-Boost regression model outputs the predicted current dam break peak flow.

3. A system for predicting earth-rock dam burst flood peaks based on Copula data enhancement and XG-Boost algorithm, applied to the method for predicting earth-rock dam burst flood peaks based on Copula data enhancement and XG-Boost algorithm described in any one of claims 1 to 2, characterized in that: include: A preprocessing module acquires historical dam body data and performs data preprocessing on the historical dam body data to obtain dam body feature data and its corresponding label data; The data enhancement module performs data enhancement on the dam feature data based on the Student-T-Copula model to obtain the initial enhanced data samples. On this basis, the enhanced samples are further fine-tuned using a perturbation mechanism based on gradient optimization to achieve dual enhancement of sample feature diversity and boundary discriminability. The enhanced data is then subjected to inverse transformation operations and label data estimation to obtain the final enhanced dataset. The model training module combines the enhanced dataset, dam feature data, and their corresponding label data into a new dataset, and uses the new dataset to build and train the XG-Boost regression model, thereby obtaining the trained XG-Boost regression model. The prediction module obtains the current dam body characteristic data, preprocesses the current dam body characteristic data and inputs the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

Images