Earth and rockfill dam outburst flood peak prediction method and system based on Copula data enhancement and XG-Boost algorithm

Abstract

The invention belongs to the technical field of machine learning, and particularly discloses an earth and rockfill dam outburst flood peak prediction method and system based on Copula data enhancement and an XG-Boost algorithm. The method comprises the following steps: S1, acquiring historical dam body data and performing data preprocessing on the historical dam body data; s2, performing data enhancement on the dam body feature data on the basis of a Student-T-Copula model, and generating an enhanced data set Xaug; s3, introducing a gradient perturbation algorithm based on model training feedback; s4, after the enhanced data Xaug and the disturbance enhanced sample Xgrad are generated, inverse transformation operation and label prediction are carried out, and a final complete enhanced data set is obtained; s5, forming a new data set by the enhanced data set, the dam body feature data and the corresponding label data, and constructing and training a regression model; and S6, inputting the current dam body characteristic data to obtain the predicted current dam break peak flow. Through the combination of data enhancement and the regression model, the problems of insufficient earth and rockfill dam burst flood peak prediction precision and insufficient data in the existing method are effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and more specifically, to a method and system for predicting earth-rock dam burst flood peaks based on Copula data enhancement and XG-Boost algorithm. Background Art

[0002] Dam break (dam failure) is one of the major disasters in water conservancy projects, usually accompanied by casualties, property losses and ecological damage. The prediction and prevention of dam break has always been a research focus in water conservancy projects. Predicting the occurrence of dam break through monitoring and data analysis of the dam body and its surrounding environment, and providing early warning and emergency response time for relevant departments are the key to ensuring public safety and reducing disaster losses. However, the occurrence of dam break is often the result of the interaction of multiple factors, including but not limited to dam structure, environmental changes (such as precipitation, temperature changes), geological conditions, external pressure, etc. The relationship between these factors is usually very complex, and there may be significant time variability and spatial heterogeneity, which makes the prediction of dam break very challenging.

[0003] Existing dam break prediction technologies mainly include physical models, statistical methods and machine learning methods. Physical models: evaluate safety by simulating the mechanical behavior of the dam body. Although the accuracy is high, it requires a large amount of input data and high-precision parameters, the calculation is complex, and real-time prediction is difficult; statistical methods: reveal the relationship between dam break and related factors by analyzing historical data, which is simple and easy to use, but it is difficult to capture complex nonlinear and interactive effects, and the accuracy is low; machine learning methods: use big data for prediction, have strong adaptive capabilities, but rely on data quality and diversity, and lack accuracy and generalization capabilities on complex problems; in addition, dam break prediction faces data imbalance and high dimensionality problems. There are relatively few dam break events, which leads to training data bias and easily affects prediction accuracy. The nonlinear relationship between multiple variables makes high-dimensional data processing more difficult. Most existing solutions use data enhancement methods to expand the data sample size. However, traditional data enhancement methods usually rely on simple transformation operations such as rotation, translation and scaling. Although these methods can expand the data set to a certain extent, the data they generate often lacks sufficient exploration of the intrinsic structure and complex patterns of the data. Traditional methods have the following shortcomings: lack of targeted optimization, limited diversity of augmented data, risk of overfitting, and low generation efficiency.

[0004] Furthermore, the above method has certain limitations in data enhancement and prediction of dam breach status, and it is difficult to provide accurate prediction results.

[0005] In view of this, the present invention proposes a method and system for predicting earth-rock dam burst flood peak based on improved Copula data enhancement and XG-Boost algorithm to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for predicting earth-rock dam burst flood peak based on Copula data enhancement and XG-Boost algorithm, comprising the following steps:

[0007] S1. Obtain historical dam body data and perform data preprocessing on the historical dam body data, thereby obtaining dam body characteristic data and its corresponding label data; the specific steps include:

[0008] Preset X={X 1 ,X 2 ,X 3 ,…,X n} represents the dam characteristic data;

[0009] Preset y={y 1 ,y 2 ,y 3 ,…,y n} represents the label data corresponding to the dam body characteristic data;

[0010] The dam characteristic data is processed using a standardized formula, which is:

[0011] ;

[0012] In the formula, represents the standardized dam characteristic data, represents the mean value of the dam characteristic data, is the standard deviation of the dam characteristic data, Indicates the characteristic data of the dam body;

[0013] The standardized dam characteristic data is converted into uniform distribution. The conversion formula is:

[0014] ;

[0015] In the formula, is the transformed dam characteristic data, is the cumulative distribution function CDF of the dam characteristic data x, It is the inverse function of the cumulative distribution function of the dam characteristic data;

[0016] S2. Based on the Student-T-Copula model, the dam feature data is enhanced and an enhanced data set is generated:

[0017] The step of performing data enhancement on the dam body characteristic data based on the Student-T-Copula model to obtain enhanced data specifically includes:

[0018] The Student-T-Copula model is used to capture the relationship between variables in the dam characteristic data, and its expression is:

[0019] ;

[0020] In the formula, is a Student's t distribution with ν degrees of freedom, is the ith marginal distribution The inverse CDF of are the parameters of the Student-T-Copula model;

[0021] Through the Student-T-Copula model and random variables sampled from uniform distribution, new dam characteristic data, namely enhanced data, are generated, and its expression is:

[0022] ;

[0023] In the formula, is the enhanced data, u=(u 1 ,u 2 ,…,u d ) is a random variable sampled from a uniform distribution, It is the inverse transformation operation of the Student-T-Copula model;

[0024] S3. In order to improve the diversity of enhanced data and the quality of boundary samples, a gradient perturbation algorithm based on model training feedback is introduced on the basis of Student-T-Copula enhanced samples. The steps are as follows:

[0025] First, use the base model f trained with the original training data 0， The model uses random forest to estimate pseudo labels for augmented samples:

[0026] ;

[0027] In the formula, x i is the enhanced sample set The i-th enhanced sample in ;

[0028] The currently trained regression model f is set to effectively fit the nonlinear mapping relationship between the dam feature data and its labels. The model is based on support vector regression (SVR) and combined with particle swarm optimization (PSO) algorithm for hyperparameter tuning;

[0029] according to The sample prediction result obtained with f, defines the loss function as the mean square error between the sample and its estimated label:

[0030] ;

[0031] In the formula, is the loss function, Represents the basic regression model trained using the original training data Pseudo-label estimation of enhanced samples; f is the regression model currently being trained, and the fitting target is the above pseudo-label;

[0032] According to the loss function, the gradient direction of the calculated sample in the input space is obtained:

[0033] ;

[0034] In the formula, is the loss function for sample x i The gradient vector of ; the gradient represents the sensitive direction of the loss function with the feature vector, which is used to guide the sample to perturb to a higher error area to improve the model's learning ability for boundary samples or easily mixed samples; Indicates that the loss function is in sample x i The gradient on

[0035] Combined with the gradient vector, the perturbation sample is generated. The perturbation method is as follows:

[0036] ;

[0037] In the formula, is the i-th disturbance enhancement sample, η is the disturbance step length, Prevent division by zero for tiny constants; is the Euclidean norm (L2 norm) of the gradient, used to normalize the direction;

[0038] Add the perturbed samples to the enhanced data to form a perturbation enhanced sample set:

[0039] ;

[0040] In the formula, Enhance the sample set for perturbation;

[0041] S4. When generating enhanced data X aug and the perturbation-enhanced sample X grad After that, perform the inverse transformation operation and make label prediction; specifically including:

[0042] The enhanced data is inversely transformed to make it conform to the original feature space; the specific inverse transformation formula is:

[0043] ;

[0044] In the formula, It is the value after matching the enhanced data back to the original distribution through CDF, that is, the result after inverse transformation; is the enhanced feature data X aug One of the sample components comes from the pseudo sample generated by the Student-T-Copula model; represents the current estimated CDF of the augmented data; It is the inverse function of the CDF of the original dam characteristic data;

[0045] The purpose of the inverse transform is to map the enhanced samples from the standardized or uniformly distributed space back to the feature space of the original data, so that the generated samples have the same physical meaning and distribution as the original data;

[0046] Use the trained regression model to predict the labels of the augmented data. The expression is:

[0047] ;

[0048] ;

[0049] In the formula, f is the regression model obtained by training with enhanced samples and pseudo labels;

[0050] Construct the final enhanced dataset and merge the original enhanced samples with the gradient perturbation enhanced samples to form the final enhanced dataset:

[0051] ;

[0052] ;

[0053] In the formula, Original augmented samples generated for Student-T-Copula; Gradient perturbation enhanced samples generated after introducing the gradient perturbation algorithm based on model training feedback; It is the collection of original enhanced samples and gradient perturbation enhanced samples; It is the collection of the predicted labels of the two parts of enhanced samples mentioned above (i.e., the collection of the predicted labels of the two parts of enhanced samples, the original enhanced samples and the gradient perturbation enhanced samples);

[0054] Finally, we get the enhanced dataset (X total ,y total ) is used for subsequent model training and testing;

[0055] S5. The enhanced data set, the dam body feature data and their corresponding label data are used together to form a new data set, and the XG-Boost regression model is constructed and trained using the new data set to obtain the trained XG-Boost regression model. The specific steps include: the final enhanced data set, the dam body feature data and their corresponding label data are used together to form a new data set, and the new data set is divided into a training set and a test set in a ratio of 8:2;

[0056] Define an XG-Boost regression model, set a base learner, and then use the training set to train the XG-Boost regression model to capture nonlinear relationships in the data;

[0057] Hyperparameter tuning of XG-Boost regression model was performed through grid search and cross validation;

[0058] Re-predict the output data in the new data set through the test set, and evaluate the XG-Boost regression model through RMSE and R² until the RMSE and R² of the XG-Boost regression model reach the preset threshold;

[0059] S6. Obtain the current dam body characteristic data, pre-process the current dam body characteristic data and input the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

[0060] Preferably, the historical dam body data include dam height, reservoir capacity, porosity and dam break peak flow, wherein the dam height, reservoir capacity and porosity are used as dam body characteristic data, and the dam break peak flow corresponding to the dam body characteristic data is used as label data.

[0061] Preferably, the step of obtaining the current dam body characteristic data, preprocessing the current dam body characteristic data and inputting the data into the trained XG-Boost regression model, i.e. obtaining the predicted current dam break peak flow, comprises:

[0062] Acquire current dam body characteristic data, and pre-process the current dam body characteristic data;

[0063] Input the preprocessed current dam feature data into the trained XG-Boost regression model.

[0064] The XG-Boost regression model outputs the predicted current dam break peak flow.

[0065] The present invention also provides a system for predicting earth-rock dam burst flood peaks based on Copula data enhancement and XG-Boost algorithm, which is applied to any of the earth-rock dam burst flood peak prediction methods based on Copula data enhancement and XG-Boost algorithm, and comprises:

[0066] A preprocessing module, which obtains historical dam body data and performs data preprocessing on the historical dam body data, thereby obtaining dam body characteristic data and its corresponding label data;

[0067] The data enhancement module performs data enhancement on the dam feature data based on the Student-T-Copula model to obtain the initial enhanced data samples. On this basis, the enhanced samples are further fine-tuned using a perturbation mechanism based on gradient optimization to achieve dual enhancement of sample feature diversity and boundary discriminability. The enhanced data are then inversely transformed and label data estimated to obtain the final enhanced data set.

[0068] The model training module combines the enhanced data set and the dam feature data and their corresponding label data into a new data set, and uses the new data set to build and train the XG-Boost regression model, thereby obtaining a trained XG-Boost regression model.

[0069] The prediction module obtains the current dam body characteristic data, preprocesses the current dam body characteristic data and inputs the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

[0070] The technical effects and advantages of the earth-rock dam burst flood peak prediction method and system based on improved Copula data enhancement and XG-Boost algorithm of the present invention are as follows:

[0071] 1. By introducing the Student-T-Copula model to construct a multivariate joint distribution, a large number of enhanced samples consistent with the original sample distribution are synthesized. On this basis, the perturbation mechanism based on gradient optimization is further used to fine-tune the enhanced samples to achieve dual enhancement of sample feature diversity and boundary discriminability. It can effectively capture the correlation between variables in the dam body characteristic data and generate a richer data set. The enhancement process can make up for the lack of actual historical dam body data and significantly improve the generalization ability and prediction accuracy of the model. By using the XG-Boost algorithm to construct a regression model, taking advantage of its advantages in processing nonlinear relationships and high-dimensional data, it can accurately capture the complex mapping relationship between dam body characteristic data and dam break peak flow, thereby achieving more accurate flood peak flow prediction;

[0072] 2. Through data preprocessing steps including standardization and distribution conversion, the impact of dimensional differences between feature variables on the training of the XG-Boost regression model can be effectively reduced. At the same time, by converting the data into a uniform distribution, the distribution uniformity of the data is improved, and the stability of the XG-Boost regression model is enhanced. By preprocessing the current dam body characteristic data and using the trained XG-Boost regression model for prediction, it is possible to quickly respond to changes in the current state of the earth-rock dam and realize real-time dam break peak flow prediction, providing an efficient and real-time decision-making basis for flood disaster prevention and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is a flow chart of a method for predicting earth-rock dam burst flood peak based on Copula data enhancement and XG-Boost algorithm of the present invention;

[0074] Figure 2 This is a diagram showing the effect of the peak flow prediction training set and test set in an embodiment of the present invention;

[0075] Figure 3 It is a structural schematic diagram of the earth-rock dam burst flood peak prediction system based on Copula data enhancement and XG-Boost algorithm of the present invention. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] Example 1

[0078] See also Figure 1 and Figure 2 As shown, the earth-rock dam burst flood peak prediction method based on Copula data enhancement and XG-Boost algorithm described in this embodiment includes:

[0079] S1. Acquire historical dam body data and perform data preprocessing on the historical dam body data, thereby obtaining dam body characteristic data and its corresponding label data;

[0080] S2. Based on the Student-T-Copula model, the dam feature data is enhanced to obtain enhanced data, and the enhanced data is inversely transformed and the label data is predicted to obtain an enhanced data set;

[0081] S3. In order to improve the diversity of enhanced data and the quality of boundary samples, a gradient perturbation algorithm based on model training feedback is introduced on the basis of Student-T-Copula enhanced samples;

[0082] S4. When generating enhanced data X aug and the perturbation-enhanced sample X grad After that, the inverse transformation operation is performed and the label prediction is performed, and finally the enhanced data set (X total ,y total ) is used for subsequent model training and testing;

[0083] S5. The final enhanced data set, the dam feature data and its corresponding label data are combined to form a new data set, and the XG-Boost regression model is constructed and trained using the new data set to obtain the trained XG-Boost regression model;

[0084] S6. Obtain the current dam body characteristic data, pre-process the current dam body characteristic data and input the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

[0085] Furthermore, the historical dam body data includes dam height (m), reservoir capacity (m³), porosity and dam break peak flow, wherein the dam height, reservoir capacity and porosity are used as dam body characteristic data, and the dam break peak flow corresponding to the dam body characteristic data is used as label data.

[0086] Furthermore, the step of acquiring historical dam body data and preprocessing the historical dam body data to obtain dam body characteristic data and corresponding label data includes:

[0087] Preset X={X 1 ,X 2 ,X 3 ,…,X n} represents the dam characteristic data;

[0088] Preset y={y 1 ,y 2 ,y 3 ,…,y n} represents the label data corresponding to the dam body characteristic data;

[0089] The dam characteristic data is processed using a standardized formula, which is:

[0090] ;

[0091] In the formula, represents the standardized dam characteristic data, represents the mean value of the dam characteristic data, is the standard deviation of the dam characteristic data, Indicates the characteristic data of the dam body;

[0092] The standardized dam characteristic data is converted into uniform distribution. The conversion formula is:

[0093] ;

[0094] In the formula, is the transformed dam characteristic data, is the cumulative distribution function (CDF) of the dam characteristic data x, It is the inverse function of the cumulative distribution function of the dam characteristic data.

[0095] Specifically, the missing values ​​of the dam characteristic data and the dam break peak flow corresponding to the dam characteristic data are processed (deleted or filled), and the abnormal values ​​therein are identified and processed, thereby ensuring the quality of the dam characteristic data and its corresponding label data;

[0096] It should be noted that the dam body characteristic data is processed through the standardized formula to make its mean 0 and variance 1 for subsequent model training; the dam body characteristic data is quantified and transformed. The purpose of the quantification transformation is to convert the characteristic data into a uniform distribution or normal distribution, which helps to reduce the skewness of the data and facilitates the use of Copula modeling.

[0097] Furthermore, the step of performing data enhancement on the dam body characteristic data based on the Student-T-Copula model to obtain enhanced data includes:

[0098] The Student-T-Copula model is used to capture the relationship between variables in the dam characteristic data (variables include dam height, reservoir capacity and porosity). The Student-T-Copula model expression is:

[0099] ;

[0100] In the formula, is a Student's t distribution with ν degrees of freedom, is the ith marginal distribution The inverse CDF of are the parameters of the Student-T-Copula model, usually related to marginal distributions and correlations;

[0101] Through the Student-T-Copula model and random variables sampled from uniform distribution, new dam characteristic data, namely enhanced data, are generated, and its expression is:

[0102] ;

[0103] In the formula, is the enhanced data, u=(u 1 ,u 2 ,…,u d ) is a random variable sampled from a uniform distribution, It is the inverse transformation operation of the Student-T-Copula model.

[0104] Specifically, the code is expressed as: [def student_t-copula sample(n,dim,nu=5):

[0105] Customize the student-T copula sampling function

[0106] - n: number of samples

[0107] - dim: data dimension

[0108] - nu degrees of freedom parameter

[0109] # Generate t-distribution samples, use standard t-distribution and perform cumulative distribution function (CDF) conversion u=t.cdf(np.random.standard_t(nu,size=(n,dim)),df=nu)

[0110] return u】;

[0111] By using t distribution to generate samples, it can have a heavier tail than normal distribution and better describe extreme events. Here we use t distribution to generate a set of samples (a set of dam feature data and its corresponding label data). np.random.standard_t(nu,size=(n,dim)) will generate n samples, each with dim dimensions, which conforms to the t distribution with nu degrees of freedom. These samples themselves are still t-distributed values, and the range may not be between [0,1], and there is no clear marginal distribution. In order to map these t-distributed samples to the interval of [0,1], we use the cumulative distribution function (CDF); in short, the role of CDF conversion is to map each t-distributed sample point to between [0,1], so that the generated samples conform to a uniform distribution;

[0112] Furthermore, the use of Student-T-Copula can effectively capture the tail dependence of the data and thus more accurately simulate the relationship between variables in extreme cases.

[0113] Furthermore, in order to improve the diversity of enhanced data and the quality of boundary samples, a gradient perturbation algorithm based on model training feedback is introduced on the basis of Student-T-Copula enhanced samples. The specific steps are as follows:

[0114] First, use the base model f trained with the original training data 0， The model uses random forest to estimate pseudo labels for augmented samples:

[0115] ;

[0116] In the formula, x i is the enhanced sample set The i-th enhanced sample in ;

[0117] The currently trained regression model is set to f(⋅) to effectively fit the nonlinear mapping relationship between the dam feature data and its label. The model is based on support vector regression (SVR) and combined with particle swarm optimization (PSO) algorithm for hyperparameter tuning;

[0118] according to The sample prediction result obtained with f, defines the loss function as the mean square error between the sample and its estimated label:

[0119] ;

[0120] In the formula, is the loss function, Represents the basic regression model f trained using the original training data 0 Pseudo-label estimation of enhanced samples; f is the regression model currently being trained, and the fitting target is the above pseudo-label;

[0121] According to the loss function, the gradient direction of the calculated sample in the input space is obtained:

[0122] ;

[0123] In the formula, is the loss function for sample x i The gradient vector of ; the gradient represents the sensitive direction of the loss function with the feature vector, which is used to guide the sample to perturb to a higher error area to improve the model's learning ability for "boundary samples" or "easy to confuse samples"; Indicates that the loss function is in sample x i The gradient on

[0124] Combined with the gradient vector, the perturbation sample is generated. The perturbation method is as follows:

[0125] ;

[0126] In the formula, is the i-th disturbance enhancement sample, η is the disturbance step length, Prevent division by zero for tiny constants; is the Euclidean norm (L2 norm) of the gradient, used to normalize the direction;

[0127] Add the perturbed samples to the enhanced data to form a perturbation enhanced sample set:

[0128] ;

[0129] In the formula, Enhance the sample set for perturbation.

[0130] Furthermore, when generating enhanced data X aug and the perturbation-enhanced sample X grad After that, perform the inverse transformation operation and perform label prediction; the specific steps include:

[0131] The enhanced data is inversely transformed to make it conform to the original feature space; the specific inverse transformation formula is:

[0132] ;

[0133] In the formula, It is the value after matching the enhanced data back to the original distribution through CDF, that is, the result after "inverse transformation"; is the enhanced feature data X aug One of the sample components comes from the pseudo sample generated by the Student-T-Copula model; is the cumulative distribution function (CDF) value of the transformed enhanced data; It is the inverse function of the CDF of the dam characteristic data.

[0134] The purpose of the inverse transform is to map the enhanced samples from the standardized or uniformly distributed space back to the feature space of the original data, so that the generated samples have the same physical meaning and distribution as the original data;

[0135] Use the trained regression model to predict the labels of the augmented data. The expression is:

[0136] ;

[0137] ;

[0138] In the formula, f is the regression model obtained by training with enhanced samples and pseudo labels;

[0139] Construct the final enhanced dataset and merge the original enhanced samples with the gradient perturbation enhanced samples to form the final enhanced dataset:

[0140] ;

[0141] ;

[0142] In the formula, Original augmented samples generated for Student-T-Copula; Gradient perturbation enhanced samples generated after introducing the gradient perturbation algorithm based on model training feedback; It is the collection of original enhanced samples and gradient perturbation enhanced samples; It is the collection of the predicted labels of the two parts of enhanced samples mentioned above (i.e., the collection of the predicted labels of the two parts of enhanced samples, the original enhanced samples and the gradient perturbation enhanced samples);

[0143] Finally, we get the enhanced dataset (X total ,y total ) is used for subsequent model training and testing;

[0144] All augmented data and their corresponding label data form an augmented dataset;

[0145] Among them, the training process of the regression model is: taking the dam body feature data as input feature data, taking the label data corresponding to the dam body feature data as output data, and using the input feature data and output data to train the regression model, that is, obtaining a trained regression model.

[0146] Specifically, the enhanced data is evenly distributed, so we need to restore it to the distribution of the dam body feature data. The code is expressed as: [#Generate enhanced data of the same size as the dam body feature data

[0147] generated_X_transformed=student_t_copula_sample(200,X.shape[1],nu=5)

[0148] #3 Restore the generated data to the original distribution

[0149] generated_X=quantile_transformer.inverse_transform(generated_X_transformed)];

[0150] Through the above-mentioned enhanced data generation process, the required sample size can be generated, and the dimension of each sample is ensured to be consistent with the dam body feature data. At the same time, the degree of freedom parameter nu (or v) is set to 5; in the t distribution, the smaller the degree of freedom, the heavier the tail, and the higher the probability of extreme values; and the larger the degree of freedom, the distribution approaches the standard normal distribution, and the tail becomes lighter. This parameter can be adjusted according to the generation effect until a satisfactory result is achieved; the generated data generated_X_transformed is uniformly distributed, while the distribution of the dam body feature data X is not uniform; therefore, we use the inverse_transform method to convert these uniformly distributed data back to a distribution form similar to the dam body feature data; the inverse transformation of the quantization transformation restores the uniformly distributed data to the original distribution, thereby ensuring that the generated enhanced data is similar to the dam body feature data in statistical characteristics, while retaining the dependencies simulated by the Copula. This step uses the QuantileTransformer trained in step S1 to implement the inverse transformation.

[0151] Furthermore, the step of forming a new data set by combining the enhanced data set, the dam body feature data and their corresponding label data, and using the new data set to construct and train an XG-Boost regression model to obtain a trained XG-Boost regression model includes:

[0152] The enhanced data set, dam feature data and their corresponding label data together form a new data set, and the new data set is divided into a training set and a test set in a ratio of 8:2;

[0153] Define an XG-Boost regression model, set a base learner (such as a decision tree), and then use the training set to train the XG-Boost regression model to capture nonlinear relationships in the data;

[0154] Hyperparameter tuning of XG-Boost regression model was performed through grid search and cross validation;

[0155] Re-predict the output data in the new data set through the test set, and evaluate the XG-Boost regression model through RMSE and R² until the RMSE and R² of the XG-Boost regression model reach the preset threshold (the performance of the XG-Boost regression model is further evaluated by visualizing the prediction results (for example: a comparison chart of the predicted value and the actual value); if the effect is not ideal, try to adjust the parameter structure of the XG-Boost regression model until a satisfactory result is achieved);

[0156] ;

[0157] In the formula, n is the number of samples, y isi is the true value, is the predicted value;

[0158] ;

[0159] In the formula, n is the number of samples, is the true value, is the predicted value, is the average of the true values.

[0160] Specifically, the code is as follows:

[0161] [param_grid={

[0162] 'learning_rate':[0.01, 0.05, 0.1, 0.2],

[0163] 'max_depth': [3, 5, 7],

[0164] 'subsample':[0.7, 0.8, 0.9],

[0165] 'colsample_bytree':[0.7, 0.8, 0.9],

[0166] 'n_estimators':[100, 200, 300]

[0167] grid_search = GridsearchCV(estimator=xgb_model, param_grid=paramgrid,cv=5,scoring='neg_mean_squared_error',verbose=1)

[0168] grid_search.fit(generated_X_train,generated_y_train)

[0169] # Print best parameters from grid search

[0170] print(f"Best hyperparameters:{grid_search.best_params_}"].

[0171] It should be noted that (Grid Search Cross Validation) GridSearchCV is a hyperparameter tuning tool provided by scikit-learn. It searches for the best parameter combination by traversing a given parameter grid (param_grid). The search parameters include learning rate (learning_rate), the maximum depth of the decision tree (max_depth), the proportion of the data subset used to train each tree (subsample), etc. The setting of each parameter plays a certain role in the performance of the model. For example, max_depth determines the complexity of each tree. A larger depth can capture more complex data patterns, but may lead to overfitting; subsample, the proportion of the data subset used to train each tree, is usually set between 0.7 and 1. A lower subsample value can help reduce overfitting.

[0172] cv=5 means using 5-fold cross validation for evaluation, that is, dividing the training data into 5 parts, using 4 parts of the data for training each time, and the remaining part of the data for validation, and repeating it 5 times. The purpose of cross validation is to ensure that the hyperparameters we choose can improve the performance of the model on unknown data, rather than just adapting to the data in the training set. By combining grid search and cross validation, we can find the best hyperparameters, and then use the negative mean square error (MSE) as the scoring indicator (scoring='neg_mean_squared_error). In regression problems, the lower the MSE value, the better the model performance. Through the above setting method, the training set part of the enhanced dataset is input again for grid search and cross validation to obtain a better parameter combination, and then use this set of parameters to refit the XG-Boost regression model.

[0173] Furthermore, the step of obtaining the current dam body characteristic data, preprocessing the current dam body characteristic data and inputting the data into the trained XG-Boost regression model, i.e. obtaining the predicted current dam break peak flow, includes:

[0174] Acquire current dam body characteristic data, and preprocess the current dam body characteristic data, wherein the preprocessing includes standardization processing (the same as the preprocessing process in S1);

[0175] Input the preprocessed current dam feature data into the trained XG-Boost regression model.

[0176] The XG-Boost regression model outputs the predicted current dam break peak flow.

[0177] Specifically, in the present invention, simulated dam break experimental data is used for research, and the dam body characteristic data (historical dam body data) includes 40 groups of experimental records, each group of data contains the following characteristics:

[0178] Dam height: the height of the dam, in meters;

[0179] Reservoir capacity: the volume of water that a dam can store, in cubic meters;

[0180] Porosity: The percentage of the volume of voids in soil or rock to the overall volume;

[0181] Dam break peak flow: the maximum flow rate of water at the breach during the dam break process, expressed in cubic meters per second.

[0182] Data preprocessing: handle missing values ​​(delete or fill); identify and handle outliers to ensure data quality; in order to increase the amount of data, we used the Copula method for data enhancement, generated an additional 200 sets of data, and finally obtained a total of 240 sets; the Copula method can retain the correlation in the dam feature data and generate more samples to improve the generalization ability of the model; after data enhancement, the data set is divided into a training set and a test set. Usually, 80% of the data is used for training and 20% of the data is used for testing.

[0183] Feature selection and model training:

[0184] Based on the characteristic variables in the data set, dam height, reservoir capacity, and porosity are selected as model input features, and the dam break peak flow is used as the target variable (label data) to build a model for predicting the dam break peak flow. During the training process, the XGBoost algorithm is first used to train the regression model. XGBoost is an ensemble learning method based on decision trees, which is efficient, accurate, and scalable, and can effectively handle multiple feature types and nonlinear relationships.

[0185] During the training process, grid search and cross-validation methods were used to select the best model hyperparameters to ensure the best performance of the model on the training set. At the same time, evaluation indicators such as mean square error (RMSE) and R² coefficient were used to evaluate the prediction performance of the model.

[0186] Dam failure prediction:

[0187] The trained XGBoost regression model is applied to the prediction of the test set data. By inputting the feature data in the test set, the model outputs the peak flow rate that may occur during the dam breach process. The results are as follows: Figure 2 As shown in the figure, the XG-Boost regression model combined with enhanced data has good performance in both the training set and the test set, with an RMSE of 0.0045 and an R² score of 0.993 for the training set and an RMSE of 0.0063 and an R² score of 0.981 for the test set, showing excellent prediction accuracy and good generalization ability;

[0188] Furthermore, the above-mentioned simulated dam break experiment effectively generated more dam break samples by introducing Copula data enhancement technology. Especially when the frequency of dam break events is low, the generated synthetic samples are kept consistent with the dam body characteristic data by maintaining the dependency structure between samples, thereby enhancing the model's recognition ability of dam break events. By combining data enhancement with the XG-Boost regression model, and using grid search and cross-validation parameter optimization methods, the prediction results are more accurate and overfitting is reduced. The root mean square error (RMSE) of the XG-Boost regression model is 0.0594 when using dam body characteristic data for prediction, while the RMSE after using enhanced data is reduced to 0.0063, and the accuracy is improved by about 89.4%. The XG-Boost algorithm is computationally efficient and suitable for rapid processing and prediction of real-time monitoring data.

[0189] In this embodiment, by introducing the Student-T-Copula model to enhance the dam body characteristic data, the correlation between variables in the dam body characteristic data can be effectively captured, and a richer data set can be generated. The enhancement process can make up for the lack of actual historical dam body data and significantly improve the generalization ability and prediction accuracy of the model. By adopting the XG-Boost algorithm to build a regression model, and taking advantage of its advantages in processing nonlinear relationships and high-dimensional data, the complex mapping relationship between the dam body characteristic data and the dam break peak flow can be accurately captured, thereby achieving a more accurate flood peak flow prediction. The data preprocessing steps include standardization and distribution conversion, which can effectively reduce the impact of dimensional differences between characteristic variables on the XG-Boost regression model training. The conversion to uniform distribution improves the uniformity of data distribution and enhances the stability of the XG-Boost regression model. By constructing a new data set, the applicability of dam break flood peak prediction can be further improved, providing accurate prediction support for different types of earth-rock dams. The hyperparameters of the XG-Boost regression model are tuned through grid search and cross-validation to ensure that the model reaches the preset thresholds in performance indicators such as RMSE and R², avoiding unreasonable parameter settings and enhancing the reliability of the prediction results. By preprocessing the current dam body feature data and using the trained XG-Boost regression model for prediction, it is possible to quickly respond to changes in the current state of the earth-rock dam and achieve real-time dam break flood peak flow prediction, providing an efficient and real-time decision-making basis for flood disaster prevention and control.

[0190] Example 2

[0191] See also Figure 3 As shown, the earth-rock dam burst flood peak prediction system based on Copula data enhancement and XG-Boost algorithm described in this embodiment includes:

[0192] A preprocessing module, which obtains historical dam body data and performs data preprocessing on the historical dam body data, thereby obtaining dam body characteristic data and its corresponding label data;

[0193] The data enhancement module performs data enhancement on the dam feature data based on the Student-T-Copula model to obtain the initial enhanced data samples. On this basis, the enhanced samples are further fine-tuned using a perturbation mechanism based on gradient optimization to achieve dual enhancement of sample feature diversity and boundary discriminability. The enhanced data are then inversely transformed and label data estimated to obtain the final enhanced data set.

[0194] The model training module combines the enhanced data set and the dam feature data and their corresponding label data into a new data set, and uses the new data set to build and train the XG-Boost regression model, thereby obtaining a trained XG-Boost regression model.

[0195] The prediction module obtains the current dam body characteristic data, preprocesses the current dam body characteristic data and inputs the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

[0196] In this embodiment, by combining data enhancement and XG-Boost regression model, the problems of insufficient accuracy and insufficient data in the prediction of earth-rock dam burst peak in the existing methods can be effectively solved. At the same time, the present invention has achieved the following technical effects and advantages based on improved Copula data enhancement and XG-Boost algorithm:

[0197] 1. By introducing the Student-T-Copula model to construct a multivariate joint distribution, a large number of enhanced samples consistent with the original sample distribution are synthesized. On this basis, the perturbation mechanism based on gradient optimization is further used to fine-tune the enhanced samples to achieve dual enhancement of sample feature diversity and boundary discriminability. It can effectively capture the correlation between variables in the dam body characteristic data and generate a richer data set. The enhancement process can make up for the lack of actual historical dam body data and significantly improve the generalization ability and prediction accuracy of the model. By using the XG-Boost algorithm to construct a regression model, taking advantage of its advantages in processing nonlinear relationships and high-dimensional data, it can accurately capture the complex mapping relationship between dam body characteristic data and dam break peak flow, thereby achieving more accurate flood peak flow prediction;

[0198] 2. Through data preprocessing steps including standardization and distribution conversion, the impact of dimensional differences between feature variables on the training of the XG-Boost regression model can be effectively reduced. At the same time, by converting the data into a uniform distribution, the distribution uniformity of the data is improved, and the stability of the XG-Boost regression model is enhanced. By preprocessing the current dam body characteristic data and using the trained XG-Boost regression model for prediction, it is possible to quickly respond to changes in the current state of the earth-rock dam and realize real-time dam break peak flow prediction, providing an efficient and real-time decision-making basis for flood disaster prevention and control.

[0199] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0200] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0201] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0202] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. The earth-rock dam burst flood peak prediction method based on Copula data enhancement and XG-Boost algorithm is characterized by: The following steps are involved: S1. Obtain historical dam body data and perform data preprocessing on the historical dam body data, thereby obtaining dam body characteristic data and its corresponding label data; the specific steps include: Assume X={X1,X2,X3,…,X n } represents the dam characteristic data; Preset y={y1,y2,y3,…,y n } represents the label data corresponding to the dam body characteristic data; The dam characteristic data is processed using a standardized formula, which is: ； In the formula, represents the standardized dam characteristic data, represents the mean value of the dam characteristic data, is the standard deviation of the dam characteristic data, Indicates the characteristic data of the dam body; The standardized dam characteristic data is converted into uniform distribution. The conversion formula is: ； In the formula, is the transformed dam characteristic data, is the cumulative distribution function CDF of the dam characteristic data x, It is the inverse function of the cumulative distribution function of the dam characteristic data; S2. Based on the Student-T-Copula model, the dam feature data is enhanced and an enhanced data set is generated: The step of performing data enhancement on the dam body characteristic data based on the Student-T-Copula model to obtain enhanced data specifically includes: The Student-T-Copula model is used to capture the relationship between variables in the dam characteristic data, and its expression is: ； In the formula, is a Student's t distribution with ν degrees of freedom, is the ith marginal distribution The inverse CDF of are the parameters of the Student-T-Copula model; Through the Student-T-Copula model and random variables sampled from uniform distribution, new dam characteristic data, namely enhanced data, are generated, and its expression is: ； In the formula, is the augmented data, u=(u1,u2,…,u d ) is a random variable sampled from a uniform distribution, It is the inverse transformation operation of the Student-T-Copula model; S3. In order to improve the diversity of enhanced data and the quality of boundary samples, a gradient perturbation algorithm based on model training feedback is introduced on the basis of Student-T-Copula enhanced samples. The steps are as follows: First, use the basic regression model f trained with the original training data 0， The model uses random forest to estimate pseudo labels for augmented samples: ； In the formula, is the pseudo label predicted by the f0 model; is the enhanced sample set The i-th enhanced sample in ; The currently trained regression model f is set to effectively fit the nonlinear mapping relationship between the dam feature data and its label. The model is based on support vector regression SVR and combined with particle swarm optimization PSO algorithm for hyperparameter tuning; according to The sample prediction result obtained with f, defines the loss function as the mean square error between the sample and its estimated label: ； In the formula, is the loss function, Represents the basic regression model trained using the original training data Pseudo-label estimation of enhanced samples; f is the regression model currently being trained, and the fitting target is the above pseudo-label; According to the loss function, the gradient direction of the calculated sample in the input space is obtained: ； In the formula, is the loss function for sample x i The gradient vector of ; the gradient represents the sensitive direction of the loss function with the feature vector, which is used to guide the sample to perturb to a higher error area to improve the model's learning ability for boundary samples or easily mixed samples; Indicates that the loss function is in sample x i The gradient on Combined with the gradient vector, the perturbation sample is generated. The perturbation method is as follows: ； In the formula, is the i-th disturbance enhancement sample, η is the disturbance step length, Prevent division by zero for tiny constants; is the Euclidean norm of the gradient, used to normalize the direction; Add the perturbed samples to the enhanced data to form a perturbation enhanced sample set: ； In the formula, Enhance the sample set for perturbation; S4. When generating enhanced data X aug and the perturbation-enhanced sample X grad After that, perform the inverse transformation operation and make label prediction; specifically including: The enhanced data is inversely transformed to make it conform to the original feature space; the specific inverse transformation formula is: ； In the formula, It is the value after matching the enhanced data back to the original distribution through CDF, that is, the result after inverse transformation; is the enhanced feature data X aug One of the sample components comes from the pseudo sample generated by the Student-T-Copula model; represents the current estimated CDF of the augmented data; It is the inverse function of the CDF of the original dam characteristic data; The purpose of the inverse transform is to map the enhanced samples from the standardized or uniformly distributed space back to the feature space of the original data, so that the generated samples have the same physical meaning and distribution as the original data; Use the trained regression model to predict the labels of the augmented data. The expression is: ； ； In the formula, f is the regression model obtained by training with enhanced samples and pseudo labels; Construct the final enhanced dataset and merge the original enhanced samples with the gradient perturbation enhanced samples to form the final enhanced dataset: ； ； In the formula, Original augmented samples generated for Student-T-Copula; Gradient perturbation enhanced samples generated after introducing the gradient perturbation algorithm based on model training feedback; It is the collection of original enhanced samples and gradient perturbation enhanced samples; is the collection of predicted labels of the two parts of enhanced samples: the original enhanced samples and the gradient perturbation enhanced samples; finally, the enhanced dataset (X total ,y total ) is used for subsequent model training and testing; S5. The enhanced data set, the dam body feature data and their corresponding label data are used together to form a new data set, and the XG-Boost regression model is constructed and trained using the new data set to obtain the trained XG-Boost regression model. The specific steps include: the final enhanced data set, the dam body feature data and their corresponding label data are used together to form a new data set, and the new data set is divided into a training set and a test set in a ratio of 8:2; Define an XG-Boost regression model, set a base learner, and then use the training set to train the XG-Boost regression model to capture nonlinear relationships in the data; Hyperparameter tuning of XG-Boost regression model was performed through grid search and cross validation; Re-predict the output data in the new data set through the test set, and evaluate the XG-Boost regression model through RMSE and R² until the RMSE and R² of the XG-Boost regression model reach the preset threshold; S6. Obtain the current dam body characteristic data, pre-process the current dam body characteristic data and input the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

2. The method for predicting earth-rock dam burst flood peak based on Copula data enhancement and XG-Boost algorithm according to claim 1 is characterized in that: The historical dam body data include dam height, reservoir capacity, porosity and dam break peak flow, wherein the dam height, reservoir capacity and porosity are used as dam body characteristic data, and the dam break peak flow corresponding to the dam body characteristic data is used as label data.

3. The method for predicting earth-rock dam burst flood peak based on Copula data enhancement and XG-Boost algorithm according to claim 1 is characterized in that: The step of obtaining the current dam body characteristic data, preprocessing the current dam body characteristic data and inputting the data into the trained XG-Boost regression model, i.e. obtaining the predicted current dam break peak flow, includes: Acquire current dam body characteristic data, and pre-process the current dam body characteristic data; The pre-processed current dam feature data is input into the trained XG-Boost regression model. The XG-Boost regression model outputs the predicted current dam break peak flow.

4. A system for predicting earth-rock dam burst flood peaks based on Copula data enhancement and XG-Boost algorithm, applied to the method for predicting earth-rock dam burst flood peaks based on Copula data enhancement and XG-Boost algorithm as claimed in any one of claims 1 to 3, characterized in that: include: A preprocessing module, which obtains historical dam body data and performs data preprocessing on the historical dam body data, thereby obtaining dam body characteristic data and its corresponding label data; The data enhancement module performs data enhancement on the dam feature data based on the Student-T-Copula model to obtain the initial enhanced data samples. On this basis, the enhanced samples are further fine-tuned using a perturbation mechanism based on gradient optimization to achieve dual enhancement of sample feature diversity and boundary discriminability. The enhanced data are then inversely transformed and label data estimated to obtain the final enhanced data set. The model training module combines the enhanced data set and the dam feature data and their corresponding label data into a new data set, and uses the new data set to build and train the XG-Boost regression model, thereby obtaining a trained XG-Boost regression model. The prediction module obtains the current dam body characteristic data, preprocesses the current dam body characteristic data and inputs the trained XG-Boost regression model to obtain the predicted current dam break peak flow.

Images