Water quality prediction method and system based on model set and storage medium

Abstract

The invention discloses a water quality prediction method and system based on a model set and a storage medium, and relates to the field of water quality index prediction. The method comprises the following steps: acquiring river-related monitoring data, and preprocessing the river-related monitoring data to form sample data; inputting the sample data into a model set which is trained in advance, the model set comprising a plurality of prediction models established by using different structures, and obtaining prediction values of a plurality of groups of different prediction indexes; and calculating a comprehensive evaluation index value of each group of evaluation index values, and determining an optimal value according to the comprehensive evaluation index values. According to the method, careful and deep analysis can be carried out according to monitoring data characteristics and historical change rules of different sections, and the prediction model more suitable for the actual condition of the section is screened out.

Description

Technical Field

[0001] The present invention relates to the field of water quality index prediction, and more specifically to a water quality prediction method, system and storage medium based on a model set. Background Art

[0002] The mainstream prediction methods in China are divided into mechanism-based water quality prediction algorithm models and non-mechanism-based water quality prediction algorithm models.

[0003] The mechanism-based water quality prediction algorithm model is constructed based on a detailed description of the physical, chemical and biological processes in the water environment, including hydrodynamic processes, material transport and diffusion, chemical reactions, biodegradation, etc. The model principles and processes are expressed through mathematical equations, including the mass conservation equation, momentum conservation equation, energy conservation equation, and partial differential equations describing various physical, chemical and biological processes. Commonly used models include SP model, WASP model, SWAT model, MIKE series model, EFDC model, Delft3D model, etc. The disadvantages of these models are: 1. Common models are all foreign water quality prediction models, which are quite different from the actual domestic situation, and the rationality of their application is questionable; 2. Mechanism models require more parameters and a large amount of computing resources, but most of the parameters are difficult to obtain accurately in reality, resulting in the inability to implement mechanism models on a large scale; 3. Due to the above-mentioned application defects, the application degree and prediction accuracy of mechanism models are low.

[0004] Non-mechanistic water quality prediction algorithm models mainly rely on a large amount of historical data, and use data analysis and machine learning algorithms to predict future water quality conditions. The model does not directly describe the physical, chemical or biological processes of water quality changes, but focuses on the relationship between input and output. Commonly used methods include mathematical statistics, gray system theory, neural network, support vector machine, random forest, long short-term memory network, etc. The disadvantages of these models are: 1. The model has poor interpretability and cannot deeply understand and reflect the specific factors of water quality changes; 2. The execution time is long and the execution efficiency is low in large-scale training samples; 3. In specific applications, the model prediction accuracy is low.

[0005] Therefore, how to solve the technical problems existing in the above model still requires further research by technical personnel in this field. Summary of the invention

[0006] In view of this, the present invention provides a water quality prediction method, system and storage medium based on a model set.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A water quality prediction method based on a model set comprises the following steps:

[0009] Obtain river-related monitoring data and form sample data after preprocessing;

[0010] Inputting the sample data into a pre-trained model set, wherein the model set includes establishing multiple prediction models using different structures to obtain prediction values ​​of multiple groups of different prediction indicators;

[0011] Calculate the comprehensive evaluation index value of each group of evaluation index values, and determine the optimal value based on the size of the comprehensive evaluation index value.

[0012] Optionally, the river-related monitoring data includes monitoring data from automatic monitoring stations at river sections, monitoring data from regional meteorological stations, monitoring data from hydrological stations and human impact data.

[0013] Optionally, preprocessing of river-related data includes frequency synchronization, outlier removal, missing value supplementation, data enhancement, and normalization.

[0014] Optionally, the model set is constructed as follows:

[0015] Building multiple prediction models using different structures;

[0016] Use swarm intelligence optimization algorithm to optimize different prediction model structure parameters and hyperparameters;

[0017] Use the training data to train different prediction models, and use forward propagation calculation to obtain the output of the model;

[0018] Calculate the loss value of the model according to the loss function, and use back propagation to calculate the gradient of the loss function to the model parameters;

[0019] Repeat the training until the preset number of training times is reached or the loss value drops within the preset range.

[0020] Optionally, the comprehensive evaluation index Z is calculated by the following formula:

[0021]

[0022] Where: y i represents the i-th positive indicator value; y j+p represents the jth reverse indicator value; w i Represents the weight of the i-th indicator, where n=p+q; p is the number of positive indicators; q is the number of negative indicators.

[0023] Optionally, the comprehensive evaluation index value is recalculated daily. The predicted value and actual value of each prediction index of all models on the previous day are obtained daily, and each group of evaluation index values ​​of each prediction index of all models is recalculated. The specific steps are as follows:

[0024] Assume that there are m models in the model set S, namely s1, s2, …, s m ;

[0025] There are n prediction indicators in the prediction indicator set X, namely x1, x2, …, x n ;

[0026] There are (p+q) evaluation indicators in the evaluation indicator set Y, namely y1, y2, …, y p ,y p+1 ,y p+2 ,…,y p+q ,

[0027] Where p is the number of positive indicators, and q is the number of negative indicators;

[0028] Then, x i (s j ) represents model s j The predicted indicator x i The value of

[0029] y k (x i (s j )) represents model s j The predicted indicator x i The corresponding evaluation index y k The value of

[0030] The entire model set can calculate m*n prediction index values ​​and m*n*(p+q) evaluation index values.

[0031] A water quality prediction system based on a model set, comprising:

[0032] Sample data acquisition module: used to obtain river-related monitoring data and form sample data after preprocessing;

[0033] Prediction value calculation module: used to input the sample data into a pre-trained model set, wherein the model set includes establishing multiple prediction models using different structures to obtain prediction values ​​of multiple groups of different prediction indicators;

[0034] Optimal value determination module: used to calculate the comprehensive evaluation index value of each group of evaluation index values, and determine the optimal value according to the size of the comprehensive evaluation index value.

[0035] A computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the water quality prediction methods based on a model set.

[0036] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a water quality prediction method, system and storage medium based on a model set, which can conduct a detailed and in-depth analysis of the monitoring data characteristics and historical change laws of different sections, and screen out a prediction model that is more suitable for the actual situation of the section. This method fully considers the uniqueness of each section, such as hydrogeological conditions, environmental factors, human activity interference, etc., thereby ensuring that the selection of the model is more accurate and effective, and improving the fit between the prediction model and the actual situation in my country, which not only improves the prediction accuracy, but also enhances the reliability and stability of the model.

[0037] In addition, this personalized model adaptation process expands the scope of application of the model, enabling it to better adapt to the prediction needs of different regions and different types of sections. This innovative approach successfully overcomes the application limitations of existing general models in specific scenarios, and provides more accurate technical support for my country's hydrological forecasting, environmental monitoring, disaster warning and other fields. At the same time, this also provides valuable data and experience accumulation for further optimization and improvement of the model, promotes the development of related disciplines and technologies, and contributes to my country's resource and environmental protection and sustainable development. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 It is a schematic diagram of the process of the present invention;

[0040] Figure 2 It is a structural schematic diagram of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The embodiment of the present invention discloses a water quality prediction method based on a model set, such as Figure 1 As shown, the following steps are included:

[0043] Obtain river-related monitoring data and form sample data after preprocessing;

[0044] Inputting the sample data into a pre-trained model set, wherein the model set includes establishing multiple prediction models using different structures to obtain prediction values ​​of multiple groups of different prediction indicators;

[0045] Calculate the comprehensive evaluation index value of each group of evaluation index values, and determine the optimal value based on the size of the comprehensive evaluation index value.

[0046] More specifically, in step 1, according to the received date to be predicted and water quality monitoring section, original data is obtained from the pre-collected and processed water quality monitoring data to form the input sample data of the model.

[0047] The data to be predicted refers to the water quality monitoring data of the water quality monitoring section to be predicted within the time span set before the date to be predicted, for example: one month, half a year, one year. The number of days in the set time span is recorded as the length of the prediction input time series.

[0048] Further, the process of collecting and processing water quality monitoring data:

[0049] S1.1: Raw data collection includes:

[0050] Monitoring data of the automatic monitoring station at the river section: dissolved oxygen, total nitrogen, total phosphorus, permanganate index, ammonia nitrogen, conductivity, water temperature, pH value, turbidity, etc.

[0051] Regional weather station monitoring data: air pressure, temperature / air temperature, relative humidity, precipitation.

[0052] Hydrological station monitoring data: water level, flow.

[0053] Other data: weather forecast data, point source pollutant emission data in the catchment area, population data, farmland data, etc.

[0054] S1.2: Data processing includes:

[0055] Data frequency synchronization: synchronize data with different acquisition frequencies, unify the step length to one day, and remove duplicates from the original data and supplement the missing step lengths.

[0056] Illegal value removal: Delete the values ​​that are not in the normal range of data.

[0057] Outlier removal: Screen and delete outliers.

[0058] Among them: The method for identifying outliers is:

[0059] Suppose the values ​​of data feature x are x1, x2, …, x n ,but Denote the average value of data x, that is

[0060]

[0061] Then the standard deviation s is

[0062]

[0063] Data that differ from the mean by more than three standard deviations are outliers. That is, the data eigenvalue x satisfies , it is considered as an outlier.

[0064] Missing values: Different methods are used to fill in missing values, including: fixed value filling, numerical calculation filling (linear, spline), time series prediction model method filling (ARIMA, diffusion model), etc.

[0065] Data enhancement includes: Feature decomposition: Decomposing the original features into time series based on Loess (local weighted regression smoothing), and decomposing each original feature into trend, season and residual components. Feature extraction: Extracting the month data in the date as a new feature.

[0066] Data normalization: Use the minimum-maximum method to normalize the data set.

[0067]

[0068] Where X is the original data, X min is the minimum value in the data, X max is the maximum value in the data, X norm is the normalized data.

[0069] Step 2: Input the sample data into multiple pre-trained models in the model set for prediction, and obtain the predicted values ​​of multiple groups of different prediction indicators.

[0070] Among them: the model set refers to a collection of multiple prediction models with the same input features, different model structures, and similar output indicators. The training process of the models in the model set is:

[0071] Furthermore, a time series data set is constructed based on the pre-collected and processed water quality monitoring data. The time series data set includes: input data: original data features and enhanced data features; output data: depending on the model, the indicators in the output data can be different, either single indicators or a combination of multiple indicators, such as dissolved oxygen, total nitrogen, total phosphorus, permanganate index, ammonia nitrogen, among which: the method of constructing a time series data set: for example, a sample data includes: the first 30 days of input data + the last 7 days of output data. Based on the pre-collected and processed water quality monitoring data, a time series sample data set is generated with a sliding window size of 37 and a step size of 1.

[0072] In addition, after the time series data set is formed, it is necessary to divide the data set: the training data, validation data, and test data are extracted from 60%, 20%, and 20% of the normalized data set respectively.

[0073] Use different structures to build multiple prediction models, such as LSTM, GRU, TCN, CNN+BiLSTM+MHA, Transformer, etc.

[0074] Swarm intelligence optimization algorithms are used to optimize the model structure parameters and hyperparameters (number of hidden layers, number of hidden layer units, number of multi-head attention heads, learning rate, batch size, etc.), including particle swarm optimization, genetic algorithm, ant colony algorithm, etc.

[0075] Initialize the model parameters and model training hyperparameters (number of training sessions, loss function, optimization algorithm, learning rate, batch size, etc.).

[0076] Input the training data into the model, perform forward propagation calculations, and obtain the output of the model.

[0077] Calculate the loss value of the model according to the loss function.

[0078] Perform back propagation to calculate the gradient of the loss function with respect to the model parameters.

[0079] Use an optimization algorithm to update the model's parameters to reduce the loss value.

[0080] Repeat the above steps until the preset number of training times is reached or the loss value no longer decreases significantly.

[0081] After each training epoch, the model is evaluated using the validation data.

[0082] Input the validation data into the model to obtain the predicted values.

[0083] Calculate the loss function value and other evaluation indicators on the validation data, such as root mean square error (RMSE), mean absolute error (MAE), mean square error (MSE), coefficient of determination (R-Square), etc.

[0084] The root mean square error (RMSE) formula is:

[0085]

[0086] Where n is the number of samples, y i is the true value, is the predicted value.

[0087] The mean absolute error (MAE) formula is:

[0088]

[0089] Where n is the number of samples, y i is the true value, is the predicted value.

[0090] The mean square error (MSE) formula is:

[0091]

[0092] Where n is the number of samples, y i is the true value, is the predicted value.

[0093] The formula for the coefficient of determination (R-Square) is:

[0094]

[0095] Where n is the number of samples, y i is the true value, is the predicted value, is the mean of the true values.

[0096] Monitor the performance indicators on the validation data to determine whether the model is overfitting or underfitting. If the performance on the validation data does not improve after a period of time, stop training early to prevent overfitting.

[0097] Use the test data to perform a final test on the trained model to ensure that the model performs well on new data.

[0098] Step 3: Optimize the predicted values ​​according to the pre-calculated multiple sets of comprehensive evaluation index values ​​of each prediction index to obtain the optimized predicted value combination. After the prediction data is processed, the final prediction result is obtained.

[0099] Among them, optimizing the predicted value according to the multiple groups of evaluation index values ​​of each prediction index calculated in advance means that multiple groups of evaluation index values ​​of each prediction index on different models are calculated in advance, the value of the comprehensive evaluation index Z of each group of evaluation index values ​​is calculated, and the prediction index value corresponding to the group of evaluation index values ​​with the largest value of the comprehensive evaluation index Z is found. This value is the optimal value of this prediction index for this prediction. The optimal values ​​of all prediction indicators for this prediction are combined together to obtain the optimized prediction value combination. The evaluation indicators of the prediction value include root mean square error (RMSE), mean absolute error (MAE), mean square error (MSE), determination coefficient (R-Square), etc.

[0100] Among them: post-prediction data processing refers to performing denormalization operations on the predicted values, etc.

[0101] In addition, the evaluation index values ​​need to be recalculated daily. The predicted value and actual value of each prediction index of all models on the previous day are obtained every day, and each group of evaluation index values ​​of each prediction index of all models is recalculated.

[0102] The specific method is as follows:

[0103] Assume that there are m models in the model set S, namely s1, s2, …, s m ;

[0104] There are n prediction indicators in the prediction indicator set X, namely x1, x2, …, x n ;

[0105] There are (p+q) evaluation indicators in the evaluation indicator set Y, namely y1, y2, …, y p ,y p+1 ,y p+2 ,…,y p+q ,

[0106] Where p is the number of positive indicators and q is the number of negative indicators.

[0107] Then, x i (s j ) represents model s j The predicted indicator x i The value of

[0108] y k (x i (s j )) represents model s j The predicted indicator x i The corresponding evaluation index y k The value of .

[0109] The entire model set can calculate m*n prediction index values ​​and m*n*(p+q) evaluation index values.

[0110] S31: Based on the historical prediction values ​​of the entire model set, m*n*(p+q) evaluation index values ​​of the entire model set are calculated.

[0111] S32: According to the following formula, calculate the comprehensive evaluation index Z of each prediction index, a total of m*n.

[0112] Among them: The comprehensive evaluation index Z is calculated by the following formula:

[0113]

[0114] in:

[0115] y iRepresents the i-th positive index value (i=1,2,...,p).

[0116] y j+p Represents the jth reverse index value (j=1,2,...,q).

[0117] w i represents the weight of the i-th indicator (i=1,2,...,n), where (n=p+q).

[0118] p is the number of positive indicators.

[0119] q is the number of inverse indicators.

[0120] In practical applications, weights can be determined by expert scoring, analytic hierarchy process (AHP), etc. Ensure that the sum of all weights is 1, that is,

[0121] Z(x i (s j )) represents model s j The predicted indicator x i The corresponding value of the evaluation index Z.

[0122] Let function maxZ(x i )=max(Z(x i (s1)),Z(x i (s2)),…,Z(x i (s m ))) represents the prediction index x predicted by all models i The corresponding maximum value of the evaluation indicator Z.

[0123] S33: Call each model in the model set to make predictions, and the latest batch of m*n prediction index values ​​can be predicted. i (s j )), that is, model s j The predicted indicator x i The latest value of .

[0124] Let the function f(Z(x i (s j )))=new(x i (s j )), then f(maxZ(x i )) represents the prediction index x in all models i The latest predicted value corresponding to the one with the largest comprehensive evaluation index Z.

[0125] Furthermore, [f(maxZ(x1)),f(maxZ(x2)),…,f(maxZ(xn ))] is the optimized prediction value combination.

[0126] and Figure 1 Corresponding to the method shown, the present invention also discloses a water quality prediction system based on a model set for Figure 1 The implementation of the method, the specific structure is as follows Figure 2 As shown, including:

[0127] Sample data acquisition module: used to obtain river-related monitoring data and form sample data after preprocessing;

[0128] Prediction value calculation module: used to input the sample data into a pre-trained model set, wherein the model set includes establishing multiple prediction models using different structures to obtain prediction values ​​of multiple groups of different prediction indicators;

[0129] Optimal value determination module: used to calculate the comprehensive evaluation index value of each group of evaluation index values, and determine the optimal value according to the size of the comprehensive evaluation index value.

[0130] Finally, this embodiment discloses a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, any one of the steps of the water quality prediction method based on a model set is implemented.

[0131] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0132] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A water quality prediction method based on a model set, characterized in that: The following steps are involved: Obtain river-related monitoring data and form sample data after preprocessing; Inputting the sample data into a pre-trained model set, wherein the model set includes establishing multiple prediction models using different structures to obtain prediction values ​​of multiple groups of different prediction indicators; Calculate the comprehensive evaluation index value of each group of evaluation index values, and determine the optimal value based on the size of the comprehensive evaluation index value.

2. A water quality prediction method based on a model set according to claim 1, characterized in that: The river-related monitoring data include monitoring data from automatic monitoring stations at river sections, monitoring data from regional meteorological stations, monitoring data from hydrological stations and data on human impact.

3. A water quality prediction method based on a model set according to claim 1, characterized in that: The preprocessing of river-related data includes frequency synchronization, outlier removal, missing value supplementation, data enhancement, and normalization.

4. The water quality prediction method based on a model set according to claim 1, characterized in that: The process of building a model set is as follows: Building multiple prediction models using different structures; Use swarm intelligence optimization algorithm to optimize different prediction model structure parameters and hyperparameters; Use the training data to train different prediction models, and use forward propagation calculation to obtain the output of the model; Calculate the loss value of the model according to the loss function, and use back propagation to calculate the gradient of the loss function to the model parameters; Repeat the training until the preset number of training times is reached or the loss value drops within the preset range.

5. The water quality prediction method based on a model set according to claim 1, characterized in that: The comprehensive evaluation index Z is calculated by the following formula: Where: y i represents the i-th positive indicator value; y j+p represents the jth reverse indicator value; w i Represents the weight of the i-th indicator, where n=p+q; p is the number of positive indicators; q is the number of negative indicators.

6. The water quality prediction method based on a model set according to claim 1, characterized in that: The comprehensive evaluation index value is recalculated daily. The predicted value and actual value of each prediction index of all models on the previous day are obtained daily. Each group of evaluation index values ​​of each prediction index of all models is recalculated. The specific steps are as follows: Assume that there are m models in the model set S, namely s1, s2, …, s m ; There are n prediction indicators in the prediction indicator set X, namely x1, x2, …, x n ; There are (p+q) evaluation indicators in the evaluation indicator set Y, namely y1, y2, …, y p ,y p+1 ,y p+2 ,…,y p+q , Where p is the number of positive indicators, and q is the number of negative indicators; Then, x i (s j ) represents model s j The predicted indicator x i The value of y k (x i (s j )) represents model s j The predicted indicator x i The corresponding evaluation index y k The value of The entire model set can calculate m*n prediction index values ​​and m*n*(p+q) evaluation index values.

7. A water quality prediction system based on a model set, characterized in that: include: Sample data acquisition module: used to obtain river-related monitoring data and form sample data after preprocessing; Prediction value calculation module: used to input the sample data into a pre-trained model set, wherein the model set includes establishing multiple prediction models using different structures to obtain prediction values ​​of multiple groups of different prediction indicators; Optimal value determination module: used to calculate the comprehensive evaluation index value of each group of evaluation index values, and determine the optimal value according to the size of the comprehensive evaluation index value.

8. A computer storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the water quality prediction method based on a model set as described in any one of claims 1 to 6 are implemented.

Images