A method for predicting river water quality based on generative data augmentation

Abstract

This invention discloses a method for predicting river water quality based on generative data augmentation, comprising the following steps: (1) setting up automatic water quality monitoring stations at each monitoring section of the river, collecting historical water quality data from multiple stations and multiple indicators, and combining meteorological driving factors to sort and preprocess the data according to time series, constructing a standardized multidimensional sequence, and obtaining measured samples; (2) establishing a water quality time series generation model based on a conditional variational autoencoder to generate virtual samples with spatiotemporal consistency; (3) applying physical constraints and distribution consistency screening to the virtual samples generated in step (2), eliminating unreasonable data, and obtaining high-quality virtual samples; (4) merging the high-quality virtual samples with the measured samples to form an augmented dataset, and re-dividing the training set, validation set, and test set; (5) training a water quality prediction model based on the augmented dataset and outputting the prediction results of water quality indicators for multiple time intervals in the future.

Description

Technical Field

[0001] This invention relates to the field of water environment monitoring and intelligent prediction technology, specifically to a river water quality prediction method based on generative data augmentation. It can be applied to dynamic water quality monitoring and multi-indicator time-series prediction of rivers, lakes and urban drainage systems, and is especially suitable for scenarios where monitoring data is scarce and samples of extreme pollution events are insufficient. Background Technology

[0002] With urbanization and economic development, river water environment problems have become increasingly prominent. The temporal changes of key water quality indicators such as dissolved oxygen, ammonia nitrogen, and total phosphorus directly reflect the ecological health level of the watershed. In order to make advance predictions of water quality change trends, the academic and engineering communities have carried out extensive research on water quality prediction, using data-driven models (machine learning and deep learning models) or mechanistic models for simulation and prediction.

[0003] While existing water quality prediction models have achieved certain results in both theory and application, they still have the following shortcomings:

[0004] 1. Data missing and sample imbalance are prominent issues.

[0005] In actual monitoring, water quality time series often have a large number of missing or noisy points due to factors such as monitoring equipment maintenance, data transmission interruption, and severe weather. Especially during periods of high pollution or sudden changes, the number of samples is extremely small, making it difficult for the model to fully learn the characteristics of abnormal operating conditions.

[0006] 2. The model has poor generalization performance under extreme events.

[0007] Most prediction models perform well under normal operating conditions, but their prediction errors rise sharply when faced with extreme pollution or sudden events. The main reason is that the distribution of the training samples is inconsistent with the target operating conditions, and the model cannot capture potential non-stationarity.

[0008] 3. Traditional data augmentation methods have obvious limitations.

[0009] Common data augmentation methods such as sliding windows, translation, or noise perturbation cannot reflect the true coupling relationship between multiple indicators and are prone to introducing spurious features; while simple random synthesis methods lack physical rationality and cannot guarantee the scientific validity of the generated data.

[0010] In recent years, the rapid development of generative artificial intelligence (GI) technology has provided new ideas for solving the problems of data scarcity and imbalance. By training generative models to learn data distributions, virtual data similar to real samples can be synthesized, thereby expanding the sample space and improving the robustness and generalization performance of the model. Common generative models include generative adversarial networks (GANs), variational autoencoders (VAEs), and diffusion models.

[0011] However, these methods are mainly applied to fields such as image and speech processing, with limited research on the generation of time series data for multiple water quality indicators. Directly applying the generative models to water quality data can easily lead to problems such as inconsistent time series, imbalanced correlations between indicators, and failure of physical constraints. Therefore, how to achieve high-quality water quality data generation while ensuring physical rationality and spatiotemporal consistency is a key challenge in current research.

[0012] In summary, existing water quality prediction methods struggle to achieve stable and accurate results under conditions of data scarcity. There is an urgent need to propose a generative data augmentation method that can maintain physical plausibility while improving the model's generalization ability. Summary of the Invention

[0013] To address the aforementioned technical problems and shortcomings in this field, this invention provides a river water quality prediction method based on generative data augmentation. This method can solve the problems of sample scarcity and performance degradation under extreme events in existing water quality prediction models. By introducing a Conditional Variational Auto-Encoder (CVAE) generative model, the method learns the potential distribution characteristics of multi-index water quality time series, generates virtual samples with spatiotemporal consistency and physical rationality, and fuses them with measured data to train the prediction model, thereby significantly improving the robustness and generalization ability of the model under different operating conditions.

[0014] A method for predicting river water quality based on generative data augmentation includes the following steps:

[0015] (1) Automatic water quality monitoring stations were set up at each monitoring section of the river to collect historical water quality data of multiple stations and multiple indicators. Combined with meteorological driving factors, the data were sorted by time series and preprocessed to construct a standardized multidimensional series and obtain measured samples.

[0016] (2) A water quality time series generation model is established based on the conditional variational autoencoder (CVAE) to generate virtual samples with spatiotemporal consistency;

[0017] (3) Apply physical constraints and distribution consistency screening to the virtual samples generated in step (2) to remove unreasonable data and obtain high-quality virtual samples;

[0018] (4) Merge high-quality virtual samples with real-world samples to form an augmented dataset, and then re-divide the training set, validation set, and test set;

[0019] (5) Train the water quality prediction model based on the augmented dataset and output the prediction results of water quality indicators for multiple time intervals in the future.

[0020] In step (1) of this invention, automatic water quality monitoring stations are set up at each monitoring section of the river to monitor and record historical water quality data and meteorological data, sort them by time series, and perform preprocessing to construct a sequence format dataset.

[0021] Further, step (1) specifically includes: obtaining the required water quality parameter data, using the IQR (InterQuartile Range) method to remove outliers from the data, using linear interpolation to fill in missing values, and performing Z-score normalization on all input features.

[0022] Furthermore, in step (2), the generated model structure includes three parts: an encoder, a latent space sampling layer, and a decoder. The input to the generated model is a time series of multiple water quality indicators. and corresponding meteorological conditions ,in t For time, n This refers to the number of sites.

[0023] Furthermore, the encoder of the generative model uses nonlinear mapping. Learning the latent representation of data:

[0024]

[0025] In the formula, For latent space variables, The probability distribution is parameterized by the neural network. It follows a normal distribution. and These represent the mean and variance functions, respectively.

[0026] Furthermore, the decoder of the generative model Generate virtual samples based on latent space variables and conditional input: .

[0027] Furthermore, the loss function of the generative model consists of two parts: reconstruction error and KL (Kullback-Leibler) divergence.

[0028]

[0029] in, As a weighting factor, is a standard normal distribution.

[0030] In step (3) of the present invention, double-constraint screening is performed on the generated samples to ensure their physical rationality and distribution consistency.

[0031] Furthermore, in step (3):

[0032] First, physical constraints are imposed. The virtual samples generated in step (2) must satisfy: DO (dissolved oxygen) > 0, 0 < NH3-N (ammonia nitrogen) < 20, 0 < TP (total phosphorus) < 5;

[0033] Then, the KL divergence and Wasserstein distance (WD) are used to measure the distribution difference between the generated data and the real data. When the difference exceeds the threshold, the generated virtual samples are excluded.

[0034] Furthermore, in step (2), the enhancement ratio is the ratio of the number of virtual samples to the number of measured samples, and is dynamically adjusted according to the scale of the real data or takes .

[0035] Furthermore, in step (4), the training set, validation set, and test set are re-divided in a ratio of 7:2:1. 4]

[0036] Furthermore, in step (5), a river water quality prediction model is trained on the enhanced data set.

[0037] Furthermore, in step (5), the prediction model is a long short-term memory network (LSTM), a spatio-temporal convolutional network (ST-ConvNet), or an improved Transformer.

[0038] Furthermore, in step (5), the loss function of the prediction model is defined as:

[0039]

[0040] where N is the total number of samples, is the true value, is the predicted value;

[0041] Furthermore, the total loss function is:

[0042]

[0043] where is the generative constraint weight.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] This invention effectively expands the training data space through generative artificial intelligence methods, overcoming problems such as scarce water quality monitoring data and insufficient extreme pollution samples. It introduces physical constraints and distribution consistency screening mechanisms to ensure the scientific rationality of the generated data. The prediction model trained on the augmented dataset exhibits stable performance under extreme conditions, with an improvement in Nash-Sutcliffe efficiency coefficient of approximately 0.08–0.12 and a reduction in root mean square error of approximately 10%–18%. This invention can significantly improve the accuracy and robustness of water quality prediction while maintaining physical consistency, and is suitable for dynamic water quality monitoring and pollution early warning in urban rivers, lakes, and drainage systems. This invention can be widely applied to dynamic river water quality monitoring, sudden pollution early warning, and watershed water environment management. Attached Figure Description

[0046] Figure 1 This is a framework diagram of the river water quality prediction method based on generative data augmentation of the present invention.

[0047] Figure 2 This is a performance graph of different water quality parameters under different prediction durations in an embodiment of the present invention.

[0048] Figure 3 This is a comparison chart of different dataset types in an embodiment of the present invention. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0050] The river basin in a certain city covers an area of ​​approximately 84 square kilometers. The abundant water resources in this basin provide a strong guarantee for the stable development of various industries in the region and surrounding areas. However, with rapid economic development and continuous improvement in urbanization, water consumption in the region has increased rapidly, and large amounts of industrial wastewater and domestic sewage have been discharged. The water quality of urban surface water bodies has deteriorated, and water quality-related water shortages have become increasingly prominent.

[0051] This embodiment of a river water quality prediction method based on generative data augmentation includes the following steps: First, automatic water quality monitoring stations are deployed at various monitoring sections of the river to collect historical water quality data and meteorological driving factors from multiple stations and multiple indicators. The data is then subjected to missing data imputation, anomaly detection, and normalization to construct a standardized time-series dataset. Second, a water quality time-series generation model is established based on a conditional variational autoencoder, and virtual water quality samples are generated by learning the potential distribution characteristics of historical data. Then, physical constraints and distribution consistency screening are applied to the generated samples to remove unreasonable data, retain high-quality virtual samples, and fuse them with measured samples to form an augmented training dataset. Finally, a water quality prediction model (improved Transformer) is trained on the augmented dataset to achieve multi-indicator predictions over multiple time intervals (12h~336h).

[0052] Specifically, the following steps are included:

[0053] (1) Automatic water quality monitoring stations were set up at seven monitoring sections in the river channel of the study area, numbered S1 to S7. The monitoring period was from January 1, 2021 to December 31, 2023, with a sampling interval of 1 hour. The monitoring indicators included dissolved oxygen (DO), ammonia nitrogen (NH3-N), total phosphorus (TP), pH, conductivity (Cond), turbidity (Turb), and water temperature (WT). Meteorological data (rainfall, air temperature, wind speed, etc.) were collected simultaneously. The monitoring data were sorted by time series to construct a time series dataset. Missing data were corrected by linear interpolation and comparison with adjacent sections. Outliers were removed by interquartile range (IQR) method. All data were input into the model after Z-score standardization.

[0054] (2) A water quality prediction model based on an improved Transformer structure was constructed, using multi-indicator time series as input to predict the water quality change trend at different future time steps. The model includes an encoder, a decoder, and a multi-head attention mechanism module to extract time-dependent features and spatially relevant information. A generative data augmentation module was introduced based on the predictive modeling, using a conditional variational autoencoder (CVAE) model to generate virtual samples. The CVAE encoder inputs multi-indicator time series and corresponding meteorological features, with the latent space dimension set to 32 and the time window length set to 24 steps. The decoder outputs generated samples with the same dimension as the original sequence, with an augmentation ratio of [missing information]. Take 0.3.

[0055] The generative model structure consists of three parts: an encoder, a latent space sampling layer, and a decoder. The input to the generative model is a time series of multiple water quality indicators. and corresponding meteorological conditions ,in t For time, n This refers to the number of sites.

[0056] The encoder of the generative model uses a nonlinear mapping Learning the latent representation of data:

[0057]

[0058] In the formula, For latent space variables, The probability distribution is parameterized by the neural network. It follows a normal distribution. and These represent the mean and variance functions, respectively.

[0059] Decoder for generative models Generate virtual samples based on latent space variables and conditional input: .

[0060] The loss function of the generative model consists of two parts: reconstruction error and KL divergence.

[0061]

[0062] in, As a weighting factor, It follows a standard normal distribution.

[0063] (3) The generated samples are screened for quality. After screening, about 83% of the generated samples are retained to form a high-quality virtual sample set.

[0064] First, physical constraints are applied. The virtual samples generated in step (2) must satisfy: DO > 0, 0 <NH3-N<20,0<TP<5;

[0065] Then, KL divergence and Wasserstein distance are used to measure the difference between the generated data and the real data distribution. When the difference exceeds the threshold, the generated virtual samples are removed.

[0066] (4) The high-quality virtual samples and the actual test samples were merged in a 4:6 ratio to construct an enhanced training dataset, and the training set, validation set and test set were divided in a 7:2:1 ratio. The size of the enhanced dataset increased by about 50% compared with the original dataset, and the number of extreme pollution event samples increased to 2.3 times that of the original.

[0067] (5) Retrain the improved Transformer prediction model on the augmented dataset. The training batch size was 16, the optimizer was Adam, and the learning rate was 0.00001.

[0068] The Nash efficiency coefficient (NSE) and root mean square error (RMSE) are used to evaluate the accuracy of the algorithm's predictions. The specific calculation formulas are as follows:

[0069]

[0070]

[0071] in, This represents the monitoring value of the water quality parameter to be predicted. This represents the predicted value of the water quality parameter to be predicted. This represents the average value of the monitored water quality parameters to be predicted. This indicates the number of data values ​​in a time series.

[0072] The above-mentioned generative data-augmented river water quality prediction model framework is as follows: Figure 1 As shown, this model can quickly and accurately predict changes in key water quality parameters of river sections. Furthermore, observing the model can extract trends and spatial information on water quality changes, which is of great significance for water environment management.

[0073] In this embodiment, to verify the predictive effectiveness of the above method, dissolved oxygen (DO), ammonia nitrogen (NH3-N), total phosphorus (TP), pH, conductivity (Cond), turbidity (Turbine), and water temperature (WT) were selected as key water quality parameters for the river section for prediction. The prediction results are as follows: Figure 2 As shown, the model's predicted values ​​and measured monitoring values ​​are basically consistent, with a uniform overall trend, timely peak response, and small errors in the steady-state segment. Verification shows that the proposed generative data augmentation method can significantly improve the fitting and stability of the prediction model. Specifically, the NSE at different time steps (12h, 24h, 120h, 168h, 336h) all exceed 0.90, with some sections achieving a maximum NSE of 0.97 in short-term predictions, indicating that the method accurately reflects the temporal changes in water quality. Furthermore, the enhanced model performs better under extreme pollution or sudden conditions, especially during the rapid increase of ammonia nitrogen (NH3-N) and total phosphorus (TP), where the predicted and measured values ​​show consistent fluctuation trends, and the peak deviation is significantly lower than that of the unenhanced model. Compared to the original model, this method effectively suppresses abnormal drift phenomena in areas with scarce samples, ensuring the continuity and physical rationality of the prediction process.

[0074] Depend on Figure 3 It can be seen that, under the same dataset and model structure, the model with generative data augmentation achieves better results in predictions at different time steps. As the prediction time step increases, the error of the traditional model tends to accumulate and amplify. However, the method introduced by this approach, through the introduction of diverse virtual samples and physical constraint screening mechanisms, enables the model to learn long-term sequence features more fully, effectively reducing the accumulated error and improving the stability of long-term predictions. This demonstrates that generative augmentation has significant advantages in long-term predictions.

[0075] Furthermore, it should be understood that after reading the above description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims.

Claims

1. A method for predicting river water quality based on generative data augmentation, characterized in that, Including the steps: (1) Automatic water quality monitoring stations are arranged at each monitoring section of the river channel, historical water quality data of multiple sites and multiple indicators are collected, combined with meteorological driving factors, sorted and preprocessed according to the time series, a standardized multi-dimensional sequence is constructed, and measured samples are obtained; (2) A water quality time series generation model is established based on a conditional variational autoencoder to generate virtual samples with spatiotemporal consistency. The generation model structure includes three parts: encoder, latent space sampling layer and decoder. The input of the generation model is the time series of multiple water quality indicators. and corresponding meteorological conditions ,in t For time, n For the number of sites; (3) Physical constraints and distribution consistency screening are imposed on the virtual samples generated in step (2), unreasonable data are剔除 (it should be "eliminated"), and high-quality virtual samples are obtained: First, physical constraints are imposed. The virtual samples generated in step (2) must satisfy: DO>0, 0<NH3-N<20, 0<TP<5; Then, the KL divergence and Wasserstein distance are used to measure the distribution difference between the generated data and the real data. When the difference exceeds the threshold, the generated virtual samples are eliminated; (4) The high-quality virtual samples and the measured samples are combined to form an enhanced data set, and the training set, validation set and test set are re-divided; 2. The river water quality prediction method based on generative data augmentation according to claim 1, characterized in that, (5) Based on the enhanced data set, a water quality prediction model is trained, and the prediction results of water quality indicators for future multi-time intervals are output.

3. The river water quality prediction method based on generative data augmentation according to claim 1, characterized in that, The encoder of the generative model uses a nonlinear mapping Learning the latent representation of data: In the formula, For latent space variables, The probability distribution is parameterized by the neural network. It follows a normal distribution. and These represent the mean and variance functions, respectively.

4. The river water quality prediction method based on generative data augmentation according to claim 3, characterized in that, Decoder for generative models Generate virtual samples based on latent space variables and conditional input: .

5. The river water quality prediction method based on generative data augmentation according to claim 4, characterized in that, Step (1) specifically includes: obtaining the required water quality parameter data, using the IQR method to remove the outliers of the data, using linear interpolation to fill the missing values, and performing Z-score normalization on all input features. The loss function of the generation model consists of two parts, the reconstruction error and the KL divergence in, As a weighting factor, It follows a standard normal distribution.

6. The river water quality prediction method based on generative data augmentation according to claim 1, characterized in that, In step (2), the enhancement ratio The ratio of the number of virtual samples to the number of actual samples is dynamically adjusted based on the actual data size. .

7. The river water quality prediction method based on generative data augmentation according to claim 1, characterized in that, Composed of:

8. The river water quality prediction method based on generative data augmentation according to claim 5, characterized in that, In step (4), the training set, validation set and test set are re-divided according to the ratio of 7:2:

1. in N The total number of samples, For the true value, This is a predicted value; In step (5), a river channel water quality prediction model is trained on the enhanced data set. The prediction model is a long short-term memory network, a spatio-temporal convolutional network or an improved Transformer. The loss function of the prediction model is defined as: The total loss function is: in These are the generative constraint weights.

Images