Lake and reservoir water quality prediction method and system based on multi-source data fusion

Abstract

The invention relates to a lake and reservoir water quality prediction method and system based on multi-source data fusion, and the method comprises the steps: obtaining multi-source and multi-site data in real time, carrying out the data preprocessing, and removing the influence of sudden change data on prediction; performing feature extraction on time sequence data in the multi-source multi-site data through a time sequence attention mechanism, and processing the time sequence data in the multi-source multi-site data to obtain a predicted value of a target index; analyzing multi-site data in the multi-source multi-site data, and obtaining upstream site information in combination with the spatial position of online monitoring equipment and the water flow direction of a reservoir to obtain trend change information of a target index; and inputting the prediction value and the trend change information into a pre-constructed meta-learner at the same time, and outputting a final prediction result of the target index through combined relation learning. Compared with the prior art, the method has the advantages of high model prediction precision and high interpretability.

Description

Technical Field

[0001] This invention relates to the field of water environment monitoring and intelligent prediction technology, and in particular to a method for predicting lake and reservoir water quality by integrating multi-source data feature engineering, machine learning model optimization and multi-model integration. It is applicable to the short-term accurate prediction and long-term trend analysis of key water quality indicators such as turbidity and chlorophyll a in complex water environments such as tidal estuary reservoirs and shallow lakes, and can provide technical support for water environment management, drinking water safety assurance and water ecological regulation. Background Technology

[0002] Reservoirs, as core infrastructure for water resource regulation, bear multiple functions including urban water supply, flood control and disaster reduction, and ecological protection. In recent years, due to the combined effects of climate change (such as extreme rainfall and high temperatures) and human activities (such as watershed pollution input and reservoir scheduling), the volatility of lake and reservoir water quality has significantly increased, with frequent problems such as abnormally high turbidity and algal blooms, seriously threatening water supply security and ecological stability. Traditional water quality prediction technologies have the following key limitations: 1. Feature level: The model does not fully integrate the time-series features of multi-source hydrological, meteorological and water quality data, lacks the construction of key derived features such as cumulative sunshine and cumulative wind value, and the model input effectiveness is insufficient. 2. Model level: Single machine learning models (such as LSTM and SVM) are difficult to simultaneously capture short-term mutations and model long-term trends. Moreover, their "black box" nature makes it impossible to explain the mechanism of variable action, making it difficult to support precise water quality control.

[0003] 3. Lack of interpretability: Most models are "black box" in nature, unable to quantify the impact of key environmental factors on water quality, making it difficult to support the scientific formulation of water quality control strategies.

[0004] While existing technologies have explored multi-source data fusion and model integration methods, they generally suffer from problems such as a disconnect between feature engineering and model optimization, and a lack of targeted fusion strategies, failing to meet the practical needs of water quality prediction in complex aquatic environments. Therefore, developing a lake and reservoir water quality prediction model that balances high accuracy, strong stability, and high interpretability has become an urgent need in the field of water environment management.

[0005] The invention disclosed in CN110598902A presents a water quality prediction method based on a combination of Support Vector Machine (SVM) and KNN, comprising the following steps: 1) preprocessing the collected water quality data; 2) pre-classifying the data using the K-nearest neighbor algorithm; 3) establishing a SVM regression model, selecting the optimal parameters of the water quality data prediction model through k-fold cross-validation, and applying them to the model; 4) training the model, using temperature and pH as prediction targets, and evaluating the prediction effect. This invention proposes a water quality prediction method combining KNN and SVM, using a combination of SVM and KNN to predict water quality time series, aiming to improve prediction accuracy. It belongs to the technical field of water quality prediction methods and has successfully achieved accurate prediction of water quality indicators. However, the model constructed by this scheme lacks adaptability and stability for short-term and long-term prediction tasks, and has low prediction accuracy and weak interpretability. Summary of the Invention

[0006] The purpose of this invention is to overcome the technical problems of insufficient feature effectiveness, weak model generalization ability, and poor interpretability in existing lake and reservoir water quality prediction technologies, and to provide a lake and reservoir water quality prediction method and system based on multi-source data fusion. Through scientific feature engineering, model optimization, and multi-model integration, it can achieve accurate prediction of indicators such as turbidity and chlorophyll a, and analyze the mechanism of action of key environmental factors, providing a practical technical solution for water quality management.

[0007] The objective of this invention can be achieved through the following technical solutions: A method for predicting lake and reservoir water quality based on multi-source data fusion, the method comprising the following steps: Step 1: Acquire multi-source, multi-site data in real time, including lake and reservoir water quality data, meteorological data, and hydrological data, and perform data preprocessing to remove the impact of abrupt changes on the prediction; Step 2: Feature extraction is performed on the time series data in the multi-source, multi-site data using a temporal attention mechanism. The time series data in the multi-source, multi-site data is processed simultaneously using two independent long short-term memory neural network structures (forward and backward) to obtain the predicted value of the target indicator. The multi-site data in the multi-source, multi-site data is analyzed, and upstream station information is obtained by combining the spatial location of the online monitoring equipment and the direction of water flow in the reservoir to further obtain the trend change information of the target indicator. Step 3: The predicted value and trend change information are simultaneously input into the pre-built meta-learner. By learning the combination relationship between the short-term prediction information and the long-term trend prediction information of the target indicator under multi-source and multi-site conditions, the final prediction result of the target indicator is output.

[0008] Furthermore, the time series data in the multi-source, multi-site data are analyzed by the fusion model, which outputs the predicted values ​​of the target indicators; the fusion model is obtained by combining the pre-built support vector machine model, the optimized long short-term memory neural network model, and the pre-built long-sequence time series prediction model. The support vector machine model is used to process the data of the target site to remove the impact of mutations on the prediction; the long-sequence time series prediction model is used to analyze the multi-site data in the multi-source multi-site data to obtain the trend change information of the target indicator. The optimization process of the optimized long short-term memory neural network model specifically includes: The temporal attention mechanism is applied to the long short-term memory neural network model to calculate the attention score at each time step and weight and aggregate the time step features to focus on key temporal information. The specific calculation formula is as follows: in, and It is a learnable parameter matrix. It is the projection vector. It is a bias term. This represents the degree of influence of time step 𝑖 on time step 𝑡. This represents the time step characteristics optimized by the TPA mechanism. For the current time step Corresponding original features Original time step Corresponding features Indicates time step The degree of influence on time step 𝑡; Time-series data is processed simultaneously using two independent long short-term memory neural network structures, one forward and one backward, enabling bidirectional information propagation and state merging. The calculation formula includes: in, and These represent the hidden states of the forward and backward LSTMs, respectively. For the current time step The corresponding input feature vector, This indicates that the forward LSTM is at time step The hidden state, This indicates that the backward LSTM is at time step The hidden state, Indicates the current time step The final hidden state.

[0009] Furthermore, an input feature set containing long-term window data and cumulative variables is constructed to train the support vector machine model, long short-term memory neural network model, and long sequence time series prediction model, specifically including: The stationarity of the target indicators is analyzed to assess the trend and stability of the time series; the long memory of the data is analyzed, and feature transformation or differencing is performed on data that are non-stationary or lack long memory; further, the moving average method is used to analyze the correlation between the water quality indicators of each monitoring point and the predictive variables of the target points under different time windows, and a long-term window dataset is constructed based on the selected data with long memory. The cumulative variables specifically include cumulative illumination variables and cumulative wind speed variables; based on Beer-Lambert's law, the cumulative illumination variables are obtained by combining the cumulative illumination values, attenuation weights, and turbidity for different time windows. By projecting wind speeds onto the east-west and north-south axes respectively, and weighting and accumulating historical time-series wind speed data, the accumulated wind speed variable is obtained to characterize the impact of the wind field on different time scales.

[0010] Furthermore, the stationarity analysis of the target index is performed using the ADF test, specifically including: Calculate the ADF statistic and compare it with the Dickey-Fuller critical value: if the ADF statistic is less than the Dickey-Fuller critical value, the time series is considered stationary; otherwise, the time series is considered non-stationary. The calculation formula is as follows: in, The unit root coefficient, The lag order is... This is the white noise error term. The unit root coefficient, This is an estimate of the unit root coefficient. Its standard error, For the intercept term, For time trend items, Time series The first lag value, Represents the first-order difference. Lagged difference explanatory variables is the coefficient of the lagged difference term.

[0011] Furthermore, the stationarity analysis of the target index is performed using the KPSS test, specifically including: Calculate the KPSS statistic and compare it with a critical value. If the KPSS statistic is greater than the critical value, the time series is considered non-stationary; otherwise, the time series is considered stationary. The calculation formula is as follows: in, It is a random walk process. Deterministic trend term For error terms, It is a time series. For the sample size, For the cumulative sum of residuals, This is the variance estimate for the error term.

[0012] Furthermore, the long memory property of the data is analyzed using the autocorrelation function, partial autocorrelation function, and power spectral density test, specifically including: The autocorrelation function is used to calculate and analyze the relationship between the current data point and the past data. The correlation between data points over a period is calculated using the following formula: in, Indicates lag The autocorrelation coefficient of the period, The mean of the time series. For the current data point lag, The mean of the time series; The partial autocorrelation function is used to calculate and analyze the direct correlation of time series under different lag orders. The calculation formula is as follows: in, Lagging The partial autocorrelation coefficient of order, The autocovariance is the result of lag order n. The variance is without lag. , Lagging Rank, The autocovariance of the order, For the front The autoregressive coefficient of order, Index for lag order; The power spectral density test is used to quantify the energy distribution of data at different frequencies to identify the long memory of the data. The calculation formula is as follows: in, It is frequency Fourier coefficients at the location, The length of the time series. The imaginary unit, Represents frequency The power intensity at a given point reflects the contribution of that frequency component to the time series.

[0013] Furthermore, step two also includes: The final input feature set is divided into a training set, a test set, and a validation set. The data is then standardized to a distribution with a mean of 0 and a standard deviation of 1, while retaining the original data distribution and ensuring that all data fall within a specific interval. The calculation formula is as follows: in, The original value, The average value of the indicator. The standard deviation of the indicator. This is the normalized value; Next, the data is transformed using changes in logarithmic returns to eliminate the influence of trends. The calculation formula is as follows: in, This represents the logarithmic return at time point X. and They represent time respectively and The original step data values ​​at each moment; Furthermore, using mean squared error as the loss function, a multi-parameter grid search method is employed to screen the optimal hyperparameter combination of pre-built random forest model, support vector machine model, long short-term memory neural network model, and long sequence time series prediction model.

[0014] Furthermore, step three also includes: analyzing the sensitivity of the fusion model, calculating the independent contribution of each input variable to the model prediction results under different combinations and the global contribution of each input variable; and further analyzing the dependency of the fusion model to reveal the interaction patterns between variables.

[0015] Furthermore, the process of sensitivity analysis of the fusion model specifically includes: The sensitivity of the fusion model was analyzed using Shapley value analysis, specifically including: The independent contribution of each input variable to the model prediction result under different combinations is calculated using the following formula: in, Indicates inclusion The prediction results Represents variables, Represents a subset of variables. Represents a subset of variables The corresponding model prediction value, Representing variables The degree of contribution; The weighted summation of the contributions of each variable is calculated using the following formula: in, Representing variables SHAP value, This represents the total number of all input variables involved in the analysis. The sensitivity of the fusion model is further analyzed using the Sobol global sensitivity analysis method. The global contribution of each input variable is calculated using the following formula: in, Represent a single variable Variance of contribution Representing variables and The joint variance contribution, This reflects the interaction between multiple variables. Reflecting variables The direct impact on the model output. As variables, The overall sensitivity index to the model output, where 𝑌 is the model output, Indicates model output The total variance; The dependencies of the fusion model are analyzed using partial dependency analysis: The partial dependency analysis method includes univariate partial dependency analysis and bivariate partial dependency analysis; the univariate partial dependency analysis method is used to calculate the mean of the model output for different values ​​of the input variable, and the calculation formula is as follows: in, Indicates except The values ​​of other variables besides the first one The mean of the prediction results for each sample is used for evaluation. The average effect under specific values; The combined effect of the two variables is analyzed using the bivariate partial dependency analysis method, and the calculation formula is as follows: in, Indicates division variable and The values ​​of other variables besides the first one The mean of the predicted results for each sample is used to assess the combined effect between variables; Based on the results of sensitivity and dependency analysis, key variables affecting the prediction accuracy of the fusion model are screened to guide the construction of the input feature set, and the fusion model is trained based on the newly constructed input feature set.

[0016] The present invention also provides a system for predicting lake and reservoir water quality based on multi-source data fusion, comprising a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of any of the methods described above.

[0017] Compared with the prior art, the present invention has the following advantages: (1) The present invention uses hyperparameter optimization to train a pre-constructed random forest model, support vector machine model, long short-term memory neural network model and long sequence time series prediction model; introduces a time series attention mechanism and bidirectional structure into the long short-term memory neural network model to obtain the TPA-Bi-LSTM model; further integrates the random forest model, support vector machine model, TPA-Bi-LSTM model and long sequence time series prediction model to construct the SVM-Informer-LSTM fusion model; The final SVM-Informer-LSTM fusion model has excellent prediction performance and is adaptable and stable for both short-term and long-term prediction tasks. In long-term rolling prediction, the RMSE is reduced by 9.8% and the NSE is improved by 12.3% compared with the single model. The short-term prediction NSE reaches 0.95 and the long-term prediction NSE remains above 0.5, which balances "short-term accuracy and long-term stability".

[0018] By combining multi-source data fusion and model optimization strategies, the advantages of different models are fully utilized, improving the adaptability and stability of short-term and long-term prediction tasks, and offering advantages such as high prediction accuracy and strong interpretability.

[0019] (2) This invention uses Shapley value analysis, Sobol global sensitivity analysis and partial dependency analysis to further analyze the sensitivity and dependency of the constructed SVM-Informer-LSTM fusion model, which helps to further screen the key variables in the water quality detection process in the input feature set and the final input feature set, and applies the optimization and improvement of the SVM-Informer-LSTM fusion model to further improve the predictive performance of the fusion model; it has outstanding interpretability, realizes the visualization of the variable action mechanism, solves the "black box" problem in machine learning models, and provides a clear scientific basis for water quality regulation.

[0020] (3) This invention constructs an input feature set containing long-term window data and cumulative variables based on multi-site and multi-source data, including lake and reservoir water quality data, meteorological data and hydrological data, and further filters the features using the random forest method to obtain the final input feature set; thus, the final constructed time series feature set has strong effectiveness, and the sensitivity of the trained model to the driving mechanism of water quality change is improved by 30%, which solves the problem of "single features" and poor prediction performance of traditional models. Attached Figure Description

[0021] Figure 1 This is a flowchart of a method for predicting lake and reservoir water quality based on multi-source data fusion provided in an embodiment of the present invention; Figure 2 This is a simplified flowchart of a lake / reservoir water quality prediction method based on multi-source data fusion provided in an embodiment of the present invention; Figure 3 This is a comparison chart of the simulation effects of various models of a lake and reservoir water quality prediction method based on multi-source data fusion provided in an embodiment of the present invention. Figure 4 This is a framework diagram of a fusion model for a lake / reservoir water quality prediction method based on multi-source data fusion provided in an embodiment of the present invention. Figure 5 This is a model prediction effect diagram of a lake and reservoir water quality prediction method based on multi-source data fusion provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0025] Definitions: SVM: A classic supervised machine learning algorithm, mainly used to solve classification and regression problems. Its core idea is to find an optimal hyperplane (a straight line in two-dimensional space) to maximize the boundary margin between data points of different classes.

[0026] Informer: A deep learning model specifically designed for long-sequence time series prediction problems. LSTM is a special type of recurrent neural network designed to solve the gradient vanishing or gradient exploding problems that are common in traditional RNNs during training, thereby enabling it to effectively learn and remember long-term dependencies in long sequence data.

[0027] RF (Ensemble Learning) is a classic machine learning algorithm based on ensemble learning, primarily used for classification and regression tasks. Its core idea is to construct and combine multiple decision trees to form a more powerful and stable model. SHAP is a model interpretation method based on the Shapley value principle in game theory. Its core idea is to treat each feature of a sample as a "collaborator". The final prediction value is the result of the "cooperation" of all features, and the SHAP value fairly allocates the contribution of each feature to the prediction result.

[0028] Sobol global sensitivity analysis is a global sensitivity analysis technique based on variance decomposition, used to quantify the contribution of the uncertainty of input parameters in a mathematical model to the total variance of the output. This method measures the impact of independent changes in a single input parameter on the output variance through the first-order Sobol exponent, and captures the joint effect of the interaction between each parameter and all other parameters through the total Sobol exponent, thus comprehensively identifying key sensitive parameters and their higher-order interactions. Its analysis is based on Monte Carlo sampling, evaluating the parameters across the full probability space, without requiring assumptions about the linearity or monotonicity of the model form or parameter distribution. Therefore, it is suitable for highly nonlinear and non-monotonic complex models, and has significant application value in fields such as environmental science, engineering design, and financial risk assessment.

[0029] Partial Dependency Analysis (PDP) is a global interpretation method used to explain complex machine learning models (especially black-box models). It aims to reveal the average marginal effect between one or two target features and the model's prediction results. Its core idea is to systematically change the value of the target feature while keeping all other feature values ​​in the dataset constant, observe and calculate the average change in the model's predictions, and thus plot the dependency curve or surface between the target feature and the prediction results.

[0030] The Alternating Current Model (ADF) test is a classic statistical hypothesis test method used to test whether a time series dataset contains a unit root. It was proposed by Dickey and Fuller and extended by Said and Dickey. Its core purpose is to determine whether a time series is stationary—that is, whether the statistical properties of the series (such as mean and variance) change over time. The test involves fitting a regression equation that includes lagged differences and examining the significance of the coefficients of the lagged level terms. If the null hypothesis is rejected (i.e., the series contains a unit root), the series is stationary; if the null hypothesis cannot be rejected, the series is non-stationary and may have a stochastic trend, requiring stationarization processing such as differencing. The ADF test is a crucial preprocessing step in time series analysis before constructing ARIMA models and performing cointegration analysis, and its results directly affect the reliability of subsequent modeling.

[0031] The KPSS test is a time series stationarity test that complements the ADF test. Its core characteristic is that the null hypothesis is opposite to that of the ADF test: the null hypothesis (H0) of the KPSS test is that the series is stationary (or has a stationary trend), while the alternative hypothesis (H1) is that the series has a unit root (non-stationary). The test constructs a model of a stationary series around a deterministic component (constant or linear trend) and examines whether its residuals are stationary. If the test statistic exceeds a given critical value, the null hypothesis of "stationarity" is rejected, and the series is inferred to be non-stationary. Because the KPSS test's hypothesis is opposite to that of the ADF test, in practice, the two are often used together for a more robust assessment of series stationarity—the confidence level is higher when both tests reach the same conclusion (e.g., both indicate non-stationarity).

[0032] Time series decomposition model: This is a classic analytical method that systematically breaks down observed time series data into several basic components. Its core idea is that the fluctuation of the series mainly stems from the combined effect of multiple identifiable intrinsic factors.

[0033] The autocorrelation function (ACF) is a statistical tool used to quantify the linear correlation between observations of the same time series at different time points (i.e., different lag orders). Its core function is to reveal the dependency structure or memory effect within the series.

[0034] Partial Autocorrelation Function (PACF): A statistical function used to measure the conditional correlation between a current observation and its k-period lag observations in a time series, given intermediate lag observations. Power spectral density test (PSD) is a time series stationarity diagnostic method based on frequency domain analysis. Its core principle is to decompose the total variance (power) of the time series into different frequency components and reveal the intensity of the periodic or seasonal fluctuations implied in the series in the form of a power spectral density function.

[0035] Moving average method: a classic time series smoothing and forecasting technique. Its core idea is to eliminate random fluctuations and highlight underlying trends by calculating the arithmetic mean of observations within a continuous time window. The basic form of this method is the simple moving average, which takes the average of the most recent n periods of data as the forecast value for the next period. Its extended forms include weighted moving average (giving more weight to recent data) and exponentially weighted moving average (weighting all historical data in an exponentially decaying manner).

[0036] Random forest method: It is a machine learning algorithm based on decision tree ensemble. Its core idea is to build a large number of independent and structurally differentiated decision trees and integrate the outputs of all trees by voting (classification) or averaging (regression) mechanism, so as to obtain a prediction model with strong generalization ability and high stability.

[0037] Temporal Attention (TPA) is a variant of the attention mechanism specifically designed for multivariate time series forecasting. It aims to adaptively capture key temporal dependency patterns at different points in time from historical data. Unlike conventional attention mechanisms, TPA not only focuses on the correlation between time points but also employs a specially designed one-dimensional convolutional layer to extract the local temporal patterns of each variable in the multivariate sequence. Then, through attention weight calculation, it identifies the most important historical time series segments and variable combinations for the current forecast.

[0038] Boosting is a machine learning framework that builds strong predictive models by sequentially integrating multiple weak learners. Its core idea is to iteratively correct errors: in each training round, the weights of the training samples or the fitting residuals are adjusted based on the prediction performance of the previous round, so that subsequent weak learners focus more on the samples that were previously mispredicted. Finally, the results of all weak learners are weighted and combined to form the final prediction.

[0039] Stacked algorithms are a high-order ensemble learning method that integrates multiple heterogeneous base learners through a hierarchical architecture. The core idea is to use a meta-learner to retrain and fuse the predictions of the primary model, thereby improving generalization performance. This method first trains multiple base learners of different types in parallel on the training set, then uses their outputs as new features to construct a meta-training set, and trains a secondary model to learn the optimal combination of the base learner predictions.

[0040] Example 1 like Figure 1 As shown in the figure, this embodiment provides a method for predicting lake and reservoir water quality based on multi-source data fusion. The method includes the following steps: S1: Real-time acquisition of multi-source, multi-site data, including lake and reservoir water quality data, meteorological data, and hydrological data, and data preprocessing to remove the impact of abrupt changes on prediction; S101: Acquire and preprocess multi-source data such as lake and reservoir water quality, meteorological and hydrological data; S1011: Acquire multi-source data of lakes and reservoirs with different variable types and spatial locations. Water quality indicators (such as turbidity, chlorophyll a, DO, pH, etc.) reflect the physicochemical properties and nutrient status of water bodies and are the core variables for model prediction; meteorological indicators (such as PAR, wind speed and wind direction, etc.) can characterize the impact of the external environment on algal growth and water disturbance; hydrological indicators (such as flow velocity, water level and its rate of change, etc.) are used to characterize the dynamics of water body scheduling in the reservoir area.

[0041] S1012: By supplementing and cleaning missing values ​​and outliers, data integrity is improved to reduce the impact of outliers on water quality prediction models.

[0042] S102: Construct an input feature set that includes derived features such as long-term windows and cumulative variables, and use the random forest method for feature selection; S1021: ADF and KPSS stationarity analyses were performed on the predictor variables to assess the trend and stability of the time series.

[0043] The ADF test is an extended form of the unit root test, mainly used to determine whether a time series has a unit root, i.e., whether it is a stationary series. Its basic idea is to construct the following regression equation and test the unit root hypothesis of the time series: In the formula, The unit root coefficient, The lag order is... This is the white noise error term. The unit root coefficient, For the intercept term, For time trend items, Represents the first-order difference Lagged difference explanatory variables for The first lag value, is the coefficient of the lagged difference term.

[0044] The core of the ADF test lies in the unit root coefficient. Perform hypothesis testing: null hypothesis The time series is assumed to have a unit root, i.e., it is non-stationary; alternative hypothesis. If the time series does not have a unit root, it is considered stationary. The formula for calculating the ADF test statistic is: In the formula, This is an estimate of the unit root coefficient. Let its standard error be denoted as . By comparing it with the Dickey-Fuller critical value, if the αi statistic is less than the critical value, the null hypothesis is rejected, indicating that the time series is stationary; otherwise, the null hypothesis is accepted, indicating that the data has a unit root and further differencing is needed to achieve stationarity.

[0045] The KPSS test and the ADF test have opposite hypotheses; its null hypothesis is... The time series is stationary, while the alternative hypothesis is... The sequence is non-stationary, and using both together can yield comprehensive stationarity analysis results.

[0046] The KPSS test is based on a time series decomposition model: In the formula, It is a random walk process. Deterministic trend term This is the error term. If... If a time series has a unit root, then... If the series is non-stationary, it is stationary; otherwise, it is stationary. The KPSS test primarily determines the stationarity of a series by testing whether its long-term variance is zero. The statistic is calculated as follows: In the formula, For the cumulative sum of residuals, This is a variance estimate for the error term. denoted as the sample size. The Kjeldahl statistic is compared with the critical value. If the Kjeldahl statistic is greater than the critical value, the null hypothesis is rejected, indicating that the series is non-stationary; if the Kjeldahl statistic is less than the critical value, the null hypothesis is accepted, indicating that the time series is stationary.

[0047] S1022: The long memory of the data is analyzed using autocorrelation function (ACF), partial autocorrelation function (PACF), and power spectral density test (PSD) to determine whether the features need feature transformation or differencing.

[0048] The autocorrelation function is defined as a time series... Its lagged terms The correlation between them is calculated as follows: In the formula, Indicates lag The autocorrelation coefficient of the period, This is the time series mean. When... A larger value indicates that the current data point is significantly different from the past data points. There is a strong correlation between data points in different periods. This characteristic can be used to construct lagged variables to improve the performance of predictive models.

[0049] The partial autocorrelation function (PACF) measures the direct correlation between time series at different lag orders; that is, after removing the influence of all intermediate lags, it represents the net correlation between the current data point and a data point at a given lag order. PACF is calculated based on multiple regression analysis and is defined as follows: In the formula, Lagging The partial autocorrelation coefficient of order, The autocovariance is the result of lag order n. The variance is without lag. , Lagging Rank, The autocovariance of the order, For the front The autoregressive coefficient of order, The index is the lag order.

[0050] The power spectral density test quantifies the energy distribution of data at different frequencies through frequency domain analysis, effectively identifying whether the data has long-term dependence and determining its periodicity or trend characteristics. The Discrete Fourier Transform is defined as follows: In the formula, It is frequency Fourier coefficients at the location, The length of the time series. The imaginary unit is used. The power spectral density is calculated by squaring the Fourier transform: In the formula, Represents frequency The power intensity at a given point reflects the contribution of that frequency component to the time series.

[0051] S1023: Features with long memory will be selected and further constructed into long-term features. The moving average method will be used to calculate the correlation between water quality indicators at each monitoring point and the predictor variables at the target point under different moving average windows. The influence range of different time windows on the predictor variables will be analyzed, and the selection of input features will be optimized accordingly.

[0052] S1024: Construct input features based on derived features such as cumulative variables.

[0053] When predicting algal biomass, photosynthetically active radiation is a key driver of algal photosynthesis; however, instantaneous light intensity alone cannot fully reflect the impact of cumulative light over a certain period on algal growth. Based on Beer-Lambert's law, and combining factors such as cumulative light values, attenuation weights, and turbidity in different time windows, cumulative light characteristics are obtained.

[0054] To assess the influence of wind direction and speed on the dynamic characteristics of a reservoir, wind speed is projected onto the east-west and north-south directions, and weighted and accumulated over historical time periods to characterize the impact of the wind field at different time scales.

[0055] Specifically, A random forest method is used for feature selection. Considering that not all features contribute positively to model performance, and some redundant or noisy features may even introduce interference, the random forest method is introduced to quantitatively evaluate feature importance. Random forest is an ensemble learning method based on decision trees. It constructs multiple decision trees and performs voting or averaging calculations. During the construction of each tree, if a feature is used to partition the data and brings a significant reduction in error, then that feature is considered highly important.

[0056] S2: Feature extraction is performed on time series data from multi-source, multi-site data using a temporal attention mechanism. The time series data from multi-source, multi-site data is processed simultaneously using two independent long short-term memory neural network structures (forward and backward) to obtain the predicted value of the target indicator. The multi-site data from multi-source, multi-site data is analyzed, and upstream station information is obtained by combining the spatial location of online monitoring equipment and the direction of water flow in the reservoir to further obtain the trend change information of the target indicator. Specifically, random forest (RF), support vector machine (SVM), long short-term memory neural network (LSTM), and long sequence time series prediction (Informer) models are built and compared, and prediction performance is improved through hyperparameter optimization; the specific modeling process includes: S201: The dataset is divided into three parts: training set, test set, and validation set, with proportions of approximately 70%, 20%, and 10%, respectively. The training set is used for model learning and parameter optimization, the test set is used to evaluate the model's generalization ability on unseen data, and the validation set is used to further test the model's predictive performance on the latest data.

[0057] S202: Standardizing the data removes unit restrictions, ensuring all data fall within a small, specific interval. In this study, standardization refers to transforming the data into a distribution with a mean of 0 and a standard deviation of 1, while preserving the original data distribution and ensuring all data fall within a specific interval. The calculation expression is: In the formula, The original value, The average value of the indicator. The standard deviation of the indicator. This is the normalized value.

[0058] S203: The original data is transformed using changes in logarithmic returns to eliminate the influence of trends, improve the stationarity of the data, and ensure that the data meets the assumptions of time series modeling. The formula for calculating logarithmic returns is: In the formula, This represents the logarithmic return at time point X. and They represent time respectively and The original step data value at each moment.

[0059] S204: Using mean squared error (MSE) as the loss function, random forest (RF), support vector machine (SVM), long short-term memory neural network (LSTM), and long-sequence time series prediction (Informer) models were built. Multi-parameter grid search was used to optimize the key hyperparameters of each model to ensure that the model operates under the optimal parameter combination and improve prediction performance. RMSE, MRE, NSE, and MAE evaluation metrics were used to measure and compare prediction performance in terms of error magnitude, model stability, and goodness of fit, respectively.

[0060] S3: The predicted value and trend change information are simultaneously input into the pre-built meta-learner. By learning the combination relationship between the short-term prediction information and the long-term trend prediction information of the target indicator under multi-source and multi-site conditions, the final prediction result of the target indicator is output.

[0061] S301: The prediction model is constructed by integrating optimization strategies and introducing a temporal attention mechanism and bidirectional structure (TPA-Bi-LSTM) into the LSTM model. S3011: Temporal Attention (TPA) is applied to the LSTM model to enhance its attention to key time steps. In the optimized model, the LSTM is responsible for extracting basic temporal features, while the TPA mechanism is used to calculate the attention score at each time step. This allows the model to dynamically adjust the level of attention at different time steps, thereby reducing the error accumulation effect in long-duration rolling prediction tasks. The specific formula is as follows: In the formula, and It is a learnable parameter matrix. It is the projection vector. It is a bias term. For the current time step Corresponding original features Original time step Corresponding features This represents the degree of influence of time step 𝑖 on time step 𝑡. This represents the time step feature optimized by the TPA mechanism. This feature can effectively highlight the information most relevant to the current prediction task, reduce information loss, and improve the model's ability to model long-range dependencies.

[0062] S3012: Bi-LSTM leverages bidirectional time-series information to enhance modeling capabilities. Bi-LSTM processes time-series data simultaneously through two independent LSTM structures (forward and backward), enabling the model to comprehensively consider past and future information, thus more accurately learning the dynamic characteristics of water quality changes. Its computational process mainly includes bidirectional information propagation and state merging. Let the input sequence be... The forward LSTM processes the input data in chronological order and calculates the hidden state, while the backward LSTM processes the time series in reverse order. Its expression is: in, For the current time step The corresponding input feature vector, This indicates that the forward LSTM is at time step The hidden state, This indicates that the backward LSTM is at time step The hidden state.

[0063] Finally, the hidden states calculated by the forward and backward LSTMs are concatenated to form the final output of the Bi-LSTM: In the formula, and These represent the hidden states of the forward and backward LSTMs, respectively. Indicates the current time step The final hidden state, and the concatenation of the two, enable the model to make predictions using both historical and future information, thereby improving its ability to capture temporal features.

[0064] S302: Integrating SVM, LSTM and Informer to build a multi-model integration framework; Based on model complementarity, SVM, LSTM, and Informer were selected. SVM first processed the data from the target site to remove the impact of mutations on predictions. Then, LSTM further extracted sequence features, and the prediction results were optimized using a boosting method. For data from other sites, Informer was used for modeling to fully utilize upstream site information and improve the predictive ability for future water quality at the target site. In the final fusion stage, the Informer and LSTM prediction results were stacked and fused using a stacking method to combine short-term and long-term prediction information, ultimately obtaining the prediction results of the fused model.

[0065] This fusion framework combines the characteristics of different models, making it more adaptable to water quality prediction tasks. In terms of performance and robustness, the introduction of SVM enhances the model's ability to handle abrupt changes, LSTM is responsible for short-term prediction and deep feature extraction, while Informer further strengthens the ability to predict long-term trends.

[0066] Preferred, We used Shapley value analysis, Sobol global sensitivity analysis, and partial dependency analysis to conduct sensitivity and dependency analysis on the fusion model to guide the selection of key variables.

[0067] The SHAP analysis method was used to analyze the model. SHAP analysis calculates the marginal contribution of each input variable to the model's prediction results under different combinations, thereby rationally allocating the importance of variables.

[0068] The SHAP computation process includes several key steps. First, for those containing... A prediction model with input variables is constructed, and a subset of variables is generated. That is, removing a certain variable The subset is then calculated, and the corresponding model prediction value 𝑓(𝑁) is calculated. The marginal contribution expression of the variable 𝑥𝑖 is: In the formula, Indicates inclusion The prediction results, and The difference between the two is the variable's contribution. To ensure fairness in variable contributions, SHAP calculates all possible combinations of variables and performs a weighted sum of the contributions of each variable, expressed as: In the formula, express SHAP value, This represents the total number of all input variables involved in the analysis. The weighting term ensures that the contribution of all variable combinations is reasonably evaluated. The larger the final SHAP value, the more significant the impact of the variable on the model's prediction results.

[0069] The Sobol global sensitivity analysis method was used to analyze the model. The Sobol method calculates the contribution of each input variable to the model output through variance decomposition. Based on the principle of variance decomposition, this method attributes the uncertainty of the model output to different combinations of input variables and calculates the contribution of each variable. Let the model output *k* be the input variable. If the function is defined, then the total variance of the model output can be decomposed into the independent contributions of each input variable and the interaction contributions between variables, as expressed by: In the formula, Represent a single variable Variance of contribution Representing variables and The joint variance contribution, This reflects the interaction between multiple variables, and 𝑌 is the model output. Indicates model output The total variance. Variance sensitivity analysis mainly includes the first-order sensitivity index and the total sensitivity index. The former measures the independent contribution of a single variable to the model output, and its calculation formula is as follows: In the formula, Reflects variables The overall sensitivity index directly impacts the model output; a higher value indicates greater importance of the variable. It measures the global contribution of a variable, including its independent effect and all possible interactions. The formula is: In the formula, For variables For the overall sensitivity index output by the model, if ≫ , describe variables The interaction significantly impacted the model output, and considering only its independent contribution may lead to an underestimation of the importance of the variable.

[0070] PDP analysis was used to analyze the model. By calculating the mean of the model output for the target variable at different values, PDP provides an intuitive visual interpretation for variable impact analysis, helps to identify the trend of the effect of key variables in different numerical ranges, and reveals the interaction patterns between variables.

[0071] The PDP calculation process includes several steps. First, the target variable is selected. And determine its range of values. For a univariate PDP, while keeping all other variables constant... Under the premise of not changing, let The model predicts the value of 'x' across different values, and then averages the predicted values ​​for all samples to assess the average impact of 'x' at a specific value. The formula for this calculation is as follows: In the formula, Indicates except The values ​​of other variables besides the first one The mean of the prediction results for each sample is used for evaluation. The average effect at specific values. For bivariate PDP, the analytical objective is the combined effect of the two variables; in the calculation, other variables are kept constant, and each variable is considered individually. and Take different values ​​and calculate the corresponding model prediction results. The formula for calculating bivariate PDP is: in Indicates division variable and The values ​​of other variables besides the first one The mean of the predicted results for each sample is used to assess their combined effect. After the univariate PDP is calculated, a curve showing the relationship between the variable values ​​and the predicted results can be plotted to analyze the local dependence characteristics of the variables and determine whether there are nonlinear or threshold effects. After the bivariate PDP is calculated, contour plots or 3D surface plots can be plotted to analyze how the two variables jointly affect the predicted results and reveal the interaction patterns between the variables.

[0072] Example 2 This embodiment provides a system for predicting lake and reservoir water quality based on multi-source data fusion, including a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of any of the methods in Embodiment 1.

[0073] This embodiment also provides a specific implementation example of a lake / reservoir water quality prediction method based on multi-source data fusion, as shown in Embodiment 1, including: Taking Reservoir A as the target, this study focuses on predicting turbidity and chlorophyll a at points in the water transfer zone (SS) to support daily reservoir operation and water quality risk early warning. 1. Data Processing and Feature Construction: The data selected were monitoring data from three locations (SS, SN, and XN) from May 2016 to December 2023, totaling 760,000 valid records, covering 19 water quality indicators, 6 meteorological indicators, and 3 hydrological indicators. The data were standardized and transformed using logarithmic rate of return according to steps 1 and 2 to ensure that the data met the stationarity requirements. 2. Feature Construction and Screening: Following step 2, time-series features and derived features were constructed. After screening using random forest, 12 key features for turbidity prediction were identified, including Wind-ac, TP, WL, and NH3-N-3d. 15 key features for chlorophyll a prediction were identified, including WT, PAR-ac, WL-7d, and pH-2d. 3. Model Building and Validation: Configure the parameters of the four types of models as in step 3, and complete the construction of RF, SVM, LSTM, and Informer models; introduce TPA and Bi-LSTM structures to complete the construction of TPA-Bi-LSTM models; 4. Multi-model fusion: Build the SVM-Informer-LSTM fusion framework according to step 4, determine the output weights of the basic model and the integration rules; use data from January to December 2023 as the validation set, output hourly prediction results for 24 hours through rolling prediction, and compare the prediction results of the basic model and the fusion model.

[0074] 5. Implementation of interpretability analysis: Perform sensitivity analysis according to step 5, calculate the SHAP value and Sobol sensitivity index of each feature, and determine key driving factors such as water level, water temperature, and cumulative wind value; Perform dependency analysis according to step 5: Plot univariate PDP curves and bivariate dependency heatmaps according to step 5, and analyze the influence patterns and synergistic mechanisms of key factors on water quality indicators.

[0075] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for predicting lake and reservoir water quality based on multi-source data fusion, characterized in that, include: Step 1: Acquire multi-source, multi-site data in real time, including lake and reservoir water quality data, meteorological data, and hydrological data, and perform data preprocessing to remove the impact of abrupt changes on the prediction; Step 2: Extract features from the time series data in the multi-source, multi-site data using a temporal attention mechanism. Simultaneously process the time series data in the multi-source, multi-site data using two independent long short-term memory neural network structures (forward and backward) to obtain the predicted value of the target indicator. The multi-site data in the multi-source multi-site data is analyzed, and the upstream station information is obtained by combining the spatial location of the online monitoring equipment and the direction of water flow in the reservoir, so as to further obtain the trend change information of the target indicator. Step 3: The predicted value and trend change information are simultaneously input into the pre-built meta-learner. By learning the combination relationship between the short-term prediction information and the long-term trend prediction information of the target indicator under multi-source and multi-site conditions, the final prediction result of the target indicator is output.

2. The method for predicting lake and reservoir water quality based on multi-source data fusion according to claim 1, characterized in that, The time series data in the multi-source, multi-site data is analyzed by the fusion model, which outputs the predicted value of the target indicator. The fusion model is obtained by combining a pre-built support vector machine model, an optimized long short-term memory neural network model, and a pre-built long-sequence time series prediction model. The support vector machine model is used to process the data of the target site to remove the impact of mutations on the prediction; the long-sequence time series prediction model is used to analyze the multi-site data in the multi-source multi-site data to obtain the trend change information of the target indicator. The optimization process of the optimized long short-term memory neural network model specifically includes: The temporal attention mechanism is applied to the long short-term memory neural network model to calculate the attention score at each time step and weight and aggregate the time step features to focus on key temporal information. The specific calculation formula is as follows: in, and It is a learnable parameter matrix. It is the projection vector. It is a bias term. This represents the degree of influence of time step 𝑖 on time step 𝑡. This represents the time step characteristics optimized by the TPA mechanism. For the current time step Corresponding original features Original time step Corresponding features Indicates time step The degree of influence on time step 𝑡; Time-series data is processed simultaneously using two independent long short-term memory neural network structures, one forward and one backward, enabling bidirectional information propagation and state merging. The calculation formula includes: in, and These represent the hidden states of the forward and backward LSTMs, respectively. For the current time step The corresponding input feature vector, This indicates that the forward LSTM is at time step The hidden state, This indicates that the backward LSTM is at time step The hidden state, Indicates the current time step The final hidden state.

3. The method for predicting lake and reservoir water quality based on multi-source data fusion according to claim 2, characterized in that, Constructing an input feature set containing long-term window data and cumulative variables to train the support vector machine model, long short-term memory neural network model, and long-sequence time series prediction model specifically includes: The stationarity of the target indicators is analyzed to assess the trend and stability of the time series; the long memory of the data is analyzed, and feature transformation or differencing is performed on data that are non-stationary or lack long memory; further, the moving average method is used to analyze the correlation between the water quality indicators of each monitoring point and the predictive variables of the target points under different time windows, and a long-term window dataset is constructed based on the selected data with long memory. The cumulative variables specifically include cumulative illumination variables and cumulative wind speed variables; based on Beer-Lambert's law, the cumulative illumination variables are obtained by combining the cumulative illumination values, attenuation weights, and turbidity for different time windows. By projecting wind speeds onto the east-west and north-south axes respectively, and weighting and accumulating historical time-series wind speed data, the accumulated wind speed variable is obtained to characterize the impact of the wind field on different time scales.

4. The method for predicting lake and reservoir water quality based on multi-source data fusion according to claim 3, characterized in that, The stationarity analysis of the target index was performed using the ADF test, specifically including: Calculate the ADF statistic and compare it with the Dickey-Fuller critical value: if the ADF statistic is less than the Dickey-Fuller critical value, the time series is considered stationary; otherwise, the time series is considered non-stationary. The calculation formula is as follows: in, The unit root coefficient, The lag order is... This is the white noise error term. The unit root coefficient, This is an estimate of the unit root coefficient. Its standard error, For the intercept term, For time trend items, Time series The first lag value, Represents the first-order difference. Lagged difference explanatory variables is the coefficient of the lagged difference term.

5. The method for predicting lake and reservoir water quality based on multi-source data fusion according to claim 3, characterized in that, The stationarity analysis of the target index was performed using the KPSS test, specifically including: Calculate the KPSS statistic and compare it with a critical value. If the KPSS statistic is greater than the critical value, the time series is considered non-stationary; otherwise, the time series is considered stationary. The calculation formula is as follows: in, It is a random walk process. Deterministic trend term For error terms, It is a time series. For the sample size, For the cumulative sum of residuals, This is the variance estimate for the error term.

6. The method for predicting lake and reservoir water quality based on multi-source data fusion according to claim 3, characterized in that, The long memory property of the data was analyzed using the autocorrelation function, partial autocorrelation function, and power spectral density test, specifically including: The autocorrelation function is used to calculate and analyze the relationship between the current data point and the past data. The correlation between data points over a period is calculated using the following formula: in, Indicates lag The autocorrelation coefficient of the period, The mean of the time series. For the current data point lag, The mean of the time series; The partial autocorrelation function is used to calculate and analyze the direct correlation of time series under different lag orders. The calculation formula is as follows: in, Lagging The partial autocorrelation coefficient of order, The autocovariance is the result of lag order n. The variance is without lag. , Lagging Rank, The autocovariance of the order, For the front The autoregressive coefficient of order, Index for lag order; The power spectral density test is used to quantify the energy distribution of data at different frequencies to identify the long memory of the data. The calculation formula is as follows: in, It is frequency Fourier coefficients at the location, The length of the time series. The imaginary unit, Represents frequency The power intensity at a given point reflects the contribution of that frequency component to the time series.

7. The method for predicting lake and reservoir water quality based on multi-source data fusion according to claim 1, characterized in that, Step two also includes: The final input feature set is divided into a training set, a test set, and a validation set. The data is then standardized to a distribution with a mean of 0 and a standard deviation of 1, while retaining the original data distribution and ensuring that all data fall within a specific interval. The calculation formula is as follows: in, The original value, The average value of the indicator. The standard deviation of the indicator. This is the normalized value; Next, the data is transformed using changes in logarithmic returns to eliminate the influence of trends. The calculation formula is as follows: in, This represents the logarithmic return at time point X. and They represent time respectively and The original step data values ​​at each moment; Furthermore, using mean squared error as the loss function, a multi-parameter grid search method is employed to screen the optimal hyperparameter combination of pre-built random forest model, support vector machine model, long short-term memory neural network model, and long sequence time series prediction model.

8. The method for predicting lake and reservoir water quality based on multi-source data fusion according to claim 1, characterized in that, Step three also includes: analyzing the sensitivity of the fusion model, calculating the independent contribution of each input variable to the model prediction results under different combinations and the global contribution of each input variable; further analyzing the dependency of the fusion model to reveal the interaction patterns between variables.

9. A method for predicting lake and reservoir water quality based on multi-source data fusion according to claim 8, characterized in that, The process of analyzing the sensitivity of the fusion model specifically includes: The sensitivity of the fusion model was analyzed using Shapley value analysis, specifically including: The independent contribution of each input variable to the model prediction result under different combinations is calculated using the following formula: in, Indicates inclusion The prediction results Represents variables, Represents a subset of variables. Represents a subset of variables The corresponding model prediction value, Representing variables The degree of contribution; The weighted summation of the contributions of each variable is calculated using the following formula: in, Representing variables SHAP value, This represents the total number of all input variables involved in the analysis. The sensitivity of the fusion model is further analyzed using the Sobol global sensitivity analysis method. The global contribution of each input variable is calculated using the following formula: in, Represent a single variable Variance of contribution Representing variables and The joint variance contribution, This reflects the interaction between multiple variables. Reflecting variables The direct impact on the model output. As variables, The overall sensitivity index to the model output, where 𝑌 is the model output, Indicates model output The total variance; The dependencies of the fusion model are analyzed using partial dependency analysis: The partial dependency analysis method includes univariate partial dependency analysis and bivariate partial dependency analysis; the univariate partial dependency analysis method is used to calculate the mean of the model output for different values ​​of the input variable, and the calculation formula is as follows: in, Indicates except The values ​​of other variables besides the first one The mean of the prediction results for each sample is used for evaluation. The average effect under specific values; The combined effect of the two variables is analyzed using the bivariate partial dependency analysis method, and the calculation formula is as follows: in, Indicates division variable and The values ​​of other variables besides the first one The mean of the predicted results for each sample is used to assess the combined effect between variables; Based on the results of sensitivity and dependency analysis, key variables affecting the prediction accuracy of the fusion model are screened to guide the construction of the input feature set, and the fusion model is trained based on the newly constructed input feature set.

10. A system for predicting lake and reservoir water quality based on multi-source data fusion, characterized in that, It includes a memory and a processor, the memory storing a computer program, the processor invoking the computer program to perform the steps of the method as described in any one of claims 1 to 9.

Images