Method for training water quality prediction model, water quality prediction method, and program product
By using multi-sub-model fusion and KL divergence weight adjustment, the problem of low water quality prediction accuracy was solved, and more accurate and adaptable water quality prediction was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 3CLEAR SCI & TECH CO LTD
- Filing Date
- 2026-02-04
- Publication Date
- 2026-06-16
AI Technical Summary
Existing technologies for water quality prediction are not accurate enough and cannot adapt to the complex changing characteristics of the water environment, especially its nonlinear and periodic fluctuation characteristics.
Multiple sub-models (such as LSTM, Prophet, and XGBoost) are used for water quality prediction. The prediction reliability of each sub-model is measured by KL divergence, the weight of each sub-model is determined, and the prediction data are fused together when the training stops to form the target water quality prediction model.
It improves the accuracy and adaptability of water quality forecasting, enabling a more comprehensive capture of the complex changing characteristics of water quality and enhancing the generalization ability of cross-basin forecasting.
Smart Images
Figure CN122221933A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of water environment governance technology, specifically to a training method for a water quality prediction model, a water quality prediction method, and a program product. Background Technology
[0002] With the acceleration of industrialization and urbanization, water pollution and algal blooms are becoming increasingly frequent, placing high demands on the accuracy and timeliness of water quality monitoring and forecasting. Water quality forecasting predicts the water quality of the environment over a future period, enabling timely and appropriate responses to changes in water quality. However, the accuracy of water quality forecasts obtained using related technologies is not high. Summary of the Invention
[0003] The purpose of this disclosure is to provide a training method for a water quality prediction model, a water quality prediction method, and a program product to improve the accuracy of water quality prediction for the aquatic environment.
[0004] To achieve the above objectives, in a first aspect, this disclosure provides a method for training a water quality prediction model, the method comprising:
[0005] Obtain a training sample set, wherein the training samples in the training sample set include sample data and labeled data, wherein the sample data includes first environmental data of the first water environment in the first time period, and the labeled data includes monitoring data of preset water quality indicators of the first water environment at multiple first moments in the second time period. The sample data is preprocessed to obtain the first information; The first information is input into multiple sub-models in the water quality prediction model to obtain the first predicted data of the preset water quality index of the first water environment at the multiple first times, respectively output by the multiple sub-models. Based on the KL divergence corresponding to each of the multiple sub-models, the first weight of each of the multiple sub-models is determined, wherein the weight of the sub-model is used to characterize the prediction confidence of the sub-model, and the KL divergence corresponding to the sub-model is obtained based on the probability distribution of the monitoring data and the probability distribution of the first prediction data output by the sub-model. A first prediction result is determined based on the first weights of each of the multiple sub-models and the first prediction data output by each of the multiple sub-models. Under the condition that training stops, the target water quality prediction model and the target weights of each of the multiple target sub-models in the target water quality prediction model are obtained.
[0006] Optionally, the plurality of sub-models includes a first sub-model, a second sub-model, and a third sub-model. The first sub-model is a Long Short-Term Memory (LSTM) network, the second sub-model is a Prophet model, and the third sub-model is an Extreme Gradient Boosting (XGBoost) model. The second sub-model includes a periodic term of the preset water quality index, which is used to characterize the changing trend of the preset water quality index.
[0007] Optionally, determining the first weight of each of the multiple sub-models based on the KL divergence corresponding to each of the multiple sub-models includes: Based on the KL divergence corresponding to the multiple sub-models, determine the second weight of each of the multiple sub-models; For each first time point, first fusion information is determined based on the second weights of each of the plurality of sub-models, the first prediction data output by each of the plurality of sub-models at the first time point, and the average prediction deviation of each of the plurality of sub-models corresponding to the first time point. For each first time point, second fusion information is determined based on the second weights of each of the plurality of sub-models, the monitoring data at the first time point, and the prediction standard deviations of each of the plurality of sub-models, wherein the prediction standard deviation of the sub-model is the standard deviation of the first prediction data at the plurality of first time points output by the sub-model; If, based on the first fusion information and the second fusion information corresponding to each of the first moments, it is determined that the preset conditions are met, then the second weight of the sub-model is taken as the first weight of the sub-model.
[0008] Optionally, determining the second weights of each of the multiple sub-models based on their respective KL divergences includes: If the number of specified sub-models is one, the second weight of the specified sub-model is determined as a preset weight, and the second weights of other sub-models besides the specified sub-model are determined according to the KL divergence corresponding to the multiple sub-models respectively. The specified sub-model includes the sub-model corresponding to the first prediction data that predicts the existence of water quality abnormalities in the first water environment during the second time period. If the number of specified sub-models is multiple and includes the first sub-model, then the second weight of the first sub-model is determined as the preset weight, and the second weights of other sub-models besides the first sub-model are determined according to the KL divergence corresponding to the multiple sub-models respectively. The first sub-model is a Long Short-Term Memory (LSTM) network.
[0009] Optionally, the second weight of each sub-model is determined based on the KL divergence corresponding to the multiple sub-models using the following formula:
[0010] in, This represents the probability distribution of the first predicted data output by the m-th sub-model. This represents the probability distribution of the monitoring data. Denotes KL divergence, This represents the preset coefficients, and M represents the number of sub-models.
[0011] Optionally, there are multiple preset water quality indicators, and the method further includes: If the training stop condition is not met, the designated sub-model is trained based on the monitoring data of multiple preset water quality indicators with coupling relationship and the first prediction data of multiple preset water quality indicators with coupling relationship output by the designated sub-model. The designated sub-model includes a sub-model that predicts the existence of water quality abnormalities in the first water environment during the second time period based on the corresponding first prediction data.
[0012] Optionally, the first environmental data includes hydrological data of the first water environment at multiple second moments within the first time period, and meteorological data at multiple third moments within the first time period, wherein the multiple third moments are a subset of the multiple second moments, and the sample data also includes the flow velocity of the first upstream water environment of the first water environment at the multiple second moments. The preprocessing of the sample data to obtain the first information includes: Identify abnormal data in the first environmental data and replace the abnormal data to obtain second environmental data of the first water environment within the first time period; Interpolate the meteorological data at the multiple third times in the second environmental data to obtain the meteorological data at the multiple second times. Determine the weighting values of the coupled meteorological and / or hydrological data at the same second time point; For each second time moment, based on the flow velocity at the second time moment and the distance between the first upstream water environment and the first water environment, the predicted arrival time of pollutants from the first upstream water environment to the first water environment at the second time moment is determined. The first information is obtained based on the weighted value, the predicted arrival time, the meteorological data at the multiple second times, and the hydrological data at the multiple second times in the second environmental data.
[0013] Optionally, determining the abnormal data in the first environmental data and replacing the abnormal data includes: The isolated forest algorithm is used to detect anomalous single data points in the first environmental data. For the hydrological data in the first environmental data, based on the standard deviation of the hydrological data at the multiple second time points, abnormal hydrological data corresponding to a first preset number of consecutive second time points are determined. For the meteorological data in the first environmental data, based on the standard deviation of the meteorological data at the multiple third times, determine the abnormal meteorological data corresponding to the second preset number of consecutive third times; Based on environmental data of other water environments within the first time period that are less than a preset distance threshold from the first water environment, the abnormal single-point data, the abnormal hydrological data, and the abnormal meteorological data are replaced.
[0014] Secondly, this disclosure provides a method for predicting water quality in aquatic environments, the method comprising: Acquire first data, which includes third environmental data of the second water environment during the third time period; The first data is preprocessed to obtain the second information; The second information is input into multiple target sub-models in the target water quality prediction model to obtain the second prediction data of the preset water quality indicators of the second water environment at multiple fourth times in the fourth time period, respectively output by the multiple target sub-models. Based on the target weights of the plurality of target sub-models and the second prediction data output by the plurality of target sub-models respectively, the target prediction results of the preset water quality indicators of the second water environment at the plurality of fourth times are determined, wherein the target water quality prediction model and the target weights of the plurality of target sub-models are obtained according to the training method of the water quality prediction model provided in the first aspect.
[0015] Thirdly, this disclosure provides a computer program product that, when executed by a processor, implements the steps of the training method for the water quality prediction model provided in the first aspect.
[0016] Fourthly, this disclosure provides a computer program product that, when executed by a processor, implements the steps of the water quality prediction method for the aquatic environment provided in the second aspect.
[0017] Through the above technical solution, the water quality prediction model can include multiple sub-models. These sub-models can be of different types and can predict water quality from different perspectives. Furthermore, based on the KL divergence of each sub-model, a first weight is determined for each sub-model. The KL divergence measures the difference between the probability distribution of the first predicted data output by the sub-model and the probability distribution of the labeled monitoring data, thus measuring the prediction reliability of the sub-model. When the training stopping condition is met, the target water quality prediction model and the target weights of each of the target sub-models within it can be obtained. In this way, the target water quality prediction model can be used for water quality prediction in the aquatic environment, and the target weights can provide a basis for fusing the prediction data output by each sub-model during water quality prediction, improving the accuracy of the fused prediction results and thus enhancing the accuracy of water quality prediction in the aquatic environment.
[0018] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description
[0019] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating an exemplary method for training a water quality prediction model.
[0020] Figure 2 This is an exemplary flowchart illustrating a method for determining the first weights of multiple sub-models based on the KL divergence corresponding to each sub-model.
[0021] Figure 3 This is a flowchart illustrating an exemplary method for preprocessing sample data.
[0022] Figure 4 This is a flowchart illustrating an exemplary method for predicting water quality in aquatic environments.
[0023] Figure 5 This is a schematic diagram illustrating an exemplary method for predicting water quality in aquatic environments. Detailed Implementation
[0024] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.
[0025] Among related technologies, there are predictive models for water quality prediction in the aquatic environment, such as the Autoregressive Integrated Moving Average (ARIMA) model or deep learning models. However, the predictive models in these technologies typically employ a single architecture, which cannot adapt to the complex water quality change characteristics of the aquatic environment, such as nonlinearity and periodic fluctuations.
[0026] This disclosure provides a training method for a water quality prediction model, a water quality prediction method, and a program product to improve the accuracy of water quality prediction.
[0027] Figure 1 This is a flowchart illustrating an exemplary method for training a water quality prediction model, such as... Figure 1 As shown, the training method for the water quality prediction model includes steps 11 to 16.
[0028] Step 11: Obtain the training sample set.
[0029] The training sample set includes sample data and labeled data. The sample data includes the first environmental data of the first water environment within the first time period, and the labeled data includes the monitoring data of preset water quality indicators of the first water environment at multiple first moments within the second time period. Among these multiple first moments, the earliest moment is later than the latest moment within the first time period, meaning the second time period is after the first time period.
[0030] The primary environmental data may include hydrological and meteorological data of the primary water environment within a primary time period. Hydrological data may include flow rate and water level data collected by sensors installed in the water environment. Meteorological data may include rainfall, temperature, air pressure, and wind speed data for the area where the primary water environment is located. Additionally, the primary environmental data may include historical water pollution information for the primary water environment. This historical information can be used to characterize past pollution events in the primary water environment, such as the time of the pollution event, the indicators exceeding standards, and the concentration of the exceeding indicators. In other words, the primary environmental data can be multi-source data to utilize multi-source characteristics for water quality prediction.
[0031] The training sample set can contain multiple training samples, which may include training samples from different water environments and different time periods. For example, training sample 1 may contain the first environmental data of water environment 1 within time period 1, and training sample 2 may contain the first environmental data of water environment 2 within time period 2. The water environment is one of the basic elements constituting the environment, encompassing various water bodies, including surface water and groundwater, such as rivers, lakes, reservoirs, and oceans.
[0032] The term "first water environment" is used to distinguish it from "second water environment" in the following text, but is not intended to limit it to a specific water environment. Similarly, the term "first time period" is used to distinguish it from "second time period," "third time period," and "fourth time period" in the following text, but is not intended to limit it to a specific time period.
[0033] In one embodiment, the training samples in the training sample set may include data from different water environments belonging to multiple watersheds. Because the characteristics of the water environments in different watersheds differ, the model can learn the water quality characteristics and water quality change characteristics of the water environments in different watersheds. Therefore, the trained target water quality prediction model can be applied to water quality prediction in different watersheds, improving the generalization ability of the target water quality prediction model for cross-watershed prediction and achieving adaptation to different watershed scenarios.
[0034] Step 12: Preprocess the sample data to obtain the first information.
[0035] For example, preprocessing may include missing value imputation, outlier detection, outlier replacement, etc., to make the first information obtained after preprocessing more complete.
[0036] Step 13: Input the first information into multiple sub-models in the water quality prediction model to obtain the first predicted data of the preset water quality indicators of the first water environment at multiple first moments, output by the multiple sub-models respectively.
[0037] In this disclosure, the water quality prediction model may include multiple sub-models, which may be different types of models. The multiple sub-models can predict water quality from different perspectives, thereby integrating the prediction data output by multiple sub-models to improve the accuracy of the prediction results.
[0038] Preset water quality indicators can include, for example, ammonia nitrogen, total phosphorus, COD (chemical oxygen demand), BOD (biochemical oxygen demand), and dissolved oxygen. Multiple preset water quality indicators can be used, meaning that data for multiple preset water quality indicators can be predicted simultaneously.
[0039] In one embodiment, the multiple sub-models include a first sub-model, a second sub-model, and a third sub-model. The first sub-model is a Long Short-Term Memory (LSTM) network, the second sub-model is a Prophet model, and the third sub-model is an Extreme Gradient Boosting (XGBoost) model.
[0040] For example, the first sub-model could be a Bidirectional Long Short-Term Memory (Bi-LSTM) network, which is an improved recurrent neural network (RNN). Bidirectional LSTM is capable of nonlinear modeling, capturing complex nonlinear relationships, and has high prediction accuracy for data with temporal relationships.
[0041] The second sub-model includes periodic terms for preset water quality indicators, which characterize the changing trends of these indicators. The Prophet model can fit and predict time-series data by additively decomposing the changing trends, describing the long-term trends and capturing periodic fluctuations. Water quality data exhibits certain periodic fluctuation characteristics, such as annual cycles (seasonal fluctuations), weekly cycles (e.g., reduced industrial wastewater discharge at weekends), and daily cycles (e.g., diurnal changes in dissolved oxygen). Periodic terms can include annual, weekly, and daily periodic terms. The Prophet model can express the periodic fluctuations of the data using Fourier series; for example, the annual periodic term can use a 5th-order Fourier series, and the weekly periodic term can use a 7th-order Fourier series. The 5th and 7th orders refer to the order of the Fourier series. The second sub-model can include periodic terms corresponding to various preset water quality indicators.
[0042] The third sub-model can be the XGBoost model. The XGBoost model can quantify the importance of features. Feature importance can characterize the contribution of features in water quality prediction, such as the contribution of rainfall, thereby screening out key factors that have a greater impact on water quality.
[0043] Step 14: Determine the first weight of each of the multiple sub-models based on the KL divergence corresponding to each sub-model.
[0044] In this embodiment, the weights of the sub-models characterize their prediction reliability. The Kullback-Leibler divergence (KL divergence) of the sub-model is obtained based on the probability distribution of the monitoring data and the probability distribution of the first predicted data output by the sub-model. KL divergence is a metric used to measure the difference between two probability distributions. In this disclosure, the KL divergence of the sub-model is determined using the probability distributions of the monitoring data and the first predicted data output by the sub-model. KL divergence measures the difference between the first predicted data output by the sub-model and the labeled monitoring data, thereby measuring the prediction reliability of the sub-model. A higher KL divergence indicates lower prediction reliability. In one embodiment, multiple monitoring data points at first time points can be normalized to a probability distribution summing to 1, and the multiple first predicted data points output by the sub-model at first time points can be normalized to a probability distribution summing to 1.
[0045] Step 15: Determine the first prediction result based on the first weights of each of the multiple sub-models and the first prediction data output by each of the multiple sub-models.
[0046] Specifically, for each first moment, the first prediction result for that first moment can be obtained by weighting and summing the first weights of each of the multiple sub-models and the first prediction data for that first moment output by the multiple sub-models. The first prediction result for that first moment can represent the preset water quality index data of the first water environment predicted by the water quality prediction model at that first moment.
[0047] Step 16: If the training stopping condition is met, obtain the target water quality prediction model and the target weights of each of the multiple target sub-models in the target water quality prediction model.
[0048] For example, the training termination condition can be met by completing the traversal of all training samples in the training sample set, or by the difference between the first prediction result and the monitoring data being less than or equal to a preset threshold, meaning the prediction accuracy of the water quality prediction model meets the requirements. The difference between the first prediction result and the monitoring data can be the average of the differences between the first prediction result and the monitoring data at each first time point.
[0049] Through the above technical solution, the water quality prediction model can include multiple sub-models. These sub-models can be of different types and can predict water quality from different perspectives, thus more comprehensively capturing the complex changing characteristics of water quality. Furthermore, based on the KL divergence corresponding to each sub-model, the first weight of each sub-model is determined. The KL divergence measures the difference between the probability distribution of the first predicted data output by the sub-model and the probability distribution of the labeled monitoring data, thereby measuring the prediction reliability of the sub-model. When the training stopping condition is met, the target water quality prediction model and the target weights of each of the target sub-models within it can be obtained. In this way, the target water quality prediction model can be used for water quality prediction in the aquatic environment, and the target weights can provide a basis for fusing the prediction data output by each sub-model during water quality prediction, improving the accuracy of the fused prediction results and thus enhancing the accuracy of water quality prediction in the aquatic environment.
[0050] Figure 2 This is an exemplary flowchart illustrating a method for determining the first weights of multiple sub-models based on their respective KL divergences, as shown below. Figure 2 As shown, step 14 includes steps 141 to 144.
[0051] Step 141: Determine the second weight of each of the multiple sub-models based on the KL divergence corresponding to each sub-model.
[0052] For example, based on the KL divergence corresponding to each of the multiple sub-models, the second weight of the sub-model can be determined by the following formula (1): (1) Let M represent the probability distribution of the first predicted data output by the m-th sub-model, and M represent the number of sub-models. This represents the probability distribution of the monitoring data. Denotes KL divergence, This indicates the preset coefficient. It can be used as a smoothing coefficient. The value can be preset, for example, it can be a value in the range [0.3, 0.7], and the preferred value can be 0.5.
[0053] Step 142: For each first time point, determine the first fusion information based on the second weights of each of the multiple sub-models, the first prediction data output by each of the multiple sub-models at the first time point, and the average prediction deviation of each of the multiple sub-models at the first time point.
[0054] For example, the first fusion information can be determined by the following formula (2): (2) This represents the first fusion information corresponding to the first moment. This represents the first predicted data at the first time step of the output of the m-th sub-model. For the preset compensation coefficient, The value can be preset, for example, it can be a value in the range [0.1, 0.3]. This represents the average prediction deviation of the m-th sub-model corresponding to the first time step. This represents the second weight of the m-th sub-model.
[0055] For example, the average prediction deviation of the m-th sub-model corresponding to the first time point can be the average of the prediction deviations of the m-th sub-model for the first time point in the second time period and for the N consecutive first times point before the first time point. The prediction deviation can be the absolute value of the difference between the monitoring data and the first prediction data.
[0056] For example, multiple first time points include time points T1, T2, T3, etc. The average prediction deviation of the m-th sub-model corresponding to time T1 can be the prediction deviation at time T1, which is the absolute value of the difference between the monitoring data at time T1 and the first predicted data at time T1 output by the m-th sub-model. The prediction deviations at other time points are calculated similarly. The average prediction deviation of the m-th sub-model corresponding to time T2 can be the average of the prediction deviations at time T1 and time T2. The average prediction deviation of the m-th sub-model corresponding to time T3 can be the average of the prediction deviations at time T1, time T2, and time T3. 25After a certain time, N can be 24. Taking the interval between two adjacent first times as 1 hour as an example, the average prediction deviation can be the average of the prediction deviations over 24 hours.
[0057] Step 143: For each first time point, determine the second fusion information based on the second weights of each of the multiple sub-models, the monitoring data at the first time point, and the prediction standard deviations of each of the multiple sub-models.
[0058] For example, the second fusion information can be determined by the following formula (3): (3) This represents the second fusion information corresponding to the first moment. This represents the monitoring data at the first moment. This represents the prediction standard deviation of the m-th sub-model. 1.96 is the preset value, corresponding to a 95% confidence level.
[0059] Here, the prediction standard deviation of the sub-model is the standard deviation of the first prediction data at multiple first time points output by the sub-model. For example, the number of first time points is R (from time T1 to T...). R (at time point), the prediction standard deviation of the m-th sub-model can be the standard deviation of the first prediction data at the first time point of the R outputs of the m-th sub-model.
[0060] Step 144: If the preset conditions are met based on the first fusion information and the second fusion information corresponding to each first time point, then the second weight of the sub-model is taken as the first weight of the sub-model.
[0061] The first fusion information can represent the data obtained by fusing the first predicted data output by the sub-model and the average prediction deviation of the sub-model using the second weight; that is, the data after bias compensation for the predicted data of the sub-model. The second fusion information can represent the data obtained by fusing the labeled monitoring data and the prediction standard deviation of the sub-model using the second weight; that is, the data after compensating for the prediction standard deviation of the monitoring data. When the second weight is accurate, the difference between the first and second fusion information should be sufficiently small. Therefore, if the difference between the first and second fusion information meets the preset conditions, it indicates that the second weight of the sub-model is appropriate, and the second weight of the sub-model can be used as the first weight of the sub-model.
[0062] Specifically, for each first time point, the absolute value of the difference between the first fused information and the second fused information corresponding to that first time point can be calculated. The average of the absolute values of the differences between the fused information corresponding to each first time point is taken as the difference information between the two fused information points. The difference information can satisfy a preset condition if the difference information is less than a preset difference value.
[0063] If, based on the first fusion information and the second fusion information corresponding to each first moment, it is determined that the preset conditions are not met, the process can return to step 13, that is, to re-predict the data using multiple sub-models and calculate new second weights until the first fusion information and the second fusion information meet the preset conditions.
[0064] Using the above technical solution, KL divergence can measure the difference between the first predicted data output by the sub-model and the labeled monitoring data, thereby accurately measuring the prediction reliability of the sub-model. The second weights can be verified using the first and second fusion information to obtain the accurate first weights for each sub-model.
[0065] In one embodiment, step 141, determining the second weights of each of the multiple sub-models based on their respective KL divergences, may include: If the number of specified sub-models is one, the second weight of the specified sub-model is determined as the preset weight, and the second weights of other sub-models besides the specified sub-model are determined according to the KL divergence corresponding to the multiple sub-models respectively. If the specified number of sub-models is multiple and includes the first sub-model, then the second weight of the first sub-model is determined as the preset weight, and the second weights of the other sub-models besides the first sub-model are determined according to the KL divergence corresponding to the multiple sub-models respectively.
[0066] The designated sub-model includes the corresponding first prediction data characterizing the sub-model that predicts the existence of water quality anomalies in the first water environment during the second time period.
[0067] For example, the second time period can be divided into multiple sub-time periods, using the formula The rate of change of the first predicted data output by the m-th sub-model within the sub-period can be determined. This represents the change in the first predicted data output by the m-th sub-model within the sub-period. Indicates the duration of the sub-period.
[0068] If the rate of change of the first predicted data output by the m-th sub-model within any sub-time period is greater than , which can represent the first predicted data output by the m-th sub-model, indicating that an anomaly in water quality was predicted in the first water environment during the second time period. This anomaly can be understood as a sudden change in the preset water quality indicators, i.e., a high rate of change. Additionally, This is just an example; it could also be... Other multiples thereof.
[0069] When there is only one designated sub-model, the second weight of the designated sub-model can be set as a preset weight, which can be greater than or equal to 0.6. Since the designated sub-model predicts water quality anomalies, these anomalies need to be given priority. Therefore, the second weight of the designated sub-model can be set to a higher value, meaning that the weight of the designated sub-model is no longer calculated using formula (1) and can be set as a preset weight. The second weights of other sub-models besides the designated sub-model can be calculated using formula (1).
[0070] If there are multiple specified sub-models, including the first sub-model, then the second weight of the first sub-model is determined as the preset weight. Since LSTM has higher prediction reliability, the second weight of the first sub-model can be determined as the preset weight. If there are multiple specified sub-models, excluding the first sub-model, the second weight of one of the specified sub-models can be randomly determined as the preset weight.
[0071] It should be noted that in this disclosure, the sum of the second weights of each of the multiple sub-models is 1. After setting the second weight of a specified sub-model to a preset weight, the weights of other sub-models can be proportionally allocated according to the values calculated by formula (1). In addition, if the second weights of multiple sub-models are all calculated by formula (1), the values calculated by formula (1) can be normalized so that the sum of the second weights of each of the multiple sub-models is 1.
[0072] Using the above technical solution, for a designated sub-model that predicts abnormal water quality phenomena, its second weight can be set to a preset weight, that is, the second weight of the designated sub-model is set to a higher value, so as to achieve attention to abnormal water quality phenomena.
[0073] The training method for the water quality prediction model provided in this disclosure may also include: If the training stop condition is not met, the specified sub-model is trained based on the monitoring data of multiple preset water quality indicators with coupling relationship, and the first prediction data of multiple preset water quality indicators with coupling relationship output by the specified sub-model.
[0074] The training stopping condition may not be met if the training samples in the training sample set have not been fully traversed, or if the difference between the first prediction result and the monitoring data exceeds a preset threshold. In this disclosure, if the training stopping condition is not met, training can be performed only on a specified sub-model to improve training speed.
[0075] Among them, LSTM and XGBoost models are typically able to predict water quality anomalies and are used as designated sub-models, while the Prophet model is not good at predicting abrupt changes and is usually not used as a designated sub-model. The Prophet model can be pre-fitted based on monitoring data of preset water quality indicators of the first water environment. For example, the parameters of the Prophet model can be optimized through maximum likelihood estimation so that the periodic terms in the Prophet model can reflect the changing trends of the preset water quality indicators of the first water environment.
[0076] In one embodiment, to improve training efficiency, the designated sub-model can be trained using the first prediction data at the first moment after the designated sub-model predicts the water quality anomaly. For example, there are R first moments in the second time period, and the sub-model predicts the water quality anomaly at moment T. r At any given time, it can be determined based on T. r Time to T R The model is trained using the first predicted data at each first time point.
[0077] For example, a loss value can be determined based on the difference between the first predicted data and the monitoring data of the same preset water quality index at the same first time point, and the difference between the first weighted value and the second weighted value, and the specified sub-model can be trained. The first weighted value is the weighted value of the monitoring data of multiple preset water quality indicators with coupling relationship at the same first time point, and the second weighted value is the weighted value of the first predicted data of multiple preset water quality indicators with coupling relationship at the same first time point. For example, to establish a coupling relationship between COD (chemical oxygen demand) and BOD (biochemical oxygen demand), the weights of COD and BOD can be preset, and the weighted values of COD and BOD can be obtained through the weights, COD data, and BOD data.
[0078] In this way, during model training, the sub-model can learn the correlation between data of multiple preset water quality indicators that have coupling relationships, thereby improving the prediction accuracy of the sub-model.
[0079] The following describes an implementation method for preprocessing sample data in this disclosure. Figure 3 This is a flowchart illustrating an exemplary method for preprocessing sample data, such as... Figure 3 As shown, step 12 includes steps 121 to 125.
[0080] Step 121: Identify the abnormal data in the first environmental data and replace the abnormal data to obtain the second environmental data of the first water environment in the first time period.
[0081] The first environmental data includes hydrological data for the first water environment at multiple second moments within the first time period, and meteorological data at multiple third moments within the first time period, where the multiple third moments are subsets of the multiple second moments. Because sensors collect hydrological data more frequently, while meteorological data is updated relatively less frequently (e.g., sensors collect hydrological data every 5 minutes, and meteorological data is updated every hour), the multiple third moments are subsets of the multiple second moments. For example, the multiple second moments include moments T1', T2', T3', and T4', and the multiple third moments include moments T1' and T3'.
[0082] In one embodiment, step 121 can be implemented as follows: The isolated forest algorithm was used to detect anomalous single data points in the first environmental data.
[0083] Among them, the isolated forest algorithm can be used to detect abnormal single data points in hydrological data at multiple second time points and meteorological data at multiple third time points. The detection method of the isolated forest algorithm can refer to relevant technologies. The isolated forest algorithm can determine the score value of each data point. When the score value is greater than the preset score threshold, it can be identified as an abnormal single data point. The preset score threshold is, for example, 0.65.
[0084] For the hydrological data in the first environmental data, based on the standard deviation of the hydrological data at multiple second time points, determine the abnormal hydrological data corresponding to a first preset number of consecutive second time points.
[0085] Taking a threshold of 3 as an example, if there are 4 consecutive second time points where the corresponding hydrological data are greater than 3 times the standard deviation of the hydrological data, these 4 hydrological data can be identified as abnormal hydrological data.
[0086] For the meteorological data in the first environmental data, based on the standard deviation of the meteorological data at multiple third time points, determine the abnormal meteorological data corresponding to a second preset number of consecutive third time points.
[0087] Taking a threshold of 3 as an example, if five consecutive meteorological data points at the third time point have a value greater than three times the standard deviation of the meteorological data, these five meteorological data points can be identified as abnormal meteorological data. The above-mentioned standard deviation of 3 times is only an example and can be set to other multiples of the standard deviation.
[0088] Based on environmental data from other water environments within the first time period that are less than a preset distance threshold from the first water environment, abnormal single-point data, abnormal hydrological data, and abnormal meteorological data are replaced.
[0089] The weight of the i-th water environment whose distance from the first water environment is less than a preset distance threshold. It can be done calculate, This represents the distance between the first water environment and the i-th water environment. It should be noted that the value calculated by distance can be normalized to obtain the weight. So that the sum of the weights of the other water environments is 1.
[0090] Replacement data can be obtained by weighting other water environment data. Specifically, for example, if the abnormal hydrological data is the value at time T1', replacement data for the abnormal hydrological data can be obtained by weighting other water environment hydrological data at time T1'. Meteorological data is similar.
[0091] Thus, since the distance between other water environments and the first water environment is less than the preset distance threshold, the other water environments have a greater impact on the first water environment. Moreover, the characteristics of the water environments are quite similar, so abnormal data can be replaced based on the environmental data of other water environments.
[0092] In addition, before step 121, missing data in the first environment data can be filled in first, for example, by using a machine learning model to predict missing data.
[0093] Step 122: Interpolate the meteorological data at multiple third time points in the second environmental data to obtain meteorological data at multiple second time points.
[0094] For example, cubic spline interpolation can be used for interpolation. For instance, based on meteorological data at times T1' and T3', meteorological data at times T2' and T4' can be fitted, i.e., interpolation yields meteorological data at times T2' and T4'. In this way, the meteorological data and hydrological data are aligned in time.
[0095] Step 123: Determine the weighting values of meteorological and / or hydrological data that are coupled at the same second time point.
[0096] For example, the coupling relationship can be established in advance, such as establishing a coupling relationship between temperature and wind speed, with the weight of temperature set to 0.7 and the weight of wind speed set to 0.3. The weighted value of temperature and wind speed can be obtained through the weights, temperature data, and wind speed data.
[0097] The establishment of coupling relationships is not limited to this example. Coupling relationships can be established between multiple meteorological data, multiple hydrological data, or even between meteorological and hydrological data. This allows for the effective use of coupled data for water quality forecasting, capturing the combined effects of coupled data on water quality, and improving forecast accuracy.
[0098] Step 124: For each second time moment, based on the flow velocity at the second time moment and the distance between the first upstream water environment and the first water environment, determine the predicted arrival time of the pollutants from the first upstream water environment to the first water environment at the second time moment.
[0099] The sample data also includes the flow velocity of the upstream water environment of the first water environment at multiple second time points, and the concentration of pollutants in the upstream water environment of the first water environment at multiple second time points, including pollutants such as total phosphorus and ammonia nitrogen. Since the upstream water environment has a significant impact on the water quality of the first water environment, and due to water flow transport, it takes a certain amount of time for pollutants from the upstream water environment to be transported to the first water environment; therefore, it can be determined using the formula... Sure, The timestamp for the second moment is the distance between the first upstream water environment and the first water environment. Let be the flow velocity of the first upstream water environment at the second moment. For example, if the pollutant concentration of the first upstream water environment is monitored at point 4, based on the distance and flow velocity, the time when the pollutants arrive at the first water environment is predicted to be point 6. That is, there is a certain time delay in the impact of the upstream on the downstream. Taking the time delay into account can make a more accurate prediction of the water quality of the first water environment.
[0100] Step 125: Obtain the first information based on the weighted value, the predicted arrival time, meteorological data at multiple second time points, and hydrological data at multiple second time points in the second environmental data.
[0101] This disclosure can also determine the degree of influence of meteorological data and hydrological data at each second time point on preset water quality indicators at each first time point, based on meteorological data at each second time point and hydrological data at each second time point in the second environmental data.
[0102] In this context, considering that the further away from the first moment when water quality is to be predicted, the smaller the impact on the water quality at that first moment, the degree of influence of hydrological and meteorological data on the preset water quality indicators at each first moment can be determined. For example, the degree of influence can be 0.95 raised to the power of k, where k is the number of hours between the second moment and the first moment. For instance, if the second moment is 2.5 hours from the first moment, and the number of hours is 2, the degree of influence of the meteorological and hydrological data at that second moment on the preset water quality indicators for that first moment can be obtained. The first information may also include the degree of influence, serving as the basis for the sub-model's water quality prediction.
[0103] The above technical solutions can preprocess the first environmental data to obtain the first information, avoid the situation where outliers cause prediction errors, align meteorological data and hydrological data in time, and enable the model to learn the correlation between data with coupling relationship, so as to improve the accuracy of water quality prediction.
[0104] In one embodiment, the first information can be feature extracted by a gated recurrent unit (GRU).
[0105] Assuming the first time period lasts 72 hours, and the interval between any two adjacent second time periods is 5 minutes, aligning the meteorological and hydrological data temporally yields 864 data points. The input feature dimension is, for example, 15, representing 15 types of data, such as water level, wind speed, and predicted arrival time. The GRU extracts associated features through its reset and update gates, compressing the output feature dimension to 8 dimensions. The GRU output features can then be used as input to LSTM and XGBoost models.
[0106] In addition, the water quality prediction model can be retrained periodically, for example, every month, using the environmental data and water quality monitoring data of that month to retrain the model in order to maintain the model accuracy.
[0107] This disclosure also provides a method for predicting water quality in aquatic environments. Figure 4 This is a flowchart illustrating an exemplary method for predicting water quality in aquatic environments, such as... Figure 4 As shown, the water quality prediction method for the water environment includes steps 41 to 44.
[0108] Step 41: Obtain the first data. The first data includes the third environmental data of the second water environment during the third time period.
[0109] It should be noted that the second water environment is used to distinguish the water environment in the model training stage and the model application stage from the first water environment mentioned above. In some embodiments, the first water environment and the second water environment can also be the same. For example, after using the data of water environment 1 and water environment 2 to train the model and obtain the target water quality prediction model, the target water quality prediction model can be used to predict the water quality of water environment 1 and water environment 2 in the future.
[0110] The third environmental data for the second water environment in the third time period may include hydrological data and meteorological data for the second water environment in the third time period.
[0111] Step 42: Preprocess the first data to obtain the second information.
[0112] The implementation of step 42 can refer to the method of preprocessing sample data, that is, refer to... Figure 3 Preprocessing is performed in this manner.
[0113] The third environmental data may include hydrological data of the second water environment at multiple fifth moments within the third time period, and meteorological data at multiple sixth moments within the third time period, wherein the multiple sixth moments are subsets of the multiple fifth moments. The first data may also include the flow velocity of the second upstream water environment of the second water environment at multiple fifth moments.
[0114] Step 43: Input the second information into multiple target sub-models in the target water quality prediction model to obtain the second prediction data of the preset water quality indicators of the second water environment at multiple fourth times in the fourth time period, output by the multiple target sub-models respectively.
[0115] The fourth time period can be a future time period, and the duration of the fourth time period is, for example, 7 days. The interval between any two adjacent fourth time periods can be 1 hour. That is, this disclosure can perform hourly water quality prediction to make timely predictions of changes in water quality.
[0116] Step 44: Based on the target weights of each of the multiple target sub-models and the second prediction data output by each of the multiple target sub-models, determine the target prediction results of the preset water quality indicators of the second water environment at multiple fourth time points.
[0117] The target weights of the target water quality prediction model and the multiple target sub-models are obtained according to the training method of the water quality prediction model provided in the embodiments of this disclosure.
[0118] For each fourth time point, the target prediction results of the preset water quality indicators at that fourth time point can be obtained by weighting the target weights of the multiple target sub-models and the second prediction data output by the multiple target sub-models at that fourth time point.
[0119] Through the above technical solution, the target weight of the target sub-model can characterize the prediction credibility of the target sub-model. Based on the target weight, the target prediction result can be obtained, and the second prediction information output by the target sub-model with high prediction credibility can be given priority. Multiple target sub-models can predict water quality from different perspectives, thereby fusing the second prediction data output by each of the multiple target sub-models, which can improve the accuracy of the target prediction results of the preset water quality indicators.
[0120] Figure 5 This is a schematic diagram illustrating an exemplary method for predicting water quality in the water environment. Taking multiple target sub-models, including a first target sub-model (LSTM model), a second target sub-model (Prophet model), and a third target sub-model (XGBoost model), as an example, the second information obtained after preprocessing the first data can be input into the LSTM model, the Prophet model, and the XGBoost model, respectively. The prediction data output by each sub-model is weighted and fused through the target weights to obtain the target prediction result.
[0121] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the above-described training method for the water quality prediction model.
[0122] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the water quality prediction method for the water environment described above.
[0123] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.
[0124] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.
[0125] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.
Claims
1. A training method for a water quality prediction model, characterized in that, The method includes: Obtain a training sample set, wherein the training samples in the training sample set include sample data and labeled data, wherein the sample data includes first environmental data of the first water environment in the first time period, and the labeled data includes monitoring data of preset water quality indicators of the first water environment at multiple first moments in the second time period. The sample data is preprocessed to obtain the first information; The first information is input into multiple sub-models in the water quality prediction model to obtain the first predicted data of the preset water quality index of the first water environment at the multiple first times, respectively output by the multiple sub-models. Based on the KL divergence corresponding to each of the multiple sub-models, the first weight of each of the multiple sub-models is determined, wherein the weight of the sub-model is used to characterize the prediction confidence of the sub-model, and the KL divergence corresponding to the sub-model is obtained based on the probability distribution of the monitoring data and the probability distribution of the first prediction data output by the sub-model. A first prediction result is determined based on the first weights of each of the multiple sub-models and the first prediction data output by each of the multiple sub-models. Under the condition that training stops, the target water quality prediction model and the target weights of each of the multiple target sub-models in the target water quality prediction model are obtained.
2. The method according to claim 1, characterized in that, The multiple sub-models include a first sub-model, a second sub-model, and a third sub-model. The first sub-model is a Long Short-Term Memory (LSTM) network, the second sub-model is a Prophet model, and the third sub-model is an Extreme Gradient Boosting (XGBoost) model. The second sub-model includes a periodic term of the preset water quality index, which is used to characterize the changing trend of the preset water quality index.
3. The method according to claim 1, characterized in that, The step of determining the first weight of each of the multiple sub-models based on the KL divergence corresponding to each of the multiple sub-models includes: Based on the KL divergence corresponding to the multiple sub-models, determine the second weight of each of the multiple sub-models; For each first time point, first fusion information is determined based on the second weights of each of the plurality of sub-models, the first prediction data output by each of the plurality of sub-models at the first time point, and the average prediction deviation of each of the plurality of sub-models corresponding to the first time point. For each first time point, second fusion information is determined based on the second weights of each of the plurality of sub-models, the monitoring data at the first time point, and the prediction standard deviations of each of the plurality of sub-models, wherein the prediction standard deviation of the sub-model is the standard deviation of the first prediction data at the plurality of first time points output by the sub-model; If, based on the first fusion information and the second fusion information corresponding to each of the first moments, it is determined that the preset conditions are met, then the second weight of the sub-model is taken as the first weight of the sub-model.
4. The method according to claim 3, characterized in that, The step of determining the second weights of each of the multiple sub-models based on their respective KL divergences includes: If the number of specified sub-models is one, the second weight of the specified sub-model is determined as a preset weight, and the second weights of other sub-models besides the specified sub-model are determined according to the KL divergence corresponding to the multiple sub-models respectively. The specified sub-model includes the sub-model corresponding to the first prediction data that predicts the existence of water quality abnormalities in the first water environment during the second time period. If the number of specified sub-models is multiple and includes the first sub-model, then the second weight of the first sub-model is determined as the preset weight, and the second weights of other sub-models besides the first sub-model are determined according to the KL divergence corresponding to the multiple sub-models respectively. The first sub-model is a Long Short-Term Memory (LSTM) network.
5. The method according to claim 3 or 4, characterized in that, Based on the KL divergence corresponding to the multiple sub-models, the second weight of each sub-model is determined using the following formula: in, This represents the probability distribution of the first predicted data output by the m-th sub-model. This represents the probability distribution of the monitoring data. Denotes KL divergence, This represents the preset coefficients, and M represents the number of sub-models.
6. The method according to claim 1, characterized in that, There are multiple preset water quality indicators, and the method further includes: If the training stop condition is not met, the designated sub-model is trained based on the monitoring data of multiple preset water quality indicators with coupling relationship and the first prediction data of multiple preset water quality indicators with coupling relationship output by the designated sub-model. The designated sub-model includes a sub-model that predicts the existence of water quality abnormalities in the first water environment during the second time period based on the corresponding first prediction data.
7. The method according to claim 1, characterized in that, The first environmental data includes hydrological data of the first water environment at multiple second moments within the first time period, and meteorological data at multiple third moments within the first time period, wherein the multiple third moments are subsets of the multiple second moments. The sample data also includes the flow velocity of the first upstream water environment of the first water environment at the multiple second moments. The preprocessing of the sample data to obtain the first information includes: Identify abnormal data in the first environmental data and replace the abnormal data to obtain second environmental data of the first water environment within the first time period; Interpolate the meteorological data at the multiple third times in the second environmental data to obtain the meteorological data at the multiple second times. Determine the weighting values of the coupled meteorological and / or hydrological data at the same second time point; For each second time moment, based on the flow velocity at the second time moment and the distance between the first upstream water environment and the first water environment, the predicted arrival time of pollutants from the first upstream water environment to the first water environment at the second time moment is determined. The first information is obtained based on the weighted value, the predicted arrival time, the meteorological data at the multiple second times, and the hydrological data at the multiple second times in the second environmental data.
8. The method according to claim 7, characterized in that, The step of determining abnormal data in the first environmental data and replacing the abnormal data includes: The isolated forest algorithm is used to detect anomalous single data points in the first environmental data. For the hydrological data in the first environmental data, based on the standard deviation of the hydrological data at the multiple second time points, abnormal hydrological data corresponding to a first preset number of consecutive second time points are determined. For the meteorological data in the first environmental data, based on the standard deviation of the meteorological data at the multiple third times, determine the abnormal meteorological data corresponding to the second preset number of consecutive third times; Based on environmental data of other water environments within the first time period that are less than a preset distance threshold from the first water environment, the abnormal single-point data, the abnormal hydrological data, and the abnormal meteorological data are replaced.
9. A method for predicting water quality in aquatic environments, characterized in that, The method includes: Acquire first data, which includes third environmental data of the second water environment during the third time period; The first data is preprocessed to obtain the second information; The second information is input into multiple target sub-models in the target water quality prediction model to obtain the second prediction data of the preset water quality indicators of the second water environment at multiple fourth times in the fourth time period, respectively output by the multiple target sub-models. Based on the target weights of each of the plurality of target sub-models and the second prediction data output by each of the plurality of target sub-models, the target prediction results of the preset water quality indicators of the second water environment at the plurality of fourth times are determined, wherein the target water quality prediction model and the target weights of each of the plurality of target sub-models are obtained by the training method of the water quality prediction model according to any one of claims 1-8.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the training method for the water quality prediction model according to any one of claims 1-8, or, when the computer program is executed by a processor, it implements the steps of the water quality prediction method for the water environment according to claim 9.