Method and system for dynamically sensing and predicting inflow load of sewage treatment plant based on deep learning

Abstract

The invention discloses a dynamic perception and prediction method and system for the inflow load of a sewage treatment plant, and the method comprises the steps: obtaining the historical data and real-time data of the inflow index of the sewage treatment plant, and generating a model database; preprocessing the data; performing data set division, normalization and the like on the cleaned historical data set; constructing a deep learning neural network model by using the processed historical data set, and determining optimal parameters of the model by using a grid search and cross validation method; verifying the trained model by using the test set to obtain an optimal model conforming to expectation; importing a to-be-tested data set into the model to obtain a prediction result; the system comprises a sewage treatment plant data acquisition module, a data preprocessing module, a model training module, a model prediction module and an interactive operation module. According to the method, the complex nonlinear relationship between the real-time water inlet index monitoring data of the sewage treatment plant and the future water inlet load of the sewage treatment plant can be obtained, so that the future water inlet load is accurately predicted; the predicted future inflow load won time for the sewage treatment plant to take emergency measures and formulate a treatment scheme, so that the risk of environmental pollution is effectively avoided; and meanwhile, a reliable basis is provided for formulating a reasonable control strategy for the sewage treatment plant, the operation management cost is reduced, and a new generation of sewage plant intelligent management platform is endowed.

Description

Technical Field

[0001] The present invention belongs to the technical field of sewage treatment, and specifically relates to a method and system for dynamic perception and prediction of influent load of a sewage treatment plant based on deep learning. Background Art

[0002] In recent years, the effluent requirements of sewage treatment plants have been continuously improved, and the management difficulty and cost of sewage treatment plants have increased significantly. It is a general trend to promote sewage treatment plants to further improve their intelligent management level. The influent indicators of sewage treatment plants are constantly fluctuating, and timely responses and adjustments are required from all parties in the plant's treatment equipment, process plans, and reagent additions to ensure the stable operation of the sewage treatment plant. At present, sewage treatment plants generally respond by monitoring the real-time influent load and adjusting the operation plan to ensure stable operation. This method cannot predict and prepare for possible influent load fluctuations, has a certain lag, and often causes waste of manpower and material resources. At the same time, the operation and management costs of sewage treatment plants remain high. Therefore, the influent prediction of sewage treatment plants is particularly important.

[0003] In recent years, with the development of data analysis science and computer technology, deep learning algorithms have far surpassed the prediction and classification accuracy of traditional machine learning algorithms for data, and have been successfully applied to modeling in various fields such as finance, medical care, and transportation. Deep learning is an efficient and intelligent fitting algorithm based on neural networks. It has strong learning ability, thus showing great flexibility and adaptability, which also provides new ideas for the construction of water volume prediction system for sewage treatment plants. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a deep learning-based method and system for dynamic perception and prediction of the influent load of a sewage treatment plant to address the deficiencies in the prior art.

[0005] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0006] On the one hand, a method for dynamic perception prediction of influent load of a sewage treatment plant based on deep learning is provided, which includes the following steps:

[0007] S1. Obtain historical and real-time data of the sewage treatment plant’s influent indicators and generate a model database;

[0008] S2. Preprocess the data in the model database to clean and filter out abnormal values ​​in historical data and supplement a small amount of missing values ​​in the data;

[0009] S3, dividing and normalizing the cleaned historical data set in the model database;

[0010] S4. Use the processed historical data set to build a deep learning neural network model, and use grid search and cross-validation methods to determine the optimal parameters of the model;

[0011] S5. Use the test set to verify the trained model and obtain the optimal model that meets expectations;

[0012] S6. Import the data set to be tested into the model to obtain the prediction result.

[0013] As described in the deep learning-based dynamic perception prediction method for the influent load of a sewage treatment plant, in step S1, the influent indicators of the sewage treatment plant include influent volume, chemical oxygen demand, five-day biochemical oxygen demand, ammonia nitrogen, total nitrogen, total phosphorus, suspended solids concentration, pH value, dissolved oxygen concentration and water temperature.

[0014] As described in the deep learning-based dynamic perception prediction method for influent load of a sewage treatment plant, step S2 includes:

[0015] Data preprocessing removes large sections of blank data that exceed a certain time;

[0016] Use 3-sigma technology to eliminate a small number of abnormal data points;

[0017] Fill a small number of missing values ​​using linear interpolation.

[0018] As described in the deep learning-based dynamic perception prediction method for influent load of a sewage treatment plant, step S3 includes:

[0019] Normalize the cleaned historical data;

[0020] The normalized data is divided into training set and test set according to a certain ratio.

[0021] As described in the deep learning-based dynamic perception prediction method for influent load of a sewage treatment plant, step S4 includes:

[0022] The deep learning neural network model consists of multiple layers of different neural networks, and the number of layers is automatically determined by grid search and cross-validation;

[0023] The main structures of the deep learning neural network model are the long short-term memory model and the convolutional neural network model, and the parameters are automatically determined by grid search and cross-validation;

[0024] Deep learning neural network model training will automatically find the optimal model parameters through multi-parameter grid search and multi-dimensional cross-validation;

[0025] The parameters of grid search include the number of neurons, the number of model layers, the learning rate, the number of running steps, the sliding window size, the pool size, the optimizer, and the ratio of the training set and the validation set.

[0026] Set up an early stopping system to monitor the training effect of the validation set and stop training when the model effect no longer improves, so as to avoid overfitting and improve model training efficiency.

[0027] As described in the deep learning-based dynamic perception prediction method for the influent load of a sewage treatment plant, the main evaluation indicator of the quality of the test set verification model in step S5 is the determination coefficient.

[0028] As described in the deep learning-based dynamic perception prediction method for the influent load of a sewage treatment plant, in which the data to be measured in step S6 is composed of data from a period of time before a future moment, and the influent data at a future moment is predicted based on the most recent existing data.

[0029] On the other hand, a deep learning-based dynamic perception prediction system for influent load of a sewage treatment plant is provided, which includes:

[0030] The sewage treatment plant data collection module collects the real-time water inflow indicators, historical logs and data manually input by staff of the sewage treatment plant to build a database;

[0031] Data preprocessing module, used for data preprocessing, including removing outliers in historical data, supplementing a small number of missing values ​​in data, dividing historical data sets, normalizing, and other processing;

[0032] Model training module, which builds models based on deep learning methods and automatically selects the best model parameters through grid search and cross-validation;

[0033] The model prediction module reads the recent sewage treatment plant inflow indicators and uses the trained model to predict the future inflow load;

[0034] The interactive operation module is used to control the display and export of data and charts, manually control the training frequency, model selection, and connect to the sewage plant smart platform.

[0035] The beneficial effects of the technical solution of the present invention are:

[0036] 1. The present invention can predict the inflow load based on real-time data reading in the traditional smart plant management of sewage treatment plants. The predicted data can buy time for sewage treatment plants to take emergency measures and formulate treatment plans, thereby effectively avoiding environmental pollution risks. At the same time, it provides a reliable basis for sewage treatment plants to formulate reasonable control strategies and reduce operation and management costs.

[0037] 2. Compared with the traditional machine learning prediction system, the present invention adopts a time series model for sewage treatment plants and innovatively introduces a convolutional neural network model to couple with it. The deep learning model based on the neural network model is more advanced, has stronger computing power, and has better prediction effect;

[0038] 3. The automatic grid search and cross-validation method adopted by the present invention does not require manual intervention in parameter adjustment during model training, and is more widely used rather than just applicable to a certain plant or station, and has broad market application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] To further illustrate the above-mentioned objectives, structural features and effects of the present invention, the present invention will be described in detail below with reference to the accompanying drawings.

[0040] Figure 1 This is a logic block diagram of a method for dynamic perception and prediction of influent load of a sewage treatment plant based on deep learning in a preferred embodiment of the present invention;

[0041] Figure 2 This is the prediction result of the model of the dynamic perception prediction method of the sewage treatment plant inlet load based on deep learning in the preferred embodiment of the present invention;

[0042] Figure 3 This is a logic block diagram of a dynamic perception and prediction system for influent load of a sewage treatment plant based on deep learning in a preferred embodiment of the present invention;

[0043] In the figure: 1. Sewage treatment plant data acquisition module; 2. Data preprocessing module; 3. Model training module; 4. Model prediction module; 5. Interactive operation module. DETAILED DESCRIPTION

[0044] The terms "invention", "present invention" and "the present invention" used in this specification are intended to refer broadly to all subject matter of this specification and any patent claims below. Statements containing these terms should not be understood to limit the subject matter described herein or to limit the meaning or scope of any patent claim below. In addition, this specification does not attempt to describe or limit the subject matter covered by any claim of any specific component, paragraph, statement or figure of this application. The subject matter should be understood with reference to the entire specification, all drawings and any claims below. The present invention may have other embodiments and be practiced or implemented in other ways. Moreover, it should be understood that the words and terms used herein are for illustrative purposes and should not be considered as limiting.

[0045] The details of the present invention will now be discussed with reference to the accompanying drawings which illustrate the present invention by way of example only. In the accompanying drawings, similar features or components may be marked with the same reference numerals.

[0046] The use of "including," "having," and "comprising" and variations thereof herein is meant to encompass the items listed thereafter and their equivalents and additional items. Although reference may be made to directions such as above, below, upward, downward, rearward, bottom, top, front, rear, etc., in describing the drawings, for convenience, reference is made relative to the drawings. These directions are not intended to literally accept or limit the present invention in any form. In addition, terms such as "first," "second," "third," etc., are used herein for illustrative purposes and are not intended to indicate or imply importance or significance.

[0047] See also Figure 1 As shown, the dynamic perception prediction method of the sewage treatment plant inlet load based on deep learning of the present invention specifically includes the following steps:

[0048] S1. Obtain historical and real-time data of the sewage treatment plant’s influent indicators and generate a model database;

[0049] S2. Preprocess the data in the model database to clean and filter out abnormal values ​​in historical data and supplement a small amount of missing values ​​in the data;

[0050] S3, dividing and normalizing the cleaned historical data set in the model database;

[0051] S4. Use the processed historical data set to build a deep learning neural network model, and use grid search and cross-validation methods to determine the optimal parameters of the model;

[0052] S5. Use the test set to verify the trained model and obtain the optimal model that meets expectations;

[0053] S6. Import the data set to be tested into the model to obtain the prediction result.

[0054] Specifically:

[0055] A. Data reading

[0056] The SCADA system is used to read the real-time online data and years of historical data from the water quality and quantity detection instruments of a sewage treatment plant to build an original database.

[0057] Optionally, employees can enter the sewage plant influent data by manually uploading it.

[0058] Specifically, in this embodiment, the data sampling interval is 1 hour.

[0059] B. Data Preprocessing

[0060] Clean the raw data in the database.

[0061] Specifically, data segments with 12 or more consecutive blank values ​​are deleted to avoid the impact of a large amount of missing data on time series prediction.

[0062] Specifically, the currently mature and reliable 3-sigma technical method is used to eliminate a small number of abnormal data points, and linear interpolation is used to fill a small number of missing values.

[0063] Among them, the 3-sigma technical method is to set the confidence interval of the data set to be within the range of less than 3 standard deviations from the mean, and its credibility reaches 99.73%. The specific calculation formula is as follows:

[0064] Pr(μ-3σ≤X≤μ+3σ)≈0.9973

[0065] Among them, the linear interpolation method can linearly fill in a small number of missing values, which is very suitable for quickly processing a small number of deleted outliers.

[0066] Specifically, the cleaned historical data is normalized.

[0067] Normalization is also called deviation standardization, which is a linear transformation of the original data of the data set. In this embodiment, the result value is mapped to the range of [-1,1] to make the complex data dimensionless, avoid numerical problems caused by too large data, and facilitate model solving and improve the model convergence speed. The specific calculation formula is as follows:

[0068]

[0069] Specifically, in this embodiment, the cleaned data is divided into a training set and a test set in a ratio of 8:2.

[0070] C. Building the model

[0071] A model structure that couples a long-short time memory model and a one-dimensional convolutional neural network model is used, and the preprocessed data is input to start training the model. The parameters of the model are crucial to the simulation prediction effect of the model. In order to achieve good prediction results, it is necessary not only to reasonably select the type of model, but also to select appropriate model parameters. Usually, the parameter adjustment process of constantly changing the model parameters to obtain a better fitting effect is done manually, which is inefficient. The method described in the present invention adopts machine automatic parameter adjustment.

[0072] Among them, the model parameters include the number of neurons, the number of model layers, the learning rate, the number of running steps, the sliding window size, the pool size, the optimizer, the ratio of the training set and the validation set, etc.

[0073] Specifically, a hyperparameter optimization technique, namely grid search and cross-validation, is used to adjust the key parameters of model construction to improve model performance, including computing speed and prediction accuracy.

[0074] Specifically, a 3-fold cross validation is used in this embodiment.

[0075] D. Training Model

[0076] The optimal model obtained by grid search is stored, and the training set data is imported to start model training.

[0077] Specifically, 20% of the data in the training set is divided into the validation set, and the validation frequency is once every 20 steps.

[0078] Specifically, callbacks are set as early stopping systems to monitor the mean absolute error (MAE) of the validation set. After 10 consecutive MAEs no longer decrease, the training is considered complete and training is stopped to avoid overfitting of the model. The specific calculation formula is as follows:

[0079]

[0080] E. Test Model

[0081] Import the test set data into the trained model for fitting, and evaluate the test set fitting results.

[0082] Specifically, R 2 Monitor with MSE and export the results. Figure 2 This is the prediction result of the model in this embodiment.

[0083] Specifically, operators can intuitively view the model test set fitting effect through the system interface, thereby understanding the model prediction effect.

[0084] Specifically, the prediction system will automatically convert the test set R 2 Models with a score higher than 85% were considered as usable models and used for actual prediction.

[0085] Specifically, operators can manually choose to abandon the model and roll back to other historical models.

[0086] F. Predicting the influent load of sewage treatment plants

[0087] The prediction system automatically reads the most recent influent data, imports it into the trained model, and predicts the future influent load of the sewage treatment plant.

[0088] Specifically, the prediction system will read the latest 24 sets of sewage plant influent water quality and quantity data in the database, that is, all data in the past 24 hours.

[0089] Specifically, the prediction system in this embodiment will predict the data for the next hour based on the data for the past 24 hours, and continue to predict the sewage plant's influent load at subsequent times based on actual demand.

[0090] G. Data observation and export

[0091] The forecasting system will display historical data, current moment data and predicted future data.

[0092] Specifically, historical data, current time data and future data are displayed in the form of line graphs and lists.

[0093] Specifically, the operator can select the chart type, data time period, etc. to be displayed through the system interface.

[0094] Specifically, the operator can export the required charts in png, cls and other formats through the system interface.

[0095] Specifically, the system can connect the prediction information with the sewage treatment plant's smart system through the network or other necessary interfaces, enabling a new generation of sewage treatment plant's smart management platform.

[0096] See also Figure 3 As shown in the figure, the deep learning-based dynamic perception prediction system for sewage treatment plant influent load includes:

[0097] The sewage treatment plant data collection module 1 collects the real-time water inflow indicators, historical logs and data manually input by staff of the sewage treatment plant to build a database;

[0098] Data preprocessing module 2 is used for data preprocessing, including removing abnormal values ​​in historical data, supplementing a small number of missing values ​​in data, dividing historical data sets, normalizing, and other processing;

[0099] Model training module 3 builds a model based on deep learning methods and automatically selects the best model parameters through grid search and cross-validation;

[0100] Model prediction module 4, reads the recent sewage treatment plant inflow indicators and uses the trained model to predict the future inflow load;

[0101] Interactive operation module 5 is used to control the display and export of data and charts, manually control the training frequency, model selection, and connect to the sewage plant smart platform.

[0102] The dynamic perception prediction method and system of the sewage treatment plant inflow load of the present invention is based on the long-term inflow index monitoring data of the sewage treatment plant, combines data analysis and deep learning, and couples the multi-level model method to construct a deep learning-based sewage treatment plant inflow load dynamic perception prediction system (SewageFlowProphet System). By learning the long-term inflow monitoring data of the sewage treatment plant, the system can obtain the complex nonlinear relationship between the real-time inflow index monitoring data of the sewage treatment plant and the future inflow load of the sewage treatment plant, thereby realizing the accurate prediction of the future inflow load; the predicted future inflow load will buy time for the sewage treatment plant to take emergency measures and formulate governance plans, thereby effectively avoiding environmental pollution risks; at the same time, it provides a reliable basis for the sewage treatment plant to formulate reasonable control strategies, reduces operation and management costs, and enables a new generation of sewage treatment plant smart management platforms.

[0103] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A dynamic perception prediction method for influent load of a sewage treatment plant based on deep learning, characterized in that: The following steps are involved: S1. Obtain historical and real-time data of the sewage treatment plant’s influent indicators and generate a model database; S2. Preprocess the data in the model database to clean and filter out abnormal values ​​in historical data and supplement a small amount of missing values ​​in the data; S3, dividing and normalizing the cleaned historical data set in the model database; S4. Use the processed historical data set to build a deep learning neural network model, and use grid search and cross-validation methods to determine the optimal parameters of the model; S5. Use the test set to verify the trained model and obtain the optimal model that meets expectations; S6. Import the data set to be tested into the model to obtain the prediction result.

2. The method for dynamic perception prediction of influent load of a sewage treatment plant based on deep learning as claimed in claim 1, characterized in that: In step S1, the influent indicators of the sewage treatment plant include influent volume, chemical oxygen demand, five-day biochemical oxygen demand, ammonia nitrogen, total nitrogen, total phosphorus, suspended solids concentration, pH value, dissolved oxygen concentration and water temperature.

3. The method for dynamic perception prediction of influent load of a sewage treatment plant based on deep learning as claimed in claim 2, characterized in that: Step S2 includes: Data preprocessing removes large sections of blank data that exceed a certain time; Use 3-sigma technology to eliminate a small number of abnormal data points; Fill a small number of missing values ​​using linear interpolation.

4. The method for dynamic perception prediction of influent load of a sewage treatment plant based on deep learning as claimed in claim 3, characterized in that: Step S3 includes: Normalize the cleaned historical data; The normalized data is divided into training set and test set according to a certain ratio.

5. The method for dynamic perception prediction of influent load of a sewage treatment plant based on deep learning as claimed in claim 4, characterized in that: Step S4 includes: The deep learning neural network model consists of multiple layers of different neural networks, and the number of layers is automatically determined by grid search and cross-validation; The main structures of the deep learning neural network model are the long short-term memory model and the convolutional neural network model, and the parameters are automatically determined by grid search and cross-validation; Deep learning neural network model training will automatically find the optimal model parameters through multi-parameter grid search and multi-dimensional cross-validation; The parameters of grid search include the number of neurons, the number of model layers, the learning rate, the number of running steps, the sliding window size, the pool size, the optimizer, and the ratio of the training set and the validation set. Set up an early stopping system to monitor the training effect of the validation set and stop training when the model effect no longer improves, so as to avoid overfitting and improve model training efficiency.

6. The method for dynamic perception prediction of influent load of a sewage treatment plant based on deep learning as claimed in claim 5, characterized in that: The main evaluation index of the test set verification model in step S5 is the determination coefficient.

7. The method for dynamic perception and prediction of influent load of a sewage treatment plant based on deep learning as described in claim 6 is characterized in that: In step S6, the data to be measured is composed of data of a period of time before the future moment, and the water inflow data at the future moment is predicted based on the most recent existing data.

8. A deep learning-based dynamic perception prediction system for influent load of a sewage treatment plant, characterized in that: include: The sewage treatment plant data collection module collects the real-time water inflow indicators, historical logs and data manually input by staff of the sewage treatment plant to build a database; Data preprocessing module, used for data preprocessing, including removing outliers in historical data, supplementing a small number of missing values ​​in data, dividing historical data sets, normalizing, and other processing; Model training module, which builds models based on deep learning methods and automatically selects the best model parameters through grid search and cross-validation; The model prediction module reads the recent sewage treatment plant inflow indicators and uses the trained model to predict the future inflow load; The interactive operation module is used to control the display and export of data and charts, manually control the training frequency, model selection, and connect to the sewage plant smart platform.

Images