Water quality detection method and system

The water quality change prediction model trained by machine learning algorithms, combined with real-time sensor data and historical data, solves the problem that existing water quality detection methods cannot reflect water quality changes in real time, and achieves prediction of future water quality trends and timely warning of sudden pollution. Improved the accuracy and emergency response capabilities of water quality detection.

CN119046750APending Publication Date: 2024-11-29SICHUAN YUNNUOXIN TESTING TECH CO LTD
View PDF 0 Cites 15 Cited by

Patent Information

Application Number
CN202411050291.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing water quality detection methods cannot reflect water quality changes in real time, especially in sudden pollution incidents, and lack the ability to predict future water quality changes trends.

Method used

Machine learning algorithms are used to train the water quality change prediction model, combine the data collected by real-time sensors and historical data, and predict future water quality parameter changes through data pretreatment and model optimization, and adjust the water treatment process parameters according to the prediction results to deal with potential pollution.

Benefits of technology

Real-time monitoring of water quality changes and future trend predictions have been achieved, timely warnings for sudden pollution incidents, improved the accuracy and emergency response capabilities of water quality detection, and protected the ecological environment and human health.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046750A_ABST
    Figure CN119046750A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of water quality detection, discloses a water quality detection method and system, and aims to overcome the limitation of a traditional water quality detection method and realize real-time prediction and monitoring of water quality change. According to the method and the system, firstly, historical and real-time water quality data are collected through a data acquisition layer, and the data cover various pollutant concentrations and environmental parameters; the data preprocessing layer is responsible for data cleaning, missing value processing and abnormal value detection, and data quality is ensured. And the model training and verification layer adopts a machine learning algorithm to train and verify the water quality change prediction model, and optimizes the performance of the model through parameter tuning. And the prediction and application layer inputs real-time data into the model, predicts future water quality change, analyzes a prediction result to identify a potential pollution event, and adjusts a water treatment strategy according to the potential pollution event. According to the invention, the timeliness and accuracy of water quality detection are improved, and water resource management and environmental protection are facilitated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of water quality detection, and in particular to a water quality detection method and system. Background Art

[0002] Water quality safety is one of the core issues of environmental protection and water resources management, and is crucial to maintaining ecological balance and human health. Currently, water quality testing mainly uses the method of manual sampling on site and sending it to the laboratory for analysis, as well as some automated monitoring stations and mobile monitoring equipment.

[0003] Although the method of manually taking samples on site and sending them to the laboratory for analysis can provide relatively accurate water quality data, its process is complicated, time-consuming, and depends on the operator's operating level and the storage conditions of the samples. This method cannot reflect the changes in water quality in real time, especially in sudden pollution incidents, and it is difficult to provide early warning information in time, which poses a challenge to pollution control and emergency response.

[0004] Traditional water quality detection methods lack the ability to predict future water quality trends. They focus mainly on the current water quality status and cannot provide information about how the water quality may change in the future. This is a significant drawback for timely warning of potential water quality problems and taking treatment measures in advance. To this end, we propose a water quality detection method and system. Summary of the invention

[0005] The purpose of the present invention is to provide a water quality detection method and system to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a water quality detection method, the method comprising:

[0007] Collect historical water quality data, specifically covering water source samples over different time periods; use water quality testing instruments to measure and record the concentrations of various pollutants in water source samples, including ammonia nitrogen, total phosphorus, heavy metals, and total bacteria; and simultaneously record the flow rate, temperature, pH value, chemical oxygen demand (COD), and biological oxygen demand (BOD) of the water source to form a complete historical water quality data set;

[0008] Preprocess the collected historical water quality data, including data cleaning, missing value processing and outlier detection;

[0009] The preprocessed historical water quality data is divided into a training set and a test set for model training and verification;

[0010] Using a machine learning algorithm, a water quality change prediction model is trained, wherein the water quality change prediction model is used to predict the change trend of water quality parameters in the future based on current water quality detection data;

[0011] Perform parameter tuning and model training on the selected machine learning algorithm, and optimize the algorithm parameters and model structure through iteration;

[0012] Use the test set to verify the trained model and evaluate the model's prediction performance and generalization ability; based on the verification results, select the model parameters with the best performance as the final water quality change prediction model parameters;

[0013] Deploy real-time sensors at water quality monitoring points to collect water quality data regularly, and ensure that the collected data is consistent with historical data in terms of parameter type and measurement unit;

[0014] The real-time collected water quality data is sent to the data center through the data transmission system; in the data center, the real-time data is preprocessed, including data cleaning and format conversion, to meet the input requirements of the machine learning model; the preprocessed real-time data is input into the trained water quality change prediction model;

[0015] The water quality change prediction model predicts the changes in water quality parameters over a period of time in the future based on historical data and the patterns learned during training;

[0016] Receive prediction results from machine learning models, analyze the prediction results, identify potential high pollution loads or harmful events; adjust the parameters and operating strategies of the water treatment process based on the prediction results and real-time water quality data.

[0017] Preferably, in preprocessing the historical water quality data, the data cleaning method includes:

[0018] Identify and remove duplicate records in a data set;

[0019] Check data consistency and correct or delete records that are incorrectly formatted or unreasonable;

[0020] Perform encoding conversion on non-numeric data and convert it into a numeric form suitable for analysis.

[0021] Preferably, in preprocessing the historical water quality data, the missing value processing method includes:

[0022] Use mean filling, median filling or mode filling methods to fill missing values ​​of numerical variables;

[0023] For categorical variables, missing values ​​were filled using the most frequently occurring category;

[0024] Apply interpolation methods, including linear interpolation and polynomial interpolation, to estimate and fill missing values ​​in time series data.

[0025] Preferably, in preprocessing the historical water quality data, the method of detecting outliers includes:

[0026] Use the Z-score method to calculate the Z-score value of each data point. If the absolute value of the Z-score is greater than the preset threshold, it is considered an outlier. The algorithm formula is:

[0027]

[0028] Among them, x is the original data, μ is the mean of the data, σ is the standard deviation of the data, and z is the Z-score value obtained by calculation.

[0029] Preferably, a long short-term memory network LSTM model is selected to construct a water quality prediction model. The specific method is: design a multi-layer LSTM neural network structure, including an input layer, one or more LSTM hidden layers, and an output layer; configure the input layer to receive and process training data of multiple time steps; in the LSTM hidden layer, set an appropriate number of LSTM units to learn and memorize the time dependency in the data and capture the trend of water quality changes; configure the output layer to generate prediction results of future water quality parameters.

[0030] Preferably, the step of initializing the parameters of the water quality prediction model includes: S1, initializing the weight parameters and bias parameters of the LSTM neural network; S2, setting the learning rate to control the step size of the network weight update to ensure the stability and convergence speed of the training; S3, determining the batch size, that is, the number of samples used in each training iteration, to balance the training speed and memory usage; S4, setting the training rounds, that is, the number of training rounds for the entire data set; S5, selecting the adaptive moment estimator Adam as the optimizer to adjust the network weights during the training process; S6, defining the mean square error MSE as the loss function to quantify the gap between the model prediction and the actual change.

[0031] Preferably, the specific method for training the water quality prediction model is:

[0032] a. Use the divided training data set to iteratively train the LSTM neural network. In each iteration, the training data is used as input and the corresponding future change value is used as the target output;

[0033] b. Calculate the output of the network through forward propagation and use the defined loss function to calculate the error between the predicted value and the actual value;

[0034] c. Apply the back-propagation algorithm and optimizer to update the network’s weight and bias parameters based on the calculated error to minimize the prediction error;

[0035] d. During the training process, the validation data set is used regularly to evaluate the performance of the model. By monitoring the loss function value and validation error, it is determined whether overfitting occurs, and the model structure is adjusted accordingly or the training is terminated early.

[0036] Preferably, a water quality detection system comprises:

[0037] The data collection layer includes real-time sensor modules, which are used to regularly collect water quality data at water quality monitoring points, covering the concentration of various pollutants, flow rate, temperature, pH value, chemical oxygen demand (COD) and biological oxygen demand (BOD) in water source samples over different time periods;

[0038] The data preprocessing layer includes a data cleaning module, a missing value processing module, and an outlier detection module, which are used to preprocess the collected water quality data to form a complete historical water quality data set, and preprocess the real-time data, including data cleaning and format conversion, to meet the input requirements of the machine learning model;

[0039] The model training and verification layer includes a machine learning algorithm module, a parameter tuning module, and a model training module, which are used to train a water quality change prediction model using a machine learning algorithm, perform parameter tuning and model training on the selected machine learning algorithm, optimize the algorithm parameters and model structure through iteration, and use the test set to verify the trained model, evaluate the prediction performance and generalization ability of the model, and select the model parameters with the best performance as the final water quality change prediction model parameters;

[0040] The prediction and application layer includes a real-time data input module, a water quality change prediction module and a result analysis module, which are used to input the pre-processed real-time data into the trained water quality change prediction model, predict the changes in water quality parameters in the future, receive the prediction results output by the machine learning model, analyze the prediction results, identify potential high pollution loads or harmful events, and adjust the parameters and operation strategies of the water treatment process according to the prediction results and real-time water quality data.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention can regularly collect water quality data and send it to the data center for analysis by deploying real-time sensors, thus achieving real-time monitoring of water quality changes. Combined with the machine learning prediction model, it can predict the trend of water quality parameters in the future, provide timely warnings for potential high pollution loads or harmful events, and effectively respond to sudden pollution events.

[0043] Traditional methods rely on manual sampling and laboratory analysis, which are complex and time-consuming. The present invention uses machine learning algorithms to conduct in-depth analysis of water quality data, thereby improving the accuracy of water quality testing. In the event of sudden pollution, the present invention can quickly provide early warning information, help to promptly initiate emergency response mechanisms, control the spread of pollution, and protect the ecological environment and human health. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A step diagram of the water quality detection method proposed by the present invention;

[0045] Figure 2 Training flow chart of water quality prediction model. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] See also Figure 1-2 The present invention provides a technical solution: The present invention provides a water quality detection method, the method comprising:

[0048] Step 1. Collect historical water quality data: Cover water source samples from different time periods to ensure data diversity and representativeness. Use water quality testing instruments to measure and record the concentrations of various pollutants in water source samples, including but not limited to ammonia nitrogen, total phosphorus, heavy metals, and total bacteria. At the same time, record the flow rate, temperature, pH value, chemical oxygen demand (COD), and biological oxygen demand (BOD) of the water source to form a complete historical water quality data set.

[0049] Step 2: Data preprocessing: Preprocess the collected historical water quality data, including data cleaning, missing value processing, and outlier detection, to ensure the quality and consistency of the data.

[0050] Step 3: Data division: Divide the preprocessed historical water quality data into training set and test set for model training and verification.

[0051] Step 4: Model training: Use machine learning algorithms to train a water quality change prediction model. This model can predict the trend of water quality parameter changes in the future based on current water quality test data.

[0052] Step 5: Parameter tuning and model training: Perform parameter tuning and model training on the selected machine learning algorithm, and iteratively optimize the algorithm parameters and model structure to improve the model's predictive performance and generalization ability.

[0053] Step 6: Model verification: Use the test set to verify the trained model and evaluate the model's prediction performance and generalization ability. Based on the verification results, select the model parameters with the best performance as the final water quality change prediction model parameters.

[0054] Step 7: Deploy real-time sensors: Deploy real-time sensors at water quality monitoring points to collect water quality data regularly. Ensure that the collected data is consistent with historical data in terms of parameter type and measurement unit.

[0055] Step 8: Data transmission and preprocessing: The real-time collected water quality data is sent to the data center through the data transmission system. In the data center, the real-time data is preprocessed, including data cleaning and format conversion, to meet the input requirements of the machine learning model.

[0056] Step 9: Real-time data prediction: Input the preprocessed real-time data into the trained water quality change prediction model. The water quality change prediction model predicts the changes in water quality parameters in the future based on historical data and the patterns learned during training.

[0057] Step 10: Result analysis and application: Receive the prediction results output by the machine learning model, analyze the prediction results, and identify potential high pollution loads or harmful events. Based on the prediction results and real-time water quality data, adjust the parameters and operation strategies of the water treatment process to deal with potential water quality problems.

[0058] The present invention will be further described below in conjunction with Examples 1 to 3:

[0059] Embodiment 1:

[0060] In the process of preprocessing historical water quality data, the implementation of data cleaning specifically includes the following steps:

[0061] Identify and remove duplicate records in the data set: Use the deduplication function in specific data processing software or programming languages ​​(such as Python, R, etc.) to compare all records in the historical water quality data set one by one. By comparing the various fields of the records (such as sampling time, sampling location, pollutant concentration, etc.), identical duplicate records are identified. The identified duplicate records are deleted from the data set to ensure that each record is unique and avoid bias in subsequent analysis.

[0062] Check data consistency, correct or delete records with incorrect or unreasonable formats: Check each record in the data set one by one to ensure that its format is correct and the data is reasonable. For records with incorrect formats, such as incorrect date format, confusing numerical format, etc., correct the format to ensure data consistency and readability. For records with unreasonable data, such as pollutant concentrations exceeding the normal range, negative flow rates, etc., further verification is carried out. If the data is confirmed to be incorrect after verification, it will be deleted from the data set.

[0063] Encode non-numeric data and convert it into a numerical form suitable for analysis: Identify non-numeric data in the dataset, such as water quality category, pollution level, and other descriptive information.

[0064] According to the characteristics of non-numerical data and analysis requirements, a reasonable coding scheme can be designed. For example, water quality categories can be coded according to their degree of quality, such as "excellent" is coded as 1, "good" is coded as 2, and "poor" is coded as 3, etc.

[0065] Coding schemes are applied to transform non-numeric data into numerical form for use in subsequent analysis and modeling.

[0066] In the process of preprocessing historical water quality data, the implementation of missing value processing specifically includes the following steps:

[0067] Filling missing values ​​for numerical variables:

[0068] First, identify the numeric variables in the dataset and check if there are any missing values ​​in these variables.

[0069] For missing values ​​of numerical variables, mean filling, median filling or mode filling methods can be used to fill them. The specific method should be determined according to the distribution characteristics of the variable and the analysis requirements. For example, if the variable is normally distributed, mean filling may be more appropriate; if the variable has a significantly skewed distribution, median or mode filling may be more appropriate. Apply the selected filling method to fill the missing values ​​of the numerical variables to ensure the integrity of the data set.

[0070] Filling missing values ​​for categorical variables:

[0071] Identify categorical variables in the dataset and check for missing values ​​in these variables. For missing values ​​of categorical variables, use the most frequently occurring category to fill in the missing values. This method is based on the principle of majority in statistics, which states that the most frequently occurring category is most likely to represent the true category of the missing value. By counting the frequency of occurrence of each category, the most frequently occurring category is determined and used to fill in the missing values ​​of the categorical variable.

[0072] Imputation and filling of missing values ​​of time series data:

[0073] Identify the time series data in the data set and check whether there are missing values ​​in these data. For missing values ​​in time series data, interpolation methods can be applied to estimate and fill them. Interpolation methods include linear interpolation and polynomial interpolation. The specific method to be selected should be determined according to the characteristics of the time series data and the analysis requirements. Linear interpolation is a simple interpolation method that assumes that the time series data changes linearly between the two observation points before and after the missing value. The slope of the two observation points is calculated and the size of the missing value is estimated accordingly. Polynomial interpolation is a more complex interpolation method that uses a polynomial function to fit the time series data and estimates the size of the missing value based on the fitting results. Polynomial interpolation can adapt to more complex time series data change patterns.

[0074] In the process of preprocessing historical water quality data, the implementation of outlier detection specifically includes the following steps:

[0075] Application of Z-score method:

[0076] First, for each numerical variable in the data set, calculate its mean (μ) and standard deviation (σ). These two statistics will be used in the subsequent Z-score calculation.

[0077] Next, for each data point x in the data set, the Z-score calculation formula is applied: z = (x-μ) / σ, where z represents the Z-score value obtained by calculation.

[0078] This formula reflects the relative position of a data point x to the mean μ of the data set, measured in units of standard deviation σ.

[0079] Identification and processing of outliers:

[0080] After obtaining the Z-score value of each data point, you need to set a preset threshold. This threshold is usually determined based on the characteristics of the data and analysis requirements. For example, you can choose 2.5, 3, or 3.5 as the threshold.

[0081] Then, check whether the absolute value of the Z-score value of each data point is greater than the preset threshold. If so, then the data point is considered an outlier.

[0082] There are many ways to deal with the identified outliers. For example, you can directly delete these outliers to maintain data consistency; or if the outliers have practical significance (such as representing a special event), you can choose to keep them and perform special processing on them.

[0083] In practical applications, the Z-score method may need to be optimized and adjusted according to the specific situation of the data and the analysis requirements. For example, the size of the preset threshold can be adjusted to more accurately identify outliers. In addition, other outlier detection methods (such as distribution-based methods, distance-based methods, etc.) can be combined to further improve the accuracy and reliability of outlier detection.

[0084] Embodiment 2:

[0085] When building a water quality prediction model, the long short-term memory network (LSTM) model is selected as the core algorithm. The specific implementation methods include:

[0086] Design a multi-layer LSTM neural network, which includes an input layer, one or more LSTM hidden layers, and an output layer. This structural design aims to make full use of the advantages of LSTM in processing time series data and capture the changing trends of water quality parameters over time.

[0087] Configuring the input layer: The input layer is configured to receive and process training data of multiple time steps. This means that the input layer can receive a series of water quality parameter data arranged in time order, such as pollutant concentrations, water temperatures, etc. at different time points.

[0088] By taking data from multiple time steps as input, the LSTM model can better understand how water quality parameters change over time and provide rich information for subsequent prediction tasks.

[0089] Set up LSTM hidden layer: In the LSTM hidden layer, set an appropriate number of LSTM units. These LSTM units are the core part of the model and are responsible for learning and memorizing the temporal dependencies in the data.

[0090] The LSTM unit controls the flow of information through its internal gating mechanism (forget gate, input gate, and output gate), thereby effectively capturing the time series characteristics of water quality parameters.

[0091] By configuring multiple LSTM hidden layers, the model's learning ability and ability to capture complex water quality change trends can be further enhanced.

[0092] Configure the output layer: The output layer is configured to generate predictions of future water quality parameters. Depending on the specific prediction task, the output layer can be one or more neurons, corresponding to different water quality parameters.

[0093] The neurons in the output layer receive the output of the LSTM hidden layer and perform further calculations and processing to ultimately generate predicted values ​​for future water quality parameters.

[0094] Model training and prediction: Use historical water quality data to train the designed LSTM model. During the training process, the model continuously learns and adjusts its internal parameters to minimize the prediction error. After the training is completed, the trained LSTM model can be used to predict future water quality parameters. By inputting the data of the new time step into the model, the model will output the corresponding prediction results.

[0095] The parameters for initializing the water quality prediction model include:

[0096] Initialize the weight parameters and bias parameters of the LSTM neural network: The weight parameters and bias parameters of the LSTM neural network are the core of network learning. They will be continuously adjusted during the training process to minimize the prediction error. When initializing these parameters, random generation is usually used, such as using normal distribution or uniform distribution to generate initial values. More complex initialization methods can also be used, such as initialization strategies based on He or Glorot, to ensure the rationality and effectiveness of parameter initialization.

[0097] Setting the learning rate: The learning rate is an important hyperparameter that controls the step size of the network weight update. Setting an appropriate learning rate is crucial to ensuring the stability and convergence speed of training. Usually, the choice of learning rate needs to be determined through experiments, and the learning rate decay strategy can be used to dynamically adjust the learning rate during training.

[0098] Determine the batch size: The batch size refers to the number of samples used in each training iteration. The selection of batch size requires a balance between training speed and memory usage. Smaller batch sizes can speed up training but may increase memory usage; larger batch sizes can reduce memory usage but may reduce training speed. The selection of batch size needs to be determined based on the specific dataset and hardware conditions.

[0099] Set the training round: The training round refers to the number of times the entire dataset is trained. Setting the appropriate training round is crucial to ensure that the model fully learns and converges. The choice of training rounds needs to be determined through experiments, and an early stopping strategy can be used to prevent overfitting.

[0100] Select optimizer: The optimizer is used to adjust the network weights during training to minimize the loss function. Adaptive moment estimation (Adam) is selected as the optimizer because it combines the advantages of momentum and RMSprop and can adaptively adjust the learning rate during training. The Adam optimizer is particularly effective for processing large-scale data and complex network structures.

[0101] Define the loss function: The loss function is used to quantify the gap between the model prediction and the actual change. Mean squared error (MSE) is defined as the loss function because it measures the squared error between the model prediction and the actual value. MSE is a commonly used regression loss function that is suitable for continuous value prediction tasks.

[0102] Embodiment 3:

[0103] A water quality detection system includes a data acquisition layer, a data preprocessing layer, a model training and verification layer, and a prediction and application layer.

[0104] For the data collection layer:

[0105] Design and implement a real-time sensor module that has the ability to interface with various water quality monitoring sensors (such as pollutant concentration sensors, flow rate sensors, temperature sensors, pH sensors, etc.) to ensure that water quality data can be collected regularly from water quality monitoring points.

[0106] The real-time sensor module is configured to cover water source samples over different time periods to ensure that the collected data includes key water quality parameters such as various pollutant concentrations, flow rate, temperature, pH value, chemical oxygen demand (COD), and biological oxygen demand (BOD).

[0107] Implement the scheduled task of data collection to ensure that the real-time sensor module can automatically collect water quality data at predetermined time intervals and store the data in a local or remote database.

[0108] For the data preprocessing layer:

[0109] Develop a data cleaning module that can pre-process the collected water quality data, including removing duplicate data, correcting erroneous data, processing inconsistent data, etc., to ensure the accuracy and consistency of the data.

[0110] Implement a missing value processing module that can identify missing values ​​in a data set and fill them with appropriate filling strategies (such as mean filling, median filling, or mode filling) to ensure the integrity of the data set.

[0111] Design and implement an outlier detection module that can apply the Z-score method or other statistical methods to detect outliers in a data set and process them appropriately (such as deletion or replacement) to ensure the normal distribution and accuracy of the data.

[0112] Develop data format conversion functions to convert water quality data after data preprocessing into the input format required by the machine learning model, including steps such as feature extraction and data normalization or standardization to ensure that the data meets the input requirements of the machine learning model.

[0113] Realize the function of constructing historical water quality data sets, organize the preprocessed water quality data according to time series, form a complete historical water quality data set, and provide data support for subsequent model training and verification.

[0114] The model training and validation layers include:

[0115] The machine learning algorithm module integrates a variety of machine learning algorithm libraries, such as TensorFlow, PyTorch, etc., to provide a rich selection of algorithms for water quality change prediction. According to the characteristics of water quality data and prediction requirements, select a suitable machine learning algorithm, such as LSTM, for model training.

[0116] The parameter tuning module designs parameter tuning strategies, including defining parameter search space, setting tuning objectives (such as minimizing prediction error), etc. Parameter tuning methods such as grid search and random search are applied to optimize the parameters of the selected machine learning algorithm. Through iterative training, the model performance under different parameter combinations is evaluated, and the optimal parameter combination is selected as the basis for model training.

[0117] The model training module uses the preprocessed historical water quality data set as training data to train the selected machine learning algorithm. During the training process, cross-validation strategies, such as K-fold cross-validation, are applied to evaluate the generalization ability of the model. Key indicators during the training process, such as loss function value and accuracy, are recorded for subsequent analysis and optimization.

[0118] The prediction and application layer includes:

[0119] Real-time data input module: Design and implement real-time data interface to ensure that it can receive pre-processed real-time water quality data from the data acquisition layer. Implement data caching mechanism to improve the access and processing efficiency of real-time data during model prediction.

[0120] Water quality change prediction module: inputs real-time data into the trained water quality change prediction model, executes the model reasoning process, and predicts the changes in water quality parameters in the future. It realizes the real-time output function of the prediction results, ensuring that the prediction results can be delivered to the subsequent processing module or user interface in a timely and accurate manner.

[0121] Result analysis module: Analyze the prediction results output by the machine learning model to identify potential high pollution loads or harmful events. Design and implement an early warning mechanism. When the prediction results exceed the preset threshold, the early warning signal is automatically triggered to remind relevant personnel to take corresponding countermeasures.

[0122] Provide visualization tools or report generation functions to display the prediction results to users in the form of charts, reports, etc., to help users intuitively understand water quality change trends and prediction results.

[0123] Decision support and application module: Provide decision support for water treatment process based on prediction results and real-time water quality data, adjust process parameters and operation strategies to optimize water quality treatment effect. Integrate into a wider water quality management system and work with other modules (such as data acquisition layer, data preprocessing layer, etc.) to achieve closed-loop management of water quality monitoring, prediction, early warning and treatment.

[0124] Provide API interface or SDK toolkit to facilitate third-party system or application integration and use of the predictive function of the water quality detection system.

[0125] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0126] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A water quality detection method, characterized in that: The method comprises: Collect historical water quality data, specifically covering water source samples over different time periods; use water quality testing instruments to measure and record the concentrations of various pollutants in water source samples, including ammonia nitrogen, total phosphorus, heavy metals, and total bacteria; and simultaneously record the flow rate, temperature, pH value, chemical oxygen demand (COD), and biological oxygen demand (BOD) of the water source to form a complete historical water quality data set; Preprocess the collected historical water quality data, including data cleaning, missing value processing and outlier detection; The preprocessed historical water quality data is divided into a training set and a test set for model training and verification; Using a machine learning algorithm, a water quality change prediction model is trained, wherein the water quality change prediction model is used to predict the change trend of water quality parameters in the future based on current water quality detection data; Perform parameter tuning and model training on the selected machine learning algorithm, and optimize the algorithm parameters and model structure through iteration; Use the test set to verify the trained model and evaluate the model's prediction performance and generalization ability; based on the verification results, select the model parameters with the best performance as the final water quality change prediction model parameters; Deploy real-time sensors at water quality monitoring points to collect water quality data regularly, and ensure that the collected data is consistent with historical data in terms of parameter type and measurement unit; The real-time collected water quality data is sent to the data center through the data transmission system; in the data center, the real-time data is preprocessed, including data cleaning and format conversion, to meet the input requirements of the machine learning model; the preprocessed real-time data is input into the trained water quality change prediction model; The water quality change prediction model predicts the changes in water quality parameters over a period of time in the future based on historical data and the patterns learned during training; Receive prediction results from machine learning models, analyze the prediction results, identify potential high pollution loads or harmful events; adjust the parameters and operating strategies of the water treatment process based on the prediction results and real-time water quality data.

2. A water quality detection method according to claim 1, characterized in that: In the preprocessing of historical water quality data, the data cleaning methods include: Identify and remove duplicate records from a data set; Check data consistency and correct or delete records that are incorrectly formatted or unreasonable; Perform encoding conversion on non-numeric data and convert it into a numeric form suitable for analysis.

3. A water quality detection method according to claim 1, characterized in that: In the preprocessing of historical water quality data, the methods of handling missing values ​​include: Use mean filling, median filling or mode filling methods to fill missing values ​​of numerical variables; For categorical variables, missing values ​​were filled using the most frequently occurring category; Apply interpolation methods, including linear interpolation and polynomial interpolation, to estimate and fill missing values ​​in time series data.

4. A water quality detection method according to claim 1, characterized in that: In the preprocessing of historical water quality data, the methods of outlier detection include: Use the Z-score method to calculate the Z-score value of each data point. If the absolute value of the Z-score is greater than the preset threshold, it is considered an outlier. The algorithm formula is: Among them, x is the original data, μ is the mean of the data, σ is the standard deviation of the data, and z is the Z-score value obtained by calculation.

5. A water quality detection method according to claim 4, characterized in that: The long short-term memory network (LSTM) model was selected to construct a water quality prediction model. The specific method was as follows: a multi-layer LSTM neural network structure was designed, including an input layer, one or more LSTM hidden layers, and an output layer; the input layer was configured to receive and process training data of multiple time steps; In the LSTM hidden layer, an appropriate number of LSTM units are set to learn and memorize the time dependency in the data and capture the trend of water quality changes; the output layer is configured to generate prediction results of future water quality parameters.

6. A water quality detection method according to claim 5, characterized in that: The steps of initializing the parameters of the water quality prediction model include: S1, initializing the weight parameters and bias parameters of the LSTM neural network; S2, setting the learning rate to control the step size of the network weight update to ensure the stability and convergence speed of the training; S3, determining the batch size, that is, the number of samples used in each training iteration, to balance the training speed and memory usage; S4, setting the training rounds, that is, the number of training rounds for the entire data set; S5, selecting the adaptive moment estimator Adam as the optimizer to adjust the network weights during the training process; S6, defining the mean square error MSE as the loss function to quantify the gap between the model prediction and the actual changes.

7. A water quality detection method according to claim 6, characterized in that: The specific method for training the water quality prediction model is: a. Use the divided training data set to iteratively train the LSTM neural network. In each iteration, the training data is used as input and the corresponding future change value is used as the target output; b. Calculate the output of the network through forward propagation and use the defined loss function to calculate the error between the predicted value and the actual value; c. Apply the back-propagation algorithm and optimizer to update the network’s weight and bias parameters based on the calculated error to minimize the prediction error; d. During the training process, the validation data set is used regularly to evaluate the performance of the model. By monitoring the loss function value and validation error, it is determined whether overfitting occurs, and the model structure is adjusted accordingly or the training is terminated early.

8. A water quality detection system, characterized in that: include: The data collection layer includes real-time sensor modules, which are used to regularly collect water quality data at water quality monitoring points, covering the concentration of various pollutants, flow rate, temperature, pH value, chemical oxygen demand (COD) and biological oxygen demand (BOD) in water source samples over different time periods; The data preprocessing layer includes a data cleaning module, a missing value processing module, and an outlier detection module, which are used to preprocess the collected water quality data to form a complete historical water quality data set, and preprocess the real-time data, including data cleaning and format conversion, to meet the input requirements of the machine learning model; The model training and verification layer includes a machine learning algorithm module, a parameter tuning module, and a model training module, which are used to train a water quality change prediction model using a machine learning algorithm, perform parameter tuning and model training on the selected machine learning algorithm, optimize the algorithm parameters and model structure through iteration, and use the test set to verify the trained model, evaluate the prediction performance and generalization ability of the model, and select the model parameters with the best performance as the final water quality change prediction model parameters; The prediction and application layer includes a real-time data input module, a water quality change prediction module and a result analysis module, which are used to input the pre-processed real-time data into the trained water quality change prediction model, predict the changes in water quality parameters in the future, receive the prediction results output by the machine learning model, analyze the prediction results, identify potential high pollution loads or harmful events, and adjust the parameters and operation strategies of the water treatment process according to the prediction results and real-time water quality data.

Citation Information

Cited By

  • Comprehensive experimental device for refrigeration and heat exchange of air conditioner

    CN118913738A

  • Sewage monitoring method and system in rainstorm scene based on deep learning

    CN119416129A

  • Sewage treatment aeration optimization control system and method

    CN119689964A

  • Water quality prediction method, device, equipment and medium

    CN119849709A

  • Wet desulphurization parameter optimization method based on data processing

    CN119905158A