Machine learning water quality monitoring model based on random forest algorithm

Abstract

The invention provides a machine learning water quality monitoring model based on a random forest algorithm. The machine learning water quality monitoring model comprises the following steps: A, preprocessing water quality monitoring data; b, selecting features related to a water quality monitoring target from the preprocessed data; c, constructing a random forest model; d, training the constructed random forest model; e, predicting the trained random forest model; f, evaluating the predicted random forest model; g, parameters of the random forest model are adjusted according to the evaluation result for optimization, through historical data training, the model can automatically recognize and predict water quality abnormal events, and pollution can be found and prevented as soon as possible. The random forest algorithm can train a plurality of decision trees at the same time to form a forest, and random sampling and random feature selection are performed on a data set, so that the accuracy and robustness of the model are improved.

Description

Technical Field

[0001] The present application relates to the technical field of water quality monitoring and prediction, and in particular to a machine learning water quality monitoring model based on a random forest algorithm. Background Art

[0002] Water quality monitoring is a key technology for protecting water resources, preventing water pollution, and safeguarding human health. However, traditional water quality monitoring methods usually rely on manual sampling and laboratory analysis, which is time-consuming and has poor real-time performance. In recent years, with the rapid development of data-driven intelligent technology, it has become possible to monitor and predict water quality using machine learning technology.

[0003] Existing water quality monitoring methods often have difficulty in detecting abnormal conditions quickly, accurately, and in real time. The water quality monitoring model based on machine learning algorithms can effectively solve this problem. It analyzes historical data, mines and establishes relationships and patterns between data, and thus predicts and monitors future water quality conditions, achieving water quality safety monitoring and early warning.

[0004] Invention patent CN113723704B discloses a method for rapid water quality prediction based on continuous and graded mixed data. This method uses the classification levels of parameters such as land use area, GDP, and population in the basin to predict water quality changes in different areas of the basin, construct a deep learning neural network, and quickly calculate the surface water quality changes and their spatial distribution under different levels of socio-economic development in the basin, as a basis for basin water environment management. The water quality prediction method of this invention avoids the problem of missing socio-economic statistical data when using socio-economic data for water quality prediction. It does not require accurate socio-economic data, and only socio-economic classification level data can be used for prediction, which is more practical. In addition, compared with traditional numerical simulation methods, water quality prediction using deep learning neural networks has faster calculation speeds and a wider range of applications.

[0005] Invention patent CN114742183B discloses a water pollution early warning method, system, equipment and storage medium based on deep learning. The method includes: setting up several monitoring points in the river channel and obtaining water quality data information of each monitoring point; extracting several pollution characteristic indicators from the water quality data information; constructing an RNN feature fusion model, inputting the pollution characteristic indicators into the RNN feature fusion model to generate fusion characteristic indicators; constructing a water quality monitoring convolutional neural network model and training the convolutional neural network model; converting the fusion characteristic indicators into fusion characteristic vectors, inputting the fusion characteristic vectors into the trained water quality monitoring convolutional neural network model, and outputting the probability value corresponding to the fusion characteristic vector; comparing the difference between the binary probability value and a preset threshold value to determine whether the water quality of the corresponding monitoring point is polluted; and issuing an early warning to the monitoring point module that detects water quality pollution, which can effectively improve the efficiency of water pollution monitoring and issue early warnings in a timely manner.

[0006] Among the existing water quality monitoring models, models based on traditional statistical methods include regression analysis, principal component analysis and other statistical analysis methods. They have the advantage of being simple and easy to understand, but they cannot handle nonlinear relationships and complex connections between features, so they are rarely used in practice.

[0007] Neural network-based models achieve data classification and prediction by imitating the connections between neurons in the human brain. They have strong nonlinear fitting capabilities and can adaptively adjust parameters to better adapt to different data characteristics. However, the training process requires a lot of time and computing resources, and may cause problems such as overfitting.

[0008] The decision tree-based model has the characteristics of easy interpretation, fast calculation speed, and can handle nonlinear relationships and non-continuous data. However, in the process of decision tree growth, it is easily affected by factors such as noise, resulting in instability of the decision tree. Therefore, it is necessary to design a machine learning water quality monitoring model based on the random forest algorithm. Summary of the invention

[0009] The present application provides a machine learning water quality monitoring model based on the random forest algorithm to solve the problems of low accuracy and robustness of the model in the prior art. It realizes the simultaneous training of multiple decision trees to form a forest, and performs random sampling and random feature selection on the data set to improve the accuracy and robustness of the model.

[0010] This application provides a machine learning water quality monitoring model based on a random forest algorithm, comprising the following steps:

[0011] A. Water quality monitoring data preprocessing;

[0012] B. Select features related to water quality monitoring objectives in the preprocessed data;

[0013] C. Random forest model construction;

[0014] D. Train the constructed random forest model;

[0015] E. Make predictions for the trained random forest model;

[0016] F. Evaluate the predicted random forest model;

[0017] G. Finally, adjust the parameters of the random forest model for optimization based on the evaluation results.

[0018] Preferably, the specific process of step A is as follows:

[0019] a. First, clean the original water quality data to remove invalid or incomplete records;

[0020] b. Use linear interpolation or nearest neighbor interpolation to handle missing values;

[0021] c. Handling outliers: When handling outliers, use Tukey quartile method or Z score method to detect and eliminate outliers;

[0022] d. Finally, the data is standardized so that the mean of each feature is 0 and the variance is 1, which can be expressed as follows:

[0023]

[0024] Among them, μ j and σ j They represent the mean and standard deviation of the j-th feature respectively.

[0025] Preferably, in step B, correlation analysis, mutual information, and recursive feature elimination methods are used to select features related to the water quality monitoring target from the preprocessed data;

[0026] m features are selected, denoted as F = {f1, f1, f3, ..., f m},use represents the matrix of selected features, and F represents the selected feature set;

[0027] Among them, the correlation analysis uses the Pearson correlation coefficient to measure the strength of the linear relationship between two variables, and its formula is:

[0028]

[0029] Among them, x i and i Respectively represent the values ​​of the two variables of the i-th sample, and They represent the means of the two variables respectively;

[0030] Mutual information measures the strength of the nonlinear relationship between two variables and is calculated using the following formula:

[0031]

[0032] Where X and Y represent two variables, p(x), p(y) and p(x, y) represent the marginal probability distribution and joint probability distribution of the two variables respectively;

[0033] Recursive feature elimination is an iterative algorithm that removes features that contribute less to the model at each iteration until the best feature set is selected.

[0034] Preferably, in step C, in the random forest model construction stage, the number of decision trees n, the selection based on information gain, and the Gini coefficient are set, and the feature selection method based on information gain and the Gini coefficient are used as the node splitting criterion, and the information gain is calculated using the following formula:

[0035] IG(Y|X j )=H(Y)-H(Y|X j );

[0036] Among them, H(Y) and H(Y|X j ) represent the entropy of the sample category and the given feature X j The conditional entropy of , the formula for calculating the Gini coefficient is:

[0037]

[0038] Among them, value(X j ) represents feature X j All possible values ​​of |X j =v| represents feature X j The number of samples with value v, G(Y|X j =v) indicates that a given feature X j =v when the Gini index of the sample category.

[0039] Preferably, the specific process of step D is as follows:

[0040] a. In the model training phase, use the Bootstrap method to extract M samples from the training set with replacement to construct a training set for each decision tree M;

[0041] b. Then, according to the feature selection method and decision tree growth strategy, the nodes are recursively divided until the stopping condition is reached;

[0042] c. Use Represents the prediction result of the i-th decision tree under the given feature matrix X* and parameter P;

[0043] The training process of the random forest model is expressed as:

[0044]

[0045] in, represents the prediction result of the random forest model for the input sample x, and T represents the number of decision trees.

[0046] quantity, Represents the prediction result of the x-th decision tree for sample x under the given feature matrix X and parameter P.

[0047] Preferably, the specific process of step E is as follows:

[0048] In the model prediction stage, the trained random forest model is used to identify and predict abnormal water quality events for new data; for a new sample The prediction results of the random forest model are composed of the prediction results of multiple decision trees, which can be expressed as follows:

[0049]

[0050] Where I(·) is an indicator function. When the condition in the brackets is met, its value is 1, otherwise it is 0; c represents the category label, and its value is {0, 1}; During the calculation process, the prediction results of each decision tree are The prediction results of all decision trees are then weighted and summed, and the category label with the highest weight is finally selected as the prediction result of the model.

[0051] Preferably, the specific process of step F is as follows:

[0052] In the model evaluation phase, some evaluation indicators, including precision, recall, and F1 score, are used to evaluate the prediction accuracy and practicality of the model; the true label of the test set is Y test , the predicted label is The calculation accuracy is as follows:

[0053]

[0054] Where I() is an indicator function, which takes the value 1 when the condition in the brackets is met, otherwise it takes the value 0; N represents the number of samples in the test set;

[0055] Recall rate indicates the ability of the model to correctly identify all true positive examples. The calculation formula is as follows:

[0056]

[0057] Among them, TP represents the number of true positive examples, FN represents the number of false negative examples, and precision represents the proportion of true positive examples in the samples predicted by the model as positive examples. The calculation formula is as follows:

[0058]

[0059] Among them, FP represents the number of false positive examples;

[0060] The F1 score is an indicator that comprehensively considers the accuracy and recall rate to more comprehensively evaluate the performance of the model. The calculation formula is as follows:

[0061]

[0062] Preferably, the step G specifically comprises: in the model optimization stage, adjusting the parameters of the random forest model according to the evaluation results, including the number of decision trees, feature selection method, and decision tree growth strategy; using grid search, random search, or Bayesian optimization method to search for the optimal parameter combination P = {p1, p1, p3, ..., p k} to improve the predictive performance of the model; when adjusting parameters, the cross-validation method is first used to evaluate the performance of different parameter combinations, and then the parameter combination with the best performance is selected as the parameters of the final model.

[0063] Beneficial effects: The present invention automatically identifies and predicts abnormal water quality events by training historical data, which helps to discover and prevent pollution events in advance. The random forest algorithm is a combination method based on Bootstrap Aggregating (Bagging) and random selection of features. It can realize real-time monitoring and prediction of water quality data by training multiple decision trees at the same time. The algorithm combines many decision trees to form a forest, and each decision tree will randomly sample the data set and select random features to improve the accuracy and robustness of the model.

[0064] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 It is the work flow chart of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification, claims and drawings of this application are intended to cover non-exclusive inclusions.

[0069] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase "embodiments" in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0070] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0071] See also Figure 1 , this application provides a machine learning water quality monitoring model based on random forest algorithm, including the following steps:

[0072] A. Water quality monitoring data preprocessing;

[0073] B. Select features related to water quality monitoring objectives in the preprocessed data;

[0074] C. Random forest model construction;

[0075] D. Train the constructed random forest model;

[0076] E. Make predictions for the trained random forest model;

[0077] F. Evaluate the predicted random forest model;

[0078] G. Finally, adjust the parameters of the random forest model for optimization based on the evaluation results.

[0079] In the present invention, the specific process of step A is as follows:

[0080] a. First, clean the original water quality data to remove invalid or incomplete records;

[0081] b. Use linear interpolation or nearest neighbor interpolation to handle missing values;

[0082] c. Handling outliers: When handling outliers, use Tukey quartile method or Z score method to detect and eliminate outliers;

[0083] d. Finally, the data is standardized so that the mean of each feature is 0 and the variance is 1, which can be expressed as follows:

[0084]

[0085] Among them, μ j and σ j They represent the mean and standard deviation of the j-th feature respectively.

[0086] In the present invention, in step B, correlation analysis, mutual information, and recursive feature elimination methods are used to select features related to the water quality monitoring target from the preprocessed data;

[0087] m features are selected, denoted as F = {f1, f1, f3, ..., f m},use represents the matrix of selected features, and F represents the selected feature set;

[0088] Among them, the correlation analysis uses the Pearson correlation coefficient to measure the strength of the linear relationship between two variables, and its formula is:

[0089]

[0090] Among them, x i and i Respectively represent the values ​​of the two variables of the i-th sample, and They represent the means of the two variables respectively;

[0091] Mutual information measures the strength of the nonlinear relationship between two variables and is calculated using the following formula:

[0092]

[0093] Where X and Y represent two variables, p(x), p(y) and p(x, y) represent the marginal probability distribution and joint probability distribution of the two variables respectively;

[0094] Recursive feature elimination is an iterative algorithm that removes features that contribute less to the model at each iteration until the best feature set is selected.

[0095] In the present invention, the above method is used to select features related to the water quality monitoring target from the pre-processed data.

[0096] In step C of the present invention, in the random forest model construction stage, the number of decision trees n, the selection based on information gain, and the Gini coefficient are set. Assuming that these parameters have been set, they are recorded as P = {p1, p1, p3, ..., p mThe random forest model consists of multiple decision trees, each of which is a classifier. Assume that there are N samples, each with m features, expressed as Use Y∈{0,1} N Represents the category label of the sample, where 0 represents normal and 1 represents abnormal.

[0097] Each decision tree is composed of a series of nodes and edges. Each node corresponds to a feature and its threshold, which is used to divide the sample into two sub-nodes on the left and right. The random forest model reduces the risk of overfitting and improves the accuracy of prediction by integrating multiple decision trees.

[0098] In the present invention, a feature selection method based on information gain and the Gini coefficient are used as the node splitting criterion, and the information gain is calculated using the following formula:

[0099] IG(Y|X j )=H(Y)-H(Y|X j );

[0100] Among them, H(Y) and H(Y|X j ) represent the entropy of the sample category and the given feature X j The conditional entropy of , the formula for calculating the Gini coefficient is:

[0101]

[0102] Among them, value(X j ) represents feature X j All possible values ​​of |X j =v| represents feature X j The number of samples with value v, G(Y|X j =v) indicates that a given feature X j =v when the Gini index of the sample category.

[0103] The training process of the random forest model is divided into two steps, namely, randomly selecting samples and features for training. For each decision tree, M samples and m' features are selected from the original data by sampling with replacement for training, where M and m' are the number of samples and the number of features of the original data, respectively. At each node, k features are randomly selected from the m' features as candidate features, and then the best feature is selected based on the information gain or Gini coefficient of the feature.

[0104] Assuming that n decision trees are trained, the random forest model is an integrated classifier composed of these n decision trees. For new samples, they are sent to each decision tree for classification, and then the final classification result is determined based on the voting results or the average probability value.

[0105] In the present invention, the specific process of step D is as follows:

[0106] a. In the model training phase, use the Bootstrap method to extract M samples from the training set with replacement to construct a training set for each decision tree M;

[0107] b. Then, according to the feature selection method and decision tree growth strategy, the nodes are recursively divided until the stopping condition is reached;

[0108] c. Use Represents the prediction result of the i-th decision tree under the given feature matrix X* and parameter P;

[0109] In random forests, some important parameters need to be determined, including the number of decision trees, maximum depth, node splitting criteria, etc. These parameters have a significant impact on the performance and generalization ability of the model. Therefore, it is necessary to determine the optimal values ​​of these parameters through methods such as cross-validation.

[0110] Assume that the training set contains n samples, each sample has m features, and Represents the feature matrix of the training set, and uses Represents the label of the training set. The samples are divided into training set and validation set according to a certain ratio, and then the training set is used to train the random forest model, and the validation set is used to evaluate the generalization ability of the model.

[0111] For each set of parameter settings, a cross-validation method can be used to evaluate the performance of the model. Specifically, the training set is divided into k folds, and each time k-1 folds are used as the training set, and the remaining fold is used as the validation set. The prediction accuracy of the model on the validation set is calculated. Repeat k times to obtain k prediction accuracy rates. Finally, the average of these accuracy rates is calculated as the performance indicator of this set of parameter settings, and the parameter settings with the best performance are selected as the parameters of the final model.

[0112] The training process of the random forest model is expressed as:

[0113]

[0114] in, represents the prediction result of the random forest model for the input sample x, T represents the number of decision trees, Represents the prediction result of the x-th decision tree for sample x under the given feature matrix X and parameter P.

[0115] Preferably, the specific process of step E is as follows:

[0116] In the model prediction stage, the trained random forest model is used to identify and predict abnormal water quality events for new data; for a new sample The prediction results of the random forest model are composed of the prediction results of multiple decision trees, which can be expressed as follows:

[0117]

[0118] Where I(·) is an indicator function. When the condition in the brackets is met, its value is 1, otherwise it is 0; c represents the category label, and its value is {0, 1}; During the calculation process, the prediction results of each decision tree are The prediction results of all decision trees are then weighted and summed, and the category label with the highest weight is finally selected as the prediction result of the model.

[0119] In the present invention, the specific process of step F is as follows:

[0120] In the model evaluation phase, some evaluation indicators, including precision, recall, and F1 score, are used to evaluate the prediction accuracy and practicality of the model; the true label of the test set is Y test , the predicted label is The calculation accuracy is as follows:

[0121]

[0122] Where I() is an indicator function, which takes the value 1 when the condition in the brackets is met, otherwise it takes the value 0; N represents the number of samples in the test set;

[0123] Recall rate indicates the ability of the model to correctly identify all true positive examples. The calculation formula is as follows:

[0124]

[0125] Among them, TP represents the number of true positive examples, FN represents the number of false negative examples, and precision represents the proportion of true positive examples in the samples predicted by the model as positive examples. The calculation formula is as follows:

[0126]

[0127] Among them, FP represents the number of false positive examples;

[0128] The F1 score is an indicator that comprehensively considers the accuracy and recall rate to more comprehensively evaluate the performance of the model. The calculation formula is as follows:

[0129]

[0130] In the present invention, step G is specifically as follows: in the model optimization stage, the parameters of the random forest model are adjusted according to the evaluation results, including the number of decision trees, feature selection method, and decision tree growth strategy; grid search, random search, or Bayesian optimization method is used to search for the optimal parameter combination P = {p1, p1, p3, ..., p k} to improve the prediction performance of the model; for the parameter adjustment of the random forest model, the following factors need to be considered:

[0131] Number of decision trees: The more decision trees there are in the random forest, the better the stability and generalization of the model will be. However, if there are too many decision trees, it will affect the running speed of the model.

[0132] Feature selection method: The feature selection method determines the number of features used and the selection method when splitting nodes on each decision tree. Different feature selection methods will affect the generalization ability and running speed of the model.

[0133] Decision tree growth strategy: Decision tree growth strategies include node splitting criteria and node pruning. Different growth strategies will affect the complexity and generalization ability of the model.

[0134] When adjusting parameters, the cross-validation method is first used to evaluate the performance of different parameter combinations, and then the parameter combination with the best performance is selected as the parameters of the final model.

[0135] Example 1

[0136] In order to further illustrate the machine learning water quality monitoring model based on the random forest algorithm in the present invention, it is assumed that a water quality data set containing the following features is collected: ammonia nitrogen (NH3-N), chemical oxygen demand (COD), dissolved oxygen (D0), total phosphorus (TP), total nitrogen (TN), pH value, etc. After the data preprocessing stage, the processed feature matrix is ​​obtained and label vector Y.

[0137] Next, using the correlation analysis method, the features that are more relevant to abnormal water quality events, such as ammonia nitrogen, chemical oxygen demand, and dissolved oxygen, are selected. Therefore, m = 3, F = {NH3-N, COD, DO}, and the matrix X* of the selected features is obtained.

[0138] For each sample in the water quality dataset, its water quality index I can be calculated using the following formula:

[0139]

[0140] Among them, x i represents the observed value of the i-th feature, is the weight coefficient, c iIt is an ideal value or reference value. In practical applications, the weight coefficient and the ideal value or reference value need to be selected and adjusted according to different water quality indicators and application scenarios.

[0141] In the random forest model building stage, the decision tree algorithm is used to build the classifier. Assuming the sample space It consists of N samples, each of which is described by m features, so each sample can be represented as an m-dimensional vector For a binary classification problem (normal / abnormal), the sample set needs to be divided into two parts: normal sample set and abnormal sample set We hope to establish a classifier h(x) so that for any sample x, h(x)∈{0,1} can be determined, that is, the sample is normal or abnormal.

[0142] In the random forest algorithm, the samples are randomly split into T parts, each containing N' samples. Then, the decision tree algorithm is used to build T decision trees based on these sample subsets to form a random forest model. When predicting, each decision tree of the new sample makes a classification decision, and finally the category of the sample is determined by voting.

[0143] In the model prediction stage, the trained random forest model is used to identify and predict abnormal water quality events for new data. Get the prediction results And compare it with the actual labels to evaluate the prediction performance of the model.

[0144] In the model evaluation phase, we calculate indicators such as accuracy, recall, and F1 score to evaluate the prediction accuracy and practicality of the model. If the evaluation results are not satisfactory, we can further optimize the model, such as adjusting the number of decision trees, feature selection methods, decision tree growth strategies, and other parameters to improve the prediction performance of the model.

[0145] Example 2

[0146] Feature selection and water quality index calculation: Assume that a water quality data set containing the following features is collected: ammonia nitrogen (NH3-N), chemical oxygen demand (COD), dissolved oxygen (DO), total phosphorus (TP), total nitrogen (TN), pH value, etc. After the data preprocessing stage, the processed feature matrix X = [x1, x1, ..., x n ] T and label vector Y = [y1, y2, ..., y n ].

[0147] Using the correlation analysis method, the features that are more relevant to abnormal water quality events, such as ammonia nitrogen, chemical oxygen demand and dissolved oxygen, were selected. Therefore, X′=[NH3-N, COD, DO], and the matrix of selected features was obtained:

[0148]

[0149] For each sample in the water quality dataset, its water quality index can be calculated using the following formula:

[0150]

[0151] Among them, x ij represents the observed value of the jth feature of the i-th sample, w j is the weight coefficient, r j It is an ideal value or reference value. In practical applications, the weight coefficient and the ideal value or reference value need to be selected and adjusted according to different water quality indicators and application scenarios.

[0152] Random forest model construction: In the random forest model construction stage, the decision tree algorithm is used to build a classifier. Assuming that the sample space consists of n samples and each sample is described by m features, each sample can be represented as an m-dimensional vector. For a binary classification problem (normal / abnormal), the sample set needs to be divided into two parts: a normal sample set and an abnormal sample set. It is hoped that a classifier can be established so that for any sample, C(x) = +1 indicates that the sample is normal, and C(x) = -1 indicates that the sample is abnormal.

[0153] In the random forest algorithm, the samples are randomly split into k parts, each containing n / k samples. Then, the decision tree algorithm is used to build T decision trees based on these sample subsets to form a random forest model. When predicting, each decision tree of the new sample makes a classification decision, and finally the category of the sample is determined by voting.

[0154] When using the CART (Classification and Regression Tree) algorithm to build a decision tree, when performing feature selection on node t, the Gini index can be used as the partitioning criterion, calculated as follows:

[0155]

[0156] Among them, p i It represents the proportion of samples of the i-th category on node t. In addition, methods such as information gain and C4.5 algorithm can also be used for feature selection.

[0157] Model prediction and evaluation: In the model prediction stage, the trained random forest model is used to identify and predict water quality anomalies for new data. For a new sample, its water quality index is first calculated based on the selected features, and then input into the trained random forest model for prediction. For a new sample x, suppose the classification results of T classifiers are obtained Among them C t (x) = 1 means that the sample is normal, C t (x) = -1 indicates that the sample is abnormal. The sample category is determined by voting:

[0158]

[0159] In the model evaluation phase, we calculate the accuracy, recall, F1 score and other indicators to evaluate the prediction accuracy and practicality of the model. Taking accuracy as an example, the definition is as follows:

[0160]

[0161] Among them, rP represents true positive examples (the number of normal samples correctly classified), TN represents true negative examples (the number of abnormal samples correctly classified), FP represents false positive examples (the number of abnormal samples incorrectly classified), and FN represents false negative examples (the number of normal samples incorrectly classified).

[0162] If the evaluation results are not satisfactory, the model can be further optimized, such as adjusting the number of decision trees, feature selection method, decision tree growth strategy and other parameters to improve the prediction performance of the model. For example, the Grid Search strategy can be used to optimize the parameters.

[0163] The machine learning water quality monitoring model based on the random forest algorithm and its supporting water quality sensor have broad application prospects. By training historical data, automatically identifying and predicting abnormal water quality events, the model can detect and prevent pollution events in advance. Using a large amount of historical data and advanced machine learning technology, the model can achieve real-time and accurate water quality monitoring in different environments and application scenarios, ensuring the safety and sustainable use of water resources.

[0164] The present invention can also optimize and improve the model to further improve its prediction performance and practicality. For example, an adaptive enhancement method is used to build a more robust classifier, or emerging technologies such as deep learning are used to improve the performance of the model. At the same time, by introducing knowledge from other related fields, such as hydrology, ecology, etc., the characteristics and parameters of the model are enriched to better adapt to actual application scenarios.

[0165] In practical applications, the model and sensor can be widely used in water resources management, environmental protection, aquaculture and other fields. For example, in urban water supply systems, abnormal water quality events in the water supply system can be monitored, so that corresponding emergency measures can be taken to prevent the spread of pollutants. In the field of industrial wastewater treatment, it can help companies evaluate the effect of wastewater treatment and optimize production processes and wastewater treatment technologies. In the field of environmental monitoring, long-term water quality monitoring of lakes, rivers and other water bodies can be carried out to timely discover pollution sources and provide strong technical support for environmental protection departments. In the field of water resources management, it can provide real-time and accurate water quality information for water resources management departments to guide the rational development and utilization of water resources.

[0166] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A machine learning water quality monitoring model based on random forest algorithm, characterized in that: The steps include: A. Water quality monitoring data preprocessing; B. Select features related to water quality monitoring objectives in the preprocessed data; C. Random forest model construction; D. Train the constructed random forest model; E. Make predictions for the trained random forest model; F. Evaluate the predicted random forest model; G. Finally, adjust the parameters of the random forest model for optimization based on the evaluation results.

2. The machine learning water quality monitoring model based on random forest algorithm according to claim 1 is characterized in that: The specific process of step A is as follows: a. First, clean the original water quality data to remove invalid or incomplete records; b. Use linear interpolation or nearest neighbor interpolation to handle missing values; c. Handling outliers: When handling outliers, use Tukey quartile method or Z score method to detect and eliminate outliers; d. Finally, the data is standardized so that the mean of each feature is 0 and the variance is 1, which can be expressed as follows: Among them, μ j and σ j They represent the mean and standard deviation of the j-th feature respectively.

3. The machine learning water quality monitoring model based on random forest algorithm according to claim 1, characterized in that: In step B, correlation analysis, mutual information, and recursive feature elimination methods are used to select features related to the water quality monitoring target from the preprocessed data; m features are selected, denoted as F = {f1, f1, f3, ..., f m },use represents the matrix of selected features, and F represents the selected feature set; Among them, the correlation analysis uses the Pearson correlation coefficient to measure the strength of the linear relationship between two variables, and its formula is: Among them, x i and i Respectively represent the values ​​of the two variables of the i-th sample, and They represent the means of the two variables respectively; Mutual information measures the strength of the nonlinear relationship between two variables and is calculated using the following formula: Where X and Y represent two variables, p(x), p(y) and p(x, y) represent the marginal probability distribution and joint probability distribution of the two variables respectively; Recursive feature elimination is an iterative algorithm that removes features that contribute less to the model at each iteration until the best feature set is selected.

4. The machine learning water quality monitoring model based on random forest algorithm according to claim 1 is characterized in that: In the step C, during the random forest model construction phase, the number of decision trees n, the selection based on information gain, and the Gini coefficient are set, and the feature selection method based on information gain and the Gini coefficient are used as the node splitting criterion, and the information gain is calculated using the following formula: IG(Y|X j )=H(Y)-H(Y|X j ); Among them, H(Y) and H(Y|X j ) represent the entropy of the sample category and the given feature X j The conditional entropy of , the formula for calculating the Gini coefficient is: Among them, value(X j ) represents feature X j All possible values ​​of |X j =v| represents feature X j The number of samples with value v, G(Y|X j =v) indicates that a given feature X j =v when the Gini index of the sample category.

5. The machine learning water quality monitoring model based on random forest algorithm according to claim 1 is characterized in that: The specific process of step D is as follows: a. In the model training phase, use the Bootstrap method to extract M samples from the training set with replacement to construct a training set for each decision tree M; b. Then, according to the feature selection method and decision tree growth strategy, the nodes are recursively divided until the stopping condition is reached; c. Use Represents the prediction result of the i-th decision tree under the given feature matrix X* and parameter P; The training process of the random forest model is expressed as: in, represents the prediction result of the random forest model for the input sample x, T represents the number of decision trees, Represents the prediction result of the x-th decision tree for sample x under the given feature matrix X and parameter P.

6. The machine learning water quality monitoring model based on random forest algorithm according to claim 1 is characterized in that: The specific process of step E is as follows: In the model prediction stage, the trained random forest model is used to identify and predict abnormal water quality events for new data; for a new sample The prediction results of the random forest model are composed of the prediction results of multiple decision trees, which can be expressed as follows: Where I(·) is an indicator function. When the condition in the brackets is met, its value is 1, otherwise it is 0; c represents the category label, and its value is {0, 1}; During the calculation process, the prediction results of each decision tree are The prediction results of all decision trees are then weighted and summed, and the category label with the highest weight is finally selected as the prediction result of the model.

7. The machine learning water quality monitoring model based on random forest algorithm according to claim 1 is characterized in that: The specific process of step F is as follows: In the model evaluation phase, some evaluation indicators, including precision, recall, and F1 score, are used to evaluate the prediction accuracy and practicality of the model; the true label of the test set is Y test , the predicted label is The calculation accuracy is as follows: Where I() is an indicator function, which takes the value 1 when the condition in the brackets is met, otherwise it takes the value 0; N represents the number of samples in the test set; Recall rate indicates the ability of the model to correctly identify all true positive examples. The calculation formula is as follows: Among them, TP represents the number of true positive examples, FN represents the number of false negative examples, and precision represents the proportion of true positive examples in the samples predicted by the model as positive examples. The calculation formula is as follows: Among them, FP represents the number of false positive examples; The F1 score is an indicator that comprehensively considers the accuracy and recall rate to more comprehensively evaluate the performance of the model. The calculation formula is as follows:

8. The machine learning water quality monitoring model based on random forest algorithm according to claim 1, characterized in that: The step G specifically includes: in the model optimization stage, adjusting the parameters of the random forest model according to the evaluation results, including the number of decision trees, feature selection method, and decision tree growth strategy; using grid search, random search, or Bayesian optimization method to search for the optimal parameter combination P = {p1, p1, p3, ..., p k } to improve the predictive performance of the model; when adjusting parameters, the cross-validation method is first used to evaluate the performance of different parameter combinations, and then the parameter combination with the best performance is selected as the parameters of the final model.

Images