Method and device for optimizing water quality evaluation based on interpretable machine learning
By using principal component analysis and various importance assessment methods, the optimal machine learning model was selected, which solved the problems of subjectivity in water quality assessment and automated model identification, and achieved efficient and accurate water quality monitoring and management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INST OF AQUATIC LIFE ACAD SINICA
- Filing Date
- 2026-01-27
- Publication Date
- 2026-05-19
AI Technical Summary
Existing water quality assessment methods suffer from strong subjectivity, difficulty in achieving automated identification of optimal models, and a lack of a unified evaluation system for key indicator selection. This leads to inconsistent evaluation results, an inability to accurately define the importance of each indicator, and affects the rationality and reliability of model construction.
Principal component analysis was used to determine the contribution of environmental indicators, and a water quality index was constructed by combining a penalty mechanism with scoring. The optimal machine learning model was selected through k-fold cross-validation and multiple importance assessment methods for water quality monitoring and evaluation.
It improves the accuracy and efficiency of water quality assessment, identifies key environmental indicators, reduces monitoring costs, and enhances the operability and real-time nature of water quality management.
Smart Images
Figure CN122067652A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of water quality assessment, and specifically to a method and apparatus for optimizing water quality assessment based on interpretable machine learning. Background Technology
[0002] Freshwater resources are a fundamental element for human survival and development, and their quality and safety are directly related to ecosystem stability, sustainable economic and social operation, and public health and well-being. However, with the expansion of industry and agriculture and the acceleration of urbanization, highly nutrient-intensive and high-load water bodies, such as domestic sewage, industrial emissions, and agricultural non-point source runoff, are constantly flowing into surface waters. This impairs the self-purification process, increases eutrophication events, and exacerbates water quality fluctuations, bringing unprecedented complexity and uncertainty to water resource management. Against this backdrop, establishing a monitoring and evaluation system that can comprehensively reflect the state of the water environment, dynamically track changes, and serve governance decisions has become an urgent task.
[0003] However, existing evaluation methods have many inherent flaws and are difficult to meet the needs of precise research and management: First, traditional single-factor evaluation methods focus on isolated indicator analysis, and the scores and contribution of each environmental factor indicator are mostly determined by experience, lacking objective quantitative basis, and are highly subjective, which can easily lead to biased evaluation results; Second, existing machine learning model prediction effect evaluations mostly use single-dimensional indicators, which cannot comprehensively measure model performance and rely on manual comparison to select the optimal model, making it difficult to achieve automated identification of the optimal prediction model, which is time-consuming, labor-intensive, and highly subjective; Third, the selection of key indicators lacks a unified and scientific evaluation system, and different models have significant differences in judging the importance of indicators, resulting in inconsistent selection results, making it impossible to accurately define the relative importance of each indicator, and thus affecting the rationality of subsequent model construction and the reliability of evaluation results. Summary of the Invention
[0004] This invention provides a method and apparatus for optimizing water quality assessment based on interpretable machine learning, in order to solve at least one of the above-mentioned technical problems.
[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A method for optimizing water quality assessment based on interpretable machine learning, comprising: S1, collect data on multiple environmental indicators from multiple monitoring points in a lake or reservoir over a long period of time to construct a water quality dataset; S2, perform principal component analysis on the water quality dataset to determine the contribution of each environmental indicator; score each environmental indicator based on the preset optimal range and the penalty mechanism for deviating from the preset optimal range, and obtain the score of each environmental indicator; calculate the water quality index of each monitoring point based on the contribution and score of each environmental indicator. S3. Using the water quality dataset and the water quality index of each monitoring point, k-fold cross-validation is performed on multiple machine learning models, and multiple performance indicators of each machine learning model are calculated after each test. The inverse average of each performance indicator of each machine learning model is weighted and averaged to obtain the comprehensive evaluation value of each machine learning model. Based on the comprehensive evaluation value, the optimal machine learning model is selected from multiple machine learning models as the prediction model. S4. Based on the water quality dataset and the water quality index of each monitoring point, a variety of importance assessment methods are used to assess the impact of each environmental indicator on the water quality index, obtain the importance of each environmental indicator in various importance assessment methods, and calculate the comprehensive importance of each environmental indicator based on the importance of each environmental indicator in various importance assessment methods. S5. Based on the order of decreasing comprehensive importance, each environmental indicator is gradually input into the prediction model for training and water quality index prediction. The model performance of the prediction model is calculated step by step to generate a model performance sequence. The optimal prediction model is determined based on the increase in model performance in the model performance sequence. The optimal prediction model and its corresponding environmental indicators are used for water quality monitoring and evaluation.
[0006] Based on the above technical solution, the present invention can be further improved as follows.
[0007] Furthermore, in step S2, principal component analysis is performed on the water quality dataset to determine the contribution of each environmental indicator, specifically including: Principal component analysis was used to perform principal component analysis on the water quality dataset to extract the top principal components and their corresponding variance contribution rates from each monitoring point whose cumulative variance was greater than or equal to a preset percentage. Based on the extracted principal components and their corresponding variance contribution rates for each monitoring point, the contribution of each environmental indicator at each monitoring point is calculated. The formula for calculating the contribution of each environmental indicator at each monitoring point is as follows: In the formula, Indicates the first The first monitoring point The contribution of each environmental indicator Indicates the extracted first The first monitoring point The variance contribution rate corresponding to each principal component This indicates the total number of principal components extracted. Indicates the extracted first The first monitoring point The principal component and the first The loading matrix between the environmental indicators, and: In the formula, Indicates the extracted first The first monitoring point Principal components, This indicates the total number of environmental indicators. Indicates the first The first monitoring point Data for each environmental indicator.
[0008] Furthermore, in S2, let the first The preset optimal range for each environmental indicator is ;in, Indicates the first The lower limit of the preset optimal range for each environmental indicator. Indicates the first The upper limit of the preset optimal range for each environmental indicator; The formula for calculating the score of each environmental indicator at each monitoring point is as follows: In the formula, Indicates the first The first monitoring point Scores for each environmental indicator; This represents a preset score constant; Let the lower bound penalty function be used. Upper limit penalty function; Indicates the degree below the lower limit, and ; Indicates the degree exceeding the upper limit, and ; Indicates the first The first monitoring point Data for each environmental indicator.
[0009] Furthermore, in step S2, the formula for calculating the water quality index at each monitoring point is as follows: In the formula, Indicates the first Water quality index at each monitoring point Indicates the first The first monitoring point The scores of each environmental indicator Indicates the first The first monitoring point The contribution of each environmental indicator This indicates the total number of environmental indicators.
[0010] Furthermore, in S3, multiple machine learning models include the lm model, ridge model, lasso model, enet model, glmnet model, bayesglm model, SVM model, svmLinear model, svmRadial model, treebag model, rf model, gbm model, xgbTree model, and earth model.
[0011] Furthermore, in S3, multiple performance metrics include , and ; The average values of each performance metric for each machine learning model are weighted and averaged. Specifically, the average values of (1-R²), MSE, and MAE for each machine learning model are weighted and averaged.
[0012] Furthermore, in S4, the formula for calculating the overall importance of each environmental indicator is: In the formula, Indicates the first The combined importance of each environmental indicator Indicates the first The environmental indicator in the first Standardization of importance in various importance assessment methods This indicates the total number of importance assessment methods.
[0013] Furthermore, in S4, various importance assessment methods include standardized regression coefficient method, random forest model method and SHAP analysis method.
[0014] Furthermore, in S5, the model performance sequence is specifically based on R... 2 The sequence of indicators, and in the model performance sequence, R 2 The rate of increase in the indicator gradually decreased; Determining the optimal prediction model specifically includes: Based on the model performance sequence, calculate R for each adjacent pair of R. 2 The second R in the indicator 2 The rate of change of the indicators is used to obtain the sequence of model performance change rates. The first R in the model performance change rate sequence that is less than a preset change rate threshold is identified. 2 The previous prediction model corresponding to the indicator is taken as the optimal prediction model.
[0015] Based on the aforementioned method for optimizing water quality assessment using interpretable machine learning, this invention also provides an apparatus for optimizing water quality assessment using interpretable machine learning.
[0016] An apparatus for optimizing water quality assessment based on interpretable machine learning includes a processor, a memory, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the method for optimizing water quality assessment based on interpretable machine learning as described above.
[0017] The beneficial effects of this invention are as follows: The method and apparatus for optimizing water quality assessment based on interpretable machine learning of this invention have the following advantages: (1) This invention takes a global perspective and uses principal component analysis to determine the contribution of each environmental indicator. For the score of each environmental indicator, a penalty mechanism is built based on the preset optimal range of each environmental indicator to accurately quantify the index score of each environmental indicator when it deviates from the preset optimal range. The water quality index is constructed by the contribution and score of each environmental indicator. Therefore, this invention can effectively improve the accuracy and efficiency of water quality assessment and overcome the limitations of traditional methods in dynamic changes and multi-parameter integration.
[0018] (2) In the process of evaluating the performance of multiple machine learning models using k-fold cross-validation, the present invention performs a weighted average after back averaging of the performance indicators of each machine learning model to obtain a comprehensive evaluation value of each machine learning model, thereby comprehensively evaluating the model performance and facilitating the automatic identification of the optimal prediction model.
[0019] (3) This invention uses multiple importance assessment methods to comprehensively assess the impact of various environmental indicators on the water quality index, thereby identifying key environmental indicators for water quality changes, providing a basis for determining the optimal prediction model, and achieving the monitoring purpose through a better combination of environmental indicators, reducing unnecessary environmental indicators. Therefore, it significantly improves the efficiency of water quality assessment while ensuring high accuracy. The optimal prediction model can effectively reduce monitoring costs and improve operability in actual water quality management.
[0020] (4) In this invention, the optimal prediction model is selected by evaluating the model performance using the rate of change of model performance, which can more efficiently and accurately identify the optimal model and realize the automatic optimization of the model. Attached Figure Description
[0021] Figure 1 This is a flowchart of a method for optimizing water quality assessment based on interpretable machine learning according to the present invention. Figure 2 This is a structural block diagram of a device for optimizing water quality assessment based on interpretable machine learning, according to the present invention. Detailed Implementation
[0022] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.
[0023] like Figure 1 As shown, a method for optimizing water quality assessment based on interpretable machine learning includes: S1, collect data on multiple environmental indicators from multiple monitoring points in a lake or reservoir over a long period of time to construct a water quality dataset; S2, perform principal component analysis on the water quality dataset to determine the contribution of each environmental indicator; score each environmental indicator based on the preset optimal range and the penalty mechanism for deviating from the preset optimal range, and obtain the score of each environmental indicator; calculate the water quality index of each monitoring point based on the contribution and score of each environmental indicator. S3. Using the water quality dataset and the water quality index of each monitoring point, k-fold cross-validation is performed on multiple machine learning models, and multiple performance indicators of each machine learning model are calculated after each test. The inverse average of each performance indicator of each machine learning model is weighted and averaged to obtain the comprehensive evaluation value of each machine learning model. Based on the comprehensive evaluation value, the optimal machine learning model is selected from multiple machine learning models as the prediction model. S4. Based on the water quality dataset and the water quality index of each monitoring point, a variety of importance assessment methods are used to assess the impact of each environmental indicator on the water quality index, obtain the importance of each environmental indicator in various importance assessment methods, and calculate the comprehensive importance of each environmental indicator based on the importance of each environmental indicator in various importance assessment methods. S5. Based on the order of decreasing comprehensive importance, each environmental indicator is gradually input into the prediction model for training and water quality index prediction. The model performance of the prediction model is calculated step by step to generate a model performance sequence. The optimal prediction model is determined based on the increase in model performance in the model performance sequence. The optimal prediction model and its corresponding environmental indicators are used for water quality monitoring and evaluation.
[0024] This invention presents a method for optimizing water quality assessment based on interpretable machine learning. It normalizes and weights multidimensional environmental indicators of water bodies using the Water Quality Index (WQI), effectively improving the accuracy and efficiency of water quality assessment. Simultaneously, it innovatively introduces a machine learning model combined with various importance assessment methods to uncover the contribution of each environmental indicator to water quality changes, thus providing interpretable decision-making basis for water quality management. By combining the machine learning model with various importance assessment methods, this invention can screen key water quality indicators and construct optimal prediction models based on these indicators, significantly reducing monitoring costs and improving the real-time performance and operability of water quality assessment.
[0025] The following is a detailed explanation of each step.
[0026] S1: Construct a water quality dataset.
[0027] To comprehensively assess the nutrient status of water bodies such as lakes and reservoirs, the first step is to conduct on-site sampling or collect long-term water quality data from several monitoring points within the lake / reservoir. This involves monitoring multiple environmental indicators, such as total phosphorus (TP), total nitrogen (TN), and phosphate (PO4). 3 -), ammonia nitrogen (NH4) + -N), nitrite nitrogen (NO2) - -N), nitrate nitrogen (NO3) - -N), permanganate index (COD) Mn The data collected included chlorophyll a (Chl.a), water temperature (WT), dissolved oxygen (DO), pH, conductivity (Cond), oxidation-reduction potential (ORP), and water transparency (SD). The collected data was then cleaned (including outlier removal and missing value handling) to create a complete water quality dataset. , Indicates the serial number of the monitoring point, and , Indicates the total number of monitoring points; Indicates the serial number of the environmental indicator, and , (This represents the total number of environmental indicators.) Indicates the first The first monitoring point Data for each environmental indicator.
[0028] S2, calculate the Water Quality Index (WQI).
[0029] (1) Determine the contribution of environmental indicators: Principal component analysis was used to analyze the water quality dataset. Principal component analysis is performed to extract the top principal components with cumulative variance greater than or equal to a preset percentage at each monitoring point, along with their corresponding variance contribution rates. For example: extract the... Among the monitoring points, the top [number] have a cumulative variance greater than or equal to 80%. Principal Components , ( (Indicates the index of the principal component): and the variance contribution rate of the corresponding principal components. 。 Therefore, the first The first monitoring point The weight of each environmental indicator is its contribution. In the formula, Indicates the extracted first The first monitoring point Principal components, This indicates the total number of environmental indicators. Indicates the first The first monitoring point One environmental indicator, Indicates the extracted first The first monitoring point The principal component and the first The loading matrix between environmental indicators is a key matrix in principal component analysis that connects the original variables (environmental indicators in this invention) with the principal components. Each element (loading coefficient) in the matrix represents the linear correlation coefficient between the corresponding original variable and a certain principal component, which essentially reflects the degree of contribution and correlation direction of the original variable to each principal component. Indicates the first The first monitoring point The contribution of each environmental indicator Indicates the extracted first The first monitoring point The variance contribution rate corresponding to each principal component This indicates the total number of principal components extracted.
[0030] (2) Determine the scores for environmental indicators: First, given the first Preset optimal range for each environmental indicator ;in, Indicates the first The lower limit of the preset optimal range for each environmental indicator. Indicates the first The upper limit of the preset optimal range for each environmental indicator.
[0031] The indicator data is within the preset optimal range. The first monitoring point The scores for each environmental indicator are (It can usually be set) =100). The indicator data is lower or higher than the preset optimal range. The first monitoring point The scores for each environmental indicator will decrease. Assumption: The degree to which the value falls below the lower limit of the preset optimal range is: ; The degree to which the value exceeds the upper limit of the preset optimal range is: ; In the formula, Indicates the degree below the lower limit; Indicates the degree exceeding the upper limit. Indicates the first The first monitoring point Data for each environmental indicator.
[0032] The penalty for falling below the lower limit of the preset optimal range is The penalty for exceeding the upper limit of the preset optimal range is .
[0033] Therefore, the first The first monitoring point The scores for each environmental indicator are: In the formula, Indicates the first The first monitoring point Scores for each environmental indicator; This represents a preset score constant; Let the lower bound penalty function be used. Upper limit penalty function; Indicates the degree below the lower limit, and ; Indicates the degree exceeding the upper limit, and ; Indicates the first The first monitoring point The data for each environmental indicator. Here... and It can be represented as and and Linear relationships, such as , It can also represent non-linear relationships, such as... , .
[0034] (3) Calculate the Water Quality Index (WQI): Based on the contribution and score of environmental indicators, the Water Quality Index (WQI) for all monitoring points is calculated. The calculation formula is as follows: (The formula is obtained by normalizing the weights and scores of all environmental indicators.) In the formula, Indicates the first Water quality index at each monitoring point Indicates the first The first monitoring point The scores of each environmental indicator Indicates the first The first monitoring point The contribution of each environmental indicator This indicates the total number of environmental indicators.
[0035] The subsequent analysis can assess the overall status and spatiotemporal variation patterns of the Water Quality Index (WQI). By analyzing the range of the WQI, the nutrient status of the water body can be qualitatively described, and the basic nutrient state of the water body can be understood. Then, R software is used to calculate the mean WQI for different time periods (annual, quarterly, monthly) and different regions / locations. Through multiple comparative analysis, significant differences in the WQI across different times, regions, and locations are evaluated, thereby identifying the spatiotemporal variation patterns of the water body's nutrient status. This provides key seasonal and regional priority monitoring points for subsequent water quality management, improving the efficiency and targeting of pollution control.
[0036] S3, Select the prediction model.
[0037] Using a water quality dataset and water quality indices from various monitoring points (the dataset is composed of the combined water quality dataset and the water quality indices from each monitoring point), various machine learning models were built using R language, including lm, ridge, lasso, enet, glmnet, bayesglm, SVM, svmLinear, svmRadial, treebag, rf, gbm, xgbTree, and earth models. First, the dataset was divided into... k The dataset is divided into 10 subsets, and k-fold cross-validation is used (e.g., the dataset is divided into 10 subsets, and 9 subsets are selected as the training set each time, with the remaining 1 subset used as the test set). This process is repeated. kNext, ensure that each subset participates in the test once. Then, calculate the performance metric for each test. For a single performance metric G, if G is a positive metric (i.e., the larger G is, the better the model's prediction performance), then the performance metric is replaced with 1-G. In this embodiment, after negatively transforming each performance metric (if the performance metric itself is a negative metric, no transformation is needed), a weighted average is taken to obtain the comprehensive evaluation value of the performance of various machine learning models. The smaller the comprehensive evaluation value, the better the model performance; conversely, the larger the value, the worse the performance. In this embodiment, the coefficient of determination (R²), mean squared error (MSE), and mean absolute error (MAE) are selected as the model's performance metrics. The calculation formulas are as follows: In the formula, Indicates the first i WQI observations at each monitoring point Indicates the first i The predicted WQI value for each monitoring point, where ya represents the mean of the observed WQI values for all monitoring points. This indicates the total number of monitoring points.
[0038] In the formula, Indicates the first i WQI observations at each monitoring point Indicates the first i The predicted WQI value for each monitoring point. This indicates the total number of monitoring points.
[0039] In the formula, Indicates the first i WQI observations at each monitoring point Indicates the first i The predicted WQI value for each monitoring point. This indicates the total number of monitoring points.
[0040] The average values of (1-R²), MSE, and MAE of each machine learning model are then weighted and averaged to obtain the comprehensive evaluation value of each machine learning model (the weights are selected according to the situation; in this embodiment, the average value is selected). The smaller the comprehensive evaluation value, the better the model fits. Therefore, the machine learning model with the smallest comprehensive evaluation value is selected as the prediction model.
[0041] S4, Calculate the overall importance of environmental indicators.
[0042] The importance of indicators calculated using different methods may vary. To better determine the impact of each environmental indicator on the water quality index, this invention combines multiple importance assessment methods, such as standardized regression coefficients, random forest models, SHAP analysis, and correlation coefficients, to comprehensively evaluate the importance of each environmental indicator. For example, methods include standardized regression coefficients, calculating feature importance based on Gini Importance, and SHAP analysis.
[0043] (1) Standardized regression coefficients: Standardized regression coefficients can be used to measure the relative contribution of each independent variable to the dependent variable; the larger the value, the greater the contribution of that environmental indicator to the Water Quality Index (WQI). The calculation formula is as follows: In the formula, It is the first Standardized regression coefficients of each environmental indicator It is the first The original regression coefficients of each environmental indicator, It is the first Standard deviation of each environmental indicator It is the standard deviation of the Water Quality Index (WQI).
[0044] (2) Random Forest Model: Using water quality index as the dependent variable and various environmental indicators as independent variables, a model was constructed. The randomForest package in R language was used, and the importance parameter was extracted to calculate the feature importance as the importance of each environmental indicator. .
[0045] (3) SHAP analysis: The SHAP (SHapley Additive exPlanations) tool is introduced to explain the contribution of each environmental indicator to the Water Quality Index (WQI) variation in the black-box model. By calculating the SHAP value of each environmental indicator, its impact on WQI variation is assessed, and the SHAP values of all parameters are ranked to identify the environmental indicators with the greatest impact on WQI. This will help provide key monitoring targets for subsequent water quality management. The formula for calculating the SHAP value is as follows: In the formula, For the first SHAP values of several environmental indicators M The total number of input features. N For the set of all features, A subset of features excluding the features themselves. For subset Size; Indicates using only subsets The features are the predicted values of the model.
[0046] Therefore, if the present invention selects ( 3) The importance assessment method, therefore the first ( Of the ) importance assessment methods, the first The environmental indicator in the first Importance in the importance assessment method is ( ), and then for the first The importance of each environmental indicator in the importance assessment method { , , , ..., Standardization yields new results ( Therefore, the first The overall importance of each environmental indicator is as follows: In the formula, Indicates the first The combined importance of each environmental indicator Indicates the first The environmental indicator in the first Standardization of importance in various importance assessment methods This indicates the total number of importance assessment methods.
[0047] S5, Determine the optimal prediction model.
[0048] The overall importance of each environmental indicator calculated in S4 is ranked (in descending order of overall importance), and each indicator is selected and gradually added to the prediction model. Assumptions: [The text abruptly ends here, so the translation stops as well.] The importance ranking is 3, 1, 2, ..., and the prediction model determined in S3 is a linear regression model, yielding z regression equations as shown in Table 1. The water quality index (WQI) is recalculated using the contribution and score calculated in S2. Finally, through linear regression analysis with the initial water quality index (WQI), the coefficient of determination (R²) is used to test whether the optimal water quality index (WQI) can effectively represent the overall nutrient status of the water body.
[0049] Table 1: Regression Equation Models and Their Performance Evaluation Metrics From this, we can obtain the model performance sequence. { , , … The model performance sequence increases non-linearly with the addition of new indicators. After reaching a certain level, the rate of increase slows significantly. The model with the best performance at this point is the optimal prediction model. Specifically: First, analyze the model performance sequence. First-order differencing is performed, which involves subtracting the performance evaluation index of the preceding adjacent model from the performance evaluation index of the following model to obtain the change in the performance evaluation index of the following model. Then, dividing this change by the performance evaluation index of the preceding adjacent model yields the rate of change of the performance evaluation index of the following model. For example, the rate of change of the performance evaluation index of the z-th model is: From this, we can obtain the rate-of-change sequence of the performance evaluation metrics for each model. ,in As the number of added variables (environmental indicators) increases, when the rate of change of the model's performance evaluation indicators gradually decreases to less than a certain value (which can be 0.05), the preceding model is considered the optimal prediction model. This model can better reflect the nutrient status of the water body. In future water quality monitoring, the environmental indicators required by the optimal prediction model can be monitored in a focused manner, saving sampling costs and improving monitoring efficiency.
[0050] This invention overcomes the limitations of traditional single-factor evaluation methods, which struggle to integrate multidimensional information and reveal spatiotemporal evolution and propagation paths. It employs a Water Quality Index (WQI) framework to normalize, weight, and aggregate multiple environmental indicators, forming a comprehensive quantitative representation that can be compared across time and space. Furthermore, it optimizes parameter selection and weighting for differences in water body types and management needs across regions, constructing a minimized indicator set (WQI) while maintaining a high correlation with the original WQI. min To reduce monitoring costs and improve the timeliness of evaluation and early warning, the system integrates historical and on-site monitoring data from multiple time periods and spatial scales. It combines time series analysis and spatiotemporal statistical methods to characterize the seasonality, trends, and abrupt changes of key indicators. Multiple machine learning models are introduced for feature selection and importance ranking. Overfitting is controlled through model integration and cross-validation, and the credibility of conclusions is improved through sensitivity analysis and uncertainty quantification.
[0051] The method of this invention can systematically compare the spatiotemporal distribution patterns and evolution trends of major water quality indices, identify key periods, hotspot water areas, and potential risk thresholds; it can simultaneously reflect the overall situation and local differences in the WQI comprehensive evaluation model, construct grading standards and health baselines, achieve comparability assessments across different times and regions, and use the results for status diagnosis and effectiveness evaluation; it uses machine learning to screen and rank the main control factors, elucidating the dominant mechanisms of natural processes and human activities on water quality changes, thereby forming an interpretable and transferable optimized WQI and monitoring sampling optimization scheme, proposing indicator combinations and frequency configurations under cost constraints, and supporting early warning triggering, source tracing identification, and scenario simulation. This invention will output an operable and reusable water quality monitoring-evaluation-decision support framework, providing a scientific basis and practical path for water environment governance, drinking water safety assurance, and watershed ecological protection, and providing scalable experience and tools for refined water resource management and adaptive governance in different regions.
[0052] Based on the aforementioned method for optimizing water quality assessment using interpretable machine learning, this invention also provides an apparatus for optimizing water quality assessment using interpretable machine learning.
[0053] like Figure 2 As shown, an apparatus for optimizing water quality assessment based on interpretable machine learning includes a processor, a memory, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the method for optimizing water quality assessment based on interpretable machine learning as described above.
[0054] In one optional embodiment, an apparatus for optimizing water quality assessment based on interpretable machine learning is provided, such as... Figure 2 As shown. Figure 2 The illustrated device for optimizing water quality assessment based on interpretable machine learning includes a processor and a memory. The processor and memory are connected, for example, via a bus. Optionally, the device may further include a transceiver, which can be used for data interaction between the device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver is not limited to one unit, and the structure of this device for optimizing water quality assessment based on interpretable machine learning does not constitute a limitation on the embodiments of the present invention.
[0055] The processor can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLC (Programmable Logic Controller), a FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0056] A bus can include a pathway for transmitting information between the aforementioned components. The bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0057] The memory may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these.
[0058] The memory stores application code (computer program) that executes the present invention, and its execution is controlled by a processor. The processor executes the application code stored in the memory to implement the content shown in the foregoing method embodiments.
[0059] The device for optimizing water quality assessment based on interpretable machine learning can also be a terminal device. The terminal device can be any device that can install applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.
[0060] It should be noted that, Figure 2 The apparatus shown for optimizing water quality assessment based on interpretable machine learning is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.
[0061] In summary, the method and apparatus for optimizing water quality assessment based on interpretable machine learning of the present invention have the following advantages: (1) This invention takes a global perspective and uses principal component analysis to determine the contribution of each environmental indicator. For the score of each environmental indicator, a penalty mechanism is built based on the preset optimal range of each environmental indicator to accurately quantify the index score of each environmental indicator when it deviates from the preset optimal range. The water quality index is constructed by the contribution and score of each environmental indicator. Therefore, this invention can effectively improve the accuracy and efficiency of water quality assessment and overcome the limitations of traditional methods in dynamic changes and multi-parameter integration.
[0062] (2) In the process of evaluating the performance of multiple machine learning models using k-fold cross-validation, the present invention performs a weighted average after back averaging of the performance indicators of each machine learning model to obtain a comprehensive evaluation value of each machine learning model, thereby comprehensively evaluating the model performance and facilitating the automatic identification of the optimal prediction model.
[0063] (3) This invention uses multiple importance assessment methods to comprehensively assess the impact of various environmental indicators on the water quality index, thereby identifying key environmental indicators for water quality changes, providing a basis for determining the optimal prediction model, and achieving the monitoring purpose through a better combination of environmental indicators, reducing unnecessary environmental indicators. Therefore, it significantly improves the efficiency of water quality assessment while ensuring high accuracy. The optimal prediction model can effectively reduce monitoring costs and improve operability in actual water quality management.
[0064] (4) In this invention, the optimal prediction model is selected by evaluating the model performance using the rate of change of model performance, which can more efficiently and accurately identify the optimal model and realize the automatic optimization of the model.
[0065] This invention is applicable to water quality monitoring and assessment in different regions. It can reduce unnecessary monitoring indicators and lower the cost and workload of water quality monitoring while ensuring the accuracy of the assessment. It provides a scientific basis and effective technical means for water environment management, drinking water safety assurance and watershed ecological protection.
[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for optimizing water quality assessment based on interpretable machine learning, characterized in that, include: S1, collect data on multiple environmental indicators from multiple monitoring points in a lake or reservoir over a long period of time to construct a water quality dataset; S2, perform principal component analysis on the water quality dataset to determine the contribution of each environmental indicator; score each environmental indicator based on the preset optimal range and the penalty mechanism for deviating from the preset optimal range, and obtain the score of each environmental indicator; calculate the water quality index of each monitoring point based on the contribution and score of each environmental indicator. S3. Using the water quality dataset and the water quality index of each monitoring point, k-fold cross-validation is performed on multiple machine learning models, and multiple performance indicators of each machine learning model are calculated after each test. The inverse average of each performance indicator of each machine learning model is weighted and averaged to obtain the comprehensive evaluation value of each machine learning model. Based on the comprehensive evaluation value, the optimal machine learning model is selected from multiple machine learning models as the prediction model. S4. Based on the water quality dataset and the water quality index of each monitoring point, a variety of importance assessment methods are used to assess the impact of each environmental indicator on the water quality index, obtain the importance of each environmental indicator in various importance assessment methods, and calculate the comprehensive importance of each environmental indicator based on the importance of each environmental indicator in various importance assessment methods. S5. Based on the order of decreasing comprehensive importance, each environmental indicator is gradually input into the prediction model for training and water quality index prediction. The model performance of the prediction model is calculated step by step to generate a model performance sequence. The optimal prediction model is determined based on the increase in model performance in the model performance sequence. The optimal prediction model and its corresponding environmental indicators are used for water quality monitoring and evaluation.
2. The method for optimizing water quality assessment based on interpretable machine learning according to claim 1, characterized in that, In step S2, principal component analysis is performed on the water quality dataset to determine the contribution of each environmental indicator, specifically including: Principal component analysis was used to perform principal component analysis on the water quality dataset to extract the top principal components and their corresponding variance contribution rates from each monitoring point whose cumulative variance was greater than or equal to a preset percentage. Based on the extracted principal components and their corresponding variance contribution rates for each monitoring point, the contribution of each environmental indicator at each monitoring point is calculated. The formula for calculating the contribution of each environmental indicator at each monitoring point is as follows: In the formula, Indicates the first The first monitoring point The contribution of each environmental indicator Indicates the extracted first The first monitoring point The variance contribution rate corresponding to each principal component This indicates the total number of principal components extracted. Indicates the extracted first The first monitoring point The principal component and the first The loading matrix between the environmental indicators, and: In the formula, Indicates the extracted first The first monitoring point Principal components, This indicates the total number of environmental indicators. Indicates the first The first monitoring point Data for each environmental indicator.
3. The method for optimizing water quality assessment based on interpretable machine learning according to claim 1, characterized in that, In S2, let the first The preset optimal range for each environmental indicator is ;in, Indicates the first The lower limit of the preset optimal range for each environmental indicator. Indicates the first The upper limit of the preset optimal range for each environmental indicator; The formula for calculating the score of each environmental indicator at each monitoring point is as follows: In the formula, Indicates the first The first monitoring point Scores for each environmental indicator; This represents a preset score constant; Let the lower bound penalty function be used. Upper limit penalty function; Indicates the degree below the lower limit, and ; Indicates the degree exceeding the upper limit, and ; Indicates the first The first monitoring point Data for each environmental indicator.
4. The method for optimizing water quality assessment based on interpretable machine learning according to claim 1, characterized in that, In S2, the formula for calculating the water quality index at each monitoring point is: In the formula, Indicates the first Water quality index at each monitoring point Indicates the first The first monitoring point The scores of each environmental indicator Indicates the first The first monitoring point The contribution of each environmental indicator This indicates the total number of environmental indicators.
5. The method for optimizing water quality assessment based on interpretable machine learning according to claim 1, characterized in that, In S3, multiple machine learning models include lm model, ridge model, lasso model, enet model, glmnet model, bayesglm model, SVM model, svmLinear model, svmRadial model, treebag model, rf model, gbm model, xgbTree model and earth model.
6. The method for optimizing water quality assessment based on interpretable machine learning according to claim 1, characterized in that, In S3, multiple performance metrics include , and ; The average values of each performance metric for each machine learning model are weighted and averaged. Specifically, the average values of (1-R²), MSE, and MAE for each machine learning model are weighted and averaged.
7. The method for optimizing water quality assessment based on interpretable machine learning according to claim 1, characterized in that, In S4, the formula for calculating the overall importance of each environmental indicator is: In the formula, Indicates the first The combined importance of each environmental indicator Indicates the first The environmental indicators in the first Standardization of importance in various importance assessment methods This indicates the total number of importance assessment methods.
8. The method for optimizing water quality assessment based on interpretable machine learning according to claim 1, characterized in that, In S4, various importance assessment methods include standardized regression coefficient method, random forest model method and SHAP analysis method.
9. The method for optimizing water quality assessment based on interpretable machine learning according to claim 1, characterized in that, In S5, the model performance sequence is specifically based on R. 2 The sequence of indicators, and in the model performance sequence, R 2 The rate of increase in the indicator gradually decreased; Determining the optimal prediction model specifically includes: Based on the model performance sequence, calculate R for each adjacent pair of R. 2 The second R in the indicator 2 The rate of change of the indicators is used to obtain the sequence of model performance change rates. The first R in the model performance change rate sequence that is less than a preset change rate threshold is identified. 2 The previous prediction model corresponding to the indicator is taken as the optimal prediction model.
10. A device for optimizing water quality assessment based on interpretable machine learning, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory, which, when executed by the processor, implements the method for optimizing water quality assessment based on interpretable machine learning as described in any one of claims 1 to 9.