Black and odorous water body early warning factor screening method based on machine learning modeling

Abstract

The invention discloses a black and odorous water body early warning factor screening method based on machine learning modeling, which is characterized by comprising the following steps: collecting a water sample, carrying out conventional water quality and spectrum detection, and establishing a machine learning modeling data set including conventional water quality data, sample labels and spectrum data of the sample; constructing different machine learning classification model structures, comparing the performance of the different machine learning classification model structures according to the machine learning modeling data set, and screening the machine learning classification model with the optimal performance as a research object; and finally, creating a shop interpreter according to the optimal machine learning classification model, calculating a Shapley value of each input feature to a positive output feature through the shop interpreter, calculating absolute average values of the Shapley values of the same input features in all samples, carrying out descending sort, screening out 3-5 input features with the maximum absolute average values, namely the black and odorous water body early warning factors, and carrying out early warning on the black and odorous water body according to the black and odorous water body early warning factors. The method has the advantage that pollution identification is more comprehensive and accurate.

Description

Technical Field

[0001] The present invention relates to a method for treating black and odorous water bodies, and in particular to a method for screening early warning factors of black and odorous water bodies based on machine learning modeling. Background Art

[0002] Black and smelly water is caused by organic pollutants exceeding the self-purification capacity of the water body, which makes the oxygen consumption rate of the aerobic decomposition process higher than the reoxygenation rate, resulting in a hypoxic state, which in turn produces smelly substances such as hydrogen sulfide, ammonia, and mercaptans, and forms black deposits. The treatment of black and smelly water bodies is a difficult problem in water pollution prevention and control, especially because of its wide distribution and easy fluctuations. After treatment, the phenomenon of "returning to black and smelly" often occurs. Therefore, effective technical means are urgently needed to fundamentally solve the black and smelly problem. In recent years, researchers have been committed to improving the monitoring accuracy of the dynamic changes of black and smelly water bodies through technical means such as remote sensing and artificial intelligence, in order to intervene in the early stages of the black and smelly phenomenon and reduce the cost of treatment.

[0003] Early warning factors for black and odorous water bodies include key water quality parameters such as dissolved oxygen, redox potential, ammonia nitrogen, chemical oxygen demand, biochemical oxygen demand, total phosphorus, total nitrogen, hydrogen sulfide and volatile organic compounds, which can provide early warnings before water quality deteriorates. Common methods for screening these factors include statistical analysis (such as principal component analysis and factor analysis), entropy method, expert consultation and multi-indicator comprehensive evaluation. However, the above methods have significant limitations: first, relying on a single indicator with limited coverage, it is difficult to fully reflect the complex causes of water pollution, and individual indicators are easily affected by environmental fluctuations, which may lead to misjudgment; second, these methods lack trend prediction capabilities, usually can only reflect the current water quality status, cannot provide early warnings, and often lag behind the development of pollution, resulting in delayed governance and increased costs. At the same time, a single indicator is difficult to reveal the interaction between multiple factors and it is difficult to accurately identify the source of pollution.

[0004] Machine learning is a computer science technology. Its core is to analyze large amounts of data by building and using algorithmic models, extract patterns from them, and achieve tasks such as prediction, classification, clustering, and decision-making. Because of its ability to automate decision-making, process massive amounts of data, and nonlinear relationships, it is widely used in image recognition, natural language processing, recommendation systems, and other fields. At present, there are no public research reports on the screening methods of black and odorous water warning factors based on machine learning modeling at home and abroad. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for screening black and odorous water body warning factors based on machine learning modeling with more comprehensive and accurate pollution identification.

[0006] The technical solution adopted by the present invention to solve the above technical problems is: a method for screening black and odorous water body early warning factors based on machine learning modeling, comprising the following steps:

[0007] Step 1: Collect water samples and conduct routine water quality and spectral testing to establish a machine learning modeling data set, including the sample's routine water quality data, sample labels, and spectral data;

[0008] Step 2: Construct different machine learning classification model structures, compare the performance of different machine learning classification model structures based on the machine learning modeling data set obtained in step 1, and select the machine learning classification model with the best performance as the research object;

[0009] Step 3. Create a shap interpreter based on the optimal machine learning classification model obtained in step 2. Calculate the Shapley value of each input feature for the positive class output feature through the shap interpreter. Calculate the absolute average of the Shapley values ​​of the same input feature in all samples and arrange them in descending order. Filter out 3 to 5 input features with the largest absolute average of Shapley values, which are the warning factors for black and odorous water bodies.

[0010] Furthermore, the conventional water quality data described in step 1 include pH, conductivity, AN, TN, NO3-N, TP, COD Mn and NPOC, the sample labels include black and odorous water bodies and non-black and odorous water bodies, and the spectral data include ultraviolet spectral features UV 254 and SUVA 254 , fluorescence spectral features HIX, FI, BIX and β / α, and 5 different fluorescence region spectral features φ obtained by EEM–FRI technology analysis i .

[0011] Furthermore, the ultraviolet spectrum characteristic UV 254 and SUVA 254 The acquisition and calculation process of is as follows: the water sample after 0.45 μm filtration is scanned in the ultraviolet wavelength range of 200-700 nm to obtain the ultraviolet spectrum characteristic UV 254 ; UV spectral characteristics SUVA 254 The calculation formula is as follows:

[0012] Among them, UV 254 is the absorbance of ultraviolet light at 254 nm, ρ NPOC It is the concentration of non-purgeable organic carbon, in mg / L.

[0013] Furthermore, the calculation process of the fluorescence spectral characteristics HIX, FI, BIX and β / α is as follows: 0.45 μm filtered water samples and ultrapure water are scanned in the fluorescence excitation wavelength range of 220 to 450 nm and the emission wavelength range of 260 to 600 nm, respectively, to obtain original fluorescence spectral data; the original fluorescence spectral data of ultrapure water is used as the baseline of the fluorescence spectrum, the Rayleigh scattering and Raman scattering of the original fluorescence spectral data are subtracted, the missing values ​​are fitted using the DOMFlour toolkit, the fluorescence spectral data are corrected, and HIX, FI, BIX and β / α are calculated using the EEMs-toolkit toolkit in the python environment.

[0014] Furthermore, the spectral characteristics of the five different fluorescence regions obtained by EEM-FRI analysis are i The calculation process of is as follows: First, the fluorescence spectrum is divided into five different fluorescence regions i using EEM-FRI technology, namely aromatic protein I region, aromatic protein II region, fulvic acid region, microbial protein region and humus region, and then the spectral characteristics of different fluorescence regions φ are calculated. i Calculate, the calculation formula is:

[0015] Φ i =MF i ∫ ex ∫ em I(λ ex λ em )dλ ex dλ em , formula (2), where Φ i is the normalized integrated volume of region i, in AU·nm 2 ; I(λ ex λ em ) is the fluorescence intensity corresponding to the excitation-emission wavelength, in AU; dλ ex is the excitation wavelength range, dλ em is the emission wavelength range, in nm; MF i is the multiplication factor, which is equal to the reciprocal of the ratio of the integrated area of ​​fluorescent region i to the integrated area of ​​the total region.

[0016] Furthermore, step 2 is as follows:

[0017] Step 2-1: Use the scikit-learn toolkit in the Python environment to build the DT, RF and XGBoost machine learning classification model structures respectively;

[0018] Step 2-2, setting the conventional water quality data and spectral data obtained in step 1 as the input features of the machine learning classification model in the python environment; setting the sample labels as the output features of the machine learning classification model in the python environment, where black and odorous water bodies are positive output features and non-black and odorous water bodies are negative output features;

[0019] Step 2-3, obtain the input features of each sample and its corresponding output features to construct a sample data set, and randomly divide the sample data set, where 80% of the sample data set is used as a training set and 20% of the sample data set is used as a test set;

[0020] Step 2-4: Import the training set into the DT, RF and XGBoost machine learning classification model structures for model training, and construct a grid search method to optimize the hyperparameters. The hyperparameters of the DT, RF and XGBoost machine learning classification models include max_depth, min_samples_split and min_samples_leaf, and the hyperparameters of the RF and XGBoost machine learning classification models also include n_estimators.

[0021] Step 2-5: Use the AUC value and F1 value of the machine learning classification model in the training set as evaluation indicators. The higher the value, the better the performance. Select the hyperparameters with the highest evaluation index value, and then determine the DT, RF and XGBoost machine learning classification models. The AUC value and F1 value are calculated using the scikit-learn toolkit in the Python environment.

[0022] Step 2-6: Import the test set into the DT, RF and XGBoost machine learning classification models determined in step 2-5 respectively, compare the AUC values ​​and F1 values ​​of the DT, RF and XGBoost machine learning classification models in the test set, and select the model with the highest evaluation index value as the optimal machine learning classification model.

[0023] Furthermore, step 3 is as follows:

[0024] Step 3-1: Use the SHAP toolkit in the Python environment to create a SHAP interpreter for the optimal machine learning classification model obtained in step 2.

[0025] Step 3-2, import the test set into the shap interpreter, and calculate the Shapley value of each input feature of each sample for the positive output feature. The larger the Shapley value, the stronger the influence of the feature on the model's prediction of the positive class result (that is, the greater the probability that the model believes that the sample belongs to the positive class);

[0026] Step 3-3, calculate the absolute average of the Shapley values ​​of the same input features in all samples and arrange them in descending order, and select 3 to 5 input features with the largest absolute average of Shapley values, which are the black and odorous water warning factors.

[0027] Compared with the prior art, the advantages of the present invention are:

[0028] 1. Multi-dimensional data integration: The present invention combines conventional water quality data (such as pH, conductivity, ammonia nitrogen, etc.) and ultraviolet and fluorescence spectral data (such as UV254, SUVA254, HIX, FI, etc.), and introduces EEM-FRI (fluorescence regional integration) technology for modeling. Spectral characteristics and conventional water quality indicators are used as input features, covering the physical, chemical and optical properties of water bodies, so that this method can accurately identify black and smelly water bodies. In particular, the fluorescence regional spectral characteristics of EEM-FRI analysis can reflect the composition and structure of organic matter in water bodies, and can more comprehensively and accurately analyze the dissolved organic matter in black and smelly water bodies, further improving the predictive ability of the model. In particular, the introduction of spectral data can provide more sensitive early warning signals than traditional water quality indicators, which is crucial for capturing early changes in black and smelly water bodies. This integration of multi-dimensional feature data not only expands the scope of application of the model, but also significantly enhances the accuracy and stability of the model.

[0029] 2. Combination of EEM-FRI spectral analysis technology and machine learning: Although EEM spectral technology and machine learning have been applied in different fields respectively, this invention introduces EEM-FRI (fluorescence regional integration) technology into the screening of early warning factors for black and odorous water bodies for the first time. This combination greatly enriches the input characteristics of the model, provides more informative data input for the machine learning algorithm, and enables the model to more accurately distinguish between black and odorous water bodies and non-black and odorous water bodies. Through the Python-based machine learning framework, combined with grid search to optimize the model hyperparameters, the model performs well in F1 value and AUC value. In particular, the RandomForest (RF) model performed very well on the test set, with an AUC value of 0.943, which has high reliability and accuracy.

[0030] 3. Feature importance analysis based on the SHAP model: Based on the machine learning classification model, the present invention further introduces the SHAP interpretability algorithm. By analyzing the SHAP value of the feature, the contribution of each input feature in the model can be clearly quantified, thereby avoiding the subjective feature selection problem common in traditional water quality monitoring methods. The introduction of the SHAP value not only improves the interpretability of the model, but also makes feature screening more scientific and data-driven, which is particularly suitable for complex environmental water pollution scenarios. Through the analysis of the SHAP value, three key indicators, NPOC, ΦIV and ΦII, were found, which played an important role in predicting black and odorous water bodies. This explanatory analysis not only improves the transparency of the model, but also provides a specific scientific basis.

[0031] 4. Technical feasibility of applying to the identification of black and odorous water bodies: The causes of the formation of black and odorous water bodies are complex and diverse, mainly related to dissolved organic matter, nitrogen, phosphorus and other substances. The present invention effectively solves the problem of single water quality indicators and insufficient monitoring methods in the traditional identification of black and odorous water bodies by combining EEM-FRI technology with machine learning models. Through actual verification of sample data sets, the model has achieved very ideal results in both F1 value and AUC value, indicating that this method has high practical feasibility in the identification of black and odorous water bodies.

[0032] In summary, the present invention provides a method for screening early warning factors for black and odorous water bodies based on machine learning modeling. This method not only covers conventional water quality indicators, but also combines multi-dimensional data such as spectral characteristics (such as ultraviolet spectrum, fluorescence spectrum), and applies EEM-FRI technology to analyze the fluorescence spectrum, which greatly broadens the scope of water quality monitoring, thereby making pollution identification more comprehensive and accurate. By applying water quality and spectral data sets to machine learning classification models, machine learning can effectively capture subtle trends in water quality changes and identify potential pollution characteristics in advance through learning of large amounts of data, thereby discovering the risk of black and odorous water bodies in advance and achieving scientific early warning factor screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flowchart of the black and odorous water body early warning factor screening method based on machine learning modeling;

[0034] Figure 2 is a fluorescence spectrum, where a is a black and smelly water body, and b is a non-black and smelly water body;

[0035] Figure 3 It is a statistical chart of Shapley values ​​of input features, where a is a summary chart of Shapley values ​​and b is a pie chart of Shapley value proportions;

[0036] Figure 4 is the Shapley value interaction diagram, where a is the relationship between NPOC and Φ IV, b is NPOC and Φ II . DETAILED DESCRIPTION

[0037] The present invention is further described in detail below with reference to the accompanying drawings.

[0038] Table 1 Chinese explanations of the English abbreviations in this invention

[0039] Abbreviations Chinese explanation unit DOM Dissolved organic matter / EEM-FRI Three-dimensional fluorescence-area integration method / AN Ammonia nitrogen mg / L TN Total Nitrogen mg / L <![CDATA[NO3-N]]> Nitrate mg / L TP Total Phosphorus mg / L <![CDATA[COD Mn ]]> Permanganate Index mg / L NPOC No organic carbon can be purged mg / L <![CDATA[UV 254 ]]> UV absorbance at 254nm AU / cm <![CDATA[SUVA 254 ]]> Characteristic UV absorbance at 254nm wavelength AUL / (m*mg) HIX Humification index / FI Fluorescence index / BIX Self-generated indicators / β / α Freshness Index / <![CDATA[φ i ]]> Normalized integrated volume of region i <![CDATA[AUnm 2 ]]> <![CDATA[I(λ ex l em )]]> Fluorescence intensity corresponding to the excitation-emission wavelength AU <![CDATA[dλ ex ]]> Excitation wavelength range nm <![CDATA[dλ em ]]> Emission wavelength range nm <![CDATA[MF i ]]> Multiplication Factor / DT Decision Tree / RF Random Forest / XGBoost Extreme Gradient Boosting / max_depth Maximum Depth / min_samples_split Minimum number of samples required to split an internal node / min_samples_leaf The minimum number of samples required for a leaf node / n_estimators The number of base learners / F1 value The harmonic mean of precision and recall / AUC value Area under the curve / Shapley value Shapler value /

[0040] A method for screening early warning factors of black and odorous water bodies based on machine learning modeling, such as Figure 1 As shown, the following steps are included:

[0041] Step 1: Collect water samples and conduct routine water quality and spectral tests to establish a machine learning modeling data set, including the sample's routine water quality data, sample labels, and spectral data, as follows:

[0042] Taking the monitoring samples of black and odorous water bodies in a certain city as an example, routine water quality testing was carried out according to the "Surface Water Environmental Quality Standard" (GB 3838-2002), and spectral testing was carried out using the UV spectrophotometer U-3900 (Hitachi, Japan) and the fluorescence photometer F-4600 (Hitachi, Japan). According to the "Guidelines for the Treatment of Rural Black and Odor Water Bodies" and the "Guidelines for the Treatment of Urban Black and Odor Water Bodies", the samples were divided into "black and odorous water bodies" and "non-black and odorous water bodies". Among the 179 samples, there were 70 black and odorous water bodies and 109 non-black and odorous water bodies. The representative EEM spectra are as follows: Figure 2 shown.

[0043] Create a machine learning modeling data set. Conventional water quality data includes pH, conductivity, AN, TN, NO3-N, TP, COD Mn and NPOC; sample labels include “black and odorous water body” and “non-black and odorous water body”; the characteristics of spectral data include ultraviolet spectral characteristics (UV 254 and SUVA 254 ), fluorescence spectral characteristics (HIX, FI, BIX and β / α), and 5 different fluorescence region spectral characteristics φ obtained by EEM-FRI technology analysis i (Characterizing various properties of dissolved organic matter such as composition, structure, and degradability).

[0044] Ultraviolet spectrum characteristics UV 254 and SUVA 254 The acquisition and calculation method of is as follows: Scan the 0.45μm filtered water sample in the ultraviolet wavelength range of 200-700nm (interval 0.5nm) to obtain the ultraviolet spectrum characteristic UV 254 , UV spectrum characteristics SUVA 254 The calculation formula is as follows:

[0045] Among them, UV 254 is the absorbance of ultraviolet light at 254 nm, ρ NPOC It is the concentration of non-purgeable organic carbon, in mg / L.

[0046] The calculation process of the fluorescence spectral features HIX, FI, BIX and β / α is as follows: scan the 0.45μm filtered water sample and ultrapure water in the fluorescence excitation wavelength range of 220-450nm (interval of 5nm) and the emission wavelength range of 260-600nm (interval of 1nm) to obtain the original fluorescence spectral data; use the original fluorescence spectral data of ultrapure water as the baseline of the fluorescence spectrum, subtract the Rayleigh scattering and Raman scattering of the original fluorescence spectral data, use the DOMFlour toolkit to fit the missing values, correct the fluorescence spectral data, and use the EEMs-toolkit toolkit in the python environment to calculate HIX, FI, BIX and β / α.

[0047] Spectral characteristics of 5 different fluorescence regions φ i The calculation process is as follows: First, the fluorescence spectrum is divided into 5 different fluorescence regions i using the EEM-FRI technology, as follows:

[0048] (1) Aromatic protein I region (R I ,λ em :260-330nm,λ ex :220-250nm);

[0049] (2) Aromatic protein II region (R II ,λ em :330-380nm,λ ex :220-250nm);

[0050] (3) Fulvic acid-like region (R III ,λ em >380nm,λ ex :220-250nm);

[0051] (4) Microbial protein region (R IV ,λ em :260-380nm,λ ex >250nm);

[0052] (5) Humus-like area (R v ,λ em >380nm,λ ex >250nm);

[0053] Then, the spectral characteristics of different fluorescence regions φi Calculate, the calculation formula is:

[0054] Φ i =MF i ∫ ex ∫ em I(λ ex λ em )dλ ex dλ em , formula (2), where Φ i is the normalized integrated volume of region i, in AU·nm 2 ; I(λ ex λ em ) is the fluorescence intensity corresponding to the excitation-emission wavelength, in AU; dλ ex is the excitation wavelength range, dλ em is the emission wavelength range, in nm; MF i is the multiplication factor, which is equal to the reciprocal of the ratio of the integrated area of ​​fluorescent region i to the integrated area of ​​the total region.

[0055] Step 2: Construct different machine learning classification model structures, compare the performance of different machine learning classification model structures based on the machine learning modeling data set obtained in step 1, and select the machine learning classification model with the best performance as the research object, as follows:

[0056] Step 2-1: Use the scikit-leam toolkit in the Python environment to build the DT, RF and XGBoost machine learning classification model structures respectively;

[0057] Step 2-2, setting the conventional water quality data and spectral data obtained in step 1 as the input features of the machine learning classification model in the python environment; setting the sample label as the output feature of the machine learning classification model in the python environment, where black and odorous water bodies are positive output features, and non-black and odorous water bodies are negative output features;

[0058] Step 2-3, obtain the input features of each sample and its corresponding output features to construct a sample data set, and randomly divide the sample data set, where 80% of the sample data set is used as a training set and 20% of the sample data set is used as a test set;

[0059] Step 2-4: Import the training set into the DT, RF and XGBoost machine learning classification model structures for model training, and construct a grid search method to optimize the hyperparameters. The hyperparameters of the DT, RF and XGBoost machine learning classification models include max_depth, min_samples_split and min_samples_leaf, and the hyperparameters of the RF and XGBoost machine learning classification models also include n_estimators.

[0060] Step 2-5, using the AUC value and F1 value of the machine learning classification model in the training set as evaluation indicators, the higher the value, the better the performance, select the hyperparameters with the highest evaluation index, and then determine each machine learning classification model. The AUC value and F1 value are calculated using the scikit-leam toolkit in the Python environment, as shown in Table 1 below;

[0061] Table 1 Evaluation indicators of each machine learning classification model in the training set

[0062]

[0063]

[0064] Step 2-6: Import the test set into the DT, RF and XGBoost machine learning classification models determined in step 2-5, compare the AUC values ​​and F1 values ​​of the DT, RF and XGBoost machine learning classification models in the test set, and select the model with the highest evaluation index as the optimal machine learning classification model. Among them, the F1 value of the RF machine learning classification model in the test set is 0.935 and the AUC value is 0.943, both of which are the highest, and it is confirmed as the optimal machine learning classification model.

[0065] Step 3: Create a shap interpreter based on the optimal machine learning classification model obtained in step 2. Use the shap interpreter to calculate the Shapley value of each input feature for the positive output feature. Calculate the proportion of the Shapley value of each input feature and arrange them in descending order. Select the 3 to 5 input features with the largest proportion of Shapley values, which are the black and odorous water warning factors. The details are as follows:

[0066] Step 3-1: Use the shap toolkit in the Python environment to create a shap interpreter for the optimal RF machine learning classification model obtained in step 2;

[0067] Step 3-2, import the test set into the shap interpreter, and calculate the Shapley value of each input feature of each sample for the positive output feature. The larger the Shapley value, the stronger the influence of the feature on the model's prediction of the positive class result (that is, the greater the probability that the model believes that the sample belongs to the positive class);

[0068] Step 3-3, calculate the absolute average of the Shapley values ​​of the same input features in all samples and arrange them in descending order, and select the three input features with the largest absolute average of the Shapley values, which are the black and odorous water warning factors. Figure 3 Figure a shows the Shapley value of each input feature for the positive output feature, and is sorted in descending order according to the absolute mean of the Shapley value.

[0069] The absolute average values ​​of all input features are added together to calculate the Shapley value ratio of each input feature. The larger the Shapley value ratio, the higher the relative importance of the feature among all features. Figure 3 As shown in b, the fluorescence region spectral features have the greatest impact on the model, accounting for 40.8%, conventional water quality features (including NPOC) account for 40.2%, and ultraviolet and fluorescence spectral features account for 18.9%. In this specific embodiment, the three features with the largest Shapley values ​​are NPOC, ΦIV, and ΦII. It is of great significance that NPOC has a higher Shapley value (14.98%), because NPOC measures all organic carbon. The contribution of ΦIV (11.16%) and ΦII (9.16%) to the Shapley value is much higher than other spectral features, and has the most important influence on determining whether the water body is determined to be a black and smelly water body.

[0070] In order to verify the practical applicability of the model, we conduct an in-depth analysis of NPOC and Φ IV and Φ II The interactive influence relationship of Shapley value is used to explore the different factors driving black and odorous water bodies with different characteristics. Figure 4 As shown in a, with Φ IV As the value increases, the NPOC value gradually increases, Φ IV The Shapley value of Φ shows a continuous nonlinear increasing trend. IV It is directly related to the activity intensity of bacteria and other microorganisms in the water body. Its high Shapley value indicates that the activity intensity of microorganisms is a key factor in reflecting the black and smelly process of water bodies. 6 <Φ IV <1.0×10 6 In the range, the Shapley value increases sharply from -0.05 to 0.05, and the sample distribution is continuous, which indicates that with the increase of Φ IVThe increase of microbial protein substances represented by the threshold data indicates that the water body is obviously becoming black and smelly. This threshold data has practical guiding significance for the management and control of black and smelly water bodies. By monitoring it, the water quality evolution stage of the water body can be accurately assessed. Figure 4 As shown in b, in the case of NPOC and Φ II In the interactive influence relationship of the Shapley value of II The Shapley value of Φ also shows similar continuity in the process of increasing, but after the Shapley value reaches the peak II There is an obvious downward trend. IV It is related to the aromatic protein components represented by tryptophan. It is a typical easily degradable organic matter in water and an important carbon source for heterotrophic microorganisms such as bacteria. However, for long-term severe black and smelly water, Φ IV The amount of the substance represented will be consumed, thus resulting in the decreasing trend of the Shapley value in the figure. The above analysis shows that this method has a strong application value in the screening of early warning factors for actual black and odorous water bodies.

[0071] The above description is not intended to limit the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by a person skilled in the art within the spirit and scope of the present invention shall also fall within the protection scope of the present invention.

Claims

1. A method for screening black and odorous water body early warning factors based on machine learning modeling, characterized in that The following steps are involved: Step 1: Collect water samples and conduct routine water quality and spectral testing to establish a machine learning modeling data set, including the sample's routine water quality data, sample labels, and spectral data; Step 2: Construct different machine learning classification model structures, compare the performance of different machine learning classification model structures based on the machine learning modeling data set obtained in step 1, and select the machine learning classification model with the best performance as the research object; Step 3. Create a shap interpreter based on the optimal machine learning classification model obtained in step 2. Calculate the Shapley value of each input feature for the positive class output feature through the shap interpreter. Calculate the absolute average of the Shapley values ​​of the same input feature in all samples and arrange them in descending order. Filter out 3 to 5 input features with the largest absolute average of Shapley values, which are the warning factors for black and odorous water bodies.

2. The method for screening black and odorous water body early warning factors based on machine learning modeling according to claim 1 is characterized in that: The general water quality data described in step 1 include pH, conductivity, AN, TN, NO3-N, TP, COD Mn and NPOC, the sample labels include black and odorous water bodies and non-black and odorous water bodies, and the spectral data include ultraviolet spectral features UV 254 and SUVA 254 , fluorescence spectral features HIX, FI, BIX and β / α, and 5 different fluorescence region spectral features φ obtained by EEM–FRI technology analysis i .

3. The method for screening black and odorous water body early warning factors based on machine learning modeling according to claim 2 is characterized in that: The ultraviolet spectrum characteristic UV 254 and SUVA 254 The acquisition and calculation process of is as follows: the water sample after 0.45 μm filtration is scanned in the ultraviolet wavelength range of 200-700 nm to obtain the ultraviolet spectrum characteristic UV 254 ; UV spectral characteristics SUVA 254 The calculation formula is as follows: Among them, UV 254 is the absorbance of ultraviolet light at 254 nm, ρ NPOC It is the concentration of non-purgeable organic carbon, in mg / L.

4. A method for screening black and odorous water body early warning factors based on machine learning modeling according to claim 2, characterized in that The calculation process of the fluorescence spectral characteristics HIX, FI, BIX and β / α is as follows: 0.45 μm filtered water samples and ultrapure water are scanned in the fluorescence excitation wavelength range of 220 to 450 nm and the emission wavelength range of 260 to 600 nm, respectively, to obtain original fluorescence spectral data; the original fluorescence spectral data of ultrapure water is used as the baseline of the fluorescence spectrum, the Rayleigh scattering and Raman scattering of the original fluorescence spectral data are subtracted, the missing values ​​are fitted using the DOMFlour toolkit, the fluorescence spectral data are corrected, and HIX, FI, BIX and β / α are calculated using the EEMs-toolkit toolkit in the python environment.

5. The method for screening black and odorous water body early warning factors based on machine learning modeling according to claim 2 is characterized in that The spectral characteristics of the five different fluorescence regions obtained by EEM-FRI analysis are shown in Figure 2. i The calculation process of is as follows: First, the fluorescence spectrum is divided into five different fluorescence regions i using EEM-FRI technology, namely aromatic protein I region, aromatic protein II region, fulvic acid region, microbial protein region and humus region, and then the spectral characteristics of different fluorescence regions φ are calculated. i Calculate, the calculation formula is: Φ i =MF i ∫ ex ∫ em I(λ ex λ em )dλ ex dλ em , formula (2), where Φi is the normalized integrated volume of region i, in units of AU·nm 2 ; I(λ ex λ em ) is the fluorescence intensity corresponding to the excitation-emission wavelength, in AU; dλ ex is the excitation wavelength range, dλ em is the emission wavelength range, in nm; MF i is the multiplication factor, which is equal to the reciprocal of the ratio of the integrated area of ​​fluorescent region i to the integrated area of ​​the total region.

6. The method for screening black and odorous water body early warning factors based on machine learning modeling according to claim 1 is characterized in that Step 2 is as follows: Step 2-1: Use the scikit-learn toolkit in the Python environment to build the DT, RF and XGBoost machine learning classification model structures respectively; Step 2-2, setting the conventional water quality data and spectral data obtained in step 1 as the input features of the machine learning classification model in the python environment; setting the sample labels as the output features of the machine learning classification model in the python environment, where black and odorous water bodies are positive output features and non-black and odorous water bodies are negative output features; Step 2-3, obtain the input features of each sample and its corresponding output features to construct a sample data set, and randomly divide the sample data set, where 80% of the sample data set is used as a training set and 20% of the sample data set is used as a test set; Step 2-4: Import the training set into the DT, RF and XGBoost machine learning classification model structures for model training, and construct a grid search method to optimize the hyperparameters. The hyperparameters of the DT, RF and XGBoost machine learning classification models include max_depth, min_samples_split and min_samples_leaf, and the hyperparameters of the RF and XGBoost machine learning classification models also include n_estimators. Step 2-5: Use the AUC value and F1 value of the machine learning classification model in the training set as evaluation indicators. The higher the value, the better the performance. Select the hyperparameters with the highest evaluation index value, and then determine the DT, RF and XGBoost machine learning classification models. The AUC value and F1 value are calculated using the scikit-learn toolkit in the Python environment. Step 2-6: Import the test set into the DT, RF and XGBoost machine learning classification models determined in step 2-5 respectively, compare the AUC values ​​and F1 values ​​of the DT, RF and XGBoost machine learning classification models in the test set, and select the model with the highest evaluation index value as the optimal machine learning classification model.

7. The method for screening black and odorous water body early warning factors based on machine learning modeling according to claim 1 is characterized in that Step 3 is as follows: Step 3-1: Use the SHAP toolkit in the Python environment to create a SHAP interpreter for the optimal machine learning classification model obtained in step 2. Step 3-2, import the test set into the shap interpreter, and calculate the Shapley value of each input feature of each sample for the positive output feature. The larger the Shapley value, the stronger the influence of the feature on the model's prediction of the positive class result (that is, the greater the probability that the model believes that the sample belongs to the positive class); Step 3-3, calculate the absolute average of the Shapley values ​​of the same input features in all samples and arrange them in descending order, and select 3 to 5 input features with the largest absolute average of Shapley values, which are the black and odorous water warning factors.

Images