Water quality assessment machine learning bayesian model based on suspended particulate matter density

Abstract

The application discloses a water quality evaluation machine learning Bayesian model based on suspended particle matter density, and comprises the following steps: collecting and testing the suspended particle matter density indexes and other multiple water quality indexes of a plurality of water quality samples; establishing an entropy weight Bayesian model for water quality evaluation; calculating entropy weight values; calculating the posterior probability P of the samples being in class I to class V water by using the distance formula with the weight; dividing the adjacent grades between class I to class V water quality grades into ten equal parts, and determining the water quality grade according to the difference between the highest probability value and the second highest probability value in the posterior probability P; matching the water quality grade of each sample with the suspended particle matter density of the sample, selecting a large number of samples as a training set, and generating a water quality evaluation Bayesian classifier based on suspended particle matter; the application simplifies the existing water quality rapid evaluation method, improves the accuracy of the evaluation by improving the calculation formula in the prior art, and realizes the rapid evaluation of the water quality grade based on the suspended particle matter density.

Description

Technical Field

[0001] This invention belongs to the field of water quality assessment, specifically a machine learning Bayesian model for water quality assessment based on the density of suspended particulate matter. Background Technology

[0002] Traditional water quality assessment methods include single-factor evaluation method and comprehensive index evaluation method. Bayesian models can quantitatively reflect the degree and confidence of the influence of other factors on water quality, so they are used in various water quality assessment methods. For example, in the prior art 1: Tang Jinping et al. Groundwater quality assessment model and application based on Bayesian theory [J], Water Saving Irrigation, No. 4, 2018, 88-91, the calculation step (2) of this method adopts the distance value method based on geometric probability, which is calculated by the reciprocal of the absolute value of the distance between the water quality index detection value and the standard value. Prior art 1 treats any water quality level as only one value. If it is higher or lower than the value, it is considered that there is still a certain distance from the level. In fact, any water quality level is a range, and the probability of belonging to the level within this range is 100%. Summary of the Invention

[0003] The purpose of this invention is to improve the traditional formula for calculating distance based on posterior probability, so that the calculation results are more accurate.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A machine learning Bayesian model for water quality assessment based on suspended particulate matter density includes the following steps:

[0006] S1. Collect and test several water quality samples. The density of suspended particulate matter was measured using indoor testing, and multiple water quality indicators for each sample were obtained using outdoor testing.

[0007] S2. Establish an entropy-weighted Bayesian model for water quality assessment;

[0008] S3. Based on the five indicators of water quality obtained from the above water quality testing, calculate the entropy weight according to the degree of disorder in the data system;

[0009] S4. Calculate the posterior probability P of the sample being in Class I to V water using a weighted distance formula;

[0010] S5. Divide the adjacent water quality grades from I to V into ten equal parts, based on the highest probability value P among the above posterior probabilities P. ik And the second highest probability value P (i±1)k The difference C is used to determine the water quality grade L:

[0011]

[0012] Where d has no specific meaning, it is only used to define the range of C, d = 1, 2, 3, 4, 5; i is the water quality grade;

[0013] S6. Based on the water quality grade of each sample calculated in S5, match the water quality grade of each sample with the suspended particulate matter density of that sample, select a large number of samples as the training set, and use the fitcnb function in the computer software MATLAB R2022a to generate a Bayesian classifier for water quality assessment based on suspended particulate matter.

[0014] S7. Select the suspended particulate matter density of the test set samples and predict the water quality level of the sample points based on the Bayesian classifier predict function.

[0015] Specifically, the suspended particulate matter density index in step S1 includes the density of detrital suspended particulate matter and the density of fibrous suspended particulate matter.

[0016] Specifically, the method for obtaining the suspended particulate matter density index in step S1 includes: obtaining suspended particulate matter samples by filtering with a 25# biological mesh; using the ZooSCAN image automatic scanning and analysis system, diluting the collected suspended particulate matter samples to a certain extent; and acquiring images and morphological data of suspended particulate matter by scanning with the instrument at a resolution of 300dp and 2400dp; uploading the suspended particulate matter images and morphological data to the database using FileZilla software; and performing classification training, manual verification, and data export on the website www.ecotaxa.cn to obtain the density of detrital suspended particulate matter and the density of fibrous suspended particulate matter.

[0017] Specifically, calculating the entropy weights in step S3 includes:

[0018] S31. Data preprocessing: Organize the raw data into an original matrix X and standardize it.

[0019]

[0020] In the formula, k is the number of stations, j is the indicator, X is the matrix composed of the original data, x is a certain item in the original matrix, Z is the standardized matrix, z is a certain item in the standardized matrix, and n is the total number of stations.

[0021] S32. Calculate the probability matrix P

[0022]

[0023] Where P is the probability matrix, and p is a term in the probability matrix;

[0024] S33. Calculate the entropy weight W

[0025]

[0026] Among them, e j It is an intermediate variable, W j is the weight of indicator j, and m is the total number of indicators.

[0027] Specifically, step S4 involves calculating the posterior probability P, which includes:

[0028] S41. The absolute distance D between the measured value and the standard value of a project at a very large indicator data site can be calculated using the following formula:

[0029] ①When i=1:

[0030]

[0031] ②When i≠1:

[0032]

[0033] The absolute distance D between the measured value and the standard value of a very small indicator data site can be calculated using the following formula:

[0034] ①When i=1:

[0035]

[0036] ②When i≠1:

[0037]

[0038] S42. Calculate relative distance

[0039]

[0040] Where D(x) jk |y ji () represents the relative distance. The smaller the value, the less likely that the water quality level of station k belongs to i; conversely, the larger the value, the greater the probability.

[0041] S43. For sample point k, targeting index j, D(x) belonging to water quality level i... jk |y ji Perform vertical normalization

[0042]

[0043] In the formula, P ijk P represents the probability that the j-th indicator at the k-th detection point belongs to the i-th water quality level. ik This represents the probability that the k-th detection point belongs to the i-th water quality level.

[0044] Specifically, in step S1, outdoor testing was used to obtain five water quality indicators for each sample, including dissolved oxygen (DO), permanganate index (COD_Mn), total nitrogen (TN), total phosphorus (TP), and ammonia nitrogen (NH3-N).

[0045] The beneficial effects of this invention are:

[0046] This invention simplifies existing rapid water quality assessment methods and improves the accuracy of assessments by refining the calculation formulas in existing technologies. It also enables rapid assessment of water quality levels based on suspended particulate matter density. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example 1

[0050] like Figure 1 As shown, the method of the present invention includes the following steps:

[0051] S1. Water samples were obtained from the drinking water source test area. The density of suspended particulate matter was measured indoors, and five water quality indicators for each sample were obtained outdoors, including dissolved oxygen (DO), permanganate index (COD_Mn), total nitrogen (TN), total phosphorus (TP), and ammonia nitrogen (NH3-N).

[0052] The density indices of suspended particulate matter were determined using indoor testing, specifically the density of detrital suspended particulate matter (D_De) and the density of fibrous suspended particulate matter (D_Fi). The method involved obtaining suspended particulate matter samples through a 25# biological mesh filter. The samples were then analyzed using a ZooSCAN image automatic scanning system (HYDROPTIC, France). A certain amount of the collected suspended particulate matter samples were diluted, and the instrument was used to acquire images and morphological data of the suspended particulate matter through preview at 300 dp resolution and scanning at 2400 dp resolution. The images and morphological data of the suspended particulate matter were uploaded to a database using FileZilla software. Classification training, manual verification, and data export were then performed on the website www.ecotaxa.cn to obtain the densities of detrital suspended particulate matter (D_De) and fibrous suspended particulate matter (D_Fi).

[0053] Outdoor testing can yield multiple water quality indicators. For ease of explanation, this embodiment only uses five indicators: dissolved oxygen (DO), permanganate index (COD_Mn), total nitrogen (TN), total phosphorus (TP), and ammonia nitrogen (NH3-N). In practical applications, the indicators can be selected based on the type of water source being sampled. The method for obtaining these five water quality indicators through outdoor testing is existing technology and will not be elaborated upon here.

[0054] S2. Based on the five indicators obtained from the above water quality tests—dissolved oxygen (DO), permanganate index (COD_Mn), total nitrogen (TN), total phosphorus (TP), and ammonia nitrogen (NH3-N)—an entropy-weighted Bayesian model for water quality evaluation is established, with the following formula:

[0055]

[0056] In the formula, P(y ji |x jk ) for x jk In the event of this, y ji The probability of occurrence. y ji x represents the standard value of index j for water quality level i. jk This represents the measured value of index j at station k, where i is the water quality level and j is the index.

[0057] Without any known data, the probability that the water quality at a certain point belongs to any grade is the same; therefore, the prior probability is:

[0058] P(y j1 )=P(y j2 )=P(y j3 )=P(y j4 )=P(y j5 ) = 1 / 5

[0059] In the formula, P(y j1 Let P(y) be the probability that index j is Class I water. j2 ), P(y j3 ), P(y j4 ), P(y j5 ) represent the probabilities of index j being Class II, Class III, Class IV, and Class V water, respectively.

[0060] S3. Based on the five water quality indicators obtained from the above testing, calculate the entropy value W according to the degree of disorder in the data system, specifically including:

[0061] S31. Data preprocessing: Organize the raw data into an original matrix X and standardize it.

[0062]

[0063] In the formula, the parameters k and j are the same as above, X is the matrix composed of the original data, and x is a certain item in the original matrix.kj The values ​​are the multiple water quality indicators of each sample detected in step S1, Z is the standardized matrix, z is a certain item in the standardized matrix, and n is the total number of stations.

[0064] S32. Calculate the probability matrix P

[0065]

[0066] Where P is a probability matrix, and p is a term in the probability matrix.

[0067] S33. Calculate the entropy weight W

[0068]

[0069] Among them, e j It is an intermediate variable, W j is the weight of indicator j, m is the total number of indicators, and other parameters are the same as above.

[0070] S4. Based on the five indicators of water quality obtained from the above water quality testing, the posterior probability P of the sample being classified as Class I to V water is calculated using a weighted distance formula, specifically including:

[0071] S41. The absolute distance D between the measured value and the standard value of a project at a very large indicator data site can be calculated using the following formula:

[0072] ①When i=1:

[0073]

[0074] ②When i≠1:

[0075]

[0076] The absolute distance D between the measured value and the standard value of a very small indicator data site can be calculated using the following formula:

[0077] ①When i=1:

[0078]

[0079] ②When i≠1:

[0080]

[0081] The distinction between maximum and minimum indicators depends on whether the value of the indicator increases or decreases with the improvement of water quality level in the national standard. For maximum indicators, the higher the value, the better, such as dissolved oxygen. For minimum indicators, the lower the value, the better, such as nitrogen, phosphorus, and permanganate.

[0082] S42. Calculate relative distance

[0083]

[0084] Where D(x) jk |y ji () represents the relative distance. The smaller the value, the less likely that the water quality level of station k belongs to i; conversely, the larger the value, the greater the probability.

[0085] S43. For sample point k, for index j, the D(x) of water quality belonging to level i (i=1,2,3,4,5) is... jk |y ji Perform vertical normalization

[0086]

[0087] In the formula, P ijk P represents the probability that the j-th indicator at the k-th detection point belongs to the i-th water quality level. ik This represents the probability that the k-th detection point belongs to the i-th water quality level.

[0088] S5. Divide the adjacent water quality grades from I to V into ten equal parts (because the calculated posterior probabilities show a small difference between different grades, it is considered that the probability difference between adjacent grades is not significant, and directly classifying them into one category would be too one-sided; therefore, grades I and II are divided into 1.1, 1.2, ..., 1.9, etc.). Based on the highest probability value P among the above posterior probabilities P... ik And the second highest probability value P (i±i)k The difference C is used to determine the water quality grade L:

[0089]

[0090] Where d has no specific meaning, it is only used to define the range of C, d = 1, 2, 3, 4, 5; i is the water quality grade.

[0091] S6. Based on the above suspended particulate matter density D and water quality grade L, the water quality grade of each sample is matched with the suspended particulate matter density of that sample. A large number of samples are selected as the training set, and the fitcnb function in the computer software MATLAB R2022a is used to generate a Bayesian classifier for water quality assessment based on suspended particulate matter.

[0092] S7. Select the suspended particulate matter density D of the test set samples, and predict the water quality level of the sample points based on the Bayesian classifier predict function. During the test, you only need to input the D_De and D_Fi values ​​of the sample to be tested, and the system will automatically obtain the specific value between which two water quality levels the water quality level is, such as 1.2 between Class I and Class II.

[0093] Example 2

[0094] In this embodiment, 1245 samples were collected and tested. The test results of five indicators for some samples are shown in Table 1 below:

[0095] Table 1 shows the test results of five indicators for some samples.

[0096] DO COD_Mn TN TP NH3-N 1 6.390 3.260 0.990 0.0115 0.2486 2 7.360 1.370 0.489 0.0097 0.0626 3 8.150 0.860 0.942 0.0083 0.1490 4 7.630 1.940 0.859 0.1030 0.3094 5 7.340 1.980 0.355 0.0880 0.2649 6 6.730 1.020 0.661 0.2212 0.0119 7 7.420 1.530 0.318 0.1612 0.0329 8 5.130 1.620 0.764 0.0692 0.2075 9 3.430 1.130 0.301 0.0484 0.1622 10 4.290 1.370 0.321 0.0121 0.1984

[0097] The posterior probability P of the samples shown in Table 1 being in Class I to V water was calculated using the distance formula assigned in step S4. The calculation results are shown in Table 2.

[0098] Table 2 Posterior probability matrix

[0099]

[0100]

[0101] In this embodiment, out of 1245 samples, 1195 samples were used as the training set and 50 samples were used as the test set. The posterior probability P and water quality grade result L of some test set samples are shown in Table 3 below.

[0102] Table 3. Water quality grade results and posterior probability of the test data.

[0103]

[0104] According to test results, the accuracy rate can reach 70% ± 5%.

[0105] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A machine learning Bayesian model for water quality assessment based on suspended particulate matter density, characterized in that, Includes the following steps: S1. Collect and test several water quality samples. The density of suspended particulate matter was measured using indoor testing, and multiple water quality indicators for each sample were obtained using outdoor testing. S2. Establish an entropy-weighted Bayesian model for water quality assessment; S3. Based on the indicator data obtained from S1 detection, calculate the entropy weight according to the degree of disorder in the data system; S4. Calculate the posterior probability P of the sample being in Class I to V water using a weighted distance formula; The calculation of the posterior probability P specifically includes: S41. The absolute distance D between the measured value and the standard value of a project at a very large indicator data site can be calculated using the following formula: ① When i = 1: ； ②When i ≠ 1: ； Among them, y ji x represents the standard value of index j for water quality level i. jk This represents the measured value of index j at station k, where i is the water quality level and j is the index. The absolute distance D between the measured value and the standard value of a very small indicator data site can be calculated using the following formula: ① When i = 1: ； ②When i ≠ 1: ； S42. Calculate relative distance: ； in, The value represents the relative distance; the smaller the value, the less likely that the water quality level of station k belongs to i; conversely, the larger the value, the greater the probability. S43. For sample point k, targeting index j, those belonging to water quality level i... Perform vertical normalization: ； ； In the formula, P ijk P represents the probability that the j-th indicator at the k-th detection point belongs to the i-th water quality level. ik W represents the probability that the k-th detection point belongs to the i-th water quality level; j It is the weight of the j-index; S5. Divide the adjacent water quality grades from I to V into ten equal parts, based on the highest probability value among the posterior probabilities P mentioned above. and the second highest probability value The difference C is used to determine the water quality grade L: ； Where d has no specific meaning, it is only used to define the range of C, d=1,2,3,4,5; i is the water quality grade; N means the number of equal parts, which is equal to 10; S6. Based on the water quality grade of each sample calculated in S5, match the water quality grade of each sample with the suspended particulate matter density of that sample, select a large number of samples as the training set, and use the fitcnb function in the computer software MATLAB R2022a to generate a Bayesian classifier for water quality assessment based on suspended particulate matter. S7. Select the suspended particulate matter density of the test set samples and predict the water quality level of the sample points based on the Bayesian classifier predict function.

2. The machine learning Bayesian model for water quality assessment based on suspended particulate matter density according to claim 1, characterized in that, The suspended particulate matter density index in step S1 includes the density of detrital suspended particulate matter and the density of fibrous suspended particulate matter.

3. The machine learning Bayesian model for water quality assessment based on suspended particulate matter density according to claim 1, characterized in that, The method for obtaining the suspended particulate matter density index in step S1 includes: obtaining suspended particulate matter samples by filtering with a 25# biological mesh; using the ZooSCAN image automatic scanning and analysis system, diluting the collected suspended particulate matter samples to a certain extent; and acquiring images and morphological data of suspended particulate matter by scanning with the instrument at a resolution of 300dp and 2400dp; uploading the suspended particulate matter images and morphological data to the database using FileZilla software; and performing classification training, manual verification, and data export on the website www.ecotaxa.cn to obtain the density of detrital suspended particulate matter and the density of fibrous suspended particulate matter.

4. The machine learning Bayesian model for water quality assessment based on suspended particulate matter density according to claim 1, characterized in that, Step S3, calculating the entropy weights, specifically includes: S31. Data preprocessing: Organize the raw data into the original matrix X and standardize it: ； In the formula, k is the station, j is the indicator, X is the matrix composed of the original data, and x is a certain item in the original matrix. kj The values ​​are the multiple water quality indicators of each sample detected in step S1, Z is the standardized matrix, z is a certain item in the standardized matrix, and n is the total number of stations. S32. Calculate the probability matrix P: ； Where P is the probability matrix, and p is a term in the probability matrix; S33. Calculate the entropy weight W ； Among them, e j It is an intermediate variable, W j is the weight of indicator j, and m is the total number of indicators.

5. The machine learning Bayesian model for water quality assessment based on suspended particulate matter density according to claim 1, characterized in that, In step S1, outdoor testing was used to obtain five water quality indicators for each sample, including dissolved oxygen (DO), permanganate index (COD_Mn), total nitrogen (TN), total phosphorus (TP), and ammonia nitrogen (NH3-N).

Images