Water quality monitoring methods and equipment

By conducting linear regression analysis and Cook distance calculation on water quality monitoring data, combining historical and real-time data correlation analysis, an abnormal data capture model is created, which solves the problem of low recognition efficiency of water quality monitoring abnormal data in the existing technology, and achieves high efficiency , accurate water quality monitoring and analysis.

CN114817851BActive Publication Date: 2024-11-01LIHE TECH (HUNAN) CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202110113745.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-27
Publication Date
2024-11-01
Estimated Expiration
2041-01-27

AI Technical Summary

Technical Problem

Existing water quality monitoring technologies are difficult to efficiently identify and process abnormal data, resulting in low data review efficiency, poor accuracy, and difficult to respond to water quality pollution problems in a timely manner.

Method used

By obtaining water quality monitoring data of different variable parameters at the same site, performing linear regression analysis and Cook distance calculation, identifying abnormal points, and combining historical data and real-time data correlation analysis, creating an abnormal data capture model, automatically filtering and predicting abnormalities data.

Benefits of technology

It improves the analysis accuracy and efficiency of water quality monitoring data, can timely identify the causes of abnormal data, reduce the amount of calculation, and has the ability to combine analysis of single monitoring indicators and multiple indicators.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114817851B_ABST
    Figure CN114817851B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a water quality monitoring method and a water quality detection device. The water quality monitoring method comprises obtaining water quality monitoring data corresponding to different variable parameters of the same site; analyzing the correlation between the variable parameters in the water quality monitoring data, and performing linear regression analysis on the variable parameter group with the correlation higher than a threshold; calculating the Cook distance according to the error vector, standard error and parameter estimation value determined by the linear regression analysis, identifying the abnormal point through the Cook distance, and marking the abnormal point with the Cook distance higher than the threshold as an abnormal value; obtaining the water quality monitoring data of the site within an update period; when the update period does not exceed a preset time range, updating the parameter estimation value according to the orthogonal unit matrix of the variable parameters determined by the previous linear regression analysis, calculating the Cook distance according to the updated parameter estimation value, the error vector and the standard error, identifying the abnormal point through the Cook distance, and marking the abnormal point with the Cook distance higher than the threshold as an abnormal value.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of water quality monitoring, and in particular to a water quality monitoring method and equipment. Background Art

[0002] Water pollution is mainly caused by pollutants generated by human activities, often by harmful chemicals that reduce or lose the use value of water. Sewage contains acids, alkalis, oxidants, compounds such as copper, cadmium, mercury, arsenic, and organic poisons such as benzene, ethylene dichloride, and ethylene glycol, which will poison aquatic organisms, affect drinking water sources, and destroy the landscape of scenic spots. When organic matter in sewage is decomposed by microorganisms, it consumes oxygen in the water, affecting the life of aquatic organisms. After the dissolved oxygen in the water is exhausted, the organic matter undergoes anaerobically decomposed, producing unpleasant gases such as hydrogen sulfide and mercaptans, which further deteriorate the water quality. Therefore, water pollution has posed a major threat to human survival and safety, and has become a major obstacle to the sustainable development of human health, economy and society.

[0003] At present, with the development of online water quality monitoring technology, it is now possible to obtain a large amount of water quality monitoring data. However, the review of data is still mainly done manually. With the increase in the amount of online data collection, the workload of manual review has increased sharply, which has increased the efficiency of water quality test data outlier identification and subsequent on-site investigation. In addition, since manual review relies on experience and senses, it is easy to make mistakes or miss reviews when faced with huge amounts of data. The review takes a long time, and it is difficult to complete the prediction in the first time after an abnormality occurs on site, which easily causes a lag in investigation.

[0004] In addition, abnormal data is often collected during the monitoring data acquisition process. The sources of abnormal data include water quality changes and some failures of the monitoring instruments themselves. How to efficiently identify the real causes of abnormal data is the primary condition for determining the authenticity of monitoring data, and it is also the cornerstone of subsequent cause analysis and solutions. Summary of the invention

[0005] In order to solve the existing technical problems, the embodiments of the present invention provide a water quality monitoring method and device that is more accurate and efficient and improves the timeliness and pertinence of the response to water pollution problems.

[0006] An embodiment of the present invention provides a water quality monitoring method, comprising:

[0007] Obtain water quality monitoring data corresponding to different variable parameters at the same site;

[0008] Analyze the correlation between the variable parameters in the water quality monitoring data, and perform linear regression analysis on the variable parameter groups whose correlation is higher than a threshold;

[0009] Calculating the Cook distance according to the error vector, standard error and parameter estimation value determined by the linear regression analysis, identifying outliers by the Cook distance, and marking outliers with a Cook distance greater than a threshold as outliers;

[0010] Obtaining water quality monitoring data of the site within an update period;

[0011] When the update period does not exceed the preset time range, the parameter estimation value is updated according to the orthogonal unit matrix of the variable parameters determined by the previous linear regression analysis, the Cook distance is calculated according to the updated parameter estimation value, the error vector and the standard error, the outlier point is identified by the Cook distance, and the outlier point with a Cook distance greater than a threshold is marked as an outlier;

[0012] When the update period exceeds the preset time range, the step of analyzing the correlation between the variable parameters in the water quality monitoring data and performing linear regression analysis on the variable parameter groups whose correlation is higher than a threshold is returned.

[0013] On the other hand, an embodiment of the present application provides a water quality monitoring device, including a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, it is the water quality monitoring method described in any embodiment of the present application.

[0014] The water quality monitoring method and equipment provided in the above embodiment collects water quality monitoring data corresponding to different variable parameters of the site, analyzes the correlation between the variable parameters in the water quality monitoring data, performs linear regression analysis on the variable parameter group with correlation higher than the threshold, identifies abnormal points by calculating the Cook distance of different variable parameter sample points, and marks the abnormal points with Cook distance greater than the threshold as abnormal values. In this way, historical data is used to analyze whether there is a correlation between different parameters of the same site. Combined with the management idea of ​​one site and one policy, when the data differs from the mathematical relationship, the system will filter out the data. Secondly, the water quality monitoring data of the site in the update period is obtained. When the update period does not exceed the preset time range, the approximate update method of Cook distance is used to calculate the water quality monitoring data in the update period to identify abnormal points. When the update period exceeds the preset time range, linear regression analysis is used to recalculate and analyze, which can efficiently identify the real cause of the abnormal data, not only ensuring the accuracy of the analysis, but also greatly reducing the amount of calculation and improving the analysis efficiency. In this way, by analyzing the rationality, logic, and relevance of historical data, real-time data, etc., an abnormal data capture model is created, suspicious data is automatically screened and predicted, and the system has the ability to judge and analyze abnormal data. It can not only provide various analysis mechanisms for a single monitoring indicator, but also has the ability to analyze multiple monitoring indicators in combination. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flow chart of a water quality monitoring method in one embodiment of the present invention;

[0016] Figure 2 A schematic diagram of determining an abnormal value by deviating from a regression line in a water quality monitoring method according to an embodiment of the present invention;

[0017] Figure 3 A schematic diagram of identifying abnormal values ​​in a time series in a water quality monitoring method according to an embodiment of the present invention;

[0018] Figure 4 A flow chart of a water quality monitoring method in another embodiment of the present invention;

[0019] Figure 5 It is a schematic structural diagram of a water quality monitoring device in another embodiment of the present invention;

[0020] Figure 6 Schematic diagram of the structure of a water quality monitoring device in one embodiment of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0023] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0024] See also Figure 1 , a water quality monitoring method provided by an embodiment of the present invention comprises the following steps:

[0025] S101, obtaining water quality monitoring data corresponding to different variable parameters at the same site.

[0026] Among them, water quality monitoring data may include the corresponding water quality monitoring data manually collected from each site, or may refer to water quality monitoring data corresponding to different variable parameters automatically collected from each site at a preset frequency. Variable parameters may include different parameters used to characterize water quality, such as water temperature, pH value, dissolved oxygen, conductivity, turbidity, permanganate index (CODMn), ammonia nitrogen, total phosphorus, total nitrogen, chlorophyll α and algae density. After the water quality monitoring data is collected, it can be sent remotely to the water quality monitoring data center for storage, and the water quality monitoring auditors can analyze the water quality monitoring data at the water quality monitoring data center.

[0027] The obtaining of water quality monitoring data corresponding to different variable parameters of the same site may be obtaining water quality monitoring data corresponding to different variable parameters collected at the same site within a set historical time period.

[0028] S102, analyzing the correlation between variable parameters in the water quality monitoring data, and performing linear regression analysis on the variable parameter groups whose correlation is higher than a threshold.

[0029] The correlation coefficient can be used to characterize the relationship between different variable parameters. The correlation between the variable parameters in the water quality monitoring data is analyzed, and the two variable parameters are represented by x and y respectively. The correlation coefficient ρ between the two variable parameters can be shown in the following formula 1.

[0030]

[0031] The two variable parameters with larger correlation coefficients are taken as the variable parameter group with correlation higher than the threshold value for linear regression analysis.

[0032] S103, calculating the Cook distance according to the error vector, standard error and parameter estimation value determined by the linear regression analysis, identifying outliers by the Cook distance, and marking outliers with a Cook distance greater than a threshold as outliers.

[0033] Please refer to Figure 2 and Figure 3 , by performing linear regression analysis, a scatter plot and a regression line can be drawn, and the Cook distance of each variable parameter can be calculated. The larger the Cook distance, the farther it deviates from the regression line. The abnormal points with a Cook distance greater than the threshold are marked as abnormal values. Optionally, the Cook distance threshold is 5 times the average value of the Cook distance of all variable parameters. The calculation of the Cook distance can be shown in the following formula 2:

[0034]

[0035] h i is the parameter estimate of the ith element, is the i-th component of the error vector, that is The i-th component of the vector, MSE refers to the standard error.

[0036] S104, obtaining water quality monitoring data of the site within an update period.

[0037] Real-time analysis is performed on the continuously updated water quality monitoring data. The update period may be the most recent collection period determined according to a pre-set collection period. Acquiring the water quality monitoring data of the site within the update period refers to performing a linear regression analysis on the water quality detection data within the previous collection period and then acquiring the water quality monitoring data within the latest collection period, so as to continuously monitor changes in water quality.

[0038] S105, when the update period does not exceed the preset time range, the parameter estimation value is updated according to the orthogonal unit matrix of the variable parameters determined by the previous linear regression analysis, the Cook distance is calculated according to the updated parameter estimation value, the error vector and the standard error, the outlier point is identified by the Cook distance, and the outlier point with a Cook distance greater than a threshold is marked as an outlier.

[0039] The preset time range can be determined according to the amount of data. The calculation of Cook's distance requires that all data be subjected to regression analysis before it can be calculated. For continuously updated data, each newly added data must be re-analyzed, which will consume a lot of computing resources and slow down the efficiency of water quality monitoring data analysis. When the update period does not exceed the preset time range, the approximate update method of Cook's distance is used to calculate the water quality monitoring data within the update period, and the abnormal points are identified by Cook's distance, and the abnormal points with Cook's distance greater than the threshold are marked as abnormal values. By analyzing the rationality, logic, and relevance of historical data, real-time data, etc. of water quality detection data, identifying and marking abnormal values, creating a capture model for abnormal data, and automatically screening and prejudging suspicious data, it has the ability to analyze and judge abnormal data.

[0040] The calculation formula of the approximate update method of Cook's distance is shown in the following formula 3:

[0041]

[0042] Among them, h i is the parameter estimate of the ith element, is the i-th component of the error vector, that is The i-th component of the vector, MSE refers to the standard error, The MSE can use the value determined by the linear regression analysis described above. iThe update can be performed based on the orthogonal unit matrix of the variable parameters determined by the previous linear regression analysis. The two variable parameters of the water quality monitoring data within the update period are represented by y0 and x0, and the calculation of the updated parameter estimate is shown in the following formula 4:

[0043]

[0044] The orthogonal unit matrix of the variable parameters determined by the linear regression analysis is (X T X) -1 .

[0045] When the update period does not exceed the preset time range, the water quality monitoring data within the update period is calculated using the approximate update method of the Cook distance. The larger the Cook distance, the further the sample deviates from the normal regression line. For example, points that are 5 times greater than the average Cook distance of all points can be marked as abnormal points of the regression relationship, and the abnormal points can be marked in the original time series. Identifying abnormal points through the Cook distance can ensure approximate accuracy, reduce the amount of calculation, and improve data analysis efficiency.

[0046] S106, when the update period exceeds the preset time range, return to the step of analyzing the correlation between the variable parameters in the water quality monitoring data, and performing linear regression analysis on the variable parameter groups whose correlation is higher than a threshold.

[0047] When the update period exceeds the preset time range, the regression relationship of the water quality monitoring data in the new period is recalculated to ensure the accuracy of the data analysis. The method of recalculating the regression relationship of the water quality monitoring data in the new period is the same as the method of the aforementioned linear regression analysis, which will not be repeated here. The preset time range can be determined according to the amount of data. For example, using a linear regression analysis of a year's water quality monitoring data, the update period can be set to one month, one quarter, etc. The water quality monitoring data within one month or one quarter is processed using the approximate update method using Cook's distance, and the water quality monitoring data after one month or one quarter is recalculated using linear regression analysis.

[0048] In the above embodiment, by collecting water quality monitoring data corresponding to different variable parameters of the site, the correlation between the variable parameters in the water quality monitoring data is analyzed, and the variable parameter group with correlation higher than the threshold is subjected to linear regression analysis. The abnormal point is identified by calculating the Cook distance of the sample points of different variable parameters, and the abnormal point with Cook distance greater than the threshold is marked as an abnormal value. In this way, the historical data is used to analyze whether there is a correlation between different parameters of the same site. Combined with the management idea of ​​one station and one policy, when the data differs from the mathematical relationship, the system will filter out the data. Secondly, the water quality monitoring data of the site in the update period is obtained. When the update period does not exceed the preset time range, the approximate update method of Cook distance is used to calculate the water quality monitoring data in the update period to identify the abnormal point. When the update period exceeds the preset time range, the linear regression analysis is used to recalculate and analyze, which can efficiently identify the real cause of the abnormal data and judge the authenticity of the monitoring data, which not only ensures the accuracy of the analysis, but also greatly reduces the amount of calculation and improves the efficiency of the analysis. In this way, by analyzing the rationality, logic, and relevance of historical data, real-time data, etc., an abnormal data capture model is created, and suspicious data is automatically screened and predicted, so that the ability to judge and analyze abnormal data can be acquired, thereby not only providing various analysis mechanisms for a single monitoring indicator, but also having the ability to analyze multiple monitoring indicators in combination.

[0049] In some embodiments, before analyzing the correlation between the variable parameters in the water quality monitoring data, the method further includes:

[0050] The water quality monitoring data is screened according to the abnormal value upper limit and abnormal value lower limit corresponding to different variable parameters, and the set variable parameter relationship to obtain effective water quality monitoring data.

[0051] Optionally, the variable parameters may include different parameters for characterizing water quality, such as water temperature, pH value, dissolved oxygen, conductivity, turbidity, permanganate index (CODMn), ammonia nitrogen, total phosphorus, total nitrogen, chlorophyll α and algae density, etc. The upper and lower limits of abnormal values ​​corresponding to the variable parameters may be shown in Table 1 below:

[0052]

[0053]

[0054] The set variable parameter relationships include: a) total nitrogen is greater than ammonia nitrogen; b) chemical oxygen demand is greater than permanganate index; c) chemical oxygen demand is greater than biochemical oxygen demand.

[0055] The upper and lower limits of abnormal values ​​corresponding to different variable parameters, as well as the set variable parameter relationships can be determined based on manually reviewed empirical values. By screening the water quality monitoring data based on the upper and lower limits of abnormal values ​​corresponding to different variable parameters and the set variable parameter relationships, some calculations that do not participate in mathematical analysis can be deleted, thereby achieving the purpose of reducing noise to improve the accuracy of mathematical analysis.

[0056] In some embodiments, performing linear regression analysis on the variable parameter group with correlation higher than a threshold value comprises:

[0057] Linear regression analysis was performed on the variable parameter groups with correlations above the threshold to establish a one-dimensional model;

[0058] Form an n-dimensional vector y according to the value of one variable parameter in the variable parameter group, form an n*p matrix X according to the value of another variable parameter, establish a linear parameter vector B with the linear parameters of the one-dimensional model, and establish a conversion relationship between the vector y, the matrix X and the linear parameter vector B according to the one-dimensional model;

[0059] The parameter estimation value, error vector and standard error of the linear parameter vector are calculated respectively according to the conversion relationship.

[0060] Taking the variable parameter group with correlation higher than the threshold value including two variable parameters x and y as an example, a linear regression analysis is performed on the variable parameter group x and y to establish a one-dimensional model as shown in the following formula 5:

[0061] y i =a+bx i +ε i ; (Formula 5)

[0062] a and b are linear parameters respectively. The values ​​corresponding to each time series of variable parameters x and y are samples. One of the indicators is marked as an n-dimensional vector y, and the other indicator and the vector of all 1s form an n*p matrix X=(1, x), where p=2, denoted as the linear parameter vector B(a, b), ε represents the n-dimensional error vector, then the matrix form of the above formula is: y=XB+∈.

[0063] The parameter estimates for B are:

[0064] The parameter estimates for y are:

[0065] Let H = X(X′X) -1 X′, the above formula can be written as: The parameter estimate is the i-th diagonal element h of the matrix H i .

[0066] The parameter estimate of the error vector ε is: Where I is the identity matrix.

[0067] The standard error is:

[0068] In the above embodiment, a corresponding scatter plot and a regression line can be drawn according to the above calculation, and a one-dimensional model is established by performing linear regression analysis on a variable parameter group whose correlation is higher than a threshold value. The water quality monitoring data acquired within the update period is monitored in real time according to the one-dimensional model, suspicious data is automatically screened and predicted, and abnormal data is captured, thereby realizing intelligent monitoring of water quality.

[0069] In some embodiments, the water quality monitoring method further comprises:

[0070] The determination coefficient is calculated according to the total variance of the vector y and the error vector, and the regression effect of the linear regression analysis is determined according to the determination coefficient.

[0071] The coefficient of determination is used to judge the regression effect. The larger the coefficient of determination, the better the regression effect. The coefficient of determination is expressed as R 2 The calculation of the coefficient of determination is shown in Formula 6:

[0072]

[0073] The total variance of vector y is sst, is the parameter estimate of the error vector ε.

[0074] In some embodiments, when the update period does not exceed a preset time range, the parameter estimation value is updated according to the orthogonal unit matrix of the variable parameters determined by the previous linear regression analysis, the Cook distance is calculated according to the updated parameter estimation value, the error vector and the standard error, the outlier point is identified by the Cook distance, and the outlier point with a Cook distance greater than a threshold is marked as an outlier, including:

[0075] When the update period does not exceed the preset time range, the orthogonal unit matrix (X T X) -1 , and a parameter vector formed by the variable parameters within the update period to calculate updated parameter estimates;

[0076] The Cook distance is calculated according to the updated parameter estimation value, error vector and standard error, outliers are identified by the Cook distance, and outliers with a Cook distance greater than a threshold are marked as outliers.

[0077] The calculation of Cook's distance requires regression analysis of all data. For the constantly updated water quality monitoring data, each new data will require a new regression analysis, which will consume a lot of computing resources. MSE,h i The two variable parameters of the original variable parameter group are y and X (in vector form). The variable parameters in the water quality monitoring data obtained during the update period are represented by y0 and x0 (here in row vector form). The regression relationship y=a+bx and the MSE of the regression have been obtained using the original data. The regression relationship obtained from the original data can be used to calculate Right now MSE uses the MSE of the original regression relationship. The above two quantities are calculated approximately, and h i The calculation formula can be shown as follows:

[0078]

[0079] It only requires that the previous regression preserves the orthogonal identity matrix (X T X) -1 , the amount of updated calculations can be greatly reduced. When using the Cook distance approximation method, ensure that The approximate accuracy of MSE is as follows: therefore, the larger the amount of data used in the original regression relationship, the higher the accuracy of the update. After updating a part of the samples, the regression relationship is recalculated again. For example, if one year of data is used for calculation before, the regression can be recalculated again after one month of update. This can reduce the amount of calculation and ensure that the new data has little impact on the regression relationship, thereby ensuring the accuracy of data analysis.

[0080] In some embodiments, it also includes:

[0081] Obtain water quality monitoring data corresponding to different variable parameters of upstream stations;

[0082] Analyze the correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals;

[0083] The time when pollution occurred at the upstream site is determined based on the correlation analysis results.

[0084] The time series of a variable parameter of the upstream site is x=(x1, x2, ...), and the time series of the same variable parameter of the downstream site is y=(y1, y2, ...). The correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals is analyzed. Taking the interval k as an example, the indicator sequence of site x can be lagged k periods to obtain x(k)=(na, na, ..., x1, x2, ...). x1 appears in the time series at the k+1 position, and then the correlation coefficient between x(k) and y is calculated. The calculation process is equivalent to deleting the samples of the first few periods where na appears. The correlation lagged k periods can more accurately discover the impact of the relevant pollution of the upstream site on the downstream after a period of time.

[0085] The analysis of the correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals includes:

[0086] A reference time period is determined according to the distribution of the abnormal values ​​of the marked current site, and a correlation analysis is performed between the water quality monitoring data of the current site in the reference time period and the water quality monitoring data of the upstream site at different intervals.

[0087] The correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals is analyzed. According to the correlation analysis results, in the process of determining the pollution occurrence time of the upstream site, if all the data are used, the effect is not obvious. Therefore, we put the investigation time period in the time period of the abnormality, and judge the influence of the upstream and downstream sites when the abnormality occurs by observing the correlation of the lag k period. According to the distribution of the marked abnormal values, the time window is expanded with the abnormal value as the pile point, the reference time period is determined, and the data in the reference time period is used to calculate the correlation between the data of the interval k period, which can improve the accuracy and efficiency of the analysis. For example, in units of days, each abnormal time point is approximated to the nearest time point at the end of the month. After such processing, the dense abnormal time points are all turned into the time at the end of the month. Then, by taking 15 days before and after the end of the month, all the abnormal times approximating to the time point at the end of this month can be covered, and then the above-mentioned correlation analysis of the front and rear sites is performed in the abnormal time period.

[0088] In some embodiments, it also includes:

[0089] For variable parameters whose correlation is lower than a threshold, an updated variable parameter is formed according to the change amount of two parameters with a time series interval of n in the variable parameter vector;

[0090] The correlations between the updated variable parameters are analyzed, and the step of performing linear regression analysis on the variable parameter groups whose correlations are higher than a threshold is returned.

[0091] In actual data analysis, the correlation coefficients between different variable parameters at the same site may be low. However, from the actual perception, the change trends have a certain correlation. For variable parameters with correlations lower than the threshold, the updated variable parameters are formed according to the changes of the two parameters with a time series interval of n in the variable parameter vector, and then the correlation analysis is performed on the updated variable parameters. Among them, the updated variable parameters are formed according to the changes of the two parameters with a time series interval of n in the variable parameter vector. The correlation coefficient can be calculated using the data after a difference. For example, the two variable parameters are (x1, x2, ..., x n ) and (y1, y2, …, y n ), a first difference refers to the change of two variable parameters with an original time series interval of 1. The updated variable parameters formed by calculating the change of two parameters with a time series interval of 1 are (x2-x1, ..., x n -x n-1 ) and (y2-y1,…,y n -y n-1 ), and then use the new data for subsequent correlation coefficient calculations and regression analysis.

[0092] Similarly, we can also use the change rate of the original time series with an interval of 1 for subsequent analysis, that is, the new data is and Furthermore, we can use the change in the original time series with an interval of k, that is, the new data is (x k+1 -x1,…,x n -x n-k ) and (y k+1 -y1,…,y n -y n-k ) is used to perform subsequent correlation coefficient calculation and regression analysis. In this embodiment, the change amount of the original time series with an interval of 1 is mainly used to perform subsequent correlation coefficient calculation and regression analysis.

[0093] In some embodiments, the water quality monitoring method further comprises:

[0094] According to the linear regression analysis results of the water quality monitoring data within the set time period, the model of each variable parameter group is established, and the latest water quality monitoring data is judged in real time according to the model to see whether it meets the model characteristics;

[0095] The data that does not conform to the characteristics of the model is screened out and provided to the auditor for manual review.

[0096] The set time period can be a time period of any length, such as one year, half a year, one quarter, one month, half a month, etc. Based on the linear regression analysis results of the water quality monitoring data within the set time period, a model of each variable parameter group is established, and the latest water quality monitoring data is judged in real time based on the model to see whether it meets the model characteristics, and the data that does not meet the model characteristics is screened out and provided to the auditor for manual review. In this way, by establishing the correlation between the various parameter indicators of the model, the model is used as the background to judge in real time whether the latest monitoring data meets the characteristics of the model. If it does not meet the requirements, the data is screened out and provided to the data auditor for reference, which can greatly improve the monitoring efficiency and accuracy.

[0097] In order to have a further overall understanding of the water quality monitoring method embodiment, please refer to Figure 4 , which is an optional specific example of a water quality detection method, comprises the following steps:

[0098] S11, calculate the correlation between different indicators at the same site; different indicators refer to different variable parameters used to characterize water quality, and the formula for calculating the correlation coefficient is:

[0099] S12, for indicators with high correlation, regression analysis is performed to establish a one-dimensional model y i =a+bx i +ε i ;

[0100] For example, all samples of an indicator are n-dimensional vectors y, and another indicator x and a vector of all 1s form an n*p matrix. ε represents an n-dimensional error vector, and the standard error MSE can be calculated to draw a scatter plot and a regression line.

[0101] S13, through the coefficient of determination R 2 To judge the regression effect, R 2 The larger the regression effect, the better; the formula for the coefficient of determination is: The total variance of sample y is sst;

[0102] S14, calculate the Cook distance of each sample point, and filter outliers according to the Cook distance; the Cook distance formula is:

[0103] S15, Cook distance online update calculation, for the data that the update period does not exceed the time range, use the (X T X) -1 , MSE,h i Perform an update and re-do a regression analysis for data that is outside the update period;

[0104] S16, calculating the correlation between different sites of the same indicator to determine the time when the pollution of the upstream site occurred; wherein the time period when the abnormality occurred is determined according to the abnormal value identified by the current site, and the correlation analysis of the previous and next sites is performed within the abnormal time period to improve the analysis efficiency and accuracy;

[0105] S17, when the correlation coefficients between different indicators at the same site are low, the correlation coefficients are calculated after a difference is performed on the two indicator data.

[0106] The water quality monitoring method provided in the embodiment of the present application uses historical data to analyze whether there is a correlation between different parameters at the same site, combined with the management idea of ​​one site one policy, when the data shows a difference from the mathematical relationship, the system will filter out the data; use historical data to analyze whether the same parameter at the same site is within a reasonable range, and by analyzing the rationality, logic, and correlation of historical data, real-time data, etc., create an abnormal data capture model, automatically filter and predict suspicious data, and use historical data to analyze the correlation between the same indicators upstream and downstream. For example, if the water quality of the upstream of the river is significantly better than that of the downstream, it may be that the number of downstream production enterprises has increased and pollution has worsened. When the data shows an abnormal violation of the relationship, the system will automatically filter out the data, so that it can have the ability to judge and analyze abnormal data, not only can it provide various analysis mechanisms for a single monitoring indicator, but also has the ability to combine and analyze multiple monitoring indicators.

[0107] On the other hand, the embodiment of the present application further provides a water quality monitoring device, including a first acquisition module 11, used to obtain water quality monitoring data corresponding to different variable parameters of the same site; an analysis module 12, used to analyze the correlation between the variable parameters in the water quality monitoring data, and perform linear regression analysis on the variable parameter group whose correlation is higher than a threshold; a marking module 13, used to calculate the Cook distance according to the error vector, standard error and parameter estimation value determined by the linear regression analysis, identify outliers through the Cook distance, and mark the outliers whose Cook distance is greater than the threshold as outliers; a second acquisition module 14, used to obtain the site within the update period water quality monitoring data; an updating module 15, for updating the parameter estimation value according to the orthogonal unit matrix of the variable parameters determined by the previous linear regression analysis when the updating period does not exceed the preset time range, calculating the Cook distance according to the updated parameter estimation value, the error vector and the standard error, identifying the abnormal points by the Cook distance, marking the abnormal points with the Cook distance greater than the threshold as abnormal values; a loop module 16, for returning to the step of analyzing the correlation between the variable parameters in the water quality monitoring data when the updating period exceeds the preset time range, and performing linear regression analysis on the variable parameter group with the correlation higher than the threshold.

[0108] The water quality monitoring device further includes a screening module for screening the water quality monitoring data according to the upper and lower limits of abnormal values ​​corresponding to different variable parameters and the set variable parameter relationship to obtain effective water quality monitoring data.

[0109] Among them, the analysis module is specifically used to perform linear regression analysis on a variable parameter group whose correlation is higher than a threshold to establish a one-dimensional model; form an n-dimensional vector y according to the value of a variable parameter in the variable parameter group, form an n*p matrix X according to the value of another variable parameter, establish a linear parameter vector B with the linear parameters of the one-dimensional model, and establish a conversion relationship between the vector y, the matrix X and the linear parameter vector B according to the one-dimensional model; and calculate the parameter estimation value, error vector and standard error of the linear parameter vector according to the conversion relationship.

[0110] The water quality monitoring device further includes a judgment module for calculating a coefficient of determination according to the total variance of the vector y and the error vector, and judging the regression effect of the linear regression analysis according to the coefficient of determination.

[0111] The marking module is further used to, when the update period does not exceed the preset time range, determine the orthogonal unit matrix (X T X) -1 , and the parameter vector formed by the variable parameters within the update period calculate the updated parameter estimate; calculate the Cook distance according to the updated parameter estimate, error vector and standard error, identify outliers through the Cook distance, and mark the outliers with a Cook distance greater than a threshold as outliers.

[0112] Among them, the water quality monitoring device also includes an association module, which is used to obtain water quality monitoring data corresponding to different variable parameters of the upstream site; analyze the correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals; and determine the time when pollution occurs at the upstream site based on the correlation analysis results.

[0113] Among them, the association module is specifically used to determine the reference time period according to the distribution of the abnormal values ​​of the marked current site, and perform correlation analysis on the water quality monitoring data of the reference time period of the current site and the water quality monitoring data of the upstream site at different intervals.

[0114] Among them, the analysis module is also used to form updated variable parameters for variable parameters whose correlation is lower than a threshold value according to the change of two parameters with a time series interval of n in the variable parameter vector; analyze the correlation between the updated variable parameters, and return to the step of performing linear regression analysis on the variable parameter group with a correlation higher than the threshold value.

[0115] Among them, the water quality monitoring device also includes a sending module, which is used to establish a model of each variable parameter group based on the linear regression analysis results of the water quality monitoring data within a set time period, and judge in real time whether the latest water quality monitoring data meets the model characteristics based on the model; the data that does not meet the model characteristics is screened out and provided to the auditor for manual review.

[0116] It should be noted that: the water quality monitoring device provided in the above embodiment only uses the division of the above program modules as an example to illustrate the water quality monitoring process. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the method steps described above. In addition, the water quality monitoring device provided in the above embodiment and the water quality monitoring method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0117] See also Figure 6 On the other hand, an embodiment of the present application further provides a water quality monitoring device, including a processor 51 and a memory 52, wherein the memory 52 stores a computer program that can be executed by the processor, and when the computer program is executed by the processor 51, the steps of the water quality monitoring method provided in any embodiment of the present application are implemented.

[0118] It will be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments of the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0119] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A water quality monitoring method, characterized in that: include: Obtain water quality monitoring data corresponding to different variable parameters at the same site; Analyze the correlation between the variable parameters in the water quality monitoring data, and perform linear regression analysis on the variable parameter groups whose correlation is higher than a threshold; Calculating the Cook distance according to the error vector, standard error and parameter estimation value determined by the linear regression analysis, identifying outliers by the Cook distance, and marking outliers with a Cook distance greater than a threshold as outliers; Obtaining water quality monitoring data of the site within an update period; When the update period does not exceed the preset time range, the parameter estimation value is updated according to the orthogonal unit matrix of the variable parameters determined by the previous linear regression analysis, the Cook distance is calculated according to the updated parameter estimation value, the error vector and the standard error, the outlier point is identified by the Cook distance, and the outlier point with a Cook distance greater than a threshold is marked as an outlier; When the update period exceeds the preset time range, the step of analyzing the correlation between the variable parameters in the water quality monitoring data and performing linear regression analysis on the variable parameter groups whose correlation is higher than a threshold is returned.

2. The water quality monitoring method according to claim 1, characterized in that: Before analyzing the correlation between the variable parameters in the water quality monitoring data, the method further includes: The water quality monitoring data is screened according to the abnormal value upper limit and abnormal value lower limit corresponding to different variable parameters, and the set variable parameter relationship to obtain effective water quality monitoring data.

3. The water quality monitoring method according to claim 1, characterized in that: The linear regression analysis of the variable parameter group with correlation higher than the threshold value comprises: Linear regression analysis was performed on the variable parameter groups with correlations above the threshold to establish a one-dimensional model; Form an n-dimensional vector y according to the value of one variable parameter in the variable parameter group, form an n*p matrix X according to the value of another variable parameter, establish a linear parameter vector B with the linear parameters of the one-dimensional model, and establish a conversion relationship between the vector y, the matrix X and the linear parameter vector B according to the one-dimensional model; The parameter estimation value, error vector and standard error of the linear parameter vector are calculated respectively according to the conversion relationship.

4. The water quality monitoring method according to claim 3, characterized in that: Also includes: The determination coefficient is calculated according to the total variance of the vector y and the error vector, and the regression effect of the linear regression analysis is determined according to the determination coefficient.

5. The water quality monitoring method according to claim 3, characterized in that: When the update period does not exceed the preset time range, the parameter estimation value is updated according to the orthogonal unit matrix of the variable parameters determined by the previous linear regression analysis, the Cook distance is calculated according to the updated parameter estimation value, the error vector and the standard error, the outlier point is identified by the Cook distance, and the outlier point with a Cook distance greater than a threshold is marked as an outlier, including: When the update period does not exceed the preset time range, the orthogonal unit matrix (X T X) -1 , and a parameter vector formed by the variable parameters within the update period to calculate updated parameter estimates; The Cook distance is calculated according to the updated parameter estimation value, error vector and standard error, outliers are identified by the Cook distance, and outliers with a Cook distance greater than a threshold are marked as outliers.

6. The water quality monitoring method according to claim 1, characterized in that: Also includes: Obtain water quality monitoring data corresponding to different variable parameters of upstream stations; Analyze the correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals; The time when pollution occurred at the upstream site is determined based on the correlation analysis results.

7. The water quality monitoring method according to claim 6, characterized in that: The analysis of the correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals includes: A reference time period is determined according to the distribution of the abnormal values ​​of the marked current site, and a correlation analysis is performed between the water quality monitoring data of the current site in the reference time period and the water quality monitoring data of the upstream site at different intervals.

8. The water quality monitoring method according to claim 1, characterized in that: Also includes: For variable parameters whose correlation is lower than a threshold, an updated variable parameter is formed according to the change amount of two parameters with a time series interval of n in the variable parameter vector; The correlations between the updated variable parameters are analyzed, and the step of performing linear regression analysis on the variable parameter groups whose correlations are higher than a threshold is returned.

9. The water quality monitoring method according to claim 1, characterized in that: Also includes: According to the linear regression analysis results of the water quality monitoring data within the set time period, the model of each variable parameter group is established, and the latest water quality monitoring data is judged in real time according to the model to see whether it meets the model characteristics; The data that does not conform to the characteristics of the model is screened out and provided to the auditor for manual review.

10. A water quality monitoring device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and when the computer program is executed by the processor, the water quality monitoring method as claimed in any one of claims 1 to 9 is implemented.

Citation Information

Patent Citations

  • Abnormal data recognition method and system based on clustering analysis, and storage medium

    CN110941648A