Water quality monitoring data analysis method, device, equipment, and storage medium

Abstract

The embodiment of the present invention discloses a water quality monitoring data analysis method and device, equipment and computer storage medium. The water quality monitoring method includes: acquiring water quality monitoring data within a preset period; performing anomaly point analysis on the water quality monitoring data and marking abnormal values ​​of the water quality detection data; determining a reference time period according to the distribution of the abnormal values, and performing correlation analysis on the water quality monitoring data of the reference time period and the water quality monitoring data of an upstream site; and determining the time when pollution occurred at the upstream site according to the correlation analysis result.

Description

Technical Field

[0001] The present invention relates to the technical field of water quality monitoring, and in particular to a water quality monitoring data analysis method and device, equipment, and computer storage medium. Background Art

[0002] Water pollution is mainly caused by pollutants generated by human activities, often by harmful chemicals that reduce or lose the use value of water. Sewage contains acids, alkalis, oxidants, compounds such as copper, cadmium, mercury, arsenic, and organic poisons such as benzene, ethylene dichloride, and ethylene glycol, which will poison aquatic organisms, affect drinking water sources, and destroy the landscape of scenic spots. When organic matter in sewage is decomposed by microorganisms, it consumes oxygen in the water, affecting the life of aquatic organisms. After the dissolved oxygen in the water is exhausted, the organic matter undergoes anaerobically decomposed, producing unpleasant gases such as hydrogen sulfide and mercaptans, which further deteriorate the water quality. Therefore, water pollution has posed a major threat to human survival and safety, and has become a major obstacle to the sustainable development of human health, economy and society.

[0003] At present, with the development of online water quality monitoring technology, it is now possible to obtain a large amount of water quality monitoring data. However, the review of data is still mainly done manually. With the increase in the amount of online data collection, the workload of manual review has increased sharply, which has increased the efficiency of water quality monitoring data outlier identification and subsequent on-site investigation. In addition, since manual review relies on experience and senses, it is easy to make mistakes or miss reviews when faced with huge amounts of data. The review takes a long time, and it is difficult to complete the prediction in the first time after an abnormality occurs on site, which easily causes a lag in investigation.

[0004] In addition, in the process of acquiring monitoring data, abnormal data is often included. The source of data anomalies comes from water quality changes and some factors come from the failure of the monitoring instrument itself. How to efficiently identify the real cause of abnormal data is the primary condition for determining the authenticity of monitoring data, and it is also the cornerstone of subsequent cause analysis and solutions. Summary of the invention

[0005] In order to solve the existing technical problems, the embodiments of the present invention provide a water quality monitoring data analysis method and device, equipment, and computer storage medium that are more accurate and efficient and improve the timeliness and pertinence of response to water pollution problems.

[0006] In a first aspect of an embodiment of the present invention, a method for analyzing water quality monitoring data is provided, comprising:

[0007] Obtain water quality monitoring data within a preset period;

[0008] Performing anomaly analysis on the water quality monitoring data and marking abnormal values ​​of the water quality monitoring data;

[0009] Determine a reference time period according to the distribution of the abnormal values, and perform a correlation analysis between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site;

[0010] The time when pollution occurred at the upstream site is determined based on the correlation analysis results.

[0011] In a second aspect of the embodiment of the present application, a water quality monitoring data analysis device is provided, comprising:

[0012] An acquisition module is used to obtain water quality monitoring data within a preset period;

[0013] An outlier extraction module, used to perform outlier point analysis on the water quality monitoring data and mark outliers of the water quality monitoring data;

[0014] A correlation analysis module, used to determine a reference time period according to the distribution of the abnormal values, and perform a correlation analysis between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site;

[0015] The determination module is used to determine the time when pollution occurs at the upstream site according to the correlation analysis result.

[0016] According to a third aspect of an embodiment of the present application, a water quality monitoring device is also provided, comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and when the computer program is executed by the processor, the water quality monitoring data analysis method described in any embodiment of the present application is implemented.

[0017] According to a fourth aspect of the embodiments of the present application, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a controller, the water quality monitoring data analysis method described in any embodiment of the present application is implemented.

[0018] The water quality monitoring data analysis method and device, water quality monitoring equipment and computer storage medium provided in the above embodiments can efficiently identify the real cause of abnormal data and judge the authenticity of monitoring data by performing abnormal point analysis on water quality monitoring data and marking abnormal values ​​of the water quality monitoring data. It can also shorten the data identification cycle and reduce the time from problem discovery to problem solving, solve key technical problems in current environmental management work, use historical data as the basis for judgment, and use algorithms as a means to promote data outlier identification, which can solve the efficiency shortcomings of water station operation and maintenance management from a management perspective and improve the overall effectiveness of online monitoring data. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Flow chart of a water quality monitoring data analysis method in one embodiment of the present invention;

[0020] Figure 2 A flow chart of a water quality monitoring data analysis method in another embodiment of the present invention;

[0021] Figure 3 A schematic diagram of determining an abnormal value by deviating from a regression line in a water quality monitoring data analysis method according to an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of identifying abnormal values ​​in a time series in a water quality monitoring data analysis method according to an embodiment of the present invention;

[0023] Figure 5 A flow chart of a water quality monitoring data analysis method in another embodiment of the present invention;

[0024] Figure 6 The linear regression graph and trend graph of the two features of turbidity and total phosphorus in September 2019 are optional specific examples;

[0025] Figure 7 The linear regression graph and trend graph of the two features of turbidity and total phosphorus in March 2020 in the optional specific example;

[0026] Figure 8 This is a data comparison chart of the upstream and downstream sites in an optional specific example with an interval of 3 periods;

[0027] Fig. 9 It is a structural schematic diagram of a water quality monitoring data analysis device in another embodiment of the present invention;

[0028] Fig.10 Schematic diagram of the structure of a water quality monitoring device in one embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0031] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] See also Figure 1, a water quality monitoring data analysis method provided by an embodiment of the present invention comprises the following steps:

[0033] S101, obtaining water quality monitoring data within a preset period.

[0034] The water quality monitoring data may include the corresponding water quality monitoring data manually collected from each site, or may refer to the water quality monitoring data corresponding to different variable parameters automatically collected from each site at a preset frequency. The preset period may refer to any set duration such as one year, half a year, one quarter or one month, and may be determined according to the amount of data within the corresponding duration.

[0035] Optionally, the water quality monitoring data includes a plurality of different variables that can characterize the water quality, and the variable parameters may include different parameters used to characterize the water quality, such as water temperature, pH value, dissolved oxygen, conductivity, turbidity, permanganate index (CODMn), ammonia nitrogen, total phosphorus, total nitrogen, chlorophyll α and algae density. After the water quality monitoring data is collected, it can be sent to the water quality monitoring data center for storage via a remote method, and the water quality monitoring auditor can analyze the water quality monitoring data at the water quality monitoring data center. The acquisition of the water quality monitoring data within the preset period may be to obtain the water quality monitoring data corresponding to different variable parameters collected at the same site within a set historical time period.

[0036] S103, performing an abnormal point analysis on the water quality monitoring data, and marking abnormal values ​​of the water quality monitoring data.

[0037] Anomalies refer to the water quality data collected at the time point that can characterize the existence of water quality problems. By acquiring water quality monitoring data, analyzing the anomalies of the water quality monitoring data, and marking the abnormal values ​​of the water quality monitoring data, it can be used to identify abnormal values ​​within a certain time and space range, automatically screen and prejudge suspicious data, and have the ability to judge and analyze abnormal data, so as to not only provide various analysis mechanisms for a single monitoring indicator, but also have the ability to combine and analyze multiple monitoring indicators.

[0038] S105, determining a reference time period according to the distribution of the abnormal values, and performing a correlation analysis between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site.

[0039] The correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals is analyzed. According to the results of the correlation analysis, in the process of determining the time of pollution occurrence at the upstream site, if all the data are used, the effect is not obvious. Therefore, we put the investigation time period in the time period when the anomaly occurs, and by observing the correlation of the lag k period, we can judge the influence of the upstream and downstream sites when the anomaly occurs. By expanding the time window with the anomaly as the stake point according to the distribution of the marked outliers, determining the reference time period, and using the data in the reference time period to calculate the correlation between the data with an interval of k periods, the accuracy and efficiency of the analysis can be improved.

[0040] S107, determining the time when pollution occurred at the upstream site according to the correlation analysis result.

[0041] For example, taking days as the unit, approximate each abnormal time point to the nearest time point at the end of the month. After this processing, the dense abnormal time points become the time at the end of the month. Then, by taking 15 days before and after the end of the month, all abnormal times approximating to the time point at the end of the month can be covered. Then, the above-mentioned correlation analysis of the previous and next sites is performed within the abnormal time period.

[0042] Optionally, determining a reference time period according to the distribution of the abnormal values, and performing a correlation analysis between the water quality monitoring data of the reference time period and the water quality monitoring data of an upstream site, comprises:

[0043] Obtain water quality monitoring data corresponding to different variable parameters of upstream stations;

[0044] The correlation between variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals is analyzed.

[0045] The time series of a variable parameter of the upstream site is , the time series of the same variable parameters at the downstream site is , the correlation between the variable parameters in the water quality monitoring data of the current station and the upstream station at different intervals is analyzed. Taking the interval k as an example, the indicator sequence of station x can be lagged k periods to obtain , the time series only appears at position k+1 Then calculate the correlation coefficient between x(k) and y. The calculation process is equivalent to deleting the samples in the first few periods where na appears. Through the correlation with a lag of k periods, it is possible to more accurately discover the impact of related pollution from upstream sites on downstream sites after a period of time.

[0046] The analysis of the correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals includes:

[0047] A reference time period is determined according to the distribution of the abnormal values ​​of the marked current site, and a correlation analysis is performed between the water quality monitoring data of the current site in the reference time period and the water quality monitoring data of the upstream site at different intervals.

[0048] In the above embodiment, the water quality monitoring data analysis method transmits the water quality monitoring data from the field to the remote data center. After the remote data center stores the data, it performs an abnormal point analysis on the water quality monitoring data and marks the abnormal values ​​of the water quality monitoring data. This can effectively identify the real cause of the abnormal data and determine the authenticity of the monitoring data. It can also shorten the data identification cycle, reduce the time from problem discovery to problem solving, solve key technical problems in current environmental management work, use historical data as the basis for judgment, and use algorithms as a means to promote data outlier identification. This can solve the efficiency shortcomings of water station operation and maintenance management from a management perspective and improve the overall effectiveness of online monitoring data.

[0049] In some embodiments, see Figure 2 The step S103, performing an abnormal point analysis on the water quality monitoring data and marking abnormal values ​​of the water quality monitoring data, includes:

[0050] S1031, decomposing the water quality monitoring data using a time series decomposition algorithm, and marking outliers in the decomposed water quality monitoring data sequence as abnormal values.

[0051] In this embodiment, the water quality monitoring data is decomposed using the time series decomposition algorithm (Standard Template Library, STL) to detect abnormal points in the time series. For a time series X, It represents the observed value of the series at time t. The series is decomposed into three parts: seasonal term (seasonal ), trend item ) and residual ), where the seasonal term represents some periodic information in the time series, and the trend term represents the trend of the time series over time. After subtracting the seasonal term and the trend term, the anomaly points are mined from the remainder.

[0052] ;

[0053] For seasonal items, the average value of the corresponding sequence of each small cycle can be used instead. For example, if the small cycle is a year, then when calculating the seasonal value of January, the average value of all January values ​​in the current data is used as the January value in the seasonal item. The length of the small cycle needs to be determined based on the granularity of the current data. For trend items, the value of the trend item can be determined by the following three methods: first, using the sliding average of the time series as the value of the trend item; second, using the median of the time series as the value of the trend item; third, using the median absolute deviation of the time series as the value of the trend item. The calculation formula for the median absolute deviation can be as follows:

[0054] ;

[0055] After subtracting the seasonal term and trend term from the time series, ESD (Extreme Studentized Deviate) can be used to detect outliers on the remaining terms. The ESD algorithm is as follows:

[0056] ESD assumes that there are at most k outliers in the data. First, calculate the statistics in the data :

[0057] ;

[0058] In calculation When , in the calculation After that, remove the The data that is maximized in is used to calculate using the remaining n-1 data , and then iterate until we get . Utilize With critical value Compare and select the largest i so that . Then take the value that makes the previous The data point with the largest value in the statistic is regarded as an outlier, and there are i outliers in total. The calculation formula is:

[0059] ;

[0060] in It is the upper critical value of the t distribution with nk-1 degrees of freedom and p significance level.

[0061] S1033: Eliminate the marked abnormal values.

[0062] S1035: Perform secondary outlier point analysis on the water quality monitoring data after the outliers are removed by a linear regression method to mark the outliers of the water quality monitoring data.

[0063] The system extracts the original data from the database, performs STL decomposition on it, calculates and extracts features from the decomposed sequence, removes the marked outliers after STL filtering, and uses linear regression method to perform secondary identification on the data, and realizes the identification of abnormal data through the pairwise relationship between variables. When the correlation between two variables is found to be inconsistent with the norm, they are paired and marked.

[0064] In the above embodiments, after using the time series decomposition algorithm to identify outliers, the linear regression method is used to perform secondary outlier point analysis after the outliers are eliminated. Thus, the time series method and the linear regression method can be used simultaneously to identify and calculate the data from two dimensions: time transformation and relationship between variables, thereby improving the accuracy of water quality monitoring data analysis.

[0065] In some embodiments, the use of a linear regression method to perform a secondary outlier point analysis on the water quality monitoring data after removing the outliers to mark the outliers of the water quality monitoring data includes:

[0066] Analyzing the correlation between the variable parameters in the water quality monitoring data after removing the abnormal values;

[0067] The abnormal data segment is identified according to the change trend of the correlation, and the abnormal value corresponding to the abnormal data segment is extracted and marked.

[0068] The correlation relationship can be represented by the value of the correlation coefficient. The two variable parameters are represented by x and y respectively, and the correlation coefficient between the two variable parameters is It can be shown as the following formula.

[0069] ;

[0070] A curve is drawn based on the correlation between variable parameters in the water quality monitoring data. When the changing trend of the water quality monitoring data in the time series does not match the changing trend of the correlation, it can be determined as an abnormal data segment, and the abnormal value corresponding to the abnormal data segment is extracted.

[0071] In some embodiments, performing anomaly analysis on the water quality monitoring data and marking abnormal values ​​of the water quality monitoring data further includes:

[0072] Shielding the abnormal values, and performing linear regression analysis on the shielded data;

[0073] Outliers are identified using the Cook distance, and outliers with Cook distance greater than the threshold are marked as outliers.

[0074] Among them, after identifying the abnormal data segment according to the changing trend of the correlation, extracting the abnormal value corresponding to the abnormal data segment and marking it, it also includes shielding the abnormal value and performing linear regression analysis on the shielded data. Optionally, performing linear regression analysis on the shielded data includes taking two variable parameters with larger correlation coefficients as a variable parameter group with correlation higher than a threshold, performing linear regression analysis, calculating the Cook distance according to the error vector, standard error and parameter estimation value determined by the linear regression analysis, identifying abnormal points through the Cook distance, and marking the abnormal points with Cook distance greater than the threshold as abnormal values.

[0075] Please refer to Figure 3 and Figure 4 , by performing linear regression analysis, a scatter plot and a regression line can be drawn, and the Cook distance of each variable parameter can be calculated. The larger the Cook distance, the farther it deviates from the regression line. The abnormal points with a Cook distance greater than the threshold are marked as abnormal values. Optionally, the Cook distance threshold is 5 times the average value of the Cook distance of all variable parameters. The calculation of the Cook distance can be shown as follows: ;

[0076] is the parameter estimate of the ith element, is the i-th component of the error vector, that is The i-th component of the vector, is the standard error.

[0077] Taking the variable parameter group with correlation higher than the threshold value including two variable parameters x and y as an example, a linear regression analysis is performed on the variable parameter group x and y to establish a one-dimensional model as shown in the following formula:

[0078] ;

[0079] a, b are linear parameters respectively. The values ​​corresponding to each time series of variable parameters x and y are samples. One of the indicators is labeled as an n-dimensional vector y, and the other indicator and the vector with all 1s form an n*p matrix. , where p=2, denoted as the linear parameter vector B (a,b), Represents an n-dimensional error vector, then the matrix form of the above formula is: .

[0080] The parameter estimates for B are: .

[0081] The parameter estimates for y are: .

[0082] remember , then the above formula can be written as: The parameter estimates are the i-th diagonal elements of the matrix H .

[0083] Error Vector The parameter estimates of are: ; where I is the identity matrix.

[0084] The standard error is: .

[0085] In the above embodiment, a corresponding scatter plot and a regression line can be drawn according to the above calculation, and a one-dimensional model is established by performing linear regression analysis on a variable parameter group whose correlation is higher than a threshold value. The water quality monitoring data acquired within the update period is monitored in real time according to the one-dimensional model, suspicious data is automatically screened and predicted, and abnormal data is captured, thereby realizing intelligent monitoring of water quality.

[0086] In some embodiments, determining a reference time period according to the distribution of the abnormal values, and performing a correlation analysis between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site, comprises:

[0087] Arrange all the marked abnormal values, take the marked abnormal values ​​as the pile points, expand the time window according to the set time frequency with the pile points as the center, and determine the reference time period according to the time window;

[0088] A correlation analysis is performed between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site.

[0089] By expanding the time window with outliers as points according to the distribution of marked outliers, determining the reference time period, and using the data in the reference time period to calculate the correlation with the data at an interval of k periods, the accuracy and efficiency of the analysis can be improved.

[0090] In some embodiments, determining the time of occurrence of pollution at the upstream site according to the correlation analysis result includes:

[0091] The water quality monitoring data of the reference time period is subjected to correlation analysis with the upstream site data at different intervals, and the time when the pollution at the upstream site occurs is determined based on the correlation coefficient with the upstream site data at different intervals.

[0092] The correlation between the variable parameters in the water quality monitoring data of the current site and the upstream site at different intervals is analyzed. According to the results of the correlation analysis, in the process of determining the time of pollution occurrence at the upstream site, if all the data are used, the effect is not obvious. Therefore, we put the investigation time period in the time period when the anomaly occurs, and by observing the correlation of the lag k period, we can judge the influence of the upstream and downstream sites when the anomaly occurs. By expanding the time window with the anomaly as the stake point according to the distribution of the marked outliers, determining the reference time period, and using the data in the reference time period to calculate the correlation between the data with an interval of k periods, the accuracy and efficiency of the analysis can be improved.

[0093] For example, taking days as the unit, approximate each abnormal time point to the nearest time point at the end of the month. After this processing, the dense abnormal time points become the time at the end of the month. Then, by taking 15 days before and after the end of the month, all abnormal times approximating to the time point at the end of the month can be covered. Then, the above-mentioned correlation analysis of the previous and next sites is performed within the abnormal time period.

[0094] Optionally, the method further includes:

[0095] For variable parameters whose correlation is lower than a threshold, an updated variable parameter is formed according to the change amount of two parameters with a time series interval of n in the variable parameter vector;

[0096] The correlations between the updated variable parameters are analyzed, and the step of performing linear regression analysis on the variable parameter groups whose correlations are higher than a threshold is returned.

[0097] In actual data analysis, the correlation coefficients between different variable parameters of the same site may be low. However, from the actual perception, the change trends have a certain correlation. For variable parameters with correlations lower than the threshold, the updated variable parameters are formed according to the changes of the two parameters with a time series interval of n in the variable parameter vector, and then the correlation analysis is performed on the updated variable parameters. Among them, the updated variable parameters are formed according to the changes of the two parameters with a time series interval of n in the variable parameter vector. The correlation coefficient can be calculated using the data after a difference. For example, the two variable parameters are ( )and , a first difference refers to the change of the two variable parameters with an interval of 1 in the original time series. The updated variable parameters formed by calculating the change of the two parameters with an interval of 1 in the time series are ( )and Then use the new data for subsequent correlation coefficient calculation and regression analysis.

[0098] Similarly, we can also use the change rate of the original time series with an interval of 1 for subsequent analysis, that is, the new data is ( )and( ), further, we can use the change in the original time series with an interval of k, that is, the new data is ( )and The subsequent correlation coefficient calculation and regression analysis are performed. In this embodiment, the change amount of the original time series with an interval of 1 is mainly used to perform the subsequent correlation coefficient calculation and regression analysis.

[0099] The step of obtaining water quality monitoring data within a preset period includes:

[0100] Obtain water quality monitoring data corresponding to multiple variable parameters at the same site within a preset period.

[0101] Optionally, the variable parameters may include different parameters for characterizing water quality, such as water temperature, pH value, dissolved oxygen, conductivity, turbidity, permanganate index (CODMn), ammonia nitrogen, total phosphorus, total nitrogen, chlorophyll α and algae density.

[0102] Optionally, the water quality monitoring data analysis method further includes:

[0103] The water quality monitoring data is screened according to the abnormal value upper limit and abnormal value lower limit corresponding to different variable parameters, and the set variable parameter relationship to obtain effective water quality monitoring data.

[0104] Take the variable parameters including water temperature, pH value, dissolved oxygen, conductivity, turbidity, permanganate index (CODMn), ammonia nitrogen, total phosphorus, total nitrogen, chlorophyll α and algae density as an example. The upper limit value and lower limit value of abnormal value corresponding to the variable parameters can be shown in the following table 1:

[0105]

[0106] The set variable parameter relationships include: a) total nitrogen is greater than ammonia nitrogen; b) chemical oxygen demand is greater than permanganate index; c) chemical oxygen demand is greater than biochemical oxygen demand.

[0107] The upper and lower limits of abnormal values ​​corresponding to different variable parameters, as well as the set variable parameter relationships can be determined based on manually reviewed empirical values. By screening the water quality monitoring data based on the upper and lower limits of abnormal values ​​corresponding to different variable parameters and the set variable parameter relationships, some calculations that do not participate in mathematical analysis can be deleted, thereby achieving the purpose of reducing noise to improve the accuracy of mathematical analysis.

[0108] In order to have a further overall understanding of the water quality monitoring data analysis method embodiment, please refer to Figure 5 , which is an optional specific example of a water quality monitoring data analysis method, comprises the following steps:

[0109] S11, perform STL decomposition on the water quality monitoring data, calculate and extract features of the decomposed sequence, and mark and extract outliers appearing in the time series features;

[0110] S12, after STL filtering, the marked outliers are removed, and the data is re-identified using the linear regression method, and the abnormal data is identified through the pairwise relationship between variables; paired marking is performed when the relationship is inconsistent with the norm;

[0111] S13, shielding the outliers, performing linear regression analysis on the shielded data, identifying outliers through the cook value, and picking out the outliers;

[0112] S14, sort out all the marked outliers, expand the time window with the marked outliers as the points, conduct correlation analysis between the data of this time period and the data of the upstream sites, determine the time period after the outliers through the correlation, and infer the possible time point of upstream pollution.

[0113] The water quality monitoring data analysis method provided in the embodiment of the present application uses the time series method and the linear regression method at the same time to identify and calculate the data from the two dimensions of time transformation and the relationship between variables. It cross-uses multiple outlier recognition technologies to realize the targeted outlier recognition function, and can establish an identification model for the existing monitoring data and apply it to the monitoring section. It can predict the outliers through remote data diagnosis and take corresponding emergency measures. It can achieve dual recognition of remote prediction and on-site verification to ensure the timeliness and pertinence of the response to environmental abnormal events. By comprehensively using linear regression and time series algorithms, starting from the internal relationship between time and variables, outliers are identified and judged. The two methods are alternately carried out according to the on-site situation during use, and parameters are optimized for specific situations. Empirical judgment is integrated into the data statistical model algorithm, and the data recognition of relatively stable monitoring sections is achieved to the degree of quantitative recognition.

[0114] In an optional specific example, the model calculation results of the water quality monitoring data analysis method of the embodiment of the present application include:

[0115] 1) Outlier Identification

[0116] For the outlier detection algorithm between different indicators at the same site, the experimental data is the monitoring data of a water system site. The water system has 4 sites, namely Site A, Site B, Site C, and Site D. The characteristics of each site are water temperature (℃), pH (dimensionless), dissolved oxygen (mg / L), conductivity (μS / cm), turbidity (NTU), permanganate index (mg / L), ammonia nitrogen (mg / L), and total phosphorus (mg / L), a total of 8 variable parameter characteristics.

[0117]

[0118] 2) Single-site multi-parameter correlation

[0119] The results are shown below using the turbidity and total phosphorus characteristics of site A from 2019 to 2020 as an example.

[0120] The linear regression and trend graphs of turbidity and total phosphorus in September 2019 are as follows: Figure 6 As shown, (in September 2019, there was no missing data for turbidity, and there were 11 missing data for total phosphorus, and the correlation coefficient between the two was 0.83434). The black dots on both sides of the straight line mark the abnormal points detected by the time series anomaly detection algorithm and the abnormal points marked by the cook distance. The regression relationship between ammonia nitrogen and total phosphorus is:

[0121] Total phosphorus (mg / L) = 0.023564 + 0.000136 x turbidity (NTU)

[0122] The linear regression and trend graphs of turbidity and total phosphorus in March 2020 are as follows: Figure 7 As shown (there was no missing data for turbidity and total phosphorus in March 2020, and the correlation coefficient between the two was 0.67543):

[0123] The regression relationship between turbidity and total phosphorus is:

[0124] Total phosphorus (mg / L) = 0.012672 + 0.000278 x turbidity (NTU)

[0125] As can be seen from the figure, most of the outliers are marked by the algorithm, which verifies the effectiveness of the algorithm.

[0126] 3) Multi-site single parameter correlation

[0127] The time frequency of the selected points is D, that is, every day is regarded as a point, the number of calculation periods is 3, the threshold of the correlation coefficient is set to 0.8, and the time window length of each abnormal point is 16 hours, that is, the 16 hours before and after each point are selected as the time window length. The length of each period is 4 hours. The analysis results of dissolved oxygen in AB are selected for display. The analysis results of dissolved oxygen are as follows:

[0128] Upstream site: A

[0129] Downstream site: B

[0130] Analytical properties: Dissolved oxygen (mg / L)

[0131] Number of outliers found: 110

[0132] The number of intervals with 0 periods is: 23

[0133] The number of intervals of 1 period is found to be: 29

[0134] The number of intervals of 2 periods is found to be: 55

[0135] The number of intervals of 3 periods is found to be: 80

[0136] Most of the data also show that the interval between the two is 3 periods. Figure 8 As shown in the figure, some specific data of the pile points are taken for display, among which the downstream station data is the data after an interval of 12 hours. It can be seen from the figure that the two have a strong correlation under this period, which can prove the effectiveness of the algorithm.

[0137] On the other hand, please refer to Fig. 9 , a water quality monitoring data analysis device is also provided, including an acquisition module 11, used to acquire water quality monitoring data within a preset period; an abnormal value extraction module 12, used to perform abnormal point analysis on the water quality monitoring data and mark the abnormal values ​​of the water quality monitoring data; a correlation analysis module 13, used to determine a reference time period according to the distribution of the abnormal values, and perform correlation analysis on the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site; a determination module 14, used to determine the time when pollution occurred at the upstream site according to the correlation analysis result.

[0138] Among them, the outlier extraction module 12 is specifically used to decompose the water quality monitoring data using a time series decomposition algorithm, mark the outliers in the decomposed water quality monitoring data sequence as outliers; remove the marked outliers; and use a linear regression method to perform a secondary outlier point analysis on the water quality monitoring data after removing the outliers to mark the outliers of the water quality monitoring data.

[0139] Among them, the outlier extraction module 12 is also used to analyze the correlation between variable parameters in the water quality monitoring data after the outliers are removed; identify the abnormal data segments according to the changing trend of the correlation, extract the abnormal values ​​corresponding to the abnormal data segments and mark them.

[0140] The outlier extraction module 12 is further used to shield the outliers and perform linear regression analysis on the shielded data; identify outliers by using the Cook distance, and mark outliers with a Cook distance greater than a threshold as outliers.

[0141] Among them, the correlation analysis module 13 is specifically used to organize all marked abnormal values, take the marked abnormal values ​​as pile points, expand the time window according to the set time frequency with the pile points as the center, and determine the reference time period according to the time window; perform correlation analysis on the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site.

[0142] The correlation analysis module 13 is further used to perform correlation analysis on the water quality monitoring data of the reference time period and the upstream site data at different intervals, and determine the time of pollution occurrence of the upstream site according to the correlation coefficient with the upstream site data at different intervals.

[0143] The acquisition module 11 is specifically used to acquire water quality monitoring data corresponding to multiple variable parameters of the same site within a preset period.

[0144] It should be noted that: the water quality monitoring data analysis device provided in the above embodiment is only illustrated by the division of the above program modules during the water quality monitoring process. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the method steps described above. In addition, the water quality monitoring data analysis device provided in the above embodiment and the water quality monitoring data analysis method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0145] See also Fig.10 On the other hand, an embodiment of the present application further provides a water quality monitoring device, including a processor 51 and a memory 52, wherein the memory 52 stores a computer program that can be executed by the processor, and when the computer program is executed by the processor 51, the steps of the water quality monitoring data analysis method provided in any embodiment of the present application are implemented.

[0146] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments of the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0147] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A water quality monitoring data analysis method, characterized in that: include: Obtain water quality monitoring data within a preset period; Performing anomaly analysis on the water quality monitoring data and marking abnormal values ​​of the water quality monitoring data; Determine a reference time period according to the distribution of the abnormal values, and perform a correlation analysis between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site; Determine the time when pollution occurs at the upstream site according to the correlation analysis results; The performing of abnormal point analysis on the water quality monitoring data and marking abnormal values ​​of the water quality monitoring data includes: Decomposing the water quality monitoring data using a time series decomposition algorithm, and marking outliers in the decomposed water quality monitoring data sequence as abnormal values; Eliminate the marked outliers; Using a linear regression method to perform a secondary outlier point analysis on the water quality monitoring data after removing the outliers, so as to mark the outliers of the water quality monitoring data; Shielding the abnormal values, and performing linear regression analysis on the shielded data; Outliers are identified using the Cook distance, and outliers with Cook distance greater than the threshold are marked as outliers.

2. The water quality monitoring data analysis method according to claim 1, characterized in that: The method of using a linear regression method to perform a secondary abnormal point analysis on the water quality monitoring data after removing the abnormal values ​​to mark the abnormal values ​​of the water quality monitoring data includes: Analyzing the correlation between the variable parameters in the water quality monitoring data after removing the abnormal values; The abnormal data segment is identified according to the change trend of the correlation, and the abnormal value corresponding to the abnormal data segment is extracted and marked.

3. The water quality monitoring data analysis method according to claim 1, characterized in that: The step of determining a reference time period according to the distribution of the abnormal values ​​and performing a correlation analysis between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site includes: Arrange all the marked abnormal values, take the marked abnormal values ​​as the pile points, expand the time window according to the set time frequency with the pile points as the center, and determine the reference time period according to the time window; A correlation analysis is performed between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site.

4. The water quality monitoring data analysis method according to claim 3, characterized in that: Determining the time of occurrence of pollution at the upstream site according to the correlation analysis result includes: The water quality monitoring data of the reference time period is subjected to correlation analysis with the upstream site data at different intervals, and the time when the pollution at the upstream site occurs is determined based on the correlation coefficient with the upstream site data at different intervals.

5. The water quality monitoring data analysis method according to claim 1, characterized in that: The step of obtaining water quality monitoring data within a preset period includes: Obtain water quality monitoring data corresponding to multiple variable parameters at the same site within a preset period.

6. A water quality monitoring data analysis device, characterized in that: include: An acquisition module is used to obtain water quality monitoring data within a preset period; An outlier extraction module, used to perform outlier point analysis on the water quality monitoring data and mark outliers of the water quality monitoring data; A correlation analysis module, used to determine a reference time period according to the distribution of the abnormal values, and perform a correlation analysis between the water quality monitoring data of the reference time period and the water quality monitoring data of the upstream site; A determination module, used to determine the time of pollution occurrence of the upstream site according to the correlation analysis result; Among them, the outlier extraction module is specifically used to decompose the water quality monitoring data using a time series decomposition algorithm, mark the outliers in the decomposed water quality monitoring data sequence as outliers; remove the marked outliers; use a linear regression method to perform a secondary outlier point analysis on the water quality monitoring data after removing the outliers to mark the outliers of the water quality monitoring data; shield the outliers and perform a linear regression analysis on the shielded data; identify outliers through Cook distance, and mark outliers with a Cook distance greater than a threshold as outliers.

7. A water quality monitoring device, characterized in that: It includes a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, it implements the water quality monitoring data analysis method as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by the controller, the water quality monitoring data analysis method as described in any one of claims 1 to 5 is implemented.

Images