Method and device for identifying abnormal water quality

Abstract

The present invention discloses a method for identifying water quality anomalies, comprising the following steps: A. acquiring raw water quality data; B. setting an initial standard deviation threshold and a moving method grouping value; C. determining the start point of the abnormal change; D. determining the extent of the abnormal change; E. determining the end point of the abnormal change; and F. reporting the start point, extent of the abnormal change, and end point of the abnormal change, and issuing an alarm based on the extent of the abnormal change. The present invention also discloses a device for identifying water quality anomalies. The device can identify "sunny day discharge and rainy day sewage discharge," controls analysis accuracy by setting algorithm parameters, and achieves "data purification" by analyzing fluctuations in detection values. This not only filters out individual abnormal values resulting from equipment errors and accidental events, but also accurately identifies "sunny day discharge" and "rainy day sewage discharge," enabling the algorithm's early warning rate for discharge events to reach over 90%, thereby improving the efficiency of relevant personnel.

Description

Technical Field

[0001] The present invention relates to the technical field of water treatment, and in particular to a method and device for identifying abnormal water quality. Background Art

[0002] Online water quality monitoring refers to the use of online water quality sensors installed at designated locations to conduct real-time detection, transmit the detection data to the terminal in real time, and conduct real-time analysis through analytical means to achieve real-time understanding and processing of the situation.

[0003] Existing water quality test data analysis methods mainly use a simple value comparison method of individual test values with various emission standards as the basis for alarms. The test value is normal if it is within the emission standard range, and abnormal if it is outside the emission range.

[0004] However, when implementing the present invention, the inventors discovered that the methods of the prior art focused on evaluating water quality, and were unable to identify whether there was "discharge behavior" (even if individual water quality parameters were normal, there might still be discharge behavior), nor could they identify "pollution discharge behavior on rainy days" (pollution discharge was carried out with the help of the dilution capacity of precipitation), nor could they identify individual abnormal values caused by equipment errors or accidental events, resulting in frequent alarms and a waste of human resources. Summary of the Invention

[0005] (1) Technical issues to be solved

[0006] The problem to be solved by the present invention is to provide a method and device for identifying abnormal water quality to overcome the defects in the prior art.

[0007] (2) Technical solution

[0008] To solve the above technical problems, the present invention provides a method for identifying abnormal water quality, which comprises the following steps:

[0009] A, obtain the raw water quality data within a pre-set time period through the sensor;

[0010] B. Sort the acquired raw water quality data in time sequence to obtain a test data set, extract a non-abnormal data set from the test data set, and set an initial standard deviation threshold and a moving method grouping value based on the non-abnormal data set;

[0011] C. Calculating the standard deviation between adjacent values and the sum of the standard deviations of the grouped values in the test data set, and comparing the calculated result with the initial standard deviation threshold, and determining the starting point of the abnormal change based on the comparison result;

[0012] D, starting from the starting point of the abnormal change, calculating the skewness between adjacent values in the test data set to obtain a slope value, and determining the degree of the abnormal change according to the slope value;

[0013] E, calculating the standard deviation sum corresponding to the value with the opposite slope to the starting point of the abnormal change, and comparing the calculated result with the initial standard deviation threshold, and determining the end point of the abnormal change according to the comparison result;

[0014] F. Report the start point, degree and end point of the abnormal change, and issue an alarm based on the degree of the abnormal change.

[0015] Furthermore, in step B, the process of extracting a non-abnormal data set from the data set to be tested includes:

[0016] B1, identifying outliers in the test data set according to the iForest algorithm, and inferring the non-outlier data set by marking the outliers.

[0017] Furthermore, in the step B1, it specifically includes:

[0018] B11, randomly select n (n≤256) data sample points from time t to form a subset Ω i ,i ∈1,2,m,(m≤100), build a tree on m subsets;

[0019] B12, set feature q = water quality parameter data, split value p, subset Ω i undergo binary fission;

[0020] B13, set the tree height to 1 and build the tree;

[0021] B14, build m trees in sequence, and define whether it is abnormal based on the average depth of m decision numbers;

[0022] B15, based on the definition of abnormal results, reverse the non-abnormal data set

[0023] D={x1,x2,...,x (t-n)}.

[0024] Furthermore, in the step B14, it specifically includes:

[0025] B141, introduce the outlier function s(x,n) to measure whether the record x is an outlier:

[0026]

[0027] Where E(h(x)) is the expected value of the path length of x in multiple trees;

[0028] c(n)=2H(n-1)-(2(n-1) / n)

[0029] H(*)=I n (*)+ξ

[0030] Where c(n) is the average path length of the tree for a dataset containing n samples, which is used to standardize the path length of record x; H(*) is the harmonic number, and ξ is the Euler constant, which is approximately 0.5772156649;

[0031] B142, based on s(x,n), defines the probability of abnormality. When E(h(x))→c(n), When E(h(x))→0, When E(h(x))→n-1, s→0, it is judged to be abnormal; when E(h(x))→n-1, s→0, it is judged to be normal.

[0032] Furthermore, in step B, the process of setting the initial standard deviation threshold and the moving method grouping value according to the non-abnormal data set specifically includes:

[0033] B2, from the non-abnormal data set D = {x1, x2, ..., x (t-n)}, determine the minimum and maximum values x in the non-abnormal data set D min with x max , calculate x min with x max The standard deviation σ m , based on the 10% data repeatability error of the sensor equipment, calculate the initial standard deviation threshold σ s =1.1*σ m ;

[0034] B3, set the moving method group value Group Size = y.

[0035] Further, the step C comprises:

[0036] C1, based on the initial set standard deviation threshold parameter δ s The standard deviation between two adjacent values is calculated by grouping the numerical parameter y with the moving method. (x i with x (i-1) are two adjacent data, and i≥2);

[0037] C2, calculate the standard deviation σ i Value and initial standard deviation threshold σ s The difference σ c :

[0038] When σ c ≥0, x iIt is an outlier, and the corresponding time node t i Marked as the starting point of abnormal change.

[0039] Furthermore, in step C, the following is also included:

[0040] C3, when σ c When <0, then x i is a normal value, and σ i Count y values D in the adjacent group array y ={x i , x (i-1) ,...,x (i-y)}(1≤(iy)≤y) y ={σ i ,σ (i-1) ,...,σ (i-y) )}, and calculate the standard deviation group σ y The standard deviation and σ within d :σ d =σ i +σ (i-1) +...+σ (i-y) ;

[0041] C4, compared with σ d and σ s The value between:

[0042] When σ d ≥σ s , that is, the standard deviation within the group number exceeds the initial standard deviation parameter, then x i is an outlier, and the corresponding time node is t i Marked as the starting point of abnormal change.

[0043] Further, the step D specifically includes:

[0044] D1, based on x i is an outlier, and the corresponding time node is t i Calibrated as the starting point of abnormal change, according to the formula Calculate x i with x (i-1) Skewness S between values i :

[0045] D2, determine the degree of abnormal change as |S i |.

[0046] Furthermore, the step D further comprises:

[0047] D3, obtaining weather parameters and classifying the weather parameters into precipitation and non-precipitation;

[0048] D4, determining the cause of the abnormal change based on the weather parameters and slope value, specifically including:

[0049] When the weather parameter is precipitation, and S i When >0, the cause of the abnormal change is determined to be an upward anomaly, which is sewage discharge on a precipitation day, and the sewage discharge multiple is S i ;

[0050] When the weather parameter is precipitation, and S i When <0, it is determined to be normal, and the precipitation dilution factor is |S i |;

[0051] When the weather parameter is non-precipitation and S i When it is greater than 0, the cause of the abnormal change is determined to be an upward anomaly, which is sewage discharge on a non-precipitation day, and the sewage discharge multiple is S i ;

[0052] When the weather parameter is non-precipitation and S i When <0, the cause of the abnormal change is determined to be a downward anomaly, which is sewage discharge on a non-precipitation day, and the sewage discharge multiple is |S i |.

[0053] Further, the step E comprises:

[0054] E1, continuous comparison x i with x (i+1) Skewness S between values (i+1) , when the skewness S i With S (i+1) When it remains consistent, it is a persistent anomaly;

[0055] E2, when the skewness S i With S (i+1) If inconsistent, calculate x i with x (i+1) The standard deviation σ (i+1) , and calculate the standard deviation σ (i+1) Value and initial standard deviation threshold σ s The difference σ (c+1) , when σ (c+1) ≥0, judge x (i+1) Normal, OK (i+1) The end point of abnormal change.

[0056] Furthermore, the step E further comprises:

[0057] E3, when σ (c+1) When <0, it is judged as a continuous abnormality, and x (i+1) Marked as the opposite slope, σ (i+1) Taking into account standard deviation and σ d ;

[0058] E4, repeat steps E2 and E3 to obtain σ d =σ (i+1) +…+σ (i+b) , when σ d ≥σ s When x (i+b) Normal, OK (i+1) The end point of abnormal change.

[0059] Furthermore, after step F, the method further includes:

[0060] G. When the time interval between the starting points of two abnormal changes exceeds the preset time threshold, repeat steps A and B, reacquire new original water quality data and set a new initial standard deviation threshold and moving method grouping value.

[0061] The present invention also provides a device for identifying abnormal water quality, comprising:

[0062] The original water quality data acquisition unit is used to acquire the original water quality data within a preset time period;

[0063] an initial standard deviation threshold and moving method grouping value setting unit, connected to the raw water quality data acquisition unit, for sorting the acquired raw water quality data in time sequence to obtain a data set to be tested, extracting a non-abnormal data set from the data set to be tested, and setting an initial standard deviation threshold and a moving method grouping value according to the non-abnormal data set;

[0064] an abnormal change starting point determination unit, connected to the original water quality data acquisition unit and the initial standard deviation threshold and moving method grouping value setting unit, respectively, for calculating the standard deviation between adjacent values in the test data set and the sum of the standard deviations of the grouped values, and comparing the calculation result with the initial standard deviation threshold, and determining the abnormal change starting point according to the comparison result;

[0065] an abnormal change degree determination unit, connected to the abnormal change starting point determination unit, for calculating the skewness between adjacent values in the test data set starting from the abnormal change starting point to obtain a slope value, and determining the degree of abnormal change according to the slope value;

[0066] an abnormal change end point determination unit, connected to the abnormal change degree determination unit, configured to calculate a standard deviation sum corresponding to a value having a slope opposite to that of the abnormal change start point, compare the calculated result with the initial standard deviation threshold, and determine the abnormal change end point based on the comparison result;

[0067] The reporting and alarm unit is respectively connected to the abnormal change starting point determination unit, the abnormal change degree determination unit and the abnormal change end point determination unit, and is used to report the abnormal change starting point, the abnormal change degree and the abnormal change end point, and to issue an alarm according to the abnormal change degree.

[0068] (3) Beneficial effects

[0069] The present invention can identify "discharge on sunny days and discharge on rainy days", use the setting of algorithm parameters to control the analysis accuracy, and realize "data purification" by analyzing the changes in the fluctuation of detection values. It can not only filter out individual abnormal values caused by equipment errors and accidental events, but also accurately identify "discharge on sunny days" and "discharge on rainy days", so that the algorithm's early warning rate for emission events can reach more than 90%, thereby improving the efficiency of relevant staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flow chart of a method for identifying abnormal water quality according to an embodiment of the present invention;

[0071] Figure 2 This is a structural diagram of a water quality abnormality identification device according to an embodiment of the present invention;

[0072] Figure 3 This is a flowchart of an implementation process of water quality anomaly identification according to an embodiment of the present invention;

[0073] Figure 4 This is a flow chart of a data analysis method according to an embodiment of the present invention;

[0074] Figure 5 This is a diagram of data anomaly recognition results based on standard deviation according to an embodiment of the present invention:

[0075] Figure 6 This is a data classification result diagram based on iForest in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0077] Example 1

[0078] A method for identifying abnormal water quality according to an embodiment of the present invention is as follows Figure 1 As shown in the figure, it analyzes abnormal changes in water quality data and identifies the degree of abnormality based on the standard deviation and slope method. Figure 1 , this embodiment includes the following steps:

[0079] Step s101, obtaining raw water quality data. In this embodiment, raw water quality data within a preset time period t is obtained by a sensor, wherein the time parameter t is manually set, and the raw water quality data includes the time and conductivity parameter value of the raw water quality data.

[0080] Step s102: Obtain a test data set and a non-abnormal data set from the original water quality data, and set an initial standard deviation threshold and a moving method grouping value based on the non-abnormal data set. In this embodiment, the obtained original water quality data is sorted in time sequence to obtain a test data set, and a non-abnormal data set is extracted from the test data set, and an initial standard deviation threshold and a moving method grouping value are set based on the non-abnormal data set. This embodiment specifically includes the following process:

[0081] First, the iForest (Isolation Forest) algorithm is used to identify outliers in the dataset to be tested. By marking the outliers, the non-outlier dataset is inferred. iForest is a fast anomaly detection method with linear time complexity and high accuracy. It uses a binary tree to partition the data. The depth of a data point in the binary tree reflects the degree of "isolation" of the data. The iForest algorithm is primarily designed for identifying outliers in continuous structured data. Specifically, this embodiment includes:

[0082] Step a: Randomly select n (n≤256) data sample points from time t to form a subset Ω i ,i∈1,2,m,(m≤100), build a tree on m subsets.

[0083] Step b, set feature q = water quality parameter data, segmentation value p, subset Ω i Perform binary fission.

[0084] Step c, set the tree height to 1 and build the tree.

[0085] Step d: Establish m trees in sequence and determine whether they are abnormal based on the average depth of the m decision numbers. In this embodiment, the process of determining whether they are abnormal specifically includes:

[0086] (1): Introduce the outlier function s(x,n) to measure whether the record x is an outlier:

[0087]

[0088] Where E(h(x)) is the expected value of the path length of x in multiple trees;

[0089] c(n)=2H(n-1)-(2(n-1) / n)

[0090] H(*)=In (*)+ξ

[0091] Where c(n) is the average path length of the tree for a dataset of n samples, which is used to standardize the path length of record x; H(*) is the harmonic number, and ξ is the Euler constant, which is approximately 0.5772156649.

[0092] (2): Define the abnormal probability according to s(x,n). When E(h(x))→c(n), When E(h(x))→0, When E(h(x))→n-1, s→0, it is judged to be abnormal; when E(h(x))→n-1, s→0, it is judged to be normal.

[0093] Step e: Based on the definition of abnormal results, reversely infer the non-abnormal data set D = {x1, x2, ..., x (t-n)}.

[0094] In this embodiment, a non-abnormal data set D is obtained through steps a to e.

[0095] Then, from the non-anomalous dataset D = {x1, x2, ..., x (t-n)}Determine the minimum and maximum values x in the non-anomalous data set D min with x max , calculate x min with x max The standard deviation σ m , based on the 10% data repeatability error of the sensor equipment, calculate the initial standard deviation threshold σ s =1.1*σ m .

[0096] Finally, set the moving method group value Group Size = y.

[0097] Step s103, calculate the standard deviation between adjacent values in the test data set and the sum of the standard deviations of the grouped values, and compare the calculated result with the initial standard deviation threshold, and determine the starting point of the abnormal change based on the comparison result. This embodiment specifically includes the following process:

[0098] First, based on the initial set standard deviation threshold parameter δ s The standard deviation between two adjacent values is calculated by grouping the numerical parameter y with the moving method. (x i with x (i-1) are two adjacent data, and i≥2);

[0099] Then, calculate the standard deviation σ i Value and initial standard deviation threshold σ s The difference σc :

[0100] When σ c ≥0, x i It is an outlier, and the corresponding time node t i Marked as the starting point of abnormal change.

[0101] When σ c When <0, then x i is a normal value, and σ i Count y values D in the adjacent group array y ={x i , x (i-1) ,...,x (i-y)}(1≤(iy)≤y) y ={σ i ,σ (i-1) ,...,σ (i-y) )}, and calculate the standard deviation group σ y The standard deviation and σ within d :σ d =σ i +σ (i-1) +...+σ (i-y) .

[0102] Compare σ d and σ s The value between σ d ≥σ s When the sum of the standard deviations within the group exceeds the initial standard deviation parameter, then x i It is an outlier, and the corresponding time node t i Marked as the starting point of abnormal change.

[0103] Step s104, starting from the starting point of the abnormal change, calculates the skewness between adjacent values in the test data set to obtain a slope value, and introduces weather parameters. Based on the slope value and weather parameters, the degree of abnormal change and the cause of the abnormal change are determined. This embodiment specifically includes the following process:

[0104] First, based on x i is an outlier, and the corresponding time node is t i Calibrated as the starting point of abnormal change, according to the formula Calculate x i with x (i-1) Skewness S between values i , and determine the degree of abnormal change as |S i |.

[0105] Then, weather parameters are obtained, the weather parameters are divided into precipitation and non-precipitation, and the cause of the abnormal change is determined based on the weather parameters and the slope value:

[0106] When the weather parameter is precipitation, and S i When >0, the cause of the abnormal change is determined to be an upward anomaly, which is sewage discharge on a precipitation day, and the sewage discharge multiple is S i ;

[0107] When the weather parameter is precipitation, and S i When <0, it is determined to be normal, and the precipitation dilution factor is |S i |;

[0108] When the weather parameter is non-precipitation and S i When it is greater than 0, the cause of the abnormal change is determined to be an upward anomaly, which is sewage discharge on a non-precipitation day, and the sewage discharge multiple is S i ;

[0109] When the weather parameter is non-precipitation and S i When <0, the cause of the abnormal change is determined to be a downward anomaly, which is sewage discharge on a non-precipitation day, and the sewage discharge multiple is |S i |.

[0110] Step s105, calculate the standard deviation corresponding to the value with the opposite slope to the starting point of the abnormal change, compare the calculated result with the initial standard deviation threshold, and determine the end point of the abnormal change based on the comparison result. This embodiment specifically includes the following process:

[0111] (1) Continuous comparison x i with x (i+1) Skewness S between values (i+1) .

[0112] (2) Comparative skewness S i With S (i+1) When the skewness S i With S (i+1) When the S i >0 and S (i+1) ≥0, or when S i <0 and S (i+1) When ≤0, it is a continuous abnormality.

[0113] (3) When the skewness S i With S (i+1) When inconsistent, that is, when S i >0 and S (i+1) ≤0, or when S i <0 and S (i+1) ≥0, calculate x i with x (i+1) The standard deviation σ (i+1) , and calculate the standard deviation σ (i+1) Value and initial standard deviation threshold σ s The difference σ(c+1) When σ (c+1) ≥0, judge x (i+1) Normal, OK (i+1) The end point of abnormal change.

[0114] (4) When σ (c+1) When <0, it is judged as a continuous abnormality, and x (i+1) Marked as the opposite slope, σ (i+1) Taking into account standard deviation and σ d .

[0115] (5) Repeat steps (3) and (4) to obtain

[0116] σ d =σ (i+1) +…+σ (i+b) , when σ d ≥σ s When x (i+b) Normal, OK (i+1) The end point of abnormal change.

[0117] Step s106: reporting the abnormal change start point, the abnormal change degree and the abnormal change end point, and issuing an alarm according to the abnormal change degree.

[0118] Step s107: When the time interval between the two abnormal change starting points exceeds the pre-set time threshold, repeat steps s101 and s102 to reacquire new raw water quality data and set a new initial standard deviation threshold and moving method grouping value. In this embodiment, when the time interval between the two abnormal change starting points exceeds 7 days, repeat steps s101 and s102 to recalculate the standard deviation parameter σ. s .

[0119] The embodiment of the present invention defines abnormal water quality data events based on the standard deviation and slope method, which subverts the traditional setting of single upper and lower threshold values for water quality data anomaly identification. It defines anomalies based on actual data changes, which is more flexible and effective.

[0120] Example 2

[0121] A water quality abnormality identification device according to an embodiment of the present invention is as follows Figure 2As shown, the device includes an original water quality data acquisition unit 21, an initial standard deviation threshold and moving method grouping value setting unit 22, an abnormal change starting point determination unit 23, an abnormal change degree determination unit 24, an abnormal change end point determination unit 25 and a reporting and alarm unit 26, wherein the original water quality data acquisition unit 21 is connected to the initial standard deviation threshold and moving method grouping value setting unit 22, the abnormal change starting point determination unit 23 is respectively connected to the original water quality data acquisition unit 21 and the initial standard deviation threshold and moving method grouping value setting unit 22, the abnormal change degree determination unit 24 is respectively connected to the abnormal change starting point determination unit 23 and the abnormal change end point determination unit 25, and the reporting and alarm unit 26 is respectively connected to the abnormal change starting point determination unit 23, the abnormal change degree determination unit 24 and the abnormal change end point determination unit 25.

[0122] The original water quality data acquisition unit 21 is used to acquire original water quality data within a preset time period.

[0123] The initial standard deviation threshold and moving method grouping value setting unit 22 is used to sort the acquired original water quality data in time sequence to obtain a data set to be tested, and extract a non-abnormal data set from the data set to be tested, and set the initial standard deviation threshold and moving method grouping value according to the non-abnormal data set.

[0124] The abnormal change starting point determination unit 23 is used to calculate the standard deviation between adjacent values and the sum of the standard deviations of grouped values in the test data set, compare the calculation result with the initial standard deviation threshold, and determine the abnormal change starting point according to the comparison result.

[0125] The abnormal change degree determination unit 24 is used to calculate the skewness between adjacent values in the test data set starting from the abnormal change starting point to obtain a slope value, and determine the degree of abnormal change according to the slope value.

[0126] The abnormal change end point determination unit 25 is used to calculate the standard deviation sum corresponding to the value with the opposite slope to the abnormal change start point, compare the calculation result with the initial standard deviation threshold, and determine the abnormal change end point according to the comparison result.

[0127] The reporting and alarm unit 26 is used to report the abnormal change starting point, the degree of the abnormal change and the abnormal change ending point, and to issue an alarm according to the degree of the abnormal change.

[0128] When the water quality abnormality identification device of this embodiment is used, the process of implementing the water quality abnormality identification is as follows: Figure 3As shown in the figure, the process is as follows: first, the detection data is obtained through the water quality sensor, and then the detection data is wirelessly transmitted to the server through NB-IoT (NarrowBand Internet of Things). The server then parses the data according to the specified protocol. The basic platform organizes the parsed data into corresponding data calculation packages according to the algorithm rules, and transmits them to the algorithm program through the interface for calculation. After receiving the data packet, the algorithm program starts calculation and transmits the calculation results to the basic platform. Finally, the basic platform receives the calculation results and displays them.

[0129] The methods used in the data analysis part are as follows: Figure 4 As shown in the figure, first, the original water quality data samples within a period of time t are collected and sorted in sequence according to the time series. The iForest algorithm is introduced to determine the maximum and minimum values of the non-abnormal data set, and based on the 10% data repeatability error of the sensor equipment, the initial standard deviation threshold is calculated, and the moving method grouping value is y; secondly, based on the initially set parameters, the standard deviation of adjacent values and the sum of the standard deviations within the grouped array are calculated. If any calculation result exceeds the set standard deviation threshold, it is marked as the beginning of an abnormal change; thirdly, weather parameters are introduced, and the cause of the start of the abnormal change and the degree of the abnormal change are determined by the standard deviation and slope values; finally, the sum of the standard deviations corresponding to the values with the opposite slope to the starting point of the abnormal change is calculated, and the calculation result is compared with the initial standard deviation threshold. The end point of the abnormal change is determined according to the comparison result.

[0130] Example 3

[0131] The data classification result based on iForest in this embodiment is as follows Figure 6 As shown, this embodiment randomly selects 125 raw conductivity data as data samples from the raw data within 7 days, sets the segmentation value p to 10, and uses the iForest algorithm to classify outliers and non-outliers, and locks in 3 outlier data.

[0132] The remaining 122 data are defined as non-abnormal data sets, and are sorted according to the data values. It is determined that the maximum value of the non-abnormal data set is 1242 and the minimum value is 984. The initial standard deviation threshold is 129, and the moving method grouping value is 6.

[0133] For the original data within 7 days, based on the initial standard deviation threshold = 129 and the parameter setting of the moving method group value as 6, the following can be obtained: Figure 5 The results of data anomaly identification based on standard deviation are shown.

[0134] The present invention can identify "discharge on sunny days and discharge on rainy days", use the setting of algorithm parameters to control the analysis accuracy, and realize "data purification" by analyzing the changes in the fluctuation of detection values. It can not only filter out individual abnormal values caused by equipment errors and accidental events, but also accurately identify "discharge on sunny days" and "discharge on rainy days", so that the algorithm's early warning rate for emission events can reach more than 90%, thereby improving the efficiency of relevant staff.

[0135] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for identifying abnormal water quality, characterized in that: The method comprises the following steps: A, obtain the raw water quality data within a pre-set time period through the sensor; B. Sort the acquired raw water quality data in time sequence to obtain a test data set, extract a non-abnormal data set from the test data set, and set an initial standard deviation threshold and a moving method grouping value based on the non-abnormal data set; C. Calculating the standard deviation between adjacent values and the sum of the standard deviations of the grouped values in the test data set, and comparing the calculated result with the initial standard deviation threshold, and determining the starting point of the abnormal change based on the comparison result; D, starting from the starting point of the abnormal change, calculating the skewness between adjacent values in the test data set to obtain a slope value, and determining the degree of the abnormal change according to the slope value; E, calculating the sum of standard deviations corresponding to the values with the opposite slopes to the starting point of the abnormal change, and comparing the calculated result with the initial standard deviation threshold, and determining the end point of the abnormal change according to the comparison result; F. Report the start point, degree and end point of the abnormal change, and issue an alarm based on the degree of the abnormal change.

2. The method for identifying abnormal water quality according to claim 1, wherein: In step B, the process of extracting a non-abnormal data set from the data set to be tested includes: B1, identifying outliers in the test data set according to the iForest algorithm, and inferring the non-outlier data set by marking the outliers.

3. The method for identifying abnormal water quality according to claim 2, wherein: In the step B1, it specifically includes: B11, randomly select n (n≤256) data sample points from time t to form a subset Ω i ,i∈1,2,……,m,(m≤100), build a tree on m subsets; B12, set feature q = water quality parameter data, split value p, subset Ω i undergo binary fission; B13, set the tree height to 1 and build the tree; B14, build m trees in sequence, and define whether it is abnormal based on the average depth of m decision numbers; B15, based on the definition of abnormal results, reverse the non-abnormal data set D = {x1, x2, ..., x (t-n) }.

4. The method for identifying abnormal water quality according to claim 3, wherein: In the step B14, it specifically includes: B141, introduce the outlier function s(x,n) to measure whether the record x is an outlier: Where E(h(x)) is the expected value of the path length of x in multiple trees; c(n)=2H(n-1)-(2(n-1) / n) H(*)=I n (*)+ξ Where c(n) is the average path length of the tree for a dataset containing n samples, which is used to standardize the path length of record x; H(*) is the harmonic number, and ξ is the Euler constant, which is approximately 0.5772156649; B142, based on the definition of abnormal probability s(x,n), when E(h(x))→c(n), s→0.5, it cannot be distinguished whether it is abnormal or not; when E(h(x))→0, s→1, it is judged to be abnormal; when E(h(x))→n-1, s→0, it is judged to be normal.

5. The method for identifying abnormal water quality according to claim 3, wherein: In step B, the process of setting the initial standard deviation threshold and the moving method grouping value based on the non-abnormal data set specifically includes: B2, from the non-abnormal data set D = {x1, x2, ..., x (t-n) }, determine the minimum and maximum values x in the non-abnormal data set D min with x max , calculate x min with x max The standard deviation σ m , based on the 10% data repeatability error of the sensor equipment, calculate the initial standard deviation threshold σ s =1.1*σ m ; B3, set the moving method group value GroupSize = y.

6. The method for identifying abnormal water quality according to claim 5, wherein: The step C comprises: C1, based on the initial set standard deviation threshold parameter δ s The standard deviation between two adjacent values is calculated by grouping the numerical parameter y with the moving method. (x i with x (i-1) are two adjacent data, and i≥2); C2, calculate the standard deviation σ i Value and initial standard deviation threshold σ s The difference σ c : When σ c ≥0, x i It is an outlier, and the corresponding time node t i Marked as the starting point of abnormal change.

7. The method for identifying abnormal water quality according to claim 6, wherein: In the step C, it also includes: C3, when σ c <0, then x i is a normal value, and σ i Count y values D in the adjacent group array y ={x i , x (i-1) ,...,x (i-y) }(1≤(iy)≤y) y ={σ i ,σ (i-1) ,...,σ (i-y) )}, and calculate the standard deviation group σ y The standard deviation and σ within d :σ d =σ i +σ (i-1) +...+σ (i-y) ; C4, compared with σ d and σ s The value between: When σ d ≥σ s , that is, the standard deviation within the group number exceeds the initial standard deviation parameter, then x i is an outlier, and the corresponding time node is t i Marked as the starting point of abnormal change.

8. The method for identifying abnormal water quality according to claim 7, wherein: The step D specifically includes: D1, based on x i is an outlier, and the corresponding time node is t i Calibrated as the starting point of abnormal change, according to the formula Calculate x i with x (i-1) Skewness S between values i : D2, determine the degree of abnormal change as |S i |.

9. The method for identifying abnormal water quality according to claim 8, wherein: The step D further comprises: D3, obtaining weather parameters and classifying the weather parameters into precipitation and non-precipitation; D4, determining the cause of the abnormal change based on the weather parameters and slope value, specifically including: When the weather parameter is precipitation, and S i When >0, the cause of the abnormal change is determined to be an upward anomaly, which is sewage discharge on a precipitation day, and the sewage discharge multiple is S i ; When the weather parameter is precipitation, and S i When <0, it is determined to be normal, and the precipitation dilution factor is |S i |; When the weather parameter is non-precipitation and S i When it is greater than 0, the cause of the abnormal change is determined to be an upward anomaly, which is sewage discharge on a non-precipitation day, and the sewage discharge multiple is S i ; When the weather parameter is non-precipitation and S i When <0, the cause of the abnormal change is determined to be a downward anomaly, which is sewage discharge on a non-precipitation day, and the sewage discharge multiple is |S i |.

10. The method for identifying abnormal water quality according to claim 8, wherein: The step E comprises: E1, continuous comparison x i with x (i+1) Skewness S between values (i+1) , when the skewness S i With S (i+1) When it remains consistent, it is a persistent anomaly; E2, when the skewness S i With S (i+1) If inconsistent, calculate x i with x (i+1) The standard deviation σ (i+1) , and calculate the standard deviation σ (i+1) Value and initial standard deviation threshold σ s The difference σ (c+1) , when σ (c+1) ≥0, judge x (i+1) Normal, OK (i+1) The end point of abnormal change.

11. The method for identifying abnormal water quality according to claim 10, wherein: The step E further comprises: E3, when σ (c+1) When <0, it is judged as a continuous abnormality, and x (i+1) Marked as the opposite slope, σ (i+1) Taking into account standard deviation and σ d ; E4, repeat steps E2 and E3 to obtain σ d =σ (i+1) +…+σ (i+b) , when σ d ≥σ s When x (i+b) Normal, OK (i+1) The end point of abnormal change.

12. The method for identifying abnormal water quality according to any one of claims 1 to 11, wherein: After step F, the method further includes: G. When the time interval between the starting points of two abnormal changes exceeds the preset time threshold, repeat steps A and B, reacquire new original water quality data and set a new initial standard deviation threshold and moving method grouping value.

13. A device for identifying abnormal water quality, characterized in that: The device comprises: The original water quality data acquisition unit is used to acquire the original water quality data within a preset time period; an initial standard deviation threshold and moving method grouping value setting unit, connected to the raw water quality data acquisition unit, for sorting the acquired raw water quality data in time sequence to obtain a data set to be tested, extracting a non-abnormal data set from the data set to be tested, and setting an initial standard deviation threshold and a moving method grouping value according to the non-abnormal data set; an abnormal change starting point determination unit, connected to the original water quality data acquisition unit and the initial standard deviation threshold and moving method grouping value setting unit, respectively, for calculating the standard deviation between adjacent values in the test data set and the sum of the standard deviations of the grouped values, and comparing the calculation result with the initial standard deviation threshold, and determining the abnormal change starting point according to the comparison result; an abnormal change degree determination unit, connected to the abnormal change starting point determination unit, for calculating the skewness between adjacent values in the test data set starting from the abnormal change starting point to obtain a slope value, and determining the degree of abnormal change according to the slope value; an abnormal change end point determination unit, connected to the abnormal change degree determination unit, configured to calculate a standard deviation sum corresponding to a value having a slope opposite to that of the abnormal change start point, compare the calculated result with the initial standard deviation threshold, and determine the abnormal change end point based on the comparison result; The reporting and alarm unit is respectively connected to the abnormal change starting point determination unit, the abnormal change degree determination unit and the abnormal change end point determination unit, and is used to report the abnormal change starting point, the abnormal change degree and the abnormal change end point, and to issue an alarm according to the abnormal change degree.

Images