Water supply network abnormal data diagnosis method based on duration time and support probability

Abstract

The invention discloses a water supply network abnormal data diagnosis method based on duration and support probability, and the method comprises the steps: calculating a local abnormal factor, quantitatively measuring the abnormality at each time point, and enabling the process of abnormality from start to end to be measured through an accumulation sum algorithm. By constructing a soft sensor, a sufficient reference sample is provided for a time sequence recorded by the sensor in real time from two aspects of time and space. By using the surplus samples, the calculation of the abnormal support rate is realized. And finally, distinguishing the fault of the sensor from the abnormal event of the water supply system by using the two indexes of the calculated abnormal duration and support rate.

Description

Technical Field

[0001] The present invention belongs to the field of abnormal diagnosis of water supply pipe networks, and in particular relates to a method for diagnosing abnormal data of water supply pipe networks based on duration and support probability. Background Art

[0002] As an important part of urban infrastructure, the monitoring and management of water supply networks rely on real-time detection data provided by sensors. However, due to the harsh environment in which the sensors are located or changes in the water supply system structures, data anomalies are a common feature of water supply network monitoring data. The data inaccuracy caused by data anomalies poses a huge challenge to the monitoring and control of water supply networks. The misdiagnosis of data anomalies may lead to waste of water resources, increased energy consumption, and increased risk of system failure. Therefore, the diagnosis of abnormal data is of great significance for the management and control of water supply systems.

[0003] The invention patent application with publication number CN109828493A discloses a control method for water supply failure of an urban water pipe network system. The present invention adopts a feedback control method of a positive Markov jump system. When a pump station valve fails during the operation of the urban water pipe network system, a positive Markov jump system model of water volume in the water supply tank of the water pipe network system is established by collecting data from the system model. Then, a reliable controller is designed for the positive Markov jump system with actuator failure, thereby obtaining a control method for water supply failure of the urban water pipe network system.

[0004] The invention patent application with the publication number CN1360200A discloses an online detection system for pipe burst faults in a city water supply network, which is composed of a control center MTU composed of a computer, a multi-channel drive receiver, a modem, and a data transmitter connected in sequence; and a remote transmission terminal RTU composed of a terminal data transmitter, a terminal modem, a terminal multi-channel drive receiver, a microprocessor, and a terminal A / D converter connected in sequence, and the terminal A / D converter is respectively connected with a water pressure sensor and a flow sensor, and is used for the remote transmission terminal to execute the online detection of pipe burst faults in the city water supply network by the control center. The present invention is provided with an online detection program for pipe burst faults in the city water supply network in the computer memory, and a database for establishing the relationship between the water pressure changes of a limited number of monitoring points and the water pressure changes of unmonitored nodes. Once a fault occurs in the water supply network, the location and degree of a type of fault such as pipe burst in the city water supply network can be easily identified by inputting the water pressure changes of a limited number of monitoring points.

[0005] The abnormality diagnosis methods in the field of water supply networks disclosed in the above two patent applications often ignore the abnormalities caused by the failure of the sensor itself. Existing methods focus on the detection and identification of abnormalities caused by events in the water supply network system, such as leaks, burst pipes, etc. However, abnormal points caused by sensor failures can easily lead to false alarms or even incorrect regulation of existing abnormality diagnosis methods.

[0006] The sensors in the water supply network have hydraulic properties that are correlated due to the hydraulic connection of the pipelines. This correlation is reflected in the correlation of the change patterns of the pressure time series or flow time series monitored by the sensors. Sensor failures usually occur randomly, and it can be assumed that multiple sensors will not fail at the same time. Therefore, the support rate of other sensors with similar change patterns can be used to distinguish random sensor failures. On the other hand, random sensor failures often occur accidentally and present data spikes. And short-duration anomalies are not enough to attract the attention of production control, so the duration of the anomaly is used as another basis for judgment. Distinguishing between sensor failures and system events is a technical gap in the current real-time water supply network monitoring and control. Summary of the invention

[0007] The present invention provides a method for diagnosing abnormal data of a water supply network based on duration and support probability, and the method can accurately diagnose abnormal data of a water supply network.

[0008] The present invention provides a method for diagnosing abnormal data of a water supply network based on duration and support probability, comprising:

[0009] The gate strategy of the local anomaly factor is used to identify the anomaly points of the latest time series. The intermediate threshold is constructed based on the threshold of the gate strategy. The anomaly point is traced back from the anomaly point until the local anomaly factor of the data point is less than the intermediate threshold. The time point when the traceback stops is taken as the starting point of the anomaly. The cumulative sum is obtained by recursively performing the operation backwards from the starting point of the anomaly. The cumulative value of each accumulation is the local anomaly factor of the current data point minus the sum of the intermediate threshold and the penalty term. The time point when the cumulative sum is zero for the first time is taken as the end point of the anomaly. The anomaly duration is constructed based on the start point and end point of the anomaly.

[0010] The probability of observing an abnormal point in the time series matrix of the soft sensor is used as the support rate. The time series matrix of the soft sensor is constructed by the latest time series monitored by the current sensor, the time series monitored by other sensors in the same group as the current sensor, and the historical subsequences of the historical time series monitored by the current sensor with the smallest time series distance from the latest time series.

[0011] An abnormality duration threshold and a support rate threshold are set, and abnormal data of the water supply network are diagnosed based on the comparison between the abnormality duration and the abnormality duration threshold, and the comparison between the support rate and the support rate threshold.

[0012] Preferably, a gate strategy of local anomaly factor is used to identify the anomalies of the latest time series, where:

[0013] Outlier = {x t |LOF(x t )>θ}

[0014] Among them, θ is the threshold of the gate strategy, LOF(x t ) is the t-th time point data x of the latest time series t The local anomaly factor.

[0015] Preferably, an intermediate threshold is constructed based on the threshold of the gate strategy, and the intermediate threshold v is:

[0016]

[0017] Among them, θ is the threshold of the gate strategy.

[0018] Preferably, the cumulative sum is obtained by recursively back-calculating from the abnormal starting point, where the cumulative sum S at the t-th time point is t for:

[0019]

[0020] δ t =LOF(x t )-v-β

[0021] β=ln(1+|LOF(x t )-v|)

[0022] Among them, LOF(x t ) is the t-th time point data x of the latest time series t The local anomaly factor, δ t is the cumulative value at the tth time, v is the intermediate threshold, β is the penalty term, t start is the starting point.

[0023] Preferably, the probability of observing an abnormal point in the time series matrix of the soft sensor is used as the support rate, and the support rate SR includes the positive support rate SR + and negative support rate SR - :

[0024]

[0025]

[0026] P(E)=P(+)=P(-), E∈{+,-}

[0027]

[0028]

[0029] Among them, the positive support rate SR +The negative support rate SR is the degree to which all sequences in the soft sensor support the occurrence of a maximum value event in sensor i at time point t. - is the degree to which all sequences in the soft sensor support the occurrence of a minimum event for sensor i at time point t; is the observation vector The geometric mean of the probability of a maximum event occurring, is the observation vector under the condition of the maximum event The geometric mean of the contribution of is the observation vector The geometric mean of the probability of a minimum event occurring, is the observation vector under the condition of the minimum event The geometric mean of the contribution of is the probability that the data monitored at the tth time point of the sth time series in the soft sensor is a larger or smaller value, representing the contribution of the sequence to larger or smaller events in the soft sensor; n is the number of time series in the soft sensor; P(E) is the probability that any observation point is a larger or smaller value. Due to the symmetry of normal distribution modeling, the probability of any point being larger is equal to the probability of being smaller; To observe the vector probability.

[0030] Preferably, when the abnormality duration is less than the abnormality duration threshold and the support rate is less than the support rate threshold, the abnormal data of the water supply network is diagnosed as a random sensor failure;

[0031] When the abnormality duration is greater than the abnormality duration threshold and the support rate is greater than the support rate threshold, the abnormal data of the water supply network is diagnosed as a system failure.

[0032] Preferably, when the abnormal duration is less than the abnormal duration threshold and the support rate is greater than the support rate threshold, the abnormal data of the water supply network is diagnosed as a temporary system fluctuation;

[0033] When the abnormal duration is greater than the abnormal duration threshold and the support rate is less than the support rate threshold, the abnormal data of the water supply network is diagnosed as an isolated persistent abnormality.

[0034] Preferably, the method for obtaining the time series monitored by other sensors in the same group as the current sensor includes:

[0035] Based on the latest time series monitored by the current sensor, the local weighted regression method is used to obtain the periodic series. The polynomial function fitted to the periodic series is multi-order differentiated to obtain the derivative time series composed of the eigenvectors of each observation point. Dynamic time warping is used to calculate the distance between the derivative time series corresponding to different sensors, thereby obtaining the time series distance matrix of different sensors. Based on the time series distance matrix of different sensors, hierarchical clustering is used to group different sensors to obtain the time series monitored by other sensors in the same group as the current sensor.

[0036] Preferably, the derivative time series of the ith sensor composed of the feature vectors of each observation point is for:

[0037]

[0038]

[0039] in represents the m-th order derivative of the polynomial function at the t-th observation point, is the feature vector of the i-th sensor at the t-th observation point, and N is the number of observation points.

[0040] Preferably, dynamic time warping is used to calculate the distances between the derivative time series corresponding to different sensors to obtain the time series distance matrix of different sensors, where the distance between the derivative time series corresponding to the i-th sensor and the derivative time series corresponding to the j-th sensor is for:

[0041]

[0042]

[0043] Where N is the number of observation points corresponding to the i-th sensor, is the number of observation points corresponding to the jth sensor.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention calculates the local abnormality factor and quantitatively measures the abnormality at each time point, so that the process from the beginning to the end of the abnormality can be measured by the cumulative sum algorithm. By constructing a soft sensor, sufficient reference samples are provided for the time series recorded by the sensor in real time from both time and space aspects. Using these redundant samples, the calculation of the abnormal support rate is realized. Finally, the two indicators of abnormal duration and support rate calculated are used to distinguish between the sensor's own failure and the abnormal event of the water supply system. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a normal function and a method for correcting its distribution function provided by a specific embodiment of the present invention, wherein: Figure 1 (a) is the graph of the modified normal distribution function. Figure 1 (b) is the cumulative distribution function graph;

[0047] Figure 2 A schematic diagram of a pipe network example provided in Example 1 of the present invention;

[0048] Figure 3 A schematic diagram of an abnormality diagnosed as a random sensor failure provided in Example 1 of the present invention;

[0049] Figure 4 A schematic diagram of an abnormality diagnosed as a short-term system event provided in Example 1 of the present invention;

[0050] Figure 5 A schematic diagram of an abnormality of a long-term low support rate that requires further analysis provided in Example 1 of the present invention;

[0051] Figure 6 A schematic diagram of an abnormality diagnosed as a system event provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0052] The specific embodiment of the present invention aims to develop a diagnostic method for distinguishing data anomalies in a water supply network using duration and support rate. First, a clustering method is used to realize the grouping of sensors and the selection of historical reference sequences to construct a soft sensor; then a gate strategy based on a local anomaly factor is used to identify anomalies; secondly, the duration of the anomaly is calculated based on the local anomaly factor; the support rate of the anomaly is calculated based on the soft sensor; and finally, the anomaly is classified according to the duration and support rate. The specific steps are as follows:

[0053] S1. Perform distance calculation and clustering on sensor time series to realize soft sensor construction.

[0054] S11. The specific steps for distance calculation and clustering of sensor time series are as follows: based on the latest time series monitored by the current sensor, a local weighted regression method is used to obtain a periodic series, multi-order derivatives are performed on the polynomial function fitted to the periodic series to obtain a derivative time series composed of the characteristic vectors of each observation point, dynamic time warping is used to calculate the distance between the derivative time series corresponding to different sensors to obtain the time series distance matrix of different sensors, and hierarchical clustering is used to group different sensors based on the time series distance matrix of different sensors.

[0055] The specific steps of step S11 provided in the specific embodiment of the present invention include:

[0056] Assume that the clustering reference window size is N. The time series of sensor i Decomposed into trend using the locally weighted regression (LOESS) method and cycle LOESS calculates the trend by fitting a linear model to the data in a local window and assigning a weight to each observation. By subtracting the trend component from the original time series, the cyclical component can be isolated

[0057]

[0058] With the original sequence In contrast, the periodic series The similar scale characteristics are shown in different time periods of long time series, which provides a basis for subsequent segmentation processing.

[0059] The piecewise polynomial fitting provided in the specific embodiment of the present invention first converts the sequence Divide into several segments, and use a polynomial function to fit each segment. For a certain segment α, its polynomial function It can be expressed as:

[0060]

[0061] Among them, β 0 ,β 1 ,β 2 ,…,β p are the coefficients of the polynomial and p is the degree of the polynomial.

[0062] In polynomial fitting, the order p is a key parameter. A small p value will lead to inaccurate fitting, while a large p value will cause overfitting and consume too much computing resources. Covariance (COV) is considered to be a suitable indicator for evaluating the similarity of the trend of the water supply system monitoring time series. A larger COV indicates that the fitting function better preserves the trend changes of the original series.

[0063] Therefore, a specific embodiment of the present invention proposes an information criterion (IC) based on COV to balance the order of the polynomial and the fitting accuracy:

[0064] IC i =-NlnCOV i +(p+1)lnN

[0065] Among them, COV i Represents the polynomial function f i (t,p) and periodic sequence The covariance between them is calculated as follows:

[0066]

[0067] in, Representation sequence The mean of represents the polynomial function f of order p for the i-th sensor at time t i The value of (t,p) is is the sample mean of the fitted function.

[0068] By minimizing the IC value, the difficulty of manually determining the polynomial order p for different time series patterns can be eliminated. The polynomial coefficients are estimated by the least squares method, which minimizes the sum of squared errors between the observations and the polynomial fit:

[0069]

[0070] By solving this minimization problem, the coefficients of the polynomial function can be determined, thereby obtaining the polynomial function f i (t).

[0071] The characteristic vector of the time series provided by the specific embodiment of the present invention is obtained from f i (t), specifically by extracting from f i For a given observation point (i.e., time point) t, the feature vector of the time series of the i-th sensor is for:

[0072]

[0073] in Represents the m-th derivative of the polynomial function at the t-th observation point.

[0074] The specific embodiment of the present invention calculates multi-order derivatives of the polynomial function at each observation point, and the time series is converted into a derivative time series (DTS). Each observation point in the DTS is represented by a feature vector Indicates that N is the number of observation points, forming the following sequence:

[0075]

[0076] The specific embodiment of the present invention calculates the distance between two DTSs by accumulating the distances between corresponding observation points through dynamic time warping (DTW). and the time series of the jth sensor Any two corresponding points t and The distance between is defined as:

[0077]

[0078] Dynamic time warping (DTW) is used to recursively calculate the distance between two derivative time series. The core concept of the DTW algorithm is to determine a warping path that defines how to correspond points in one sequence to points in another sequence so as to minimize the total distance between all corresponding points.

[0079] The distance between the derivative time series corresponding to the i-th sensor and the derivative time series corresponding to the j-th sensor for:

[0080]

[0081]

[0082] Where N is the number of observation points corresponding to the i-th sensor, is the number of observation points corresponding to the jth sensor. The distance d between sensor i and sensor j ij is the distance between the derivative time series corresponding to the i-th sensor and the derivative time series corresponding to the j-th sensor

[0083] After calculating the distances between all sensors, a symmetrical distance matrix is ​​obtained as follows:

[0084]

[0085] where d ij represents the distance between sensor i and sensor j, and q is the number of sensors.

[0086] In order to effectively group sensors based on similarity, the distance matrix is ​​used to cluster them using the hierarchical clustering (HC) method. The HC method initially treats each sensor as an independent cluster, and then iteratively merges the two clusters with the smallest distance in the distance matrix into a new cluster. This process continues until all sensors are successfully grouped into one cluster.

[0087] S12, constructing a time series matrix of the soft sensor based on the latest time series monitored by the current sensor, the time series monitored by other sensors in the same group as the current sensor, and the historical subsequences with the smallest time series distance to the latest time series in the historical time series monitored by the current sensor.

[0088] Furthermore, the specific steps of step S12 are as follows:

[0089] In the specific embodiment of the present invention, the window size of the soft sensor is set to W. For sensor i, its historical time series is divided into a subsequence every t data points, and the distance calculation between each subsequence is the same as step S11. The l subsequences with the smallest distance to the front q are selected and added to the soft sensor corresponding to sensor i.

[0090] The soft sensor contains its own record sequence, namely the latest time series, the time series of other sensors in the same group, and similar historical subsequences. It is a real-time updated data structure. The specific structure is as follows:

[0091]

[0092] Where W is the window size of the soft sensor, is the time series of length W recently acquired by sensor i c 1 、c 2 ,…c k are other sensors that are highly correlated with the time series of sensor i and are grouped into the same group, t 1 ,t 2 ,…t l is the time series of the current window of sensor i The starting point of the other window sequences in the history of highly correlated sensor i.

[0093] The data in the soft sensor provided by the specific embodiment of the present invention is updated in real time over time, and the sensor group c 1 、c 2 ,…c k With the historical sequence t 1 ,t 2 ,…t l Reselection is carried out regularly.

[0094] S2. Detecting anomalies in the sensor time series through local anomaly factors: Using the gate strategy of local anomaly factors to identify anomalies in the latest time series.

[0095] Specifically, for the data point x t , its local reachable density LRD can be expressed as:

[0096]

[0097] where x n Represents x t The data points in the neighborhood of N k (x t ) represents x t The k-distance neighborhood of |N k (xt )| is the number of data points in the neighborhood; the local reachable density rd(x t ,x n ) is defined as x n The distance to its kth nearest neighbor and point x t Arrival x n The actual distance is the larger of the two distances.

[0098] x t The LOF value of the local outlier factor is obtained by comparing x t The LRD of and the LRD of its neighboring points are calculated as follows:

[0099]

[0100] The gate strategy based on LOF value is used to identify outliers, and the threshold of the gate strategy is θ:

[0101] Outlier = {x t |LOF(x t )>θ}

[0102] S3. A cumulative sum algorithm based on local anomaly factors is proposed to calculate the duration of anomalies: an intermediate threshold is constructed based on the threshold of the gate strategy, and the traceback is started from the anomaly point until the local anomaly factor of the data point is less than the intermediate threshold. The time point when the traceback is stopped is taken as the starting point of the anomaly, and the cumulative sum is obtained recursively from the starting point of the anomaly. The cumulative value of each accumulation is the local anomaly factor of the current data point minus the sum of the intermediate threshold and the penalty term. The time point when the cumulative sum is zero for the first time is taken as the end point of the anomaly, and the duration of the anomaly is constructed based on the starting point and the end point of the anomaly.

[0103] Specifically, the threshold of the local outlier factor LOF for identifying outliers is θ, and the intermediate threshold ν of the cumulative sum algorithm is as follows:

[0104]

[0105] For the detected anomalies From the time point Go back. If the time point satisfy:

[0106]

[0107] From the time point Continue to trace forward until the above conditions are no longer met and stop tracing forward. The time point when tracing forward stops is regarded as the starting point of the anomaly t start . Starting from the starting point and accumulating backwards, the cumulative value is calculated as follows:

[0108] δ t =LOF(xt )-v-β

[0109] Where β is the penalty term, defined as:

[0110] β=ln(1+|LOF(x t )-v|)

[0111] The cumulative sum S of the recursive calculation t As shown below:

[0112]

[0113] When the cumulative sum equals 0 for the first time, the time point is considered to be the abnormal end point t end :

[0114] t end =min{t i |S t =0}

[0115] Then the abnormal duration T is:

[0116] T=t end -t start .

[0117] S4. An abnormal support probability algorithm based on soft sensors is proposed to calculate the abnormal confidence. The probability of observing abnormal points in the time series matrix of soft sensors, that is, the degree to which abnormal points can be observed, is taken as the support rate:

[0118] First, the normal distribution model is built for the time series of sensor i, and the probability density function g(x i )for:

[0119]

[0120] + represents a maximum value event that is higher than the normal value, and - represents a minimum value event that is lower than the normal value. For the observation value of sensor i at the tth time point The degree of its large or small value is used to measure its contribution to the maximum or minimum events:

[0121]

[0122]

[0123] in Represents the observed value For the contribution rate of maximum value events +, Represents the observed value For the minimum event, the contribution rate of -.

[0124] In order to reduce the impact of a single maximum or minimum value on the overall support probability, the cumulative distribution function of the normal distribution is modified. Figure 1 (a) and Figure 1 (b) shows the modified normal distribution function and cumulative distribution function. The minimum value of the cumulative distribution function is P min for:

[0125] P min =P(μ i -k min σ i |+)=P(μ i +k min σ i |-)

[0126] where μ i , σ i are the mean and variance of the time series of sensor i, k min is the parameter that controls the minimum value of the cumulative distribution function.

[0127] The middle area of ​​the normal distribution function is cut, and the cumulative distribution function value P of the cut area is mid for:

[0128]

[0129] where k mid Parameters for adjusting the size of the middle cropping area.

[0130] For data points in the clipped area Their cumulative distribution functions are:

[0131]

[0132] Then the maximum value of the modified cumulative distribution function is:

[0133] P max =1-P min -P mid

[0134] For the soft sensor corresponding to sensor i The different sequences in the soft sensor are considered to have independent measurements and failure probabilities. In the sequence, the event E is a large event or a small event. For any type of event E, E∈(+,-), the joint probability for:

[0135]

[0136] in Indicates the soft sensor corresponding to sensor i The tth column, n is the soft sensor If the probability of event E occurring in the corresponding history of the soft sensor and its data in the same group is large, it means that it is a system event. If it is small, it means that it is a single sensor failure.

[0137] In order to prevent the numerical overflow caused by the different number of sequences in different soft sensors, the joint probability Smoothing using geometric mean:

[0138]

[0139] According to Bayes' theorem, when we observe The probability of event E occurring later can be expressed as:

[0140]

[0141] According to the total probability formula, expand

[0142]

[0143] is the observation vector under the condition of the maximum event The contribution degree of P(+) is the probability that any observation point is a large value. is the observation vector under the condition of the minimum event The contribution degree of P(-) is the probability that any observation point is a small value.

[0144] Due to the symmetry of the normal distribution:

[0145] P(E)=P(+)=P(-)

[0146] Therefore, we can finally get that when the observation value is When , the support rate SR is calculated as follows:

[0147]

[0148]

[0149] Among them, the positive support rate SR + The negative support rate SR is the degree to which all sequences in the soft sensor support the occurrence of a maximum value event in sensor i at time point t. - is the degree to which all sequences in the soft sensor support the occurrence of a minimum event for sensor i at time point t; is the observation vector The geometric mean of the probability of a maximum event occurring, is the observation vector under the condition of the maximum event The geometric mean of the contribution of is the observation vector The geometric mean of the probability of a minimum event occurring, is the observation vector under the condition of the minimum event The geometric mean of the contribution of is the probability that the data monitored at the tth time point of the sth time series in the soft sensor is a larger or smaller value, representing the contribution of the sequence to larger or smaller events in the soft sensor; n is the number of time series in the soft sensor; P(E) is the probability that any observation point is a larger or smaller value. Due to the symmetry of normal distribution modeling, the probability of any point being larger is equal to the probability of being smaller; To observe the vector probability.

[0150] S5. Classify anomalies using duration and confidence:

[0151] Specifically, the duration threshold of the abnormality is set to T th , the support rate threshold is SR th By combining the duration T with the support rate SR and based on their comparison with the threshold T th and SR th According to the relationship between them, anomalies can be divided into four types.

[0152] Sensor failures occur sporadically and lack support from other sequences in soft sensors. Therefore, an anomaly can be classified as a sensor failure if the following conditions are met:

[0153] Random sensor failure = {abnormal|T<T th ∧SR<SR th}

[0154] Real water supply system events (such as pipe bursts) are caused by changes in system infrastructure. Such events usually last a relatively long time from occurrence to repair and are confirmed by multiple sensor responses. Therefore, the rules for determining system events are as follows:

[0155] Real system event = {abnormal|T≥T th ∧SR≥SR th}

[0156] Anomalies classified as random sensor failures are treated as invalid data and masked through subsequent data interpolation. Anomalies identified as true system events are reported to users to prompt immediate action to resolve system issues.

[0157] For anomalies that are short in duration but have high support rates, they are classified as temporary system fluctuations. This type of anomaly may be caused by system adjustments or unusual water usage patterns, and although they are outside the normal range, they will soon return to normal. Due to their short duration, these anomalies are not considered sufficient to cause alarm to users, but are still available for engineers to review if necessary.

[0158] Temporary system fluctuation = {abnormal|T<T th ∧SR≥SR th}

[0159] Anomalies with long duration but low support are classified as isolated persistent anomalies. Such anomalies may be caused by persistent long-term sensor failures (such as sensor drift causing significant deviations in monitored values). Alternatively, they may be caused by sparse sensor deployment in the area where the system event occurred, resulting in only a single sensor recording a response.

[0160] Isolated persistent exception = {Exception|T≥T th ∧SR <SR th}

[0161] The present invention belongs to a method for diagnosing abnormalities in monitoring data of a water supply network. Traditional abnormal event diagnosis in this field often ignores the impact of abnormalities of the sensor itself. These methods only focus on the abnormal patterns of monitoring data, but ignore the usefulness of the characteristics of monitoring data as time series for abnormal detection and diagnosis. In a water supply network, how to utilize the correlation generated by the hydraulic connection of multiple sensors in the network and fuse the characteristics of time series for abnormal diagnosis of the system is a blank that has not yet been explored in this field. This method calculates the local abnormal factor and quantitatively measures the abnormality at each time point, so that the process from the beginning to the end of the abnormality can be measured by the cumulative sum algorithm. By constructing a soft sensor, sufficient reference samples are provided for the time series recorded by the sensor in real time in terms of time and space. Using these redundant samples, the calculation of abnormal support rate is realized. Finally, the two indicators of abnormal duration and support rate calculated are used to distinguish between sensor faults and abnormal events in the water supply system.

[0162] Example 1: The method provided in the specific embodiment of the present invention is applied to a certain actual water supply network, which is equipped with 58 pressure sensors and 36 flow sensors. Figure 2 shown.

[0163] Duration threshold T L Set to 120 minutes, support rate threshold SR LThe value is set to 0.8. The invented method is used to diagnose anomalies into four types: (1) short duration, low support rate; (2) short duration, high support rate; (3) long duration, low support rate; (4) long duration, high support rate.

[0164] (1) Short-duration, low-support anomalies: These anomalies occur suddenly and lack verification from other sequence data in the same soft sensor, so they are classified as random sensor failures. In this case, data padding is used to mask the outliers. Figure 3 Shows an anomaly diagnosed as a random sensor failure. This anomaly appears as a spike-like extremely low value that occurs suddenly during smooth operation without the same trend as other sensors. Therefore, it appears as a short duration and low support rate on the diagnostic indicator.

[0165] (2) Long-duration, high-support anomalies: This type of anomaly reflects that multiple sensors have been out of their normal mode for a long time and synchronously, so it is diagnosed as a water supply network system event. This type of anomaly reflects the coordinated response of the sensor group to the event in a certain area. Figure 4 This is an abnormality diagnosed as a water supply network system event. This abnormality reflects the occurrence of a pipe burst event. First, the pipe is blocked and the pressure gradually rises. Then the pressure drops sharply after the pipe bursts. After 990 minutes, it returns to the normal change mode after maintenance by the staff. In this process, the sequence from three different sensors in the soft sensor responds synchronously, showing a high support rate.

[0166] (3) Short duration, high support rate anomaly Figure 5 Shows anomalies diagnosed as short-term system events. The anomaly is significantly higher than the normal peak, but unlike the spike-like sensor anomaly, there are multiple time points of increasing and decreasing data, and multiple sensors respond with the same trend. Although this type of anomaly is beyond the usual range, it is short-lived and poses no significant threat to the system, so no intervention is required. They are consistent with data from other sensors or historical records, indicating that these anomalies are not random sensor failures, but may be caused by transient conditions such as water hammer effects.

[0167] (4) Long-duration, low-support anomalies: Regardless of the support rate, anomalies that persist for a long time should be reported to the control end. These long-duration anomalies require the attention of engineers and further analysis, where the support rate is an important indicator in the analysis process. This type of anomaly may indicate a persistent sensor failure or a local network event occurring in a sparsely monitored area. Figure 6 As shown in the anomaly, the sensor was damaged and no longer transmitted monitoring data normally after several extremely large monitoring values ​​appeared, reflecting a serious and long-term sensor failure.

[0168] In summary, the anomaly classification method proposed in the present invention can be effectively used for anomaly diagnosis of water supply system sensor monitoring data.

Claims

1. A water supply network abnormal data diagnosis method based on duration and support probability, characterized in that: include: The gate strategy of the local anomaly factor is used to identify the anomaly points of the latest time series. The intermediate threshold is constructed based on the threshold of the gate strategy. The anomaly point is traced back from the anomaly point until the local anomaly factor of the data point is less than the intermediate threshold. The time point when the traceback stops is taken as the starting point of the anomaly. The cumulative sum is obtained by recursively performing the operation backwards from the starting point of the anomaly. The cumulative value of each accumulation is the local anomaly factor of the current data point minus the sum of the intermediate threshold and the penalty term. The time point when the cumulative sum is zero for the first time is taken as the end point of the anomaly. The anomaly duration is constructed based on the start point and end point of the anomaly. The probability of observing an abnormal point in the time series matrix of the soft sensor is used as the support rate. The time series matrix of the soft sensor is constructed by the latest time series monitored by the current sensor, the time series monitored by other sensors in the same group as the current sensor, and the historical subsequences of the historical time series monitored by the current sensor with the smallest time series distance from the latest time series. An abnormality duration threshold and a support rate threshold are set, and abnormal data of the water supply network are diagnosed based on the comparison between the abnormality duration and the abnormality duration threshold, and the comparison between the support rate and the support rate threshold.

2. The water supply network abnormal data diagnosis method based on duration and support probability according to claim 1 is characterized in that: The gate strategy of local anomaly factor is used to identify the anomalies of the latest time series, and the anomalies are: Outlier = {x t |LOF(x t )>θ} in, θ is the threshold of the gate strategy, LOF(x t ) is the t-th time point data x of the latest time series t The local anomaly factor.

3. The water supply network abnormal data diagnosis method based on duration and support probability according to claim 1 is characterized in that: The intermediate threshold v is constructed based on the threshold of the gate strategy, and the intermediate threshold v is: Among them, θ is the threshold of the gate strategy.

4. The water supply network abnormal data diagnosis method based on duration and support probability according to claim 1 is characterized in that: The cumulative sum is obtained by recursively moving backward from the abnormal starting point, where the cumulative sum S at the tth time point is t for: δ t =LOF(x t )-v-β β=ln(1+|LOF(x t )-v|) Among them, LOF(x t ) is the t-th time point data x of the latest time series t The local anomaly factor, δ t is the cumulative value at the tth time, v is the intermediate threshold, β is the penalty term, t start is the starting point.

5. The water supply network abnormal data diagnosis method based on duration and support probability according to claim 1 is characterized in that: The probability of observing an abnormal point in the time series matrix of the soft sensor is taken as the support rate, and the support rate SR includes the positive support rate SR + and negative support rate SR - : P(E)=P(+)=P(-),E∈{+,-} Among them, the positive support rate SR + The negative support rate SR is the degree to which all sequences in the soft sensor support the occurrence of a maximum value event in sensor i at time point t. - is the degree to which all sequences in the soft sensor support the occurrence of a minimum event for sensor i at time point t; is the observation vector The geometric mean of the probability of a maximum event occurring, is the observation vector under the condition of the maximum event The geometric mean of the contribution of is the observation vector The geometric mean of the probability of a minimum event occurring, is the observation vector under the condition of the minimum event The geometric mean of the contribution of is the probability that the data monitored at the tth time point of the sth time series in the soft sensor is a larger or smaller value, representing the contribution of the sequence to larger or smaller events in the soft sensor; n is the number of time series in the soft sensor; P(E) is the probability that any observation point is a larger or smaller value. Due to the symmetry of normal distribution modeling, the probability of any point being larger is equal to the probability of being smaller; To observe the vector probability.

6. The water supply network abnormal data diagnosis method based on duration and support probability according to claim 1 is characterized in that: When the abnormal duration is less than the abnormal duration threshold and the support rate is less than the support rate threshold, the abnormal data of the water supply network is diagnosed as a random sensor failure; When the abnormality duration is greater than the abnormality duration threshold and the support rate is greater than the support rate threshold, the abnormal data of the water supply network is diagnosed as a system failure.

7. The water supply network abnormal data diagnosis method based on duration and support probability according to claim 6 is characterized in that: When the abnormal duration is less than the abnormal duration threshold and the support rate is greater than the support rate threshold, the abnormal data of the water supply network is diagnosed as a temporary system fluctuation; When the abnormal duration is greater than the abnormal duration threshold and the support rate is less than the support rate threshold, the abnormal data of the water supply network is diagnosed as an isolated persistent abnormality.

8. The water supply network abnormal data diagnosis method based on duration and support probability according to claim 1 is characterized in that: The method for obtaining the time series monitored by other sensors in the same group as the current sensor includes: Based on the latest time series monitored by the current sensor, the local weighted regression method is used to obtain the periodic series. The polynomial function fitted to the periodic series is multi-order differentiated to obtain the derivative time series composed of the eigenvectors of each observation point. Dynamic time warping is used to calculate the distance between the derivative time series corresponding to different sensors, thereby obtaining the time series distance matrix of different sensors. Based on the time series distance matrix of different sensors, hierarchical clustering is used to group different sensors to obtain the time series monitored by other sensors in the same group as the current sensor.

9. The water supply network sensor data recovery method based on soft sensor according to claim 8, characterized in that: The derivative time series of the i-th sensor composed of the feature vectors of each observation point for: in represents the m-th order derivative of the polynomial function at the t-th observation point, is the feature vector of the i-th sensor at the t-th observation point, and N is the number of observation points.

10. The water supply network sensor data recovery method based on soft sensor according to claim 8, characterized in that: Dynamic time warping is used to calculate the distance between the derivative time series corresponding to different sensors to obtain the time series distance matrix of different sensors, where the distance between the derivative time series corresponding to the i-th sensor and the derivative time series corresponding to the j-th sensor is for: Where N is the number of observation points corresponding to the i-th sensor, is the number of observation points corresponding to the jth sensor.

Images