Drainage system anomaly monitoring method and device based on binary classification and clustering algorithm, and storage medium

Abstract

The application discloses a kind of based on two classification and clustering algorithm's drainage system anomaly monitoring method, equipment and storage medium, include: obtaining monitoring data sequence;Define trend estimation sequence;According to trend estimation sequence and monitoring data sequence, calculate mean square deviation value;The value of each time in trend estimation sequence is respectively added and subtracted mean square deviation value, obtain the abnormal upper and lower boundary of liquid level monitoring value in this time period;Two classifications are carried out to all point positions in monitoring data sequence;Abnormal data is clustered in time dimension, and the dynamically numbered abnormal cluster is obtained;Abnormal sample data in abnormal cluster is arranged in ascending order according to time, and the abnormal value of the most middle time is selected as typical abnormal value.The two classification and clustering algorithm-based drainage system anomaly monitoring method provided in the application is based on single time sequence liquid level monitoring data, uses big data analysis algorithm, scientifically determines the abnormal value in liquid level monitoring data, and divides the dynamic abnormal time period range based on this.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of urban drainage system monitoring, and particularly relates to a drainage system anomaly monitoring method and device based on a binary classification and clustering algorithm and a storage medium. BACKGROUND

[0002] As the top priority of urban infrastructure construction, the drainage pipe network system not only guarantees the daily life of residents, but also shoulders the heavy responsibilities of flood control and sewage treatment. With the explosive development of Internet technology in recent years and the deep-rooted concept of "intelligent drainage", more and more water engineering projects introduce sensing devices to monitor the pipe network liquid level value to monitor the daily operation of the drainage system. On this basis, the operation state of the drainage system is tried to be judged by analyzing the change of the liquid level value, and the problem is timely located and solved.

[0003] However, how to determine whether the liquid level value is abnormal becomes a big problem. The past method often determines whether the liquid level is in a normal state based on a simple mutation value or sets a fixed upper and lower limit. This method is too simple for complex drainage systems, and the risk of misjudgment and omission is extremely high. At the same time, it is difficult to scientifically divide the abnormal period range after determining the abnormal value. All monitoring information determined to be abnormal is often pushed to the user in the form of an alarm, making the messages received by the user redundant and without focus. Therefore, in order to accurately and scientifically identify the abnormal operation state of the drainage system and optimize the user experience, it is necessary to propose a method that can efficiently identify abnormal monitoring values and scientifically and dynamically divide the abnormal period range. SUMMARY

[0004] The application provides a drainage system anomaly monitoring method based on a binary classification and clustering algorithm to solve the above-mentioned technical problems, specifically adopting the following technical scheme:

[0005] A drainage system anomaly monitoring method based on a binary classification and clustering algorithm, comprising:

[0006] Obtaining drainage pipe liquid level meter historical monitoring data, obtaining a monitoring data sequence in a certain time period from the historical monitoring data, and the detection samples in the monitoring data sequence have a time interval;

[0007] Defining a trend estimation sequence with the same length as the monitoring data sequence according to the dependent transmission of time series data;

[0008] Calculating the mean square deviation value of the liquid level change in the time period according to the trend estimation sequence and the monitoring data sequence;

[0009] Adding and subtracting the mean square deviation value to the value of each time in the trend estimation sequence to obtain the abnormal upper and lower boundaries of the liquid level monitoring value in the time period;

[0010] Taking the abnormal upper and lower boundaries as the discrimination condition, all points in the monitoring data sequence are classified into two categories, and the abnormal values in the monitoring data sequence are screened out;

[0011] The abnormal data are clustered in the time dimension to obtain a dynamic number of abnormal clusters, and one abnormal cluster is an abnormal time period range;

[0012] The abnormal sample data in the abnormal cluster are arranged in ascending order of time, and the abnormal value at the most middle time is selected as the representative sample, i.e. the typical abnormal value.

[0013] Further, the length of the monitoring data sequence is 30 days, and the time interval of the data is 1 minute, wherein V1 is the first monitoring value of the monitoring data sequence, V n is the last monitoring value of the monitoring data sequence, and the monitoring data sequence is represented as:

[0014] {V1, V2, V3, V4, …, V i , …, V n-1 , V n}

[0015] Further, the specific method for defining a trend estimation sequence with the same length as the monitoring data sequence according to the dependence transmission of time series data is:

[0016] According to the time series dependence transmission, the trend estimation sequence of the liquid level data in the time period is fitted, and the discrete trend estimation values are connected to form the trend estimation sequence, and the specific formula is as follows:

[0017]

[0018]

[0019]

[0020] The linear transformation with time represents the dependence coefficient sequence of the liquid level value at a historical time i on the liquid level value at the current T time, and the trend estimation value of the liquid level value at T time can be obtained by weighted calculation, which is represented by E T , wherein V i represents the actual liquid level value at i time, and finally the trend estimation values at each time are connected into a line to form the trend estimation sequence E, which is represented as:

[0021] {E1, E2, E3, E4, …, E i , …, E n-1 , E n}

[0022] Further, the mean square deviation value is calculated according to the following formula,

[0023]

[0024] Further, the DBSCAN method is used to perform time dimension clustering on the abnormal data to determine different abnormal clusters.

[0025] Further, the specific method of using the DBSCAN method to perform time dimension clustering on the abnormal data is as follows:

[0026] A point is selected from the sample, and a radius of 0.025 and a minimum number of points in the circle of 5 are given;

[0027] If the point meets the condition that there are at least 5 neighboring points in the neighborhood circle with a radius of 0.025, the center of the circle is moved to the next sample point;

[0028] If the sample point does not meet the above condition, the sample point is reselected, and iterative clustering is performed according to the set radius of 0.025 and the minimum number of points of 5, until all the abnormal clusters are obtained.

[0029] Further, after the typical abnormal value is obtained, the drainage system abnormal monitoring method based on the binary classification and clustering algorithm further comprises:

[0030] The obtained typical abnormal value is linked to the business system, and the liquid level size, occurrence time and other related information of the typical abnormal value are pushed to the user.

[0031] An electronic device comprising a memory and a processor, the memory being used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the foregoing drainage system abnormal monitoring method based on the binary classification and clustering algorithm.

[0032] A computer readable storage medium storing a computer program, wherein the computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the foregoing drainage system abnormal monitoring method based on the binary classification and clustering algorithm.

[0033] The drainage system abnormal monitoring method, device and storage medium based on the binary classification and clustering algorithm have the advantages that based on single time sequence liquid level monitoring data, a big data analysis algorithm is used to scientifically determine abnormal values in the liquid level monitoring data, and a dynamic abnormal time period range is divided based on the abnormal values, so that the drainage system abnormal operation state is scientifically guided to be determined. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0035] Figure 1 is a schematic diagram of a drainage system anomaly monitoring method based on binary classification and clustering algorithm of the present application;

[0036] Figure 2 is a change diagram of the selected monitoring data sequence;

[0037] Figure 3 is a comparison diagram of the calculated trend estimation sequence and the monitoring data sequence;

[0038] Figure 4 is a schematic diagram of the upper and lower boundaries of the anomaly binary classification method obtained according to the mean square deviation value;

[0039] Figure 5 is an anomaly value clustering analysis result diagram;

[0040] Figure 6 is a selected typical anomaly value result diagram;

[0041] Figure 7 is a schematic diagram of an electronic device for implementing a drainage system anomaly monitoring method based on binary classification and clustering algorithm of the present application. DETAILED DESCRIPTION

[0042] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0043] As Figure 1 is a drainage system anomaly monitoring method based on binary classification and clustering algorithm of the present application, comprising:

[0044] S101: Obtain the historical monitoring data of the drainage pipeline liquid level meter, and obtain the monitoring data sequence within a certain time period from the monitoring data sequence, and the detection samples in the monitoring data sequence have time intervals.

[0045] Specifically, the time interval of the liquid level monitoring data can be selected according to the equipment condition, and the minimum data collection step of the equipment is appropriate to ensure the sample quantity. In this application, the data acquisition time is from April 19, 2022 to May 18, 2022, a total of 30 days. The data interval is 1 minute, that is, 1 piece per minute, a total of 43200 pieces of data. V1 is the first monitoring value of the monitoring data sequence, and V n is the last monitoring value of the monitoring data sequence. The monitoring data sequence is represented as:

[0046] {V1, V2, V3, V4, …, V i , …, V n-1 , V n}

[0047] The time sequence change process of the monitoring data sequence V is shown in Figure 2 .

[0048] S102: According to the time sequence data, the same length of the trend estimation sequence is defined according to the dependent transmission.

[0049] Specifically, according to the time sequence dependent transmission, the trend estimation sequence E of the liquid level data in this period is fitted. Then the discrete trend estimation values are connected to form the trend estimation sequence, and the specific formula is as follows:

[0050]

[0051]

[0052]

[0053] The trend estimation value of the liquid level value at time T is obtained by weighted calculation, which is represented by E T , wherein V i represents the actual liquid level value at time i, and finally the trend estimation values at each time are connected to form a line to form the trend estimation sequence E, which is represented as:

[0054] {E1, E2, E3, E4, …, E i , …, E n-1 , E n}

[0055] It can be understood that the nth value of the sequence E is determined by the first n values of the sequence V, which can be expressed by the formula as follows:

[0056]

[0057] Substitute the embodiment data of the application into the above formula, the following results can be obtained:

[0058] E1 = 1 x V1

[0059]

[0060]

[0061]

[0062]

[0063] At this point, according to the known liquid level observation value sequence V, the corresponding trend estimation sequence E is calculated. The comparison of the two sequences is shown as Figure 3 .

[0064] S103: Calculate the mean square deviation value of the liquid level change in the time period according to the trend estimation sequence and the monitoring data sequence.

[0065] Calculate the mean square deviation value B between the sequence V and the sequence E, and calculate the mean square deviation value according to the following formula,

[0066]

[0067] Substitute the liquid level observation value sequence V and the trend estimation sequence E calculated in step two into the above formula, and take n as the total length of the sequence 43200. Finally, the mean square deviation value of this embodiment is calculated as 0.583 meters.

[0068] S104: Add and subtract the values of each time in the trend estimation sequence to obtain the abnormal upper and lower boundaries of the liquid level monitoring value in the time period.

[0069] Specifically, the trend estimation sequence E obtained in step S102 is added and subtracted by the mean square deviation value 0.583 of the liquid level change in the time period calculated in step S103 to obtain the upper and lower classification boundaries of the two classification method, wherein the upper boundary is represented by the sequence U and the lower boundary is represented by the sequence D. The upper and lower boundaries of this embodiment are shown as the dashed lines in Figure 4 , the upper dashed line is the upper boundary of the two classification, and the lower dashed line is the lower boundary of the two classification.

[0070] {U1, U2, U3, U4, …, U i , …, U 43199 , U 43200}

[0071] {D1, D2, D3, D4, …, D i , …, D 43199 , D 43200}

[0072] S105: Taking the abnormal upper and lower boundaries as the discrimination condition, performing binary classification on all points in the monitoring data sequence to identify abnormal values in the monitoring data sequence.

[0073] Specifically, the observation sequence V of the original liquid level value is compared with the binary classification upper and lower boundaries obtained in step S104, and the liquid level value higher than the upper boundary or lower than the lower boundary is marked as an abnormal liquid level value. The sample set of the abnormal liquid level value is denoted by Z, and in this embodiment, Z contains a total of 26784 abnormal monitoring values,

[0074] Z = {Z1, Z2, Z3, Z4,..., Z i ,..., Z 26783 , Z 26784}

[0075] S106: Time-dimension clustering of the abnormal data is performed to obtain a dynamic number of abnormal clusters, and one abnormal cluster is one abnormal time period range.

[0076] In the embodiments of the present application, the DBSCAN method is used to perform time-dimension clustering of the abnormal data to determine different abnormal clusters.

[0077] As a preferred embodiment, the specific method of using the DBSCAN method to perform time-dimension clustering of the abnormal data is as follows:

[0078] A point is selected from the samples, and a radius of 0.025 and a minimum number of neighboring points in the circle of 5 are given. If the point satisfies that there are at least 5 neighboring points in the neighborhood circle with a radius of 0.025, the center of the circle is moved to the next sample point. If the sample point does not satisfy the above condition, the sample point is reselected, and iterative clustering is performed according to the set radius of 0.025 and the minimum number of neighboring points of 5, until all abnormal clusters are obtained.

[0079] In the present application, the clustering radius R = 0.025 and the minimum number of neighboring points N = 5 in the circle are selected, and different abnormal liquid level values are divided into different abnormal clusters. Finally, the 26784 abnormal values of this embodiment are divided into 4 abnormal clusters, and the abnormal time period ranges obtained by specific clustering are as shown in Figure 5 .

[0080] S107: The abnormal sample data in the abnormal cluster are arranged in ascending order of time, and the abnormal value at the most middle time is selected as the representative sample, i.e., the typical abnormal value.

[0081] The samples at the middle time of each abnormal cluster in step S106 are arranged in ascending order of time, and the samples at the middle time of each abnormal cluster are selected as typical abnormal values. In this embodiment, the typical abnormal values of the four abnormal clusters are 2.155, 2.417, 2.932, and 2.443, and the corresponding time points are 2022-04-24 17:15, 2022-04-27 20:23, 2022-05-01 05:06, and 2022-05-11 11:47, respectively. The specific typical abnormal values are shown in Figure 6 .

[0082] As a preferred embodiment, after obtaining the typical abnormal values, the drainage system abnormal monitoring method based on the binary classification and clustering algorithm further comprises:

[0083] S108: Linking the obtained typical abnormal values with the business system, and pushing the liquid level size, occurrence time and other related information of the typical abnormal values to the user.

[0084] As shown in Figure 7 , an electronic device of the present application comprises a memory 401 and a processor 402, the memory 401 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 402 to implement the above-mentioned drainage system abnormal monitoring method based on the binary classification and clustering algorithm.

[0085] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic device described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0086] The present application also discloses a computer readable storage medium storing a computer program, which is executed by the processor 402 to implement the above-mentioned drainage system abnormal monitoring method based on the binary classification and clustering algorithm.

[0087] Specifically, the computer program can be divided into one or more modules / units, one or more modules / units are stored in the memory 401 and executed by the processor 402, and the input and output of data is completed by the input and output interface 405 and the output interface 406, so as to complete the present application. One or more modules / units can be a series of computer program instruction segments that can complete a specific function, which is used to describe the execution process of the computer program in the computer device.

[0088] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The computer device can include, but is not limited to, a memory 401 and a processor 402. Those skilled in the art can understand that the embodiments are merely examples of the computer device, and do not limit the computer device, and the computer device can include more or fewer components, or combine certain components, or different components, for example, the computer device can also include an inputter 407, a network access device, a bus and the like.

[0089] The processor 402 can be a central processing unit (CPU), and can also be other general-purpose processors 402, digital signal processors 402 (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. The general-purpose processor 402 can be a microprocessor or can also be any conventional processor 402 and the like.

[0090] The memory 401 can be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. The memory 401 can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card and the like. Further, the memory 401 can include both the internal storage unit and the external storage device of the computer device. The memory 401 is used to store computer programs and other programs and data required by the computer device, and can also be used to temporarily store the outputter 408, and the aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM) 403, a random access memory (RAM) 404, a disk or an optical disk and the like. Various storage medium that can store program codes.

[0091] The basic principles, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the above embodiments do not limit the present application in any form, and any technical solutions obtained by equivalent replacement or equivalent transformation fall within the protection scope of the present application.

Claims

1. A method for anomaly monitoring of a drainage system based on binary classification and clustering algorithms, characterized in that, Include: Historical monitoring data of the level gauge in the drainage pipeline is obtained, and a monitoring data sequence within a certain time period is obtained from it, wherein there is a time interval between the detection samples in the monitoring data sequence; Based on the dependency transitivity of time series data, a trend estimation sequence of the same length as the monitored data sequence is defined; The mean square deviation of the liquid level change during this time period is calculated based on the trend estimation sequence and the monitoring data sequence. By adding or subtracting the mean square deviation value from the value at each time point in the trend estimation sequence, the abnormal upper and lower boundaries of the liquid level monitoring value within that time period are obtained. Using the upper and lower boundaries of anomalies as discrimination criteria, all points in the monitoring data sequence are classified into two categories to identify outliers in the monitoring data sequence. Clustering of abnormal data along the time dimension yields a dynamic number of abnormal clusters, with each abnormal cluster representing an abnormal time period. The abnormal sample data in the abnormal cluster are arranged in ascending order of time, and the abnormal value at the middle time is selected as the representative sample, that is, the typical abnormal value. The monitoring data sequence is 30 days long, with a data interval of 1 minute, where V1 is the first monitoring value in the monitoring data sequence, V n The last monitored value in the monitoring data sequence is represented as: ； The specific method for defining a trend estimation sequence of the same length as the monitoring data sequence based on the dependency transitivity of time series data is as follows: Modeling is performed based on time-series dependency propagation to fit a trend estimation sequence of liquid level data within the specified time period. The discrete trend estimates are then concatenated to form the trend estimation sequence, as shown in the following formula: ； E represents a linear transformation over time, indicating the dependence coefficient sequence of the liquid level value at a historical time i on the current liquid level value at time T. Through weighted calculation, a trend estimate of the liquid level value at time T can be obtained. This value is denoted by Ei. T It means that V i Let i represent the actual liquid level value at time i. Finally, connect the trend estimates at each time point into a line to form the trend estimation sequence E, which is represented as: ； The mean square deviation value is calculated using the following formula. 。 2. The drainage system anomaly monitoring method based on binary classification and clustering algorithms according to claim 1, characterized in that, The DBSCAN method is used to cluster the abnormal data along the time dimension to identify different abnormal clusters.

3. The drainage system anomaly monitoring method based on binary classification and clustering algorithms according to claim 2, characterized in that, The specific method for using DBSCAN to perform time-dimensional clustering of outlier data is as follows: Select a point from the sample, given a radius of 0.025 and a minimum number of nearest neighbors of 5 within the circle; If a point satisfies the condition that it has at least 5 nearest neighbors within a neighborhood circle with a radius of 0.025, then the center of the circle is moved to the next sample point. If the sample points do not meet the above conditions, the sample points are reselected, and iterative clustering is performed according to the set radius of 0.025 and minimum neighbor number of 5 until all the abnormal clusters are obtained.

4. The drainage system anomaly monitoring method based on binary classification and clustering algorithms according to claim 1, characterized in that, After obtaining the typical outliers, the drainage system anomaly monitoring method based on binary classification and clustering algorithms further includes: The obtained typical outliers are linked with the business system, and the liquid level and occurrence time of the typical outliers are pushed to the user.

5. An electronic device, characterized in that, The system includes a memory and a processor, the memory being used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the drainage system anomaly monitoring method based on binary classification and clustering algorithms as described in any one of claims 1-4.

6. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the drainage system anomaly monitoring method based on binary classification and clustering algorithms as described in any one of claims 1 to 4.

Images