A method and device for detecting abnormality of municipal sewer network liquid level data

Abstract

The present application relates to the field of sewer network monitoring, and particularly relates to a municipal sewer network liquid level data anomaly detection method and device. The present application uses multi-scale feature extraction to ensure feature extraction effect while significantly reducing computational complexity. The multi-class support vector machine is used to finely classify liquid level anomalies, thereby comprehensively and efficiently detecting anomalies. Compared with the single type detection method of the prior art, the detection range is significantly expanded, and various abnormal conditions such as sensor failure, data transmission error and pipe rupture can be effectively identified. Finally, the present application forms a complete anomaly management closed loop from data acquisition, anomaly detection, classification determination to early warning output, and the anomaly log database provides data support for subsequent pipe network health assessment and operation and maintenance decision.

Description

Technical Field

[0001] This invention relates to the field of drainage network monitoring, and in particular to a method and equipment for detecting anomalies in liquid level data of municipal drainage networks. Background Technology

[0002] With the continuous advancement of urbanization, the scale of municipal drainage pipe networks is constantly expanding, and the effective monitoring and intelligent management of their operational status has become an important research topic in the field of urban infrastructure operation and maintenance. Drainage pipe network liquid level data, as a key parameter reflecting the operational status of the network, directly relates to the fluid dynamics characteristics within the network, playing an irreplaceable role in assessing drainage capacity, predicting urban flooding risks, and ensuring urban water environment safety. Real-time acquisition of accurate liquid level monitoring data and the efficient detection of abnormal conditions are the core technological foundation for realizing intelligent operation and maintenance of drainage pipe networks, and have significant practical implications for preventing urban flooding, avoiding pipe blockages, and preventing sudden accidents such as sewage overflows.

[0003] Anomaly detection technology for drainage pipe network liquid level data aims to identify abnormal data points that deviate from normal operating conditions through the analysis and processing of monitoring data, thereby supporting the assessment and decision-making of pipe network operation status. Several existing methods exist for anomaly detection of liquid level data. Among them, the box plot method based on statistical principles determines outliers by setting a fixed threshold. This method is computationally simple but has poor adaptability to different pipe network conditions. Methods based on deep learning prediction models construct time-series prediction networks and use the degree of deviation between predicted and actual values ​​as anomaly criteria. This method has good detection performance in specific scenarios, but its applicability is limited and it is difficult to handle complex operating conditions.

[0004] Taking the intelligent analysis method for overflow early warning of drainage pipe network with publication number CN113902161A as an example, this scheme is based on box plots for outlier detection and uses a one-dimensional convolutional neural network to build a prediction model. However, this method has the following technical defects: First, the anomaly judgment rules based on fixed coefficient box plots lack the ability to adapt to different pipe network structures and different operating conditions, making it difficult to effectively cope with complex and ever-changing actual application scenarios; Second, the original design of this method focuses on the single scenario of overflow early warning, and lacks general detection capabilities for other types of liquid level anomalies such as sensor failure, data transmission errors, and pipe rupture; Third, the time window setting of using the data from the previous 2 hours to predict the liquid level for the next 1 hour has a significant delay in application scenarios with high real-time requirements, and cannot meet the actual needs of rapid response.

[0005] Therefore, existing technologies still have many shortcomings in terms of coverage of different types of anomalies, algorithm adaptability to complex working conditions, and real-time detection. There is an urgent need for a more comprehensive and efficient method for detecting anomalies in liquid level data. Summary of the Invention

[0006] The purpose of this invention is to overcome the above-mentioned deficiencies in the prior art and provide a method for detecting anomalies in municipal drainage network liquid level data.

[0007] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0008] A method for detecting anomalies in municipal drainage network liquid level data includes the following steps: S1: Real-time acquisition of liquid level data of the municipal drainage pipe network to be detected, and preprocessing of the liquid level data to generate liquid level time series data; the preprocessing includes missing value detection, format verification and sliding window filtering; S2: The liquid level time series data is decomposed into multi-scale features by discrete wavelet transform to obtain multi-scale feature vectors; S3: The multi-scale feature vector is judged by a multi-class support vector machine and a preset anomaly judgment threshold to obtain the liquid level anomaly judgment result; S4: Write the liquid level time series data, the feature values ​​of the corresponding multi-scale feature vector, and the corresponding liquid level anomaly determination results into the anomaly log database, and trigger the early warning mechanism corresponding to the liquid level anomaly determination results; the anomaly log database is used to adaptively update the anomaly determination threshold.

[0009] As a preferred embodiment of the present invention, the sampling frequency for real-time acquisition of liquid level data in S1 is a preset time interval, and the liquid level data includes liquid level height value, timestamp and sensor status code.

[0010] As a preferred embodiment of the present invention, the anomaly determination threshold in S3 includes a change amplitude threshold and a cumulative change threshold; the liquid level anomaly determination result includes no anomaly, abrupt anomaly, slow change anomaly, and missing anomaly; the judgment criteria are as follows: When the change amplitude of the high-frequency detail coefficients in the multi-scale feature vector corresponding to a single sampling point is greater than the preset change amplitude threshold, it is judged as a sudden anomaly. When the low-frequency approximation coefficients in the multi-scale feature vector corresponding to M consecutive sampling points show a monotonically increasing or monotonically decreasing trend, and the cumulative change is greater than the preset cumulative change threshold, it is judged as a slow-change anomaly; where M is the preset value. When the original liquid level data of K consecutive sampling points is zero or has an abnormal format, it is judged as a missing anomaly; where K is a preset value; Otherwise, it is judged as no abnormality.

[0011] As a preferred embodiment of the present invention, S3 further includes pipeline rupture detection; the pipeline rupture detection includes the following steps: When the liquid level anomaly determination result is a sudden anomaly; Check whether there is a slowly changing anomaly within a preset time range after the sampling point corresponding to the abrupt anomaly; If a slow-change anomaly is present, it is determined to be a pipeline rupture anomaly.

[0012] In a preferred embodiment of the present invention, the update period of the anomaly determination threshold is a preset time period, and the update of the anomaly determination threshold includes the following steps: Calculate the statistical parameters for the day based on the liquid level data from the previous time period; The anomaly detection threshold is updated based on the statistical parameters of the day and the seasonal adjustment coefficient to obtain the updated anomaly detection threshold. Its expression is: ; in, This represents the mean of the feature components corresponding to the multi-scale feature vector in the liquid level data. As a confidence factor, This represents the standard deviation of the feature components corresponding to the multi-scale feature vector in the liquid level data. This is a seasonal adjustment factor.

[0013] As a preferred embodiment of the present invention, the update expressions for the statistical parameters of the liquid level data in different feature components are as follows: ; ; in, express Time of the first The first multi-scale eigenvector The mean of the eigenvalues ​​of the dimension, The preset forgetting factor, express Time of the first The first multi-scale eigenvector 3D eigenvalues express Time of the first The first multi-scale eigenvector The variance of the eigenvalues.

[0014] As a preferred embodiment of the present invention, the seasonal adjustment coefficient is set to different values ​​in different seasons, with a first coefficient value set during the rainy season and flood season, and a second coefficient value set during the dry season and low water season; the first coefficient value is greater than the second coefficient value.

[0015] As a preferred embodiment of the present invention, the triggering rule of the early warning mechanism is as follows: When the liquid level anomaly determination result is a sudden anomaly, a level one warning is triggered and the operation and maintenance personnel are notified; When the liquid level anomaly determination result is a slow-change anomaly, a level-two warning is triggered and the operation and maintenance personnel are notified within a preset response time. When the liquid level anomaly determination result is a missing anomaly, a level 3 warning is triggered.

[0016] As a preferred embodiment of the present invention, S4 further includes performing a quality score on the sensor data using data stored in the anomaly log database; the sensor data quality score The formula for calculation is:

[0017] in, This represents the normal percentage of data. This represents the frequency of anomaly detection. As a data continuity indicator, As a data quality consistency indicator, , , and The preset weighting coefficients satisfy... ; When the score is lower than the preset score threshold, the corresponding sensor will be marked as a suspected faulty sensor, and the sensor calibration or replacement process will be automatically triggered, generating the corresponding maintenance work order.

[0018] An anomaly detection device for municipal drainage network liquid level data includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform any of the above-described methods for anomaly detection of municipal drainage network liquid level data.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention employs multi-scale feature extraction, significantly reducing computational complexity while ensuring effective feature extraction. It utilizes a multi-class support vector machine for refined classification of liquid level anomalies, enabling comprehensive and efficient anomaly detection. Compared to existing single-type detection methods, the detection range is significantly expanded, effectively identifying various anomalies such as sensor malfunctions, data transmission errors, and pipeline ruptures. Finally, this invention forms a complete anomaly management closed loop from data acquisition, anomaly detection, classification, and early warning output. The anomaly log database provides data support for subsequent pipeline health assessments and maintenance decisions. Attached Figure Description

[0020] Figure 1This is a flowchart illustrating an anomaly detection method for municipal drainage network liquid level data as described in Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of the structure of an anomaly detection device for municipal drainage network liquid level data as described in Embodiment 4 of the present invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to experimental examples and specific embodiments. However, this should not be construed as limiting the scope of the above-mentioned subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0022] Example 1 like Figure 1 As shown, a method for detecting anomalies in municipal drainage network liquid level data includes the following steps: S1: Real-time acquisition of liquid level data of the municipal drainage network to be detected, and preprocessing of the liquid level data to generate liquid level time series data; the preprocessing includes missing value detection, format verification and sliding window filtering.

[0023] S2: The liquid level time series data is decomposed into multi-scale features by discrete wavelet transform to obtain multi-scale feature vectors.

[0024] S3: The multi-scale feature vector is judged by a multi-class support vector machine and a preset anomaly judgment threshold to obtain the liquid level anomaly judgment result, and the anomaly judgment threshold is adaptively updated according to the liquid level anomaly judgment result and the anomaly log database.

[0025] S4: Write the liquid level time series data, the feature values ​​of the corresponding multi-scale feature vector, and the corresponding liquid level anomaly determination results into the anomaly log database, and trigger the early warning mechanism corresponding to the liquid level anomaly determination results; the anomaly log database is used to adaptively update the anomaly determination threshold.

[0026] Example 2 This embodiment is a specific implementation of the method for detecting anomalies in municipal drainage network liquid level data described in Embodiment 1, including the following steps: S1: Real-time acquisition of liquid level data of the municipal drainage network to be detected, and preprocessing of the liquid level data to generate liquid level time series data; the preprocessing includes missing value detection, format verification and sliding window filtering.

[0027] Furthermore, in this embodiment, liquid level data is collected in real time by a liquid level sensor in the municipal drainage network; the sampling frequency for real-time acquisition of liquid level data is a preset time interval, and the liquid level data includes liquid level height value, timestamp, and sensor status code.

[0028] This embodiment obtains high-quality, stable, and reliable liquid level time-series data through this step, laying the data foundation for subsequent feature extraction and anomaly detection. Preferably, the liquid level sensor is a pressure-type or ultrasonic sensor with high-precision measurement capabilities, a measurement accuracy better than ±0.1% of full scale, and a response time of less than 100 milliseconds. The sensor establishes a connection with the data acquisition terminal through a wired communication interface or a wireless communication module, and the acquisition frequency is set to a preset time interval, such as acquiring liquid level data once every 30 seconds. Each acquired data record includes three core fields: liquid level height value, timestamp, and sensor status code. The liquid level height value is digitally represented in millimeters, the timestamp is accurate to the millisecond level, and the sensor status code is used to identify the current working status of the sensor, including status types such as normal, communication interruption, power supply abnormality, and probe obstruction.

[0029] When preprocessing liquid level data, a missing value detection operation is first performed to check whether the time interval between two consecutive timestamps exceeds a preset acquisition interval threshold. If it does, data is considered missing. For the detected missing locations, a linear interpolation method is used to fill in the data. Specifically, the arithmetic mean of the two valid data points before and after the missing location is taken as the filler value. If the number of missing data points exceeds the preset maximum allowable number (e.g., no data for 10 consecutive acquisition cycles), the data segment is marked as a long-term missing area and is directly passed to the subsequent processing stage without filling. The format validation operation checks the field integrity, data type, and numerical range of each data record to ensure that the liquid level height value is within a reasonable range (usually 0 to the pipe diameter height), the timestamp format conforms to the ISO 8601 standard, and the sensor status code is a predefined valid enumeration value.

[0030] After missing value detection and format verification, a sliding window filtering algorithm is used to remove noise interference from the data. The implementation mechanism of the sliding window filtering algorithm is as follows: The window length L is set to a preset number of sampling points (e.g., L is 5 or 7). The window slides sequentially on the time axis, and the median or weighted average of the L data points within the window is calculated as the filtered output for the current moment. The setting of the filtering coefficient affects the balance between filtering effect and real-time performance. The filtering coefficient α is a real number between 0 and 1. The larger the α value, the stronger the ability of the filtered output to track the original data; the smaller the α value, the more obvious the smoothing effect of the filtering. In a typical configuration, α is set to 0.3, which effectively suppresses high-frequency noise components while preserving the trend changes of the liquid level data. The liquid level time series data after sliding window filtering is output to S2 for further processing.

[0031] S2: The liquid level time series data is decomposed into multi-scale features by discrete wavelet transform to obtain multi-scale feature vectors.

[0032] This embodiment performs multi-scale decomposition on the preprocessed liquid level time series data. Discrete wavelet transform is used to extract liquid level time series data at different scales, constructing multi-scale feature vectors that characterize the essential properties of the liquid level data. The core advantage of discrete wavelet transform lies in its ability to provide good localization characteristics in both the time and frequency domains, making it suitable for analyzing non-stationary liquid level time series data. Pre-defined wavelet basis functions are selected, with preference given to the Daubechies wavelet family, which possesses tight support and approximate symmetry, such as db4 or db6 wavelets, whose vanishing moment order meets the requirements for feature extraction from liquid level time series data. The number of decomposition layers is set to a preset number. For typical drainage network liquid level monitoring scenarios, the number of decomposition layers is usually set to 3 or 4. Too few decomposition layers may lead to insufficient feature extraction, while too many decomposition layers will increase computational complexity and may introduce boundary effects. The specific decomposition process is as follows: The liquid level time series data undergoes a first-level decomposition. The original signal is filtered through a low-pass filter and a high-pass filter to obtain the first-level low-frequency approximation coefficients and the first-level high-frequency detail coefficients, respectively. The low-frequency approximation coefficients contain the main trend information of the signal, while the high-frequency detail coefficients contain abrupt changes and edge information. A second-level decomposition is then performed on the first-level low-frequency approximation coefficients, repeating the filtering and downsampling operations to obtain the second-level low-frequency approximation coefficients and the second-level high-frequency detail coefficients. A third-level decomposition is then performed on the second-level low-frequency approximation coefficients to obtain the third-level low-frequency approximation coefficients and the third-level high-frequency detail coefficients. If the decomposition level is set to four levels, a fourth-level decomposition is performed on the third-level low-frequency approximation coefficients. The downsampling operation follows the Dyadic rule, halving the signal length with each decomposition level.

[0033] Furthermore, the construction of the multi-scale feature vector is based on the energy value calculation of the coefficients obtained from each layer of decomposition. For the high-frequency detail coefficients obtained from each layer of decomposition, their energy value is calculated as a feature component, and the calculation formula is the square root of the sum of squares of the coefficient sequence of that layer. At the same time, the energy value of the low-frequency approximation coefficient is extracted as a trend feature. For a 3-layer decomposition, a total of 3 high-frequency detail coefficient energy features and 1 low-frequency approximation coefficient energy feature can be obtained. Adding the energy value of the original signal, a 4-dimensional or more dimensional feature vector is formed. In addition, the following statistical features can be extracted to enhance the representational ability of the feature vector: the mean, variance, kurtosis, skewness of the coefficient sequence of each layer, and the ratio of the energy values ​​of coefficients at adjacent scales. The constructed multi-dimensional multi-scale feature vector is used as the input of the adaptive threshold anomaly detection model in step 3.

[0034] S3: The multi-scale feature vector is judged by a multi-class support vector machine and a preset anomaly judgment threshold to obtain the liquid level anomaly judgment result, and the anomaly judgment threshold is adaptively updated according to the liquid level anomaly judgment result and the anomaly log database.

[0035] Furthermore, the anomaly detection threshold includes a change amplitude threshold and a cumulative change threshold. The liquid level anomaly detection results include no anomaly, abrupt anomaly, gradual change anomaly, and missing anomaly; the judgment criteria include: (1) Abrupt anomaly: When the change amplitude of the high-frequency detail coefficient in the multi-scale feature vector corresponding to a single sampling point is greater than the preset change amplitude threshold, it is determined to be an abrupt anomaly.

[0036] Specifically, when the absolute difference between the high-frequency detail coefficients in the multi-scale feature vector corresponding to the liquid level value at a certain sampling moment and the high-frequency detail coefficients in the multi-scale feature vector corresponding to the liquid level value at the previous moment exceeds a preset change amplitude threshold, that point is determined to be a sudden anomaly. The preset change amplitude threshold is calculated based on the anomaly judgment threshold, and is typically 3 to 5 times the standard deviation of the high-frequency detail coefficients in the multi-scale feature vector corresponding to the liquid level data. Sudden anomalies are usually caused by instantaneous sensor failure, signal transmission interference, or sudden pipeline damage, and manifest as peaks or dips on the liquid level curve.

[0037] (2) Slowly changing anomaly: When the low-frequency approximation coefficients in the multi-scale feature vectors corresponding to M consecutive sampling points show a monotonically increasing or monotonically decreasing trend, and the cumulative change is greater than the preset cumulative change threshold, it is judged as a slowly changing anomaly; where M is the preset value.

[0038] Specifically, the system continuously monitors the direction of change of low-frequency approximation coefficients in the multi-scale feature vectors corresponding to the liquid level data from the most recent N sampling points. If at least M points (M < N) maintain the same direction of change (e.g., continuous rise or continuous fall), and the cumulative change exceeds the cumulative change threshold, it is identified as a slow-change anomaly. Typical parameter configurations include N ranging from 12 to 24 sampling points, M being more than 80% of N, and the cumulative change threshold being set to 10% to 20% of the average liquid level. Slow-change anomalies are usually caused by slow pipeline leakage, increased siltation, or abnormal pump station operation, manifesting as a gradual deviation of the liquid level curve.

[0039] (3) Missing anomaly: When the original liquid level data of K consecutive sampling points is zero or there is a format anomaly, it is determined to be a missing anomaly; where K is a preset value.

[0040] Specifically, when the liquid level data at K consecutive sampling points is zero or at the lower limit of the equipment's range, and the sensor status code does not indicate equipment failure, a potential data acquisition anomaly is identified. If the detected data format does not conform to predefined specifications (e.g., duplicate timestamps, excessive liquid level values, incorrect data types, etc.), it is also marked as a missing data anomaly. In a typical configuration, K is taken as 3 to 5 consecutive sampling periods. Missing data anomalies are usually caused by sensor probe blockage, communication link interruption, or data acquisition software failure.

[0041] (4) No abnormality: Otherwise, it is judged as no abnormality.

[0042] Furthermore, in this embodiment, the multi-class support vector machine employs a one-to-one strategy to construct multiple binary classifiers: a classifier for mutated anomalies and normal classes, a classifier for slowly changing anomalies and normal classes, and a classifier for missing anomalies and normal classes. Each binary classifier uses a radial basis function kernel as its kernel function, with the kernel function parameters set to preset values. The kernel width and penalty coefficient are optimized using grid search and cross-validation methods. The kernel width can range from 0.01 to 1, and the penalty coefficient can range from 1 to 100. The advantage of the one-to-one strategy is that each classifier only needs to distinguish between two classes of samples, resulting in higher classification accuracy and faster training speed.

[0043] In this embodiment, the anomaly detection threshold is adaptively updated based on the liquid level anomaly determination result and the anomaly log database. Furthermore, an online learning algorithm is used to update the threshold parameters in real time, enabling adaptive anomaly detection for different pipeline networks and operating conditions. The dynamic threshold update strategy includes two core mechanisms: periodic statistical parameter recalculation and seasonal adjustment coefficient correction.

[0044] Furthermore, the update cycle of the anomaly determination threshold is a preset time period, and the initial value of the anomaly determination threshold is calculated based on initial statistical parameters. The update of the anomaly determination threshold includes the following steps: (1) Calculate the statistical parameters for the day based on the liquid level data of the previous time period; For example, at midnight each day, the statistical parameters for that day are recalculated based on liquid level data from a preset historical period. This period is typically set to the past 7 or 30 days, with the specific value determined based on the pipeline network size and the amount of historical data. The statistical parameters include the mean of the feature components corresponding to the multi-scale feature vectors in the liquid level data. and standard deviation These statistical parameters are used to calculate the base values ​​for anomaly detection thresholds, such as setting anomaly detection thresholds based on the 3σ principle.

[0045] (2) Update the anomaly determination threshold according to the statistical parameters of the day and the seasonal adjustment coefficient to obtain the updated anomaly determination threshold.

[0046] Furthermore, the seasonal adjustment coefficient is set to different values ​​in different seasons, with the first coefficient value set during the rainy season and flood season, and the second coefficient value set during the dry season and low water season; the first coefficient value is greater than the second coefficient value.

[0047] This embodiment considers the periodic impact of external factors such as rainfall and temperature on liquid level data by introducing a seasonal adjustment coefficient. The baseline value and fluctuation range of liquid level differ significantly across seasons; liquid level fluctuations are larger during the rainy summer season, while the liquid level is relatively stable during the dry winter season. The base threshold is adjusted according to the season type (spring, summer, autumn, winter) of the current date, using a seasonal adjustment coefficient. The coefficient is set to a value greater than 1, taking a larger value (e.g., 1.2 to 1.5) during the rainy and flood seasons, and a smaller value (e.g., 1.0 to 1.1) during the dry season and low-water seasons. The updated threshold expression can be represented as: ; in, The confidence factor is typically set between 2.5 and 3.5 and is adjusted based on actual detection results.

[0048] The online learning algorithm uses an exponentially weighted moving average method to update the threshold in real time. The forgetting factor assigns higher weight to recent data while gradually attenuating the influence of older data. Let the current time be... Then, the formula for updating the statistical distribution parameters of this feature based on the exponentially weighted moving average is: ; ; in, This is a preset forgetting factor, with a value ranging from 0 to 1, typically between 0.95 and 0.99. express Time of the first The first multi-scale eigenvector 3D eigenvalues express Time of the first The first multi-scale eigenvector The mean of the eigenvalues ​​of the dimension, express Time of the first The first multi-scale eigenvector The variance of the eigenvalues.

[0049] Through the above recursive formula, the parameter of the anomaly detection threshold can continuously track the statistical distribution changes of liquid level data and adapt to changes in the pipeline network's operating status. When the pipeline network's operating status undergoes gradual changes (e.g., a slow rise in the liquid level baseline due to pipeline siltation), the anomaly detection threshold can be dynamically adjusted accordingly, avoiding missed or false detections caused by a fixed threshold.

[0050] Furthermore, the abnormal liquid level determination result also includes pipeline rupture determination; the pipeline rupture determination includes the following steps: When the liquid level anomaly determination result is a sudden anomaly; Check whether there is a slowly changing anomaly within a preset time range after the sampling point corresponding to the abrupt anomaly; If a slow-change anomaly is present, it is determined to be a pipeline rupture anomaly.

[0051] In this embodiment, pipeline rupture detection is achieved through post-processing of the classification results from a multi-class support vector machine. Specifically, upon detecting a sudden anomaly, the system automatically checks whether a slowly changing anomaly exists within a preset time range (e.g., 30 minutes to 2 hours) after the anomaly. If slowly changing anomaly characteristics are continuously detected for a period of time after the sudden anomaly occurs, the two are combined and classified as a pipeline rupture anomaly, because pipeline rupture typically manifests as an initial sudden large-scale leakage (sudden anomaly) followed by continuous leakage (slowly changing anomaly). This post-processing mechanism significantly improves the detection accuracy of composite anomaly types.

[0052] S4: Write the liquid level time series data, the feature values ​​of the corresponding multi-scale feature vector, and the corresponding liquid level anomaly determination results into the anomaly log database, and trigger the early warning mechanism corresponding to the liquid level anomaly determination results; the anomaly log database is used to adaptively update the anomaly determination threshold.

[0053] Furthermore, the triggering rules for the early warning mechanism are as follows: When the liquid level anomaly determination result is a sudden anomaly, a level one warning is triggered and the operation and maintenance personnel are notified; When the liquid level anomaly determination result is a slow-change anomaly, a level-two warning is triggered and the operation and maintenance personnel are notified within a preset response time. When the liquid level anomaly determination result is a missing anomaly, a level 3 warning is triggered.

[0054] In this embodiment, the liquid level time-series data, the feature values ​​of the corresponding multi-scale feature vectors, and the corresponding liquid level anomaly determination results are written into the anomaly log database, triggering the corresponding early warning mechanism. The anomaly log database is stored using a time-series database, which is optimized for high write rates and time-series queries of time-series data, enabling efficient storage and retrieval of a large number of anomaly records. A single record contains fields including anomaly number, timestamp, liquid level sensor number, anomaly type, anomaly feature value, confidence level, and processing status. The anomaly number uses a UUID (Universally Unique Identifier) ​​format to ensure global uniqueness. A UUID consists of 128 digits in a hexadecimal string formatted as 8-4-4-4-12, for example, 550e8400-e29b-41d4-a716-446655440000. The timestamp is accurate to the millisecond level. The liquid level sensor number uniquely identifies the sensor device that detected the anomaly. The anomaly type is stored in the form of an enumeration value (e.g., 0 represents normal, 1 represents sudden anomaly, 2 represents gradual anomaly, and 3 represents missing anomaly). The anomaly feature value records the key components of the multi-scale feature vector corresponding to the anomaly. The confidence level is a floating-point number between 0 and 1, representing the degree of confidence of the classification model in the judgment result. The processing status records whether the anomaly has been confirmed and processed by the operation and maintenance personnel.

[0055] The early warning mechanism triggers different levels of warning signals based on the type of anomaly. Sudden-onset anomalies trigger a Level 1 warning, which is the highest priority, immediately notifying maintenance personnel via SMS, telephone, or mobile application push notifications, requiring immediate on-site verification. Gradually changing anomalies trigger a Level 2 warning, which is a medium priority, notifying maintenance personnel via email or instant messaging tools within a preset response time (e.g., 30 minutes), requiring them to complete the investigation within the specified time. Missing anomalies trigger a Level 3 warning, which is a lower priority, generating a work order and assigning it to relevant maintenance personnel for sensor equipment inspection. The warning message includes the anomaly type, occurrence time, sensor location, and preliminary analysis suggestions to facilitate rapid response from maintenance personnel.

[0056] Based on historical records in the anomaly log database, the frequency, duration, and spatial distribution characteristics of various liquid level anomalies are statistically analyzed across daily, weekly, and monthly time dimensions. Daily reports reflect the occurrence of liquid level anomalies within a single day, used for routine operation and maintenance monitoring; weekly reports reflect the periodic patterns of liquid level anomalies, used for operation and maintenance scheduling; and monthly reports reflect the overall trend of the pipeline network's operational status, used for management decision support. Spatial distribution analysis maps the locations of liquid level anomalies onto the pipeline network topology map, identifying high-incidence and low-incidence areas of liquid level anomalies, providing data support for pipeline network renovation and optimization. The generated liquid level anomaly statistical reports include numerical statistics, chart visualizations, and analytical conclusions, output in a standardized format for relevant personnel to review.

[0057] Furthermore, health assessment of the level sensor is a crucial step in ensuring reliable anomaly detection. A sensor data quality score is calculated based on data stored in the anomaly log database. The sensor data quality score calculation comprehensively considers indicators such as the proportion of normal data, the number of anomaly detections, the distribution of anomaly types, and data continuity within the most recent statistical period. The sensor data quality score formula uses a weighted summation method, and the weights of each indicator are adjusted according to the actual application scenario. The calculation formula is:

[0058] in, This represents the normal percentage of data. This represents the frequency of anomaly detection. As a data continuity indicator, As a data quality consistency indicator, , , and The preset weighting coefficients satisfy... When the score is lower than the preset score threshold (e.g., 0.6), the sensor is marked as a suspected faulty sensor, and the sensor calibration or replacement process is automatically triggered, generating the corresponding maintenance work order.

[0059] This invention employs a multi-class support vector machine for refined classification of liquid level anomalies, simultaneously detecting three types: abrupt anomalies, gradual anomalies, and missing anomalies. Compared to existing single-type detection methods, the detection range is significantly expanded, effectively identifying various anomalies such as sensor malfunctions, data transmission errors, and pipeline ruptures. Furthermore, by adopting a one-to-one strategy and optimized kernel function parameters, it improves inference speed while maintaining classification accuracy, meeting the needs of large-scale concurrent sensor processing. Through intelligent post-processing mechanism for correlation analysis of different anomaly types, it can identify composite anomalies such as pipeline ruptures, improving the accuracy of anomaly determination. This invention also achieves automatic updating and adaptive adjustment of anomaly determination thresholds through dynamic threshold calculation and online learning algorithms, avoiding the problem of poor adaptability of fixed thresholds to different pipe networks and operating conditions. It can adjust detection parameters in real time according to changes in the pipe network's operating status. Moreover, a seasonal adjustment coefficient is introduced to consider the impact of external factors such as rainfall on liquid level data, further enhancing the rationality of the thresholds. It also employs discrete wavelet transform for multi-scale feature extraction, which has low computational complexity and good feature extraction results; a complete anomaly management closed loop is formed from data acquisition, anomaly detection, classification and judgment to early warning output, and the anomaly log database provides data support for subsequent pipeline health assessment and operation and maintenance decisions.

[0060] Example 3 This embodiment represents multiple implementations of the method for detecting anomalies in municipal drainage network liquid level data described in Embodiment 2, as detailed below: In one embodiment, the method of the present invention is applied to an intelligent monitoring platform for municipal drainage pipe networks, supporting concurrent data processing from a preset number of liquid level sensors. The platform can adopt a distributed architecture design, with the data acquisition layer, feature extraction layer, anomaly detection layer, and early warning output layer deployed on different computing nodes, and data transmission between layers achieved through message queues. The anomaly detection latency of a single sensor is less than a preset time threshold (e.g., 500 milliseconds), and the entire process latency from data acquisition to anomaly output is less than 2 seconds, meeting real-time monitoring requirements. The platform supports horizontal scaling; as the number of connected sensors increases, processing capacity can be linearly increased by adding computing nodes.

[0061] A specific application example is illustrated using a city's drainage network. This network is equipped with 200 level sensors, covering the main drainage pipes and flood-prone areas of the city. The sensor sampling frequency is set to once per minute, with a sampling interval of 60 seconds. The sliding window filter has a window length of 5 sampling points, and the filtering coefficient α is set to 0.3. The discrete wavelet transform uses the db4 wavelet basis function, with a decomposition level of 3. The dynamic threshold is updated daily at 2:00 AM. Statistical parameters are calculated based on historical data from the past 30 days, and the seasonal adjustment coefficient is set to 1.3 during the rainy season and 1.05 during the dry season. The forgetting factor for the online learning algorithm is set to 0.97. The multi-class support vector machine uses a one-to-one strategy, with the radial basis kernel width set to 0.05 and the penalty coefficient set to 10. The threshold for identifying abrupt anomalies is four times the standard deviation of the liquid level. For slowly changing anomalies, the threshold parameters N are 12 sampling points, M is 10 points, and the cumulative change threshold is 15% of the average liquid level. For missing anomalies, the threshold parameter K is 3 consecutive sampling periods. The confidence threshold is set to 0.75; detection results below this threshold will not trigger an alert.

[0062] During actual operation, at 14:23:15 on a certain day, a sudden change in the liquid level value of sensor No. 3 was detected, with a single-point change of 0.45 meters, exceeding the sudden change threshold of 0.38 meters, and was judged as a sudden anomaly with a confidence level of 0.89. This immediately triggered a Level 1 alert, which was pushed to the mobile devices of maintenance personnel. Maintenance personnel arrived at the site at 14:35 to verify and found that the pipeline did indeed have a sudden leak caused by a disconnected interface. Continuous monitoring of the sensor data showed that the liquid level showed a continuous and slow downward trend over the next 30 minutes, consistent with the characteristics of a gradual change anomaly. The post-processing module merged the sudden anomaly and the gradual change anomaly, classifying it as a pipeline rupture anomaly, and generated a complete anomaly event report. This report includes the time sequence of the anomaly occurrence, anomaly characteristic data, on-site handling records, and subsequent maintenance recommendations, providing complete data support for pipeline repair and preventative maintenance.

[0063] In another implementation, this embodiment optimizes parameter configuration for specific application scenarios, making it suitable for scenarios involving high-density sensor networks and edge computing node deployments.

[0064] Specifically, in high-density sensor network scenarios, a single monitoring area may deploy dozens or even hundreds of liquid level sensors, resulting in a high data acquisition frequency (e.g., once every 10 seconds) and a massive amount of data. To address this scenario, this invention optimizes the following parameters: the data acquisition frequency is increased to once every 10 seconds, and the window length of the sliding window filter is adjusted accordingly to 10 sampling points to maintain the same temporal coverage of the filter window. The number of decomposition layers in the discrete wavelet transform remains at 3, but the more computationally efficient Haar wavelet is used instead of the Daubechies wavelet, significantly reducing computational complexity within an acceptable range of feature extraction accuracy loss.

[0065] The update cycle for the anomaly detection threshold has been shortened to once every 6 hours, and statistical parameter calculations are based on data from the past 7 days to adapt to the rapid changes in pipeline network status under high-density sensor scenarios. The forgetting factor of the online learning algorithm has been adjusted to 0.99 to increase the weight of recent data, enabling the threshold to track pipeline network status changes more quickly. The multi-class support vector machine adopts an incremental learning method instead of a batch learning method, updating model parameters in real time when new data arrives instead of periodically retraining the entire dataset.

[0066] The edge computing node deployment configuration is optimized for edge devices with limited computing and storage resources. Edge nodes are deployed in racks at the monitoring site, responsible for data acquisition and preliminary processing from all sensors within the area, uploading only detected anomaly data and statistical features to the cloud server. The anomaly detection algorithm running on the edge nodes adopts a lightweight design: sliding window filtering is replaced with simple moving average filtering, discrete wavelet transform is replaced with single-level decomposition to reduce computational load, and multi-class support vector machines are replaced with rule-based classifiers to reduce model storage space. The single detection latency of the lightweight algorithm running on the edge node is controlled within 50 milliseconds, meeting real-time requirements. The cloud server receives the anomaly data and statistical features uploaded by the edge nodes and performs global anomaly correlation analysis and pipeline health status assessment. When an edge node detects a suspected faulty sensor, the cloud server comprehensively analyzes data from adjacent sensors to confirm the sensor fault determination, avoiding misjudgments caused by measurement deviations of a single sensor. The parameter optimization configuration of this embodiment can adapt to the needs of high-density sensor networks and edge computing scenarios, achieving efficient utilization of computing resources and network bandwidth while ensuring the accuracy of anomaly detection.

[0067] Example 4 like Figure 2As shown, an anomaly detection device for municipal drainage network liquid level data includes at least one processor, a memory communicatively connected to the at least one processor, and at least one input / output interface communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enables the at least one processor to perform the anomaly detection method for municipal drainage network liquid level data described in the foregoing embodiments. The input / output interface may include a display, keyboard, mouse, and USB interface for inputting and outputting data.

[0068] Furthermore, the anomaly detection device for municipal drainage network liquid level data can be a desktop computer, mobile phone, tablet computer, wearable municipal drainage network liquid level data anomaly detection device, or any device capable of deep information recognition.

[0069] Furthermore, the processor may include one or more processing cores. The processor connects to various parts of the anomaly detection device for municipal drainage network liquid level data using various interfaces and lines. It executes various functions and processes data within the device by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, the processor may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate one or more of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also be implemented separately as a communication chip, without being integrated into the processor.

[0070] The memory may include random access memory (RAM) or read-only memory (ROM). The memory can be used to store instructions, programs, code, code sets, or instruction sets, such as instructions or code sets used to implement an anomaly detection method for municipal drainage network liquid level data provided in this application embodiment. The memory may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described above, etc. The data storage area may also store data created during the use of the municipal drainage network liquid level data anomaly detection device (such as a modulation sequence-depth mapping table, image data, spectrogram data, etc.).

[0071] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory, magnetic disks, or optical disks.

[0072] When the integrated units of the present invention are implemented as software functional units and sold or used as independent products, they can also be stored in a computer-readable storage medium. The computer-readable storage medium stores program code, which can be called by a processor to execute the methods described in the above method embodiments. Based on this understanding, the technical solution of the embodiments of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes electronic memories such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for program code that executes any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code can be compressed, for example, in an appropriate form.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting anomalies in liquid level data of municipal drainage pipe networks, characterized in that, Includes the following steps: S1: Real-time acquisition of liquid level data of the municipal drainage pipe network to be detected, and preprocessing of the liquid level data to generate liquid level time series data; the preprocessing includes missing value detection, format verification and sliding window filtering; S2: The liquid level time series data is decomposed into multi-scale features by discrete wavelet transform to obtain multi-scale feature vectors; S3: The multi-scale feature vector is judged by a multi-class support vector machine and a preset anomaly judgment threshold to obtain the liquid level anomaly judgment result; S4: Write the liquid level time series data, the feature values ​​of the corresponding multi-scale feature vector, and the corresponding liquid level anomaly determination results into the anomaly log database, and trigger the early warning mechanism corresponding to the liquid level anomaly determination results; the anomaly log database is used to adaptively update the anomaly determination threshold.

2. The method for detecting anomalies in municipal drainage network liquid level data according to claim 1, characterized in that, The sampling frequency for real-time acquisition of liquid level data in S1 is a preset time interval, and the liquid level data includes liquid level height value, timestamp and sensor status code.

3. The method for detecting anomalies in municipal drainage network liquid level data according to claim 2, characterized in that, The anomaly determination threshold in S3 includes a change amplitude threshold and a cumulative change threshold; the liquid level anomaly determination result includes no anomaly, abrupt anomaly, slow change anomaly, and missing anomaly; The criteria for judgment are: When the change amplitude of the high-frequency detail coefficients in the multi-scale feature vector corresponding to a single sampling point is greater than the preset change amplitude threshold, it is determined to be a sudden anomaly. When the low-frequency approximation coefficients in the multi-scale feature vector corresponding to M consecutive sampling points show a monotonically increasing or monotonically decreasing trend, and the cumulative change is greater than the preset cumulative change threshold, it is judged as a slow-change anomaly; where M is the preset value. When the original liquid level data of K consecutive sampling points is zero or has an abnormal format, it is judged as a missing anomaly; where K is a preset value; Otherwise, it is judged as no abnormality.

4. The method for detecting anomalies in municipal drainage network liquid level data according to claim 3, characterized in that, S3 further includes pipeline rupture detection; the pipeline rupture detection includes the following steps: When the liquid level anomaly determination result is a sudden anomaly; Check whether there is a slowly changing anomaly within a preset time range after the sampling point corresponding to the abrupt anomaly; If a slow-change anomaly is present, it is determined to be a pipeline rupture anomaly.

5. The method for detecting anomalies in municipal drainage network liquid level data according to claim 3, characterized in that, The update cycle for the anomaly detection threshold is a preset time period, and the update of the anomaly detection threshold includes the following steps: Calculate the statistical parameters for the day based on the liquid level data from the previous time period; The anomaly detection threshold is updated based on the statistical parameters of the day and the seasonal adjustment coefficient to obtain the updated anomaly detection threshold. Its expression is: ； in, This represents the mean of the feature components corresponding to the multi-scale feature vector in the liquid level data. As the confidence factor, This represents the standard deviation of the feature components corresponding to the multi-scale feature vector in the liquid level data. This is a seasonal adjustment factor.

6. The method for detecting anomalies in municipal drainage network liquid level data according to claim 5, characterized in that, The update expressions for the statistical parameters of the liquid level data in different feature components are as follows: ； ； in, express Time of the first The first multi-scale eigenvector The mean of the eigenvalues ​​of the dimension, The preset forgetting factor, express Time of the first The first multi-scale eigenvector 3D eigenvalues express Time of the first The first multi-scale eigenvector The variance of the eigenvalues.

7. The method for detecting anomalies in municipal drainage network liquid level data according to claim 6, characterized in that, The seasonal adjustment coefficient is set to different values ​​in different seasons. It is set to a first coefficient value during the rainy season and flood season, and a second coefficient value during the dry season and low water season. The first coefficient value is greater than the second coefficient value.

8. The method for detecting anomalies in municipal drainage network liquid level data according to claim 1, characterized in that, The triggering rules for the aforementioned early warning mechanism are as follows: When the liquid level anomaly determination result is a sudden anomaly, a level one warning is triggered and the operation and maintenance personnel are notified; When the liquid level anomaly determination result is a slow-change anomaly, a level-two warning is triggered and the operation and maintenance personnel are notified within a preset response time. When the liquid level anomaly determination result is a missing anomaly, a level 3 warning is triggered.

9. The method for detecting anomalies in municipal drainage network liquid level data according to claim 1, characterized in that, S4 further includes performing a quality score on the sensor data using data stored in the anomaly log database; the sensor data quality score The formula for calculation is: in, This represents the normal percentage of data. This represents the frequency of anomaly detection. As a data continuity indicator, As a data quality consistency indicator, , , and The preset weighting coefficients satisfy... ; When the score is lower than the preset score threshold, the corresponding sensor will be marked as a suspected faulty sensor, and the sensor calibration or replacement process will be automatically triggered, generating the corresponding maintenance work order.

10. An anomaly detection device for liquid level data in municipal drainage pipe networks, characterized in that, The device includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform an anomaly detection method for liquid level data in a municipal drainage network according to any one of claims 1 to 9.

Images