Anomaly Duration Clustering for Enduring Event Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection systems fail to efficiently identify and address enduring anomalies, which persist over extended periods and require attention, often overshadowed by transient anomalies.
Innovation Solution
A method involving obtaining time series of anomaly score data, detecting segments with predefined criteria, determining anomaly durations and scores, and performing clustering to identify the smallest clusters, which likely contain enduring anomalies, allowing for targeted adjustments or outputs to manage these anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If anomaly detection is performed on all detected anomalies, then comprehensive anomaly identification is achieved, but computational resources and processing time are wasted on transient anomalies that do not require attention
Solution Approach 1:
The patent segments anomalies into different categories based on duration characteristics. By dividing the anomaly detection task into transient anomaly filtering and enduring anomaly analysis, the system processes only relevant anomalies in detail, improving both accuracy and efficiency.
Solution Approach 2:
The patent applies different processing qualities to different anomaly types. Enduring anomalies receive comprehensive analysis and attention, while transient anomalies are filtered out with minimal processing. This local differentiation of processing quality resolves the contradiction between comprehensive detection and efficient resource usage.
2Reliability
If all anomaly events are investigated in detail, then no enduring anomalies are missed, but the system becomes overwhelmed by transient anomalies that do not require corrective action
Solution Approach 1:
The patent extracts enduring anomalies from the mixture of all anomaly events by using duration as a discriminating feature. Through clustering analysis on anomaly duration and score, the system separates enduring anomalies (which require attention) from transient anomalies (which do not), simplifying the investigation process.
Solution Approach 2:
The patent introduces anomaly duration as an additional dimension for analysis alongside anomaly score. By clustering in the two-dimensional space of (duration, score), the system effectively separates different anomaly types, reducing the complexity of identifying which anomalies require detailed investigation.
3Speed
If anomaly detection focuses only on short-term anomalies, then rapid response to immediate issues is achieved, but enduring anomalies that require corrective intervention are overlooked
Solution Approach 1:
The patent performs preliminary filtering and clustering analysis to identify enduring anomalies before detailed investigation. By pre-separating enduring anomalies based on duration characteristics, the system ensures they are not overlooked while maintaining rapid response capability for immediate issues.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
According to an aspect, there is provided a computer- implemented method comprising the following. Initially, information on a plurality of anomaly events relating to operation of a target system is obtained. The information comprises one or more time series of anomaly score data. Segments satisfying one or more pre-defined criteria for anomalous operation are detected from the one or more time series. The one or more pre-defined criteria are defined to exclude fully non-anomalous anomaly score data. Anomaly durations and standardized anomaly scores are determined for the segments. Partition or density based clustering is performed in a two-dimensional space formed by the standardized anomaly scores and the anomaly durations to form n clusters, and m smallest clusters of the n clusters are identified. At least one of the following is performed: outputting information on the m smallest clusters or causing adjusting of operation of the target system based on the m smallest clusters.