Anomaly Score Clustering for Enduring Anomaly Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection systems fail to efficiently identify enduring anomalies, which persist over extended periods and require attention, often overshadowed by transient anomalies.
Innovation Solution
A method involving obtaining time series of anomaly score data, detecting segments with predefined criteria, determining anomaly durations and scores, performing clustering in a two-dimensional space, and identifying the smallest clusters to focus on enduring anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If anomaly detection is performed on all detected anomaly events, then comprehensive anomaly identification is achieved, but computational complexity increases and enduring anomalies are overshadowed by transient anomalies
Solution Approach 1:
The patent segments anomaly events into two distinct categories: transient anomalies and enduring anomalies, based on their temporal characteristics. This segmentation allows the system to apply different processing strategies to each type, reducing overall computational complexity while maintaining detection accuracy for both categories.
Solution Approach 2:
The patent introduces a temporal dimension (anomaly duration) to the anomaly detection process by analyzing the time-series characteristics of anomaly scores. This additional dimension enables the system to distinguish between transient and enduring anomalies, allowing for more efficient filtering and prioritization without compromising comprehensive detection.
2Loss of information
If all anomaly events are processed equally, then no information is lost, but enduring anomalies cannot be efficiently identified among transient anomalies
Solution Approach 1:
The patent extracts enduring anomalies from the mixed pool of all anomaly events by applying duration-based filtering criteria. This extraction process isolates the specific subset of enduring anomalies that require attention, preventing information loss while dramatically improving detection efficiency for this critical category.
Solution Approach 2:
The patent changes the evaluation parameter from treating all anomalies uniformly to using duration-based classification. By introducing duration as a key parameter and applying threshold-based filtering, the system efficiently identifies enduring anomalies without losing information about transient anomalies, which are handled differently.
3Reliability
If comprehensive anomaly analysis is performed, then all anomaly types are identified, but timely corrective actions for enduring anomalies are delayed
Solution Approach 1:
The patent performs preliminary classification of anomalies by duration before full analysis is completed. By identifying enduring anomalies early in the process based on their temporal characteristics, the system can trigger timely corrective actions for these critical cases while still maintaining comprehensive analysis for all anomaly types.
Solution Approach 2:
The patent maintains continuous monitoring and analysis of all anomaly events while simultaneously enabling expedited processing for enduring anomalies. This continuous action ensures that no anomaly information is lost while allowing the system to prioritize and respond to enduring anomalies without delay.
Data Source
AI summary
According to an aspect, there is provided a computer-implemented method comprising the following. Initially, information on a plurality of anomaly events relating to operation of a target system is obtained. The information comprises one or more time series of anomaly score data. Segments satisfying one or more pre-defined criteria for anomalous operation are detected from the one or more time series. The one or more pre-defined criteria are defined to exclude fully non-anomalous anomaly score data. Anomaly durations and standardized anomaly scores are determined for the segments. Partition or density based clustering is performed in a two-dimensional space formed by the standardized anomaly scores and the anomaly durations to form n clusters, and m smallest clusters of the n clusters are identified. At least one of the following is performed: outputting information on the m smallest clusters or causing adjusting of operation of the target system based on the m smallest clusters.