Persistent Anomaly Detection via Sliding Window Voting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for predicting failures in cloud native applications fail to accurately distinguish between transient and persistent anomalies, leading to high false positive rates due to their inability to consider the temporal evolution of anomalies.
Innovation Solution
A method and system that combines supervised and unsupervised anomaly detection techniques, using a binary classification model and clustering to identify persistent anomalies by segmenting time series data into sliding windows and analyzing their growth over time, with a voting mechanism to classify anomalies as persistent or not.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing anomaly detection systems use only a set of anomalies (point, context, and collective) without considering temporal evolution, then the detection process is simple, but the false positive rate increases due to inability to distinguish persistent from transient anomalies
Solution Approach 1:
The patent segments the anomaly detection process into multiple independent detectors: point anomaly detector, context anomaly detector, and collective anomaly detector. Each detector operates independently on different aspects of the data, and their results are combined through voting mechanisms. This segmentation allows complex temporal evolution analysis to be broken down into manageable components while improving overall reliability.
Solution Approach 2:
The patent introduces a temporal dimension to anomaly detection by analyzing anomalies across multiple time windows (t-1, t, t+1). Instead of treating anomalies as static points, the system examines their evolution over time, adding a temporal dimension that enables distinction between persistent and transient anomalies, thereby reducing false positives while maintaining computational feasibility.
2Measurement precision
If the system analyzes temporal evolution of anomalies across multiple sliding windows, then false positives are reduced, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary actions by pre-defining the number of sliding windows (N) and the threshold (M) before processing actual anomaly data. This allows the system to prepare the detection framework in advance, reducing runtime computational overhead. The voting mechanism thresholds are established beforehand, enabling quick comparison during real-time anomaly detection without repeated complex calculations.
Solution Approach 2:
The system implements feedback mechanisms where the results from multiple sliding windows feed back into the final anomaly classification. The voting mechanism uses feedback from individual window detections to reach a consensus on persistence, allowing the system to refine its determination of persistent anomalies while maintaining efficient processing through structured feedback loops.
3Reliability
If the system combines supervised and unsupervised anomaly detection approaches, then detection reliability improves, but the system complexity and implementation difficulty increase
Solution Approach 1:
The patent merges multiple detection approaches (supervised and unsupervised anomaly detection) into a unified system where results are combined through voting mechanisms. The point, context, and collective anomaly detectors operate in parallel and their outputs are aggregated, creating a hybrid system that leverages the strengths of both supervised and unsupervised methods while managing complexity through modular integration.
Solution Approach 2:
The voting mechanism serves multiple functions simultaneously: it aggregates results from different detectors, applies persistence thresholds, and produces final anomaly classifications. This multi-functionality reduces the need for separate processing stages, simplifying the overall system structure while maintaining high reliability through the universal voting framework.
Data Source
AI summary
A computer-implemented method and a computer system for identifying persistent anomalies for failure prediction. The computer system receives a time series data stream. The computer system received a predetermined number N and a predetermined number M which is a fraction of N. The computer system segments the time series data stream into N consecutive sliding windows. The computer system performs supervised persistent anomaly detection to determine whether anomalies across the N consecutive sliding windows are persistent, by using a binary classification model. The computer system performs unsupervised persistent anomaly detection to determine whether the anomalies across the N consecutive sliding windows are persistent. The computer system combines results of the supervised persistent anomaly detection and results of the unsupervised persistent anomaly detection to determine persistent anomalies.