Anomaly Detection Using Multiple Statistical Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for detecting anomalies in time series data from software applications often rely on static rules and thresholds, which are inadequate for identifying subtle changes and can lead to false positives, making it difficult to accurately detect anomalies, especially in complex application management scenarios.
Innovation Solution
An anomaly detection system that uses multiple statistical models to predict data points and identifies anomalies by comparing actual values to predicted values, with a voting mechanism to confirm anomalies based on significant differences, allowing for dynamic adjustment of models according to application activity levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If static rules and thresholds are used for anomaly detection, then the system is simple to operate, but the measurement precision of anomaly detection deteriorates
Solution Approach 1:
The patent transitions from static thresholds to dynamic thresholds that adapt based on historical data and statistical models. The system automatically adjusts detection thresholds according to learned patterns in the time series data, eliminating the need for manual threshold configuration while improving detection accuracy.
Solution Approach 2:
The system changes the parameters used for anomaly detection from fixed static values to dynamic statistical parameters derived from multiple models. By incorporating multiple statistical models with different parameters (mean, standard deviation, trends), the system achieves higher measurement precision without sacrificing operational simplicity.
2Measurement precision
If multiple statistical models are used to improve anomaly detection accuracy, then the measurement precision improves, but the device complexity increases
Solution Approach 1:
The patent segments the anomaly detection task into multiple independent statistical models, each analyzing the time series from different perspectives (different time windows, different statistical properties). This segmentation allows the system to achieve high precision through model ensembling while keeping each individual model relatively simple.
Solution Approach 2:
The system employs multiple statistical models that serve universal purposes in anomaly detection. Each model contributes to the overall detection capability, and their results are aggregated through voting mechanisms. This multi-functionality approach improves measurement precision while the modular architecture manages complexity.
3Ease of operation
If static thresholds are applied to thousands of time series, then the ease of operation is maintained, but the measurement precision deteriorates due to inability to detect subtle changes
Solution Approach 1:
The system implements self-service by automatically learning and adapting thresholds for each time series independently. The statistical models automatically adjust to the characteristics of each specific time series, enabling the system to handle thousands of series without manual intervention while maintaining high detection precision for subtle changes.
Solution Approach 2:
The system applies dynamic threshold adjustment for each time series based on its historical behavior and statistical properties. This allows the system to automatically adapt to subtle changes in each series without requiring manual threshold specification, achieving both ease of operation and high measurement precision across thousands of time series.
Data Source
AI summary
A time series is analyzed by multiple functions simultaneously to identify an anomaly for a data point in the series. Data point values are predicted by the multiple functions. An anomaly occurs when an actual data point in the series differs significantly from the data point's predicted value as generated by the functions. If enough statistical models detect an anomaly has occurred for a data point, an anomaly event is generated. The set of functions can include different types of functions, the same function type configured with different constants, or a combination of these.