Correlation-Based Time-Series Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current root cause analysis methods in software telemetry, such as state-driven regression models, fail to accurately predict anomalies due to neglecting seasonal context and relying on supervised training, making them inefficient in complex systems with many potential failure points and requiring human intervention.
Innovation Solution
The development of predictive correlation models that use unsupervised learning to model temporal variations in metric correlations, accounting for trends and seasonality, and automatically detecting changes in correlation patterns without human guidance, allowing for independent analysis of metric relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If state-driven regression models are used for root cause analysis, then human intervention can guide the analysis process, but the models fail to account for seasonal context and temporal variations in metric correlations
Solution Approach 1:
The patent implements dynamic correlation modeling that adapts to temporal variations in metric relationships. The system learns and models how correlations between metrics change over time, including seasonal patterns and trending behaviors, rather than assuming static correlations. This allows the anomaly detection to account for temporal dynamics while maintaining automated operation.
Solution Approach 2:
The patent changes the parameters being modeled from individual metric values to correlation coefficients between metrics. By focusing on how metrics relate to each other over time rather than their absolute values, the system can detect anomalies in correlation patterns that indicate root causes, while automatically adapting to seasonal and temporal parameter changes in the system behavior.
2Reliability
If aggregate analytic models are built by linking diagnostic models based on system topology, then diagnostic information can be produced, but the models are tightly coupled to system topology and require changes when topology changes
Solution Approach 1:
The patent creates a universal correlation-based anomaly detection framework that can be applied to any software system regardless of its specific topology. Instead of building custom diagnostic models for each system configuration, the system uses general-purpose correlation analysis that automatically adapts to different system structures, making it versatile across varying topologies while maintaining reliable anomaly detection.
Solution Approach 2:
The system performs self-service by automatically learning correlation patterns from historical metric data without requiring manual model construction or topology configuration. The correlation models adapt automatically to the system's structure and behavior, eliminating the need for administrators to manually update models when system topology changes.
3Measurement precision
If administrators manually analyze various metrics to determine root causes, then detailed inspection of each metric is possible, but the process is time-consuming and complex metrics are difficult to interpret
Solution Approach 1:
The patent extracts the complex task of multi-metric analysis by automatically computing correlation coefficients between metrics and identifying anomalous correlation patterns. This extraction separates the time-consuming computational analysis from the administrator's review process, allowing detailed metric inspection to be performed automatically while administrators only need to review the generated anomaly reports.
Solution Approach 2:
The patent introduces correlation coefficients as an intermediary metric that simplifies the relationship between multiple system metrics. Instead of requiring administrators to directly analyze complex interrelationships between numerous metrics, the correlation coefficients serve as intermediate representations that capture these relationships in a single, interpretable value that can be easily analyzed for anomalies.
4Extent of automation
If unsupervised learning is used to model temporal variations in metric correlations, then automated anomaly detection is achieved, but the models require significant computational resources to learn and evaluate correlations
Solution Approach 1:
The patent applies partial action by focusing computational resources on computing correlations only for metric pairs that are likely to be related based on domain knowledge or preliminary analysis. Rather than computing all possible pairwise correlations across all metrics, the system selectively computes correlations for relevant metric pairs, reducing computational overhead while maintaining effective automated anomaly detection.
Data Source
AI summary
Techniques are described for modeling variations in correlation to facilitate analytic operations. In one or more embodiments, at least one computing device receives first metric data that tracks a first metric for a first target resource and second metric data that tracks a second metric for a second target resource. In response to receiving the first metric data and the second metric data, the at least one computing device generates a time-series of correlation values that tracks correlation between the first metric and the second metric over time. Based at least in part on the time-series of correlation data, an expected correlation is determined and compared to an observed correlation. If the observed correlation falls outside of a threshold range or otherwise does not satisfy the expected correlation, then an alert and/or other output may be generated.