Segment-Based Anomaly Detection for Computer Services
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for detecting performance anomalies in large-scale distributed web services rely on threshold-based approaches, which can be challenging due to varying normal performance modes and multiple monitored metrics, often requiring manual or automatic threshold setting, and may not accurately account for temporal relationships and multimodal performance behaviors.
Innovation Solution
A segment-based approach using information-theoretic and statistical learning techniques to detect anomalies by clustering contiguous time samples into segments, modeling them with Gaussian mixture distributions, and analyzing statistical distances to identify outliers and performance changes, allowing for timely and accurate diagnosis and recovery from service performance issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If threshold-based approaches are used for anomaly detection, then the detection process is simple, but detection accuracy deteriorates due to varying normal performance modes and multiple monitored metrics
Solution Approach 1:
The patent segments the time series data into multiple modes using Gaussian Mixture Models, where each mode represents a distinct normal performance pattern. This segmentation allows the system to differentiate between varying normal performance modes and actual anomalies, thereby improving detection accuracy while maintaining computational feasibility through automated mode identification.
Solution Approach 2:
The patent transforms the detection approach by changing from fixed threshold parameters to dynamic statistical parameters derived from historical data. By using percentile-based thresholds (e.g., 5th and 95th percentiles) and standard deviations calculated from mode-specific data, the system adapts to different performance modes and improves accuracy across multiple monitored metrics.
2Ease of operation
If manual threshold setting is used, then threshold customization is possible, but the process becomes challenging and time-consuming due to different normal performance modes and large number of metrics
Solution Approach 1:
The patent implements self-service by enabling the system to automatically identify performance modes and calculate appropriate thresholds using historical data. The Gaussian Mixture Model automatically clusters data into modes, and thresholds are computed based on mode-specific statistics, eliminating the need for manual threshold setting while maintaining customization capability through automated adaptation to service-specific patterns.
Solution Approach 2:
The patent performs preliminary action by pre-processing historical performance data to establish baseline modes and thresholds before actual anomaly detection begins. This preliminary analysis of historical measurements allows the system to be ready for immediate anomaly detection without requiring manual configuration at runtime, significantly reducing setup time while preserving customization through data-driven parameter selection.
3Extent of automation
If automated threshold determination using statistical distribution is used, then threshold setting becomes automatic, but it may not accurately capture temporal relationships and multimodal performance behaviors
Solution Approach 1:
The patent introduces dynamics by transitioning from static threshold values to dynamic, mode-dependent thresholds that adapt to changing performance conditions. The system continuously identifies performance modes and adjusts thresholds accordingly, allowing automated threshold determination to accurately capture temporal relationships and multimodal behaviors through real-time adaptation to the current performance state.
Solution Approach 2:
The patent adds another dimension by incorporating mode identification as an additional layer in the detection process. Instead of applying single global thresholds, the system first classifies the current performance state into a specific mode dimension, then applies mode-appropriate thresholds. This dimensional approach enables automated determination to accurately capture temporal relationships and multimodal patterns by considering both time and performance state dimensions.
4Measurement precision
If segment-based approach with clustering is used, then detection accuracy improves by considering temporal relationships, but computational complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the continuous time series into discrete modes using Gaussian Mixture Models. This segmentation reduces computational complexity by transforming an infinite-state problem into a finite-state problem with a manageable number of modes. Each segment (mode) can be analyzed independently with its own characteristics, improving detection accuracy while keeping computational requirements tractable through the finite mode representation.
Solution Approach 2:
The patent manages computational complexity through parameter changes by limiting the number of Gaussian components in the mixture model and using efficient algorithms for mode identification. By adjusting parameters such as the number of components and convergence criteria, the system achieves an optimal balance between detection accuracy (capturing temporal relationships) and computational complexity, making the segment-based approach feasible for production environments.
Data Source
AI summary
A technique includes sampling at least one performance metric of a computer-based service to form time samples of the metric(s) and detecting an occurrence of an anomaly or a performance mode change in the service. The detection includes arranging the time samples in segments based on a statistical analysis of the time samples.