Anomaly Alert Filtering for Cloud Time Series Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud computing environments, the large-scale multitenancy generates a high volume of noisy alerts from user interface logs, overwhelming system administrators and making it infeasible to distinguish between actual errors and false positives, due to the complexity of user interactions and variable performance metrics across thousands of organizations.
Innovation Solution
A data processing pipeline is implemented using modular representation of raw log data to generate performance metrics, incorporating schema inference and log data validation to detect errors and filter alerts based on service level indicators (SLIs) and objectives (SLOs), reducing the number of noisy alerts while ensuring critical errors are not missed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If anomaly detection is performed on all time series data from user interface logs in a multitenant cloud environment, then the coverage of error detection is improved, but the number of false positive alerts increases significantly
Solution Approach 1:
The patent segments the multitenant environment into multiple tenant-specific time series data sets, each analyzed independently. This segmentation allows anomaly detection to be performed on per-tenant basis rather than aggregating all tenants' data, reducing the false positive rate while maintaining comprehensive error detection coverage across all tenants.
Solution Approach 2:
The patent applies local quality by customizing anomaly detection parameters and thresholds for each individual tenant based on their specific usage patterns and performance characteristics. This allows the system to adapt to local conditions of each tenant, improving detection accuracy and reducing false positives compared to a uniform approach applied to all tenants.
2Measurement precision
If anomaly detection parameters are tuned for individual data sets, then the detection accuracy for that specific data set is improved, but the system complexity increases
Solution Approach 1:
The patent implements self-service by automatically tuning anomaly detection parameters for each tenant's time series data using machine learning techniques. The system learns from historical data and automatically adjusts detection thresholds and parameters without requiring manual configuration, thereby maintaining high detection accuracy while reducing system complexity and administrative burden.
3Manufacturing precision
If all alerts are processed and analyzed in detail, then the thoroughness of error analysis is improved, but the time required for administrator response increases
Solution Approach 1:
The patent applies partial action by filtering and prioritizing alerts based on their significance and likelihood of being true anomalies. Instead of analyzing all alerts in equal detail, the system focuses thorough analysis on high-priority alerts while applying quicker filtering to lower-priority ones, thereby maintaining error analysis thoroughness for critical issues while reducing overall response time.
Data Source
AI summary
Systems and methods are described for applying a plurality of data points of a time series data set representing values of a metric measuring performance of a cloud computing service to a machine learning model to predict a forecast of a most likely value of the metric at a selected future time. The method includes determining whether the plurality of data points of the time series data set are anomalies according to the machine learning model and the forecast and generating a collective anomaly from the anomalies when the plurality of data points is determined to be anomalies. The method further includes determining whether the collective anomaly does not meet one or more cloud computing service level objective (SLO) threshold requirements and sending an alert when the collective anomaly does not meet one or more cloud computing SLO threshold requirements.