On-Demand Anomaly Detection for Counter-Based Asset Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large scale systems generate vast amounts of operational data, and identifying the cause of failures promptly to reduce Mean Time to Repair (MTTR) is challenging due to the complexity and volume of data.
Innovation Solution
A metrics-based anomaly detection system that utilizes time series data analysis and machine learning models to identify anomalous metrics by converting counter metrics to gauge metrics, training models specific to each metric, and detecting anomalies through anomaly scores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If engineers manually analyze operational data to identify failure causes, then measurement precision may be improved, but loss of time increases significantly
Solution Approach 1:
The system performs preliminary actions by continuously collecting and storing operational metrics in time-series format, pre-processing data to detect counter metrics, and training anomaly detection models before failures occur. This preparation enables rapid anomaly detection when failures happen, reducing the time engineers need to spend analyzing operational data while maintaining high detection accuracy through pre-trained models specific to each metric type.
2Ease of operation
If the system processes all metrics uniformly, then ease of operation is improved, but manufacturing precision of anomaly detection decreases
Solution Approach 1:
The system applies local quality by detecting the type of each metric (counter vs. non-counter) and applying type-specific anomaly detection models accordingly. Counter metrics undergo seasonal differencing and use counter-specific models, while other metrics use standard models. This localized approach tailored to each metric's characteristics improves anomaly detection precision while the automated type detection and model selection maintain system ease of operation.
3Measurement precision
If the system uses specialized models for different metric types, then anomaly detection precision is improved, but device complexity increases
Solution Approach 1:
The system segments metrics into different types (counter and non-counter) based on their characteristics, and trains separate anomaly detection models for each type. Counter metrics are identified through seasonal differencing and linear regression analysis, then processed with counter-specific models that account for their monotonically increasing nature. This segmentation improves anomaly detection precision by applying appropriate models to each metric type while managing complexity through automated classification and modular model architecture.
Data Source
AI summary
A method for metrics-based anomaly detection involves receiving an anomaly analysis request for an asset and obtaining metrics associated with the asset. Each of the metrics includes time series data. The method further involves detecting that one of the metrics is a counter. The detection involves seasonally differencing the metric, obtaining a regression line by performing a linear regression on the metric, and determining that an angle of the regression line exceeds a predetermined threshold angle. The method also involves training models for the metrics, the training including training a counter-specific model for the metric that is a counter. The method further involves determining, using the models after the training, at least one metric that is anomalous.