Robust Forecasting for Computer System Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional forecasting methods are ineffective in handling anomalies, particularly when they span multiple data points or occur recently, leading to inaccurate forecasts and false alarms in computer system monitoring, and are computationally intensive, making real-time monitoring of complex systems challenging.
Innovation Solution
Robust forecasting techniques that derive seasonal profiles by excluding anomalies and using multiple independent forecasts to generate predictions representative of normal operating conditions, ensuring accuracy even in the presence of anomalies and reducing computational intensity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional forecasting methods (exponential smoothing, Holt-Winters) are used to predict system metrics, then the forecasts can be generated quickly with simple computations, but the forecasts become inaccurate when anomalies are present in the historical data
Solution Approach 1:
The patent segments the historical data into normal data points and anomalous data points. By identifying and separating anomalies from the historical data used for forecasting, the method maintains forecast accuracy without requiring complex anomaly detection algorithms. The forecasting model only uses segmented normal data points, effectively excluding the harmful influence of anomalies while keeping computations simple.
Solution Approach 2:
The patent extracts and removes anomalous data points from the historical data set before performing forecasting. By taking out the harmful anomaly elements and using only the clean normal data points for forecast calculations, the method achieves accurate forecasts without needing complex robust forecasting algorithms, thus resolving the contradiction between reliability and computational complexity.
2Reliability
If robust forecasting techniques that exclude anomalies are used, then forecast accuracy improves in the presence of anomalies, but the computational intensity increases making real-time monitoring difficult
Solution Approach 1:
The patent applies segmentation by dividing historical data into normal and anomalous segments. This segmentation allows the use of simple forecasting algorithms on cleaned data, achieving robust accuracy without computational overhead. The segmented approach enables real-time processing because the anomaly exclusion is done through efficient data filtering rather than complex computations.
Solution Approach 2:
The patent uses a disposable approach to anomaly handling by identifying and discarding anomalous data points rather than attempting to correct or model them. This cheap, simple anomaly removal strategy enables real-time processing speeds while maintaining forecast accuracy, as it avoids the computational intensity of sophisticated robust forecasting methods.
3Productivity
If simple exponential smoothing is used for forecasting, then computational complexity is low and real-time monitoring is feasible, but the method produces inaccurate forecasts when anomalies are present in recent observations
Solution Approach 1:
The patent applies preliminary action by pre-processing the historical data to remove anomalies before feeding it to the simple exponential smoothing algorithm. This preliminary cleaning step ensures that the computationally simple forecasting method receives clean input data, thereby maintaining both real-time processing capability and forecast accuracy without requiring complex algorithms.
Solution Approach 2:
The patent extracts and removes anomalous data points from the historical data set before performing exponential smoothing forecasting. By taking out the harmful anomaly elements in advance, the simple forecasting algorithm can operate efficiently in real-time while producing accurate forecasts, thus resolving the contradiction between productivity and reliability.
4Reliability
If Holt-Winters seasonal forecasting is used to account for periodic variations, then forecast accuracy improves for seasonal metrics, but the method remains vulnerable to anomalies and produces poor forecasts for several cycles after an anomaly occurs
Solution Approach 1:
The patent segments the historical data used for seasonal forecasting by identifying and excluding anomalous data points. By using only segmented normal data points to build the seasonal patterns in Holt-Winters forecasting, the method achieves accurate seasonal forecasts that are not contaminated by anomalies, thereby reducing the recovery time after anomalies occur.
Solution Approach 2:
The patent applies preliminary action by pre-cleaning the historical data to remove anomalies before performing seasonal forecasting. This preliminary anomaly removal ensures that the seasonal patterns learned by Holt-Winters are based on normal operations only, enabling the forecast to recover quickly after anomalies without being dragged down by contaminated historical data.
Data Source
AI summary
Robust forecasting techniques are relatively immune from anomalies or outliers in observed data, such as a stream of data values reflective of the operation or use of a computer system. One robust technique provides a relatively accurate forecast of seasonal behavior even in the presence of an anomaly in corresponding historical data. Another robust forecasting technique provides a relatively accurate forecast even in the presence of an anomaly that spans multiple recent observations. In one embodiment, both techniques are used in combination to automatically detect anomalies in the operation and/or use of a multi-user computer system.