Anomaly Detection Using Workload Velocity Baselines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current programmatic performance anomaly detection systems often generate false positives and require the system to be in an anomalous mode to collect valuable data, which can lead to delayed detection of performance issues in high-speed computing environments.
Innovation Solution
The method involves combining workload manager (WLM) velocity data with Predictive Failure Analysis (PFA) to model expected velocity values for address spaces, comparing current velocity to a factor of the expected value, and generating remedial actions when the current velocity is lower, thereby detecting performance anomalies proactively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system operates in anomalous mode to collect valuable data for anomaly detection, then the detection accuracy improves, but the detection time is delayed
Solution Approach 1:
The system performs preliminary actions by continuously collecting and analyzing velocity data during normal operation to establish baseline performance metrics. This preliminary data collection and model training enables the system to detect anomalies immediately when they occur, rather than waiting for the system to enter an anomalous mode. The baseline model is prepared in advance through continuous learning from normal operational data.
Solution Approach 2:
The system implements continuous feedback by monitoring current velocity values against the baseline model in real-time. When deviations are detected, the system immediately generates alerts and can trigger remedial actions. This closed-loop feedback mechanism ensures that anomalies are detected and responded to promptly, eliminating the time delay associated with waiting for anomalous modes to occur for data collection.
2Reliability
If traditional anomaly detection methods are used, then false positive information increases, but the system complexity remains low
Solution Approach 1:
The system segments the anomaly detection process into distinct components: velocity data collection from workload manager, baseline model creation through statistical analysis, current velocity measurement, comparison logic against the baseline, and alert generation. This segmentation allows each component to be optimized independently and improves overall detection reliability by ensuring thorough analysis at each stage.
Solution Approach 2:
The baseline model serves as an intermediary between raw velocity data and anomaly detection decisions. Instead of directly comparing current values to fixed thresholds, the system uses the baseline model (created from historical normal operation data) as a mediator to evaluate whether current velocity represents a true anomaly. This intermediary layer significantly reduces false positives by providing context-aware comparison.
3Reliability
If velocity data is continuously monitored and compared against baseline models, then anomaly detection reliability improves, but the computational resources increase
Solution Approach 1:
The system applies partial action by monitoring only the velocity metric rather than analyzing all possible system parameters. The baseline model compares current velocity values against expected ranges without requiring full system state analysis. This selective monitoring approach maintains high detection reliability for performance anomalies while minimizing computational resource consumption compared to comprehensive system-wide monitoring.
Data Source
AI summary
A method, system, and computer program product are provided for performance anomaly detection. Velocity data is periodically received from a workload manager for one or more address spaces. An expected velocity value is created for each of the one or more address spaces. A factor of the expected velocity value is compared to a current velocity value from the velocity data. Based on the current velocity value being lower than the factor, a remedial action is generated indicating an anomaly.