Storage Protocol Connection Anomaly Detection Without Alert Fatigue
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face issues with over-alerting due to temporary and sustained spikes in storage protocol connections, leading to alert fatigue and inefficient resource allocation, as current brute force approaches fail to differentiate between normal spikes and anomalous patterns.
Innovation Solution
Implementing a machine-learning model trained on historical customer telemetry data to categorize storage protocol connection counts as anomalous or not, using algorithms like Isolation Forest or Random Cut Forest, and applying these models in real-time to generate customer-specific and workload-specific alerts, filtering out expected seasonal spikes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If brute force approaches are used to monitor storage protocol connections, then all spikes in connections are detected and alerted, but this leads to over-alerting and alert fatigue
Solution Approach 1:
The patent transforms the monitoring approach by changing parameters from fixed thresholds to dynamic, learned patterns. The system uses machine learning models trained on historical data to adaptively determine what constitutes anomalous behavior, allowing the detection parameters to evolve with changing system characteristics and workload patterns, thereby reducing false alerts while maintaining reliable anomaly detection
Solution Approach 2:
The system performs preliminary training actions by collecting and analyzing historical connection data before deploying anomaly detection. Machine learning models are pre-trained on normal operational patterns and seasonal variations, enabling the system to distinguish between expected spikes (like seasonal increases) and true anomalies, thus preventing alert fatigue before it occurs
2Reliability
If machine-learning models are implemented to differentiate anomalies, then alert relevance improves, but system complexity increases
Solution Approach 1:
The patent introduces machine learning models as intermediary components between raw connection data and alert generation. These models act as intelligent mediators that process complex patterns and translate them into meaningful anomaly detections, shielding the rest of the system from complexity while improving alert relevance through learned behavioral patterns
3Productivity
If customer-specific and workload-specific alerts are generated, then resource allocation efficiency improves, but data processing requirements increase
Solution Approach 1:
The patent segments the monitoring approach by creating separate machine learning models for different customers and workload types. This segmentation allows the system to process data in manageable, specialized units rather than attempting to analyze all data uniformly, improving resource allocation efficiency while distributing data processing requirements across multiple targeted models rather than requiring excessive centralized processing
Data Source
AI summary
Systems and methods for anomaly detection within a storage system based on a number of storage protocol connections are provided. In one example, alerting thresholds based on storage protocol connections are made more relevant and customer-specific by taking into consideration, among other things, one or more of the type/class/model of storage system, the configuration of the storage system, the type of the workload making use of the storage system, and various windows of time. Based on the customer-specific nature of the alerts generated by the proposed alerting system, potential performance issues being experienced by the storage system may be more efficiently identified and remediated, for example, by customer service/support, the customer, or possibly by automated storage system (on-box) functionality.