Storage QOS Monitoring via Step-Function Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in efficiently monitoring and analyzing Quality of Service (QOS) performance, particularly in identifying and addressing incidents that deviate from predefined response time and throughput targets, as they expand in size and operating speeds.
Innovation Solution
A method is implemented to collect QOS data from storage volumes, analyze it for step-up or step-down functions, generate an expected range for future data, and monitor current data to detect deviations, identifying incidents and proposing remediation plans to maintain optimal performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If storage systems expand in size and operating speeds, then processing capacity and throughput increase, but monitoring and analyzing QOS data becomes more difficult and complex
Solution Approach 1:
The patent segments QOS monitoring into multiple hierarchical levels (storage device level, storage system level, and client level). Each level has dedicated monitoring components that collect and analyze QOS data locally, then aggregate results upward. This segmentation reduces the complexity at each individual level while maintaining comprehensive system-wide monitoring capability.
Solution Approach 2:
The patent introduces intermediary components including QOS monitoring agents deployed on storage devices and a central QOS analysis server. These intermediaries facilitate data collection, processing, and analysis, reducing the direct complexity burden on the expanded storage system by distributing monitoring functions across multiple specialized components.
2Reliability
If traditional monitoring methods are used, then system simplicity is maintained, but incidents deviating from QOS targets cannot be efficiently identified
Solution Approach 1:
The patent implements feedback mechanisms where QOS data is continuously collected, analyzed against predefined targets and thresholds, and used to generate alerts or automated responses. The system monitors actual QOS performance and feeds this information back to stakeholders or control systems, enabling efficient incident identification when deviations occur while maintaining manageable complexity through automated feedback loops.
Solution Approach 2:
The patent monitors multiple QOS parameters (response time, throughput, IOPS) and dynamically adjusts monitoring thresholds and analysis methods based on changing system conditions and incident patterns. By changing monitoring parameters adaptively rather than using fixed simple thresholds, the system achieves higher incident detection accuracy while managing complexity through intelligent parameter adjustment.
3Loss of information
If comprehensive QOS data collection is implemented, then complete performance visibility is achieved, but data processing and analysis overhead increases
Solution Approach 1:
The patent performs preliminary actions by pre-defining QOS targets, thresholds, and analysis rules before data collection begins. QOS data is collected and preliminarily processed at the source with basic filtering and aggregation, so that when data reaches the central analysis system, it is already partially processed. This preliminary action reduces the processing overhead at later stages while maintaining complete QOS information.
Solution Approach 2:
The patent implements partial processing at multiple levels rather than complete centralized processing. Each storage device and monitoring agent performs partial analysis of QOS data locally, filtering out normal variations and only escalating anomalies requiring full analysis. This partial action approach achieves complete QOS visibility while reducing overall processing time by distributing work and avoiding unnecessary full-system analysis for every data point.
Data Source
AI summary
Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes are provided. QOS data is collected for the plurality of storage volumes and includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. The process determines an average of N collected QOS data points at any given time; and iteratively analyzes each QOS data point to detect if a step-up or a step-down function has occurred, where a step-up function represents an unpredictable increase in value of a data point and a step-down function is an unpredictable decrease in value of the data point. A subset of the N QOS data points based on when the step-up function or step-down function occurs is selected for analysis and an expected range for future QOS data based on the subset of the N QOS data points is generated.