Host Issue Queue Size Control for Storage QoS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing QoS management techniques for storage arrays are inefficient due to centralized throttling solutions that lead to substantial loss in utilization and non-work-conserving behavior, particularly in distributed systems where storage arrays appear as 'black boxes' and are not easily configurable for resource allocation.
Innovation Solution
A control algorithm that manages issue queue size based on a moving average latency calculation, using an Exponentially Weighted Moving Average (EWMA) to adjust window sizes dynamically, allowing for proportional resource allocation to hosts without requiring inter-host communication or special information from the storage array, thereby ensuring fair distribution of services.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If centralized throttling solutions are used to manage QoS for storage arrays, then resource allocation control is simplified, but utilization loss increases and work-conserving behavior is compromised
Solution Approach 1:
The host bus adapter performs QoS management autonomously by monitoring its own issue queue depth and dynamically adjusting window sizes based on observed latency. This self-service approach eliminates the need for centralized throttling while maintaining work-conserving behavior and high storage utilization efficiency.
Solution Approach 2:
The system implements feedback mechanisms where the host bus adapter continuously monitors issue queue depth and latency observations to dynamically adjust window sizes. This closed-loop control enables adaptive QoS management that responds to real-time conditions without centralized intervention.
2Adaptability or versatility
If host-based throttling solutions with centralized monitoring are used, then QoS control is distributed, but communication costs increase substantially
Solution Approach 1:
Each host bus adapter independently monitors its own issue queue depth and performs local QoS control without requiring communication with centralized monitoring systems. This self-service approach achieves distributed QoS control while eliminating substantial communication overhead.
Solution Approach 2:
The QoS control function is segmented and distributed to individual host bus adapters, each managing its own window sizes and issue queue depth independently. This segmentation eliminates the need for centralized monitoring and reduces communication costs significantly.
3Reliability
If strict throttling solutions are applied to ensure QoS compliance, then service level guarantees are improved, but efficiency losses increase and work-conserving behavior is compromised
Solution Approach 1:
The system uses dynamic window size adjustment based on observed latency and issue queue depth rather than static throttling thresholds. This dynamic approach maintains QoS guarantees while adapting to changing conditions to preserve work-conserving behavior and storage access efficiency.
Solution Approach 2:
The system changes operational parameters (window sizes) dynamically based on real-time latency observations and issue queue depth, replacing strict fixed throttling with adaptive parameter adjustment that maintains both QoS compliance and efficiency.
Data Source
AI summary
In one embodiment, a latency value is determined for an input/output IO request in a host computer of a plurality of host computers based on an amount of time the IO request spent in the host computer's issue queue. The issue queue of the host computer is used to transmit IO requests to a storage system shared by the plurality of host computers. The method determines a host specific value assigned to the host computer based in proportion on a number of shares assigned to the host in a quality of service policy for IO requests. The size for the host computer's issue queue is determined based on the latency value and the host specific value to control a number of IO requests that are added to the host computer's issue queue where other hosts in the plurality of hosts independently determine respective sizes for respective issue queues.