VSAN Composite Scheduler for Distributed Storage Bandwidth Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face performance issues due to contention between multiple clients accessing shared storage resources, leading to reduced overall performance and higher latency, particularly when different types of storage I/O operations contend for resources without fair allocation of bandwidth.
Innovation Solution
A virtual storage area network (VSAN) module with a composite scheduler that manages storage I/O requests using a fair scheduling algorithm and backpressure congestion control to ensure equitable processing of different classes of storage I/O requests, preventing resource congestion and optimizing bandwidth allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple clients simultaneously access shared storage resources in a distributed storage system, then storage capacity and scalability are improved, but latency increases and overall performance deteriorates due to resource contention
Solution Approach 1:
The patent segments storage I/O requests into different classes (e.g., sync I/O requests and async I/O requests) and processes them through separate queues with different scheduling priorities. This segmentation allows the system to handle different types of storage operations independently, preventing high-priority sync operations from being blocked by lower-priority async operations, thereby reducing latency while maintaining storage capacity
Solution Approach 2:
The patent implements dynamic bandwidth allocation that adjusts resource allocation based on current system conditions and request characteristics. The fair scheduling algorithm dynamically determines the proportion of storage I/O bandwidth allocated to different client I/O operations, allowing the system to adapt to changing workload patterns and minimize latency under varying access conditions
2Device complexity
If different classes of storage I/O requests are processed without fair scheduling, then device complexity is reduced, but performance fairness and overall system efficiency deteriorate
Solution Approach 1:
The fair scheduling algorithm dynamically adjusts bandwidth allocation proportions for different storage I/O classes based on current system state and request characteristics. This dynamic approach ensures fair resource distribution without requiring complex static configuration, maintaining relative simplicity while improving performance fairness and overall system efficiency
Solution Approach 2:
The system monitors storage I/O operations and uses feedback mechanisms to adjust scheduling decisions in real-time. The fair scheduling algorithm receives feedback about request completion, queue depths, and resource utilization to continuously optimize bandwidth allocation, improving performance without proportionally increasing complexity
Data Source
AI summary
System and method for managing storage requests issued from multiple sources in a distributed storage system utilizes different queues at a host computer in the distributed storage system to place different classes of storage requests for access to a virtual storage area network. The storage requests in the queues are processed using a fair scheduling algorithm. For each queue, when the storage requests in the queue exceeds a threshold, a backpressure signal is generated and transmitted to at least one source for a class of storage requests queued in one of the queues corresponding to that backpressure signal to delay issuance of new storage requests of that class of storage requests.