Storage Load Balancing via Client Throttling and Zone Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage architectures face issues with hot spots where some drives are over-utilized while others are under-utilized, leading to inconsistent client performance, and quality of service prioritization does not guarantee a consistent level of performance or allow administrators to control system load effectively.
Innovation Solution
A method for determining client metrics and system load in a storage system, adjusting performance based on calculated load values, and dynamically managing access to data by throttling clients to balance load across all drives, ensuring even distribution of data and consistent aggregate performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is stored on a small percentage of drives in a storage cluster, then client data access is simplified, but hot spots occur where portions of the cluster are over-utilized while other portions are under-utilized
Solution Approach 1:
The patent segments the storage cluster into multiple zones or regions, and distributes client data across these segments rather than concentrating it on a small percentage of drives. This segmentation approach balances the load across the entire cluster, preventing hot spots while maintaining manageable data access through organized zone-based routing.
2Ease of operation
If quality of service prioritization is used to improve client experience, then high-priority clients receive better service, but a single client's effects on performance are not capped and the system always runs slow when stressed
Solution Approach 1:
The patent introduces throttling parameters that dynamically adjust client performance limits based on system conditions. Instead of unlimited prioritization, the system applies caps and thresholds to client I/O operations, transforming the unbounded priority system into a controlled one where performance parameters are adjusted in real-time to maintain overall system responsiveness even when stressed.
Solution Approach 2:
The patent implements feedback mechanisms that monitor system load and client performance metrics, then adjust throttling levels accordingly. When the system detects overload conditions, it feeds this information back to the throttling controller, which then reduces client performance limits to prevent system collapse, ensuring the system remains responsive under stress.
3Ease of operation
If priority levels are assigned to clients, then access requests are dispatched based on cluster load and client priority, but it is difficult for customers to understand the actual performance they are receiving
Solution Approach 1:
The patent employs visual indicators or status markers (analogous to color changes) that display actual performance metrics to customers in an easily understandable format. Instead of abstract priority levels, the system presents performance information through intuitive visual cues that show customers their actual performance standing, making the information transparent and comprehensible.
4Speed
If data is concentrated on specific drives, then client data access is streamlined, but some clients experience good performance while others experience poor performance inconsistently
Solution Approach 1:
The patent applies local quality by allowing different zones or regions of the storage cluster to have specialized characteristics optimized for different types of workloads. Instead of uniform data distribution, the system creates locally optimized storage regions that can handle specific access patterns efficiently, while the overall system maintains performance consistency through intelligent routing and load balancing across these specialized zones.
Data Source
Figure 1A~1B
Figure 2~3
Figure 4~5
AI summary
Disclosed are systems, computer-readable mediums, and methods for determining client metrics of a volume in a storage system for a first client of a plurality of clients. The storage system stores data from the plurality of clients. System metrics of a cluster in the storage system are determined based upon use of the storage system by the plurality of clients. A load value of the storage system is determined based upon the system metrics and the client metrics. The load value is determined to be above a predefined threshold. A target performance value is calculated based upon the load value, a minimum quality of service value, and a maximum quality of service value. Performance of the storage system is adjusted for the client based upon the target performance value and the determining the load value is above the predefined threshold.