Distributed Storage QoS Manager via Priority Choking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face challenges in managing quality of service effectively in distributed storage environments, leading to inefficiencies and potential bottlenecks in data access and processing.
Innovation Solution
The implementation of a distributed electronic storage system (DESS) with quality of service management capabilities, which includes a centralized priority manager and choking processes to regulate file system requests based on resource load and priority levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional data storage systems are used without quality of service management, then system simplicity is maintained, but resource utilization efficiency deteriorates and bottlenecks occur in data access and processing
Solution Approach 1:
A quality of service manager is introduced as an intermediary component between storage clients and storage resources. This manager receives quality of service parameters from clients, determines appropriate storage policies, and coordinates with storage resources to implement differentiated service levels. The intermediary resolves the contradiction by adding intelligence to improve resource utilization while maintaining relative system simplicity through centralized policy management.
Solution Approach 2:
The system implements dynamic quality of service management where storage policies and resource allocation are adjusted in real-time based on current system state and client requirements. The quality of service manager continuously monitors resource utilization and modifies storage operations accordingly, enabling the system to adapt to changing conditions and optimize performance without requiring complete system redesign.
2Reliability
If quality of service management with priority levels is implemented, then high-priority request servicing is improved, but system complexity increases due to centralized priority manager and choking processes
Solution Approach 1:
The quality of service management function is segmented into distinct components: quality of service parameters reception, policy determination, and storage resource coordination. This segmentation allows each component to be optimized independently and simplifies the overall management structure by dividing complex functions into manageable modules that can be implemented and maintained separately.
Solution Approach 2:
The system uses parameter-based quality of service management where service levels are controlled through configurable parameters rather than hard-coded logic. By changing parameters in the quality of service manager, the system can adjust priority levels and resource allocation without modifying the underlying management structure, thereby maintaining simplicity while improving service reliability.
3Productivity
If choking processes regulate file system requests based on resource load, then congestion management is improved, but request processing time increases due to additional regulation steps
Solution Approach 1:
The quality of service manager performs preliminary assessment of storage requests before they are fully processed. By evaluating resource availability and applying appropriate choking policies in advance, the system prevents congestion from developing rather than reacting to it. This preliminary action reduces the need for time-consuming regulation steps during actual request processing.
Solution Approach 2:
The system implements feedback mechanisms where the quality of service manager continuously monitors resource load and adjusts choking levels dynamically. When resources are abundant, choking is relaxed to speed up processing; when congestion is detected, choking is tightened to manage load. This feedback loop optimizes the balance between congestion control and processing speed, minimizing unnecessary delays.
Data Source
AI summary
One or more computing devices may comprise congestion management circuitry, one or more client file system request buffers, and DESS interface circuitry. The one or more client file system request buffers is/are operable to queue first client file system requests of a first priority level and second client file system requests of a second priority level, where the first priority level is higher priority than the second priority level. The DESS interface circuitry is operable to determine a choking level according to the load on a plurality of DESS resources. Individual load values of the DESS resources are mapped to a composite load value using a first function. The composite load value is mapped to a congestion contribution using a second function. And, the congestion contribution is mapped to a choking level using a third function.