Distributed Compute Node Storage Access Budgeting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-tenant storage systems, there is a need to efficiently control IO requests and throughput across compute nodes accessing shared storage entities without wasting resources, ensuring that performance limits are not exceeded and minimum guarantees are met, while avoiding the bottlenecks and communication burdens associated with central orchestration.
Innovation Solution
Each compute node determines its current performance requirements, updates shared storage values independently, and calculates access budgets based on performance policies and limitations, allowing for efficient access to storage entities without requiring a central orchestrating entity or synchronization among nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a central orchestrating entity is used to control access performance, then performance limits can be enforced, but system complexity and communication overhead increase
Solution Approach 1:
Each compute node independently determines its own performance requirements, updates shared storage values, and calculates access budgets without requiring a central orchestrating entity. The system uses self-service mechanisms where nodes autonomously manage their own access control while collectively maintaining performance limits through shared state visibility.
Solution Approach 2:
The central control function is segmented and distributed to individual compute nodes. Each node independently performs the control operations that would traditionally be centralized, dividing the monolithic control architecture into distributed autonomous units that collectively achieve the same performance guarantee.
2Reliability
If frequent synchronization is implemented among compute nodes, then performance limit compliance is improved, but communication overhead and processing time increase
Solution Approach 1:
Compute nodes determine their performance requirements and calculate access budgets in advance for each access cycle, before actual storage operations begin. This preliminary action allows nodes to pre-compute their budgets based on current system state, avoiding the need for frequent synchronization during active operations and reducing real-time communication overhead.
3Measurement precision
If performance control mechanisms are added to each compute node, then access control precision is improved, but device complexity and resource consumption increase
Solution Approach 1:
The same compute nodes that perform storage operations also perform performance control functions. Each node's processor executes both data processing tasks and budget calculation/monitoring tasks, making the control mechanisms multi-functional rather than adding separate dedicated control hardware, thereby avoiding increased device complexity.
4Productivity
If shared storage space is made accessible by all compute nodes, then resource utilization is improved, but access coordination complexity increases
Solution Approach 1:
Each compute node reads the shared storage value that reflects total performance requirements from all nodes, uses this feedback information to calculate its own budget, and updates the shared storage with its contribution. This feedback loop enables automatic coordination without complex centralized control, as each node adjusts its behavior based on the aggregate state visible in shared storage.
Data Source
AI summary
A method for accessing stored entities (SEs) that are stored in a storage unit of a storage system, the method may include determining in a cyclic manner, by each compute node (CN) of a group of compute nodes, CN SEs budgets to be used in a cycle, based on a shared storage space that stores performance requests of Ces of the group.