Virtual Resource Pool Token System for Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional distributed storage systems face challenges in managing resources effectively, particularly in ensuring performance isolation among applications and scaling resource management without central policy bottlenecks, while also failing to guarantee minimum resource availability and varying performance requirements.
Innovation Solution
A virtual resource pool system that uses tokens to represent resource attributes, allowing applications to consume guaranteed minimum and maximum resources, with local allocation pools on storage devices enforcing these policies, enabling scalable and flexible resource management across multiple storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If resource management is centralized in a central management server, then consistent global policies can be ensured, but the processing load on the central management server limits system performance and scalability
Solution Approach 1:
The patent segments resource management functions by distributing them across multiple storage servers rather than centralizing them in a single management server. Each storage server independently manages its own resources and enforces performance limits locally, eliminating the bottleneck at the central management server while maintaining policy consistency through a standardized token-based enforcement mechanism across all servers.
2Productivity
If resource management is decentralized to individual storage servers, then system performance can scale with system size, but consistent global policies become difficult to ensure
Solution Approach 1:
The patent implements a universal token-based resource management mechanism that operates consistently across all decentralized storage servers. The token, which encapsulates performance limits and service levels, serves as a universal enforcement tool that each server can independently validate and apply, ensuring global policy consistency without requiring centralized coordination for each resource management decision.
3Stability of the object's composition
If performance limits are associated with data items, then all accesses to the item are controlled, but the mechanism cannot adapt to different applications or clients issuing requests
Solution Approach 1:
The patent applies local quality by making performance limits application-specific and context-dependent. Instead of a single uniform limit for all accesses to a data item, the system establishes different performance limits tailored to each application's requirements. The token mechanism enables each application to negotiate and receive customized performance guarantees based on its specific needs, while the storage server enforces these customized limits locally for that application's requests.
4Adaptability or versatility
If sessions are used to negotiate performance limits, then application-specific control is achieved, but the session is tied to a network transport connection and cannot be easily migrated
Solution Approach 1:
The patent extracts the performance limit enforcement from the network transport session layer and embeds it directly into the storage protocol at the storage server level. By incorporating performance limit validation and enforcement into the storage protocol tokens, the system decouples performance management from session management, allowing applications to maintain performance guarantees even when network connections or sessions are migrated or reestablished.
Data Source
AI summary
A resource management system uses a virtual resource pool distributed across a set of storage devices to manage resources in a distributed storage system. The storage devices dedicate a resource in an allocation pool available to the virtual resource pool. The virtual resource pool is a virtual storage server in which an application receives at least a predetermined amount of storage capacity, a predetermined level of performance, or a predetermined reliability, represented by tokens. A virtual resource pool manager gives the tokens to an application. The application transmits the tokens along with the commands to the storage device. The token represents a right to consume up to some limit of resources on a specific storage device in a computing session. The storage device locally tracks resource consumption through the token.