Disaggregated Storage Pod Allocation Areas for Multi-Node Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems suffer from sub-optimal infrastructure architectures that do not effectively utilize compute, network, and storage resources, leading to storage silos and inefficient management of storage space.
Innovation Solution
Implementing a disaggregated storage pod with a global physical volume block number (PVBN) space accessible to all nodes, using dynamically extensible file systems (DEFS) that allow nodes to request and transfer allocation areas (AAs) dynamically, enabling independent file system operations and eliminating storage silos.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If each node of a distributed storage system is associated with a dedicated pool of storage space (node-level aggregate), then each node has independent storage management, but storage silos are created and storage space utilization becomes inefficient
Solution Approach 1:
The patent segments storage space into allocation areas (AAs) that can be independently assigned to different nodes. Instead of dedicating entire storage pools to each node, the storage space is divided into smaller units (AAs) that can be dynamically allocated, allowing multiple nodes to share the same physical storage space while maintaining independent management capabilities.
Solution Approach 2:
The patent implements dynamic allocation of storage space where nodes can request and receive allocation areas based on their current needs. The system monitors storage space availability and automatically transfers AAs between nodes, enabling the storage architecture to adapt dynamically to changing requirements rather than being static and dedicated.
2Device complexity
If storage space is dedicated to each node, then storage management is simplified, but the system cannot accommodate addition or removal of nodes and disks on demand
Solution Approach 1:
The system employs dynamic allocation areas that can be automatically assigned or reassigned when nodes are added or removed. When a new node joins the cluster, it can automatically receive AAs from the global pool or from other nodes, enabling on-demand accommodation without complex manual reconfiguration.
Solution Approach 2:
The global PVBN space serves multiple functions: it acts as both the physical storage space and the dynamic allocation pool. The same storage infrastructure supports both dedicated node operations and flexible resource sharing, allowing the system to handle various scenarios (node addition, removal, load balancing) through a unified mechanism.
3Productivity
If a global PVBN space is implemented for disaggregated storage, then storage utilization improves and silos are eliminated, but system complexity increases
Solution Approach 1:
The patent introduces allocation areas (AAs) as an intermediary layer between the global PVBN space and individual nodes. This intermediary abstraction simplifies management by providing a standardized unit of allocation that mediates between the complex global space and simple node requirements, making the system easier to manage despite the underlying complexity.
Solution Approach 2:
The system implements automatic monitoring and transfer mechanisms where nodes independently monitor their storage needs and automatically request or receive AAs from other nodes without manual intervention. This self-service approach reduces operational complexity by eliminating the need for administrators to manually manage storage allocation across the distributed system.
Data Source
AI summary
Systems and methods are provided for implementation and use of disaggregated storage by a distributed storage system. In various examples described herein, the association of blocks of storage space within a storage pod with a given dynamically extensible file system (DEFS) may be in large chunks of one or more gigabytes (GB), which may be referred to herein as “allocation areas” (AAs) that each include multiple RAID stripes. The use of large, multi-GB chunks, as the unit of space allocation/assignment to DEFSs of a distributed storage system facilitates ease of management and independence of write allocation. The use of AAs described herein further allows disk space to be used more fluidly across individual (nodes) storage systems of a distributed storage system (cluster of nodes working together), thereby eliminating silos of storage; and processing resource (e.g., CPU) load may be distributed across the cluster.