Multi-Tier Storage Scaling via Segmented Volumes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems face scalability challenges due to hardware limitations, requiring monolithic changes and inefficient resource utilization when scaling up storage capabilities.
Innovation Solution
A scalable multi-tier storage structure with independent scaling of each tier, comprising a directory volume for directory metadata, a file volume for file metadata, and a shard volume for file data, allowing for seamless expansion and resource optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional monolithic storage systems are used, then hardware resources are consolidated, but scalability is limited due to hardware constraints
Solution Approach 1:
The storage system is divided into three independent tiers: directory volume (first tier), file volume (second tier), and shard volume (third tier). Each tier can be scaled independently based on specific needs, allowing the system to adapt to different storage requirements without scaling the entire system. This segmentation enables flexible resource allocation and avoids the hardware constraints of monolithic systems.
2Quantity of substance
If storage capacity is increased by adding more hardware, then storage capabilities expand, but resource utilization becomes inefficient
Solution Approach 1:
The multi-tier architecture allows dynamic scaling where each tier can be independently adjusted based on actual storage needs. The directory volume, file volume, and shard volume can be scaled differently to match the specific requirements of metadata storage versus data storage, ensuring that resources are allocated efficiently without over-provisioning any single component.
3Quantity of substance
If the storage system is scaled up to meet growing data requirements, then storage capacity increases, but access performance may deteriorate
Solution Approach 1:
By separating the storage system into three tiers with distinct functions, the patent enables optimized access paths for different types of operations. Directory metadata access, file metadata access, and shard data access can occur in parallel through different pathways, maintaining access speed even as total storage capacity grows across multiple clusters.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Examples described herein generally relate to a scalable multi-tier storage system. An entry may be added and/or deleted within the storage system. To delete an entry, the storage system may determine whether the entry corresponds to the file or the directory based on directory metadata, and request deletion of the directory metadata associated with the entry from the directory volume based on determining that the entry corresponds to the directory, and further requesting deletion of the file from a file volume based on a determination that the entry corresponds to the file. To add a file, the storage system may generate directory metadata associated with the entry in the directory volume based on a determination that the entry corresponds to the directory, and may further allocate file metadata in the file volume based on a determination that the entry corresponds to the file.