Multi-Tiered Storage Policy Segmentation for Distributed File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional distributed file systems lack flexibility in setting up storage policies, as they do not allow service providers to set one policy that coexists with a customer-selected policy, limiting control over data distribution across nodes and affecting efficiency and reliability.
Innovation Solution
A multi-tiered storage policy approach is implemented, where different data distribution tiers can apply distinct storage policies, such as hierarchical, random, or multi-tenancy policies, allowing for flexible data placement and minimizing the complexity and maintenance of the distributed file system service by using the same data distribution module across tiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional distributed file systems use random distribution to balance files across nodes, then node balancing is achieved, but flexibility in setting up storage policies is lost
Solution Approach 1:
The storage policy configuration is segmented into multiple independent tiers (first tier, second tier, third tier), each with its own distribution policy. This allows different policies to be applied at different levels without interfering with each other, providing flexibility while maintaining manageable complexity through modular organization.
Solution Approach 2:
The patent introduces a hierarchical dimension to storage policy configuration, organizing policies across multiple tiers rather than a single flat level. This dimensional structure enables simultaneous application of different distribution strategies (random, hierarchical, multi-tenancy) at different levels of the storage hierarchy.
2Adaptability or versatility
If multiple different data distribution policies are applied at different tiers, then flexibility and control over data distribution is improved, but system complexity increases
Solution Approach 1:
A universal data distribution module is implemented that can apply multiple different distribution policies (random, hierarchical, multi-tenancy) across different tiers. This single module serves multiple functions by configuring different policies at different levels, reducing overall system complexity while maintaining high flexibility in data distribution control.
3Adaptability or versatility
If traditional storage systems use single-level storage policies, then system simplicity is maintained, but flexibility for service providers and customers to set coexisting policies is lost
Solution Approach 1:
The policy configuration interface is segmented into multiple tiers, allowing service providers to set policies at the first tier while customers can select and apply their own policies at lower tiers. This segmentation enables coexisting policies to be configured independently and applied simultaneously, maintaining ease of operation through clear hierarchical separation.
Solution Approach 2:
The system provides dynamic policy selection where customers can choose different distribution policies (random, hierarchical, multi-tenancy) at different tiers based on their specific needs. This dynamic configuration capability allows flexible coexistence of multiple policies while maintaining user-friendly operation through intuitive policy selection interfaces.
Data Source
AI summary
A processing device identifies a file (or data block) and selects a storage location for the file based on a plurality of data distribution policies. To select the storage location, the processing device applies, at a top distribution layer, a first data distribution policy to select a first storage pool from a plurality of storage pools based on a first storage criterion. The processing device then applies, at a bottom distribution layer, a second data distribution policy associated with the first storage pool to select a first storage node from a plurality of storage nodes in the first storage pool based on a second storage criterion. The processing device stores at least a portion of the file in the first storage node of the first storage pool.