Directory-Based Storage Tiering in Parallel File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage tiering arrangements in parallel file systems are inflexible, limiting the ability to dynamically adjust data placement based on access frequency and other factors.
Innovation Solution
Implementing directory-based storage tiering by associating parallel file system directories with specific sets of storage servers and devices, allowing data movement between tiers using a SYNCer module or similar data mover modules, and utilizing extended attributes to determine data placement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional storage tiering arrangements are used in parallel file systems, then storage systems can provide multiple storage tiers with different IO performance levels, but the storage tiering arrangements are unduly inflexible and cannot dynamically adjust data placement
Solution Approach 1:
The patent segments the storage system by associating specific directories with specific storage tiers through extended attributes. Instead of treating the entire file system as a single tiered structure, it divides the namespace into directory-level segments that can be independently assigned to different storage tiers (e.g., SSD tier, HDD tier, archive tier). This segmentation enables flexible, fine-grained control over data placement while maintaining manageable system complexity.
Solution Approach 2:
The patent implements dynamic storage tiering by allowing the association between directories and storage tiers to be changed at runtime through extended attributes. The system can dynamically migrate data between tiers based on access patterns, storage capacity, and performance requirements without requiring system reconfiguration. This dynamic capability resolves the contradiction by providing adaptability while using standardized file system mechanisms to avoid excessive complexity.
2Productivity
If data is moved between storage tiers based on access frequency, then storage performance is optimized, but conventional arrangements lack the flexibility to implement this in parallel file systems
Solution Approach 1:
The patent implements feedback-based storage tiering by monitoring data access patterns and using this information to dynamically adjust directory-to-tier associations through extended attributes. The system continuously gathers feedback on access frequency, access time, and storage utilization, then uses this feedback to make intelligent decisions about data migration between tiers. This feedback mechanism enables optimized data access performance while maintaining the flexibility to adapt to changing workloads.
Solution Approach 2:
The patent changes the parameter of directory-to-tier association from fixed to variable by using extended attributes that can be modified at runtime. This allows the system to change storage tier assignments based on evolving access patterns and requirements. The extended attributes serve as flexible parameters that can be adjusted without altering the underlying file system structure, enabling both performance optimization and adaptability.
3Loss of energy
If multiple storage tiers with different IO performance are used, then storage costs are reduced by placing less frequently accessed data on slower tiers, but conventional tiering arrangements are too rigid for parallel file systems
Solution Approach 1:
The patent enables self-service storage tiering by implementing automated policies that monitor storage usage, access patterns, and tier capacity, then automatically migrate data between tiers without manual intervention. The system uses extended attributes to track and manage tier associations, and automated processes to enforce retention policies and optimize data placement. This self-service capability reduces storage costs through intelligent tiering while eliminating the operational complexity of manual tier management.
Data Source
AI summary
An apparatus in one embodiment comprises a storage system having at least first and second storage tiers each comprising a plurality of storage devices. The storage system implements a parallel file system comprising at least first and second sets of storage servers. The first set of storage servers is configured to access directories of the parallel file system that are stored in storage devices of the first storage tier, and the second set of storage servers is configured to access directories of the parallel file system that are stored in storage devices of the second storage tier. The storage system is further configured to move data between the first and second storage tiers by moving data files between different directories of the respective first and second storage tiers. One or more servers of the first set may be generated as respective new server instances on one or more compute nodes.