Multi-tier Storage File Set Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage management systems lack an efficient mechanism to identify and move infrequently accessed files from higher to lower cost storage tiers without disrupting user access or requiring modifications to applications and databases.
Innovation Solution
The creation of a second file set with separate inodes for each tier allows for dynamic allocation and movement of files based on access metrics, enabling efficient identification and relocation of 'cold' files to lower cost storage tiers, while maintaining transparent access and supporting heterogeneous infrastructure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If all data is stored in high-end storage (higher tiers), then performance is improved, but storage cost increases significantly
Solution Approach 1:
The patent segments the storage system into multiple tiers (higher tier and lower tier) and segments files into hot files and cold files based on access frequency. By dividing the storage workload across different tiers, the system achieves high performance for frequently accessed files while storing less frequently accessed files in lower-cost storage, thus resolving the contradiction between performance and cost.
2Quantity of substance
If cold files are moved to lower tier storage, then storage cost is reduced, but identifying cold files becomes complex without efficient mechanisms
Solution Approach 1:
The patent implements a feedback mechanism by creating a second file set that monitors access patterns of files. The system continuously tracks which files are frequently accessed (hot) and which are infrequently accessed (cold), using this feedback information to automatically move files between tiers. This feedback-driven approach simplifies the identification of cold files and enables automated cost-optimized storage management.
Solution Approach 2:
The system enables files to essentially self-identify their access pattern characteristics through the second file set mechanism. By monitoring access patterns and automatically categorizing files as hot or cold based on observed behavior, the system eliminates the need for complex external analysis mechanisms, allowing files to effectively self-service their own storage tier placement decisions.
3Quantity of substance
If files are dynamically moved between tiers, then storage optimization is achieved, but transparency to applications must be maintained
Solution Approach 1:
The patent introduces an intermediary layer (the file set mechanism with inodes) that sits between the application and the physical storage tiers. This intermediary handles all the complexity of file movement, tier selection, and access pattern analysis transparently. Applications interact with files through this intermediary without needing to know whether files are stored in higher or lower tiers, thus maintaining ease of operation while achieving storage optimization.
Data Source
AI summary
Storage locations in a first tier of a multi-tier storage system are allocated to a first set of data structures (e.g., inodes) in a first file set. A file that is stored in the first tier is associated with a first data structure of the first set. In response to determining that data in the file should be moved to a second tier of the multi-tier storage system, the file is associated with a second data structure in a second file set. The second data structure is allocated a storage location in the second tier. Consequently, two data structures are associated with the file. The data is copied from the first tier to the storage location in the second tier, and can be subsequently accessed using the second data structure.