Tiered Storage Block Retiering via Access Statistics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tiered storage systems face challenges in efficiently managing data movement between storage tiers based on access frequency and other factors, often requiring manual intervention or periodic automated processes that can be complex and time-consuming.
Innovation Solution
A method for storing and retiering data in a tiered storage system that involves receiving policies for data stores, storing data as blocks on multiple storage tiers with different performance levels, generating statistics on block access, and dynamically moving blocks between tiers based on their access patterns and defined policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If manual intervention or periodic automated processes are used for data movement between storage tiers, then data can be moved between tiers, but the process becomes complex and time-consuming
Solution Approach 1:
The system implements self-service by automatically monitoring block access patterns and autonomously making retiering decisions without manual intervention. The retiering process is triggered automatically when access statistics indicate a need for optimization, and the system self-manages the entire process of evaluating, deciding, and executing data movement between tiers.
Solution Approach 2:
The system continuously collects feedback on block access patterns and uses this information to dynamically adjust retiering decisions. Access statistics are monitored in real-time, and this feedback drives automated decisions about which blocks should be moved between storage tiers, creating a closed-loop control system that adapts to changing data access patterns.
2Speed
If data is stored on higher performance tiers, then access speed is improved, but storage cost increases
Solution Approach 1:
The system dynamically adjusts data placement between storage tiers based on changing access patterns rather than using static placement. Blocks are continuously evaluated and moved between tiers as their access characteristics evolve, allowing the system to optimize the balance between performance and cost in real-time rather than relying on fixed tier assignments.
Solution Approach 2:
The system changes the storage tier parameter for each block based on its access statistics. By monitoring access patterns and adjusting the storage location parameter dynamically, the system optimizes performance for frequently accessed data while moving less frequently accessed data to lower-cost tiers, thereby balancing speed and cost.
3Productivity
If periodic automated retiering processes are used, then data can be optimized for performance, but system complexity increases
Solution Approach 1:
The retiering system operates autonomously by self-monitoring access patterns and self-making optimization decisions without requiring complex external control mechanisms. This self-service approach simplifies the overall system architecture while maintaining effective performance optimization.
Solution Approach 2:
The system introduces an intermediary layer that monitors access patterns and manages retiering decisions, separating the complexity of optimization logic from both the data access operations and the physical storage management. This intermediary component handles the complexity internally while presenting a simplified interface to users and applications.
Data Source
AI summary
A method to store data in tiered storage includes receiving for data stores to be stored on the tiered storage. The tiered storage may include multiple storage tiers, each storage tier having a different performance level. Each policy defines a default storage tier for the corresponding data store that has the policy. The method includes storing the data stores on the tiered storage as blocks. Each of the blocks is stored to a corresponding one of the storage tiers based at least on the policy of the data store to which the block belongs.