Storage Tier Data Promotion via I/O Statistics Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face inefficiencies in optimizing data movement between storage tiers, particularly in identifying and promoting data with non-sequential access patterns from lower performance tiers to higher performance tiers, which affects overall I/O performance and storage utilization.
Innovation Solution
A method that tracks I/O statistics for extents stored on lower performance tiers, determines which extents have non-sequential access patterns, and promotes only cached data from these tiers to higher performance tiers, such as SCM, by copying data from the cache without additional reads, and updates mapping information to reflect the new storage location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is moved between storage tiers based on traditional methods, then storage capacity is utilized, but I/O performance is not optimized because data with non-sequential access patterns is not identified and promoted
Solution Approach 1:
The system proactively identifies data with non-sequential access patterns and promotes it to higher performance tiers before it is actually needed, based on I/O statistics tracking. This preliminary action prevents performance degradation by ensuring hot data is already in optimal storage locations.
Solution Approach 2:
The system continuously monitors I/O statistics for each extent and uses this feedback to dynamically adjust data placement decisions. The feedback loop tracks access patterns and triggers promotion/demotion operations to optimize performance based on actual usage rather than static rules.
2Productivity
If all data is promoted to higher performance tiers, then I/O performance improves, but storage costs increase and storage efficiency decreases
Solution Approach 1:
The system applies different storage quality levels to different data portions based on their access characteristics. Data with non-sequential access patterns is promoted to high-performance tiers, while sequential access data remains in capacity-optimized tiers, creating localized optimization rather than uniform treatment.
Solution Approach 2:
The system changes the storage tier parameter dynamically based on detected access patterns. When non-sequential access is detected for an extent, the system transitions that data to a higher performance tier, and when sequential patterns return, demotes it back, adapting storage parameters to actual usage.
3Speed
If data is promoted from lower to higher performance tiers, then access speed improves, but additional read operations are required which consume extra I/O resources
Solution Approach 1:
The system performs data promotion in advance based on detected access patterns, so that when data is actually accessed, it is already in the high-performance tier. This eliminates the need for additional read operations to fetch data from lower tiers, as the promotion happens proactively based on I/O statistics.
4Productivity
If traditional data movement methods are used, then storage operations are simple, but storage efficiency is reduced due to inability to optimize data placement based on access patterns
Solution Approach 1:
The system automatically monitors I/O statistics, identifies non-sequential access patterns, and performs promotion/demotion operations without manual intervention. The storage system serves itself by making intelligent placement decisions based on observed behavior, eliminating the need for complex external management while improving efficiency.
Data Source
AI summary
Techniques for performing data movements may include tracking sets of I/O statistics for a extents, wherein each of the sets of I/O statistics characterizes I/O operations directed to a corresponding one of the extents having data stored on a first storage tier of first non-volatile storage devices; determining, in accordance with the sets of I/O statistics, a list that includes one or more of the extents qualified for promotion from the first storage tier to a second storage tier, wherein the second storage tier includes second non-volatile storage devices have a higher performance ranking that the first non-volatile storage devices of the first storage tier; selecting, from the list, a first extent to be promoted from the first storage tier to the second storage tier; and performing first processing that promotes only cached data of the first extent from the first storage tier to the second storage tier.