Storage Tier Migration via Activity Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face inefficiencies in optimizing data movement between different storage tiers based on activity levels, leading to suboptimal performance and resource utilization.
Innovation Solution
A method for generating hints based on aggregated activity data, which involves ranking data portions by intensity and entropy metrics to determine optimal storage tier movements, promoting active data to higher performance tiers and demoting less active data to lower tiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data storage systems use traditional static storage tiering, then storage capacity is sufficient, but performance optimization based on activity levels is insufficient
Solution Approach 1:
The patent implements dynamic storage tiering by continuously monitoring data access patterns and activity levels, then automatically migrating data between different storage tiers (e.g., SSD to HDD) based on real-time performance requirements. This dynamic approach allows the storage system to adapt to changing workload patterns, promoting frequently accessed data to higher-performance tiers and demoting less active data to lower-cost tiers, thereby resolving the contradiction between maintaining high performance and adapting to varying activity levels.
Solution Approach 2:
The system employs feedback mechanisms where performance data and activity metrics are collected, analyzed, and used to adjust storage tier assignments. The feedback loop includes monitoring I/O operations, calculating activity scores, and using these results to trigger data migration decisions. This closed-loop feedback enables the system to learn from historical patterns and make intelligent storage optimization decisions, improving both performance and adaptability.
2Loss of time
If data is always stored in high-performance storage tiers, then response time is minimized, but storage cost increases
Solution Approach 1:
The patent applies local quality by differentiating storage treatment based on individual data characteristics and access patterns. Instead of uniformly storing all data in high-performance tiers, the system identifies locally optimal storage locations for different data types and access frequencies. Frequently accessed critical data resides in high-performance tiers, while less active data is placed in cost-effective lower-tier storage, achieving optimal resource allocation that balances response time requirements with storage cost constraints.
Solution Approach 2:
The storage system is segmented into multiple performance tiers (e.g., cache tier, hot tier, warm tier, cold tier) with different performance and cost characteristics. The patent implements segmentation strategies that divide data based on activity levels, allowing each segment to be stored in the most appropriate tier. This segmentation enables fine-grained control over storage resources, ensuring that high-performance storage capacity is allocated only where needed to minimize response time while controlling overall storage costs.
3Extent of automation
If manual data movement is used, then control is maintained, but automation level is insufficient
Solution Approach 1:
The system implements self-service automation where the storage system autonomously monitors its own performance metrics, analyzes activity patterns, and executes data migration decisions without manual intervention. The automation framework includes self-diagnosis capabilities that detect performance degradation or optimization opportunities, self-planning that determines optimal migration strategies, and self-execution that performs the actual data movement. This self-service approach significantly increases automation extent while managing complexity through standardized, rule-based decision logic and automated policy enforcement.
4Measurement precision
If historical activity data is aggregated for analysis, then prediction accuracy improves, but processing time increases
Solution Approach 1:
The patent implements preliminary action by pre-processing and aggregating historical activity data in advance of actual migration decisions. The system performs preliminary analysis of past access patterns, calculates activity scores, and identifies migration candidates before new data arrives or before migration execution. This advance preparation enables more accurate predictions about future data access patterns while reducing real-time processing requirements, as the heavy computational tasks are performed beforehand during off-peak periods or in batch modes.
Data Source
AI summary
Described are techniques for providing hints. Activity data for data portions is received. For each data portion, activity data is aggregated for each of a plurality of time periods and a value for a metric is determined for the data portion in accordance with the aggregated activity data. The values for the metric are ranked. A set of one or more data portions are identified based on the ranked metric values. Hint generation processing is performed to generate a hint for each data portion in said set. The hint may be used in connection with performing data storage movement optimizations to move selected data portions between different storage tiers having different performance classifications. The metric may be determined using one or more criteria including standard deviation, entropy and/or a maximum probability for a time period having the largest portion of I/O wait time for a data portion.