Auto-tiering Data Movement Selection via Workload Skew Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face inefficiencies in optimizing data movement across different storage tiers based on criteria beyond I/O workload, leading to suboptimal performance and resource utilization.
Innovation Solution
A method and system for ranking and selecting data movements to optimize data placement across multiple storage tiers by considering characteristics such as file system metadata, application criticality, I/O workload, and quality of service levels, and implementing a subset of the highest ranked movements to improve performance and resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If all proposed data movements are implemented based on I/O workload ranking, then I/O performance is improved, but system resources are overwhelmed and productivity decreases
Solution Approach 1:
The patent applies partial action by selecting and implementing only a subset of the most critical data movements from the ranked list, rather than executing all proposed movements. This selective approach focuses resources on high-impact movements while avoiding resource exhaustion, thereby maintaining I/O performance improvements without overwhelming the system.
Solution Approach 2:
The system implements feedback mechanisms by monitoring resource utilization and data movement outcomes, using this information to dynamically adjust which data movements are selected for implementation. This feedback loop ensures that the system adapts to current resource conditions while continuing to optimize I/O performance through strategic data placement.
2Reliability
If multiple criteria are used for ranking data movements, then data placement optimization is improved, but system complexity increases
Solution Approach 1:
The patent segments the ranking criteria into distinct, manageable components such as I/O workload metrics, data characteristics, storage tier properties, and application priorities. Each criterion can be independently calculated and weighted, making the overall complex ranking system more manageable and maintainable while still achieving comprehensive data placement optimization.
Solution Approach 2:
The system utilizes parameter changes by allowing the weights and thresholds of different ranking criteria to be dynamically adjusted based on system conditions and optimization goals. This flexibility enables the system to adapt to changing requirements without redesigning the entire ranking mechanism, thereby managing complexity while maintaining optimization effectiveness.
3Use of energy by moving object
If a subset of data movements is selected for implementation, then resource utilization is improved, but I/O performance optimization is reduced
Solution Approach 1:
The patent applies local quality by selecting data movements based on their specific characteristics and the local conditions of the storage system. Rather than uniformly treating all data movements, the system identifies and prioritizes movements that will have the most significant local impact on I/O performance while considering resource constraints, thereby achieving effective optimization with limited resources.
Data Source
AI summary
Described are techniques for performing data movement optimization processing comprising: receiving a list of proposed data movements; ranking the list in accordance with one or more criteria associated with each of the proposed data movements of the list, wherein the one or more criteria for each proposed data movement, that moves a data portion to a target storage tier, includes at least one characteristic about each proposed data movement other than I/O workload directed to the data portion; selecting a subset of the proposed data movements of the list; implementing the subset of the proposed data movements by performing each of the proposed data movements of the subset; and revising the list to remove proposed data movements of the subset implemented in said implementing. A new list may be obtained each time period. A subset of the ranked list may be implemented each time period in a cycle.