Storage Tiering via Application Activity Predictors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current tiered storage systems face performance limitations due to thrashing caused by solely activity-based promotion and demotion decisions, leading to inefficient use of storage resources and potential system-wide performance issues, especially when dealing with significantly different performance characteristics between tiers.
Innovation Solution
Implementing application awareness to monitor and adjust data movement between storage tiers based on both actual data access activity and future application activity indicators, using growth and decay functions to control the apparent activity level, thereby preventing thrashing and optimizing storage resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data movement decisions are based solely on monitored activity, then storage resource utilization is simplified, but thrashing occurs and performance deteriorates
Solution Approach 1:
The system implements feedback mechanisms by continuously monitoring application activity indicators and using this information to adjust data movement decisions. The storage system receives feedback about application state (active/inactive) and modifies promotion/demotion behavior accordingly, preventing thrashing by adjusting movement decisions based on observed performance outcomes.
Solution Approach 2:
The system performs preliminary actions by proactively promoting data to higher storage tiers when applications are detected to be in an active state, before the data is actually needed. This anticipatory data movement prepares the system for upcoming access patterns, improving performance by having data ready in advance rather than waiting for actual access to occur.
2Reliability
If growth and decay functions are used to control data movement, then thrashing is reduced, but data occupies higher tier resources for extended periods even when applications are inactive
Solution Approach 1:
The system applies dynamic control by adjusting data movement behavior based on real-time application state. When applications transition from active to inactive state, the system dynamically changes from proactive promotion to aggressive demotion, allowing data to leave higher tiers quickly. This dynamic adaptation resolves the contradiction by making the storage system flexible rather than static in its data movement decisions.
Solution Approach 2:
The system changes parameters by modifying promotion and demotion thresholds based on application activity indicators. When applications are inactive, the system lowers demotion thresholds to enable faster data egress from higher tiers. When applications become active, the system raises promotion thresholds to prepare data in advance. These parameter adjustments allow the system to optimize both stability and resource utilization under different operating conditions.
3Adaptability or versatility
If significant performance differences exist between storage tiers, then cost-effectiveness is improved, but system-wide performance problems occur when low tiers are overloaded
Solution Approach 1:
The system uses application state information as an intermediary to mediate between storage tier performance characteristics and data movement decisions. This intermediary information allows the system to make intelligent routing decisions that prevent overload of lower tiers by keeping frequently accessed application data in appropriate tiers, thus maintaining system-wide performance while preserving the cost benefits of tiered storage.
Data Source
AI summary
Predictors of future application activity/inactivity are used in combination with monitored data access activity to prompt movement of data between tiers of storage. The apparent amount of activity for data associated with an application that is expected to become active to be increased even before data associated with that application becomes active. Further, the apparent amount of activity for data associated with an application that has become inactive is rapidly decreased by overriding algorithms that control rate of change to counter thrashing.