Storage Tiering Engine Using Pool State Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face challenges in efficiently managing storage tiers based on predicted activity and capacity demands, leading to suboptimal performance and service level objective compliance, with existing systems often promoting or demoting data unnecessarily, resulting in degraded states and performance issues.
Innovation Solution
A data storage system with a data movement engine that assigns states to storage pools based on activity and capacity demands, promoting extents to higher performance tiers when activity is unacceptable and demoting to lower tiers when capacity is unacceptable, while maintaining local and global event records and historical databases to inform tiering decisions and ensure service level objective compliance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If extents are promoted or demoted based on predicted activity alone, then storage tier performance is optimized, but unnecessary promotions and demotions occur leading to degraded states
Solution Approach 1:
The system monitors the actual state of storage pools after promotion/demotion decisions and uses this feedback to adjust future decisions. When a storage pool enters a degraded state, the system detects this through continuous monitoring of demanded activity and capacity, and reverses the problematic promotion/demotion action, creating a closed-loop control system that prevents unstable oscillations.
Solution Approach 2:
The system evaluates whether a storage pool can tolerate further promotions or demotions before making such decisions. By assessing the current state and capacity margins, the system prevents actions that would push the pool into a degraded state, cushioning against future problems before they occur.
2Productivity
If storage pools accept high demanded activity, then service level objectives are met, but capacity may become unacceptable leading to second degraded state
Solution Approach 1:
The system dynamically adjusts the balance between demanded activity and demanded capacity based on current pool state. When capacity becomes constrained, the system prioritizes capacity management over activity maximization, and vice versa, allowing flexible adaptation to changing conditions rather than rigidly prioritizing one metric.
3Quantity of substance
If extents are demoted to lower performance tiers to free capacity, then storage pool capacity becomes acceptable, but performance may degrade
Solution Approach 1:
After demoting extents to free up capacity, the system monitors whether the storage pool remains in a healthy state. If performance degradation occurs, the feedback mechanism triggers reverse actions to restore the pool to a healthy state, ensuring that capacity optimization does not compromise service level objectives.
Data Source
AI summary
Promotion and demotion of extents for each storage pool is determined based on a storage pool state defined by demanded activity and demanded capacity. Promotion and demotion of extents for each storage group is determined based on whether the storage group is performing at SLO, above SLO or below SLO. Events are identified and stored in local records using hash tables. The locally stored events are used to update a database of historical extent activity. A moving average of extent activity is used to calculate the demanded activity.