Storage Tiering Prediction Using State Space Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in predicting future data usage patterns, leading to inefficient storage tier management, where data may be misplaced on slower or more expensive storage tiers just before it is needed, due to reliance on immediate past usage frequency.
Innovation Solution
A system that uses a state space model to predict future data usage by evaluating characteristics of data sections and moving them between different storage tiers based on anticipated usage, with software managing logical devices to map sections to appropriate physical storage areas with distinct properties.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is moved between storage tiers based on immediate past usage frequency, then storage cost is reduced by placing less frequently used data on cheaper tiers, but data may be misplaced on slower or more expensive storage tiers just before it is needed
Solution Approach 1:
The system performs preliminary actions by predicting future data usage patterns and proactively moving data to appropriate storage tiers before actual access occurs. The state space model analyzes historical access patterns to forecast when data will be needed, enabling advance placement on optimal tiers and avoiding last-minute data movements that would place data on incorrect tiers.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring actual data access patterns and comparing them with predicted patterns. This feedback loop allows the state space model to refine its predictions over time, improving the accuracy of data placement decisions and ensuring that data is positioned on appropriate tiers based on both historical and real-time usage information.
2Reliability
If data is moved to faster storage tiers to ensure availability, then data access reliability is improved, but storage cost increases due to placement on more expensive tiers
Solution Approach 1:
The system performs preliminary actions by predicting future data usage patterns and proactively moving data to appropriate storage tiers before actual access occurs. The state space model analyzes historical access patterns to forecast when data will be needed, enabling advance placement on optimal tiers and avoiding last-minute data movements that would place data on incorrect tiers.
Solution Approach 2:
The system dynamically changes the parameter of data placement decisions by using the state space model to determine optimal storage tier selection. Instead of static placement rules, the system adjusts data placement parameters based on predicted usage patterns, allowing data to be positioned on the most cost-effective tier that still ensures availability when needed.
3Measurement precision
If predictions are made for each hour of the day or for each data object, then prediction accuracy is improved, but computational complexity increases
Solution Approach 1:
The system applies segmentation by dividing prediction tasks into manageable units - either time-based segments (hourly predictions) or data-object-based segments (individual file or volume predictions). This segmentation allows the state space model to process predictions in discrete, independent units, improving accuracy while enabling parallel processing and reducing overall computational complexity.
Data Source
AI summary
Managing data includes providing at least one logical device that maps sections of the logical device to sections of at least two physical storage areas, where the at least two physical storage areas have different physical storage properties, evaluating characteristics of data associated with at least one of the sections of the logical device, where at least some of the characteristics depend upon predictions as a function of time of future data usage based on past data usage and on heuristic values, and moving the at least one of the sections of the logical device between the at least two storage areas from a first location to a second location according to the characteristics of the data. Each of the at least two physical storage areas may correspond to a different storage tier. Predictions of future data usage may be provided by a state space model.