ARIMA-Based Workload Forecasting for Multi-Tier Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face inefficiencies in optimizing data storage tiers, leading to suboptimal performance and capacity utilization due to inadequate prediction and movement of data portions based on I/O statistics, resulting in response time objectives not being met and exceeding storage capacity limits.
Innovation Solution
A method utilizing auto-regressive integrate moving average models to predict I/O statistics and determine data movement from one storage tier to another, optimizing data placement across different performance tiers based on response time objectives, workload characteristics, and capacity limits, with a data storage optimizer managing the movement of data portions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in lower performance storage tiers to optimize capacity utilization, then storage capacity is improved, but response time deteriorates
Solution Approach 1:
The patent implements dynamic data placement that automatically adjusts storage tier allocation based on real-time I/O workload characteristics and predicted future access patterns. The system continuously monitors I/O statistics and uses adaptive algorithms to move data between performance tiers, ensuring that frequently accessed data resides in high-performance storage while capacity-constrained data moves to lower-performance tiers, thus dynamically balancing capacity utilization and response time performance
Solution Approach 2:
The system performs preliminary data movement operations based on predicted I/O workloads before performance degradation occurs. By analyzing historical I/O patterns and predicting future access requirements, the system proactively positions data in appropriate storage tiers in advance, preventing response time violations rather than reacting after performance targets are missed
2Device complexity
If data movement is delayed to reduce processing overhead, then device complexity is reduced, but productivity deteriorates
Solution Approach 1:
The patent implements periodic evaluation cycles where the storage system assesses I/O workload characteristics and determines data movement operations at scheduled intervals. This periodic approach balances processing overhead by not continuously analyzing every I/O operation, while still maintaining storage optimization efficiency through regular reassessment of data placement based on accumulated I/O statistics and workload pattern recognition
3Quantity of substance
If storage capacity is expanded to accommodate all data, then capacity limits are avoided, but response time performance deteriorates
Solution Approach 1:
The patent applies local quality optimization by assigning different storage tier qualities to different data portions based on their specific access patterns and I/O workload characteristics. Instead of uniformly storing all data in high-performance storage, the system identifies and places only the most frequently accessed data in premium tiers, while less critical data resides in capacity-oriented lower-performance tiers, thus achieving response time objectives for critical data while maximizing overall capacity utilization
Data Source
AI summary
Data storage optimization techniques determine predicted values for I/O statistics using an ARIMA (auto-regressive integrated moving average) model. The ARIMA model may be used to capture periodic patterns and trends of workload I/O access to predict the future load demand. A current set of I/O statistics is collected for a current time period T. Using the current set and one or more ARIMA models, a predicted set of I/O statistics is determined for a next time period T+1. Each of the ARIMA models is characterized by model parameters including P denoting a number of auto-regressive terms, D denoting a number of nonseasonal difference needed for stationarity, and Q denoting a number of lagged forecast errors of prediction. A data storage optimizer may determine one or more data portions for movement from a current storage tier to a target storage tier using the predicted set of I/O statistics.