ML Storage Object Temperature Forecasting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face inefficiencies in predicting and managing the activity levels of storage objects, leading to suboptimal data tiering and prefetching strategies, which result in increased IO latency and reduced performance.
Innovation Solution
The use of machine learning models, specifically supervised regression models like random forest and gradient boosting algorithms, to forecast storage object temperatures based on IO operations and bandwidth, allowing for data tiering and prefetching optimizations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistical approaches (simple moving average, weighted moving average, exponential moving average) are used to forecast storage object temperatures, then implementation simplicity is maintained, but forecasting accuracy is limited
Solution Approach 1:
The patent replaces traditional statistical forecasting methods (mechanical/mathematical systems based on simple averaging) with machine learning models that can capture complex, non-linear patterns in storage object activity. This substitution enables significantly improved forecasting accuracy while the system manages model complexity through automated training and evaluation processes.
2Productivity
If data tiering and prefetching strategies are optimized based on accurate temperature forecasting, then IO latency is reduced and performance is improved, but computational overhead increases
Solution Approach 1:
The system performs preliminary forecasting of storage object temperatures using machine learning models to predict future activity levels. Based on these predictions, data tiering and prefetching actions are taken in advance - hot data is proactively moved to faster storage tiers and prefetching is triggered before actual access patterns occur. This preliminary action reduces IO latency and improves performance while the computational overhead is managed through efficient model inference and selective application of optimization strategies.
3Measurement precision
If machine learning models are used to forecast storage object temperatures, then forecasting accuracy is improved, but system complexity and computational resources required increase
Solution Approach 1:
The machine learning system is designed to be self-service through automated model training, evaluation, and selection processes. The system automatically trains multiple ML models on historical storage object activity data, evaluates their performance using validation datasets, and selects the best-performing models for deployment. This self-service approach improves forecasting accuracy while managing system complexity by eliminating manual model tuning and selection, allowing the system to adapt automatically to changing storage patterns.
Data Source
AI summary
Techniques for forecasting temperatures of storage objects in a storage system using machine learning (ML). The techniques can include forecasting at least one temperature of a storage object using at least one ML model, modifying storage of the storage object based on the at least one temperature of the storage object, and, having modified storage of the storage object, obtaining at least one performance metric associated with the storage object. The techniques can further include, based on the performance metric(s), varying a frequency of forecasting the at least one temperature of the storage object, retraining the at least one ML model used in forecasting the at least one temperature, and/or adjusting at least one operational parameter of the system. The techniques provide increased accuracy over known statistical approaches to forecasting temperatures of storage objects, leading to increased performance gains in terms of IO latency, IO operations per second, and bandwidth.