DPU ARIMA Forecasting for Hot-Cold Storage Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer storage systems face inefficiencies in data compression and decompression due to constant processing of frequently accessed data, impacting system response time and resource consumption, particularly when CPU resources are overloaded under heavy I/O workloads.
Innovation Solution
Implementing autoregressive integrated moving average (ARIMA) models using data processing units (DPUs) for forecasting I/O activity in storage arrays to determine hot and cold data, offloading CPU-intensive tasks and optimizing data compression decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is applied to storage arrays, then storage efficiency is improved, but system response time deteriorates due to constant compression and decompression of frequently accessed data
Solution Approach 1:
The system performs preliminary classification of storage locations into hot and cold categories using ARIMA forecasting before compression operations. By predicting future I/O activity patterns and pre-classifying data locations, the system determines which data should be compressed in advance, avoiding compression of frequently accessed data and thus maintaining fast response times while still achieving storage efficiency gains.
2Productivity
If CPU resources are used for I/O workload processing, then data processing capability is improved, but resource consumption deteriorates under heavy I/O workloads
Solution Approach 1:
The system introduces an intermediary forecasting mechanism using ARIMA models that runs independently of the main CPU I/O processing path. This intermediary component analyzes I/O patterns and generates temperature scores that guide compression decisions, allowing the CPU to focus on data processing while the forecasting mechanism handles optimization decisions, thereby reducing overall CPU resource consumption under heavy workloads.
3Device complexity
If compression decisions are made without forecasting, then system complexity is reduced, but data reduction effectiveness deteriorates
Solution Approach 1:
The system changes the parameter used for compression decisions from static or simple access-based criteria to dynamic temperature scores generated by ARIMA forecasting. This parameter change allows the system to identify truly cold data that will remain inactive, improving data reduction effectiveness. The forecasting adds computational complexity but provides more accurate predictions of data coldness, enabling better compression decisions.
Data Source
AI summary
A system can maintain a group of data processing units and a storage array that comprises a group of sub-logical unit numbers of storage. The system can collect, by a central processing unit, first data indicative of input and output events for the storage array. The system can process, by respective data processing units, respective autoregressive integrated moving average models for respective sub-logical unit numbers of the group of sub-logical unit numbers with the first data, to generate respective statuses that indicate respective frequencies of access of the respective sub-logical unit numbers. The system can determine, by the central processing unit, respective classifications for respective sub-logical unit numbers of the group of sub-logical unit numbers of storage based on the respective statuses. The system can compress, by a compression engine, second data stored in at least some of the respective sub-logical unit numbers based on the respective classifications.