Storage Array Data Tiering via Exponential Moving Average
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage arrays face performance limitations due to the need for manual management of data tiers, which requires significant effort to categorize and maintain data placement based on access patterns, and are constrained by the use of identical disks, limiting the ability to differentiate between hot and cold data.
Innovation Solution
A method that analyzes I/O data to compute an exponential moving average of data access activity, allowing for automatic tiering of data based on access patterns, enabling migration of hot data to faster storage and cold data to slower storage, thereby optimizing performance and cost without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual management of data tiers is implemented, then data placement based on access patterns can be achieved, but significant effort and time are required for categorization and maintenance
Solution Approach 1:
The system automatically monitors I/O activity and performs data migration between tiers without manual intervention. The storage system self-manages the categorization and placement of data based on real-time access patterns, eliminating the need for administrators to manually categorize and maintain data tiers.
Solution Approach 2:
The system continuously monitors I/O activity metrics and uses this feedback to dynamically adjust data placement. By measuring data access activity and using exponential moving averages, the system adapts to changing access patterns and automatically migrates data between hot and cold tiers based on current system state.
2Adaptability or versatility
If identical disks are used in the storage array, then system simplicity is maintained, but the ability to differentiate between hot and cold data storage is limited
Solution Approach 1:
The system creates different storage tiers with distinct performance characteristics by using identical physical disks but organizing them into logical groups with different properties. Fast tier disks are configured for high-performance access while slow tier disks are configured for capacity-oriented storage, effectively creating local quality differences without requiring different hardware.
Solution Approach 2:
The system differentiates storage tiers by changing operational parameters such as I/O scheduling priorities, cache allocation, and access patterns rather than requiring different hardware. By modifying these parameters, the same physical disks can serve different functional roles for hot and cold data.
3Productivity
If computational overhead for tracking data access patterns is reduced, then system efficiency improves, but the precision of data temperature classification may be affected
Solution Approach 1:
The system tracks only the most relevant I/O activity metrics rather than all possible access patterns. By focusing on key indicators such as read/write frequency and data temperature changes, the system achieves sufficient classification precision without the computational burden of comprehensive monitoring.
Solution Approach 2:
The system uses exponential moving averages to smooth I/O activity measurements over time, transforming raw fluctuating data into stable temperature classifications. This parameter transformation maintains classification accuracy while reducing the computational complexity of real-time analysis.
Data Source
AI summary
A method is used in measuring data access activity. I/O data is analyzed that describes I/O activity for a slice of a logical volume. Based on the I/O data, a first value of data access activity is determined for the slice corresponding to a first time period, and a second value of data access activity is determined corresponding to a second time period. From the first and second values, an exponential moving average of data access activity is derived for the slice.