Correlation Filter for Composite Disk Data Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In composite disk systems, frequently accessed data blocks on slower magnetic hard drives are sometimes misidentified, leading to inefficient data migration to faster solid-state drives, resulting in suboptimal performance due to false positive determinations during data eviction and migration.
Innovation Solution
A correlation filter is employed to track recent accesses using a hash-indexed correlation array, filtering out correlated references within a specified time frame to accurately identify frequently accessed data blocks and reduce I/O overhead during data migration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If frequent access detection is used to identify data blocks for migration, then data migration speed is improved, but false positive determinations increase leading to reduced migration accuracy
Solution Approach 1:
The patent segments the access detection process into multiple independent data structures: a correlation array for tracking recent accesses, a migration unit structure for grouping blocks, and a composite disk management structure for overseeing the entire system. This segmentation allows each component to specialize in specific aspects of access detection, improving overall accuracy while maintaining speed.
Solution Approach 2:
The patent introduces intermediary structures between the raw access detection and the final migration decision. The correlation array acts as an intermediary that filters and processes access patterns before they reach the migration decision logic, reducing false positives by intermediate processing of access frequency data.
2Loss of energy
If multiple data blocks are stored in a single migration unit, then I/O overhead is reduced, but the complexity of tracking and managing migration units increases
Solution Approach 1:
The patent merges multiple data blocks into single migration units, combining their access tracking into a unified structure. The migration unit structure contains arrays that aggregate access counts and correlation data for multiple blocks, reducing the number of individual I/O operations needed while managing complexity through structured organization.
Solution Approach 2:
The migration unit structure serves multiple functions simultaneously: it groups data blocks for efficient I/O operations, tracks access patterns for each contained block, manages correlation data, and facilitates bulk migration operations. This multi-functionality reduces overall system complexity by consolidating what would otherwise require separate structures.
3Measurement precision
If a correlation array tracks all migration unit addresses, then correlated accesses are filtered accurately, but the memory space required increases
Solution Approach 1:
The correlation array is segmented into buckets, with each bucket containing a limited set of migration unit addresses. This segmentation divides the large memory requirement into smaller, manageable portions while maintaining the ability to track all necessary addresses through distributed storage across multiple buckets.
Solution Approach 2:
The patent transforms the one-dimensional problem of storing all addresses into a multi-dimensional structure using buckets and indices. Addresses are distributed across buckets based on hash functions or other indexing mechanisms, effectively adding dimensional organization that reduces the contiguous memory space required while maintaining comprehensive tracking capability.
Data Source
AI summary
In one embodiment, the correlation filter can use one of several data structure to track each migration unit and reject successive accesses within a period of time to each migration unit. In one embodiment, the correlation filter uses a space efficient data structure, such as a hash indexed correlation array to store the address of referenced migration units, and to filter accesses to a single migration unit that are correlated accesses resulting from multiple accesses to the same migration unit during a sequential I/O stream. In one embodiment, the correlation array contains a global timeout, which resets each element to a default value, clearing all store migration unit address values from the correlation array. In one embodiment, each element of the migration array can time-out separately.