Extent Migration Speed via Stride-Level IO Interception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The speed of extent migration in tiered storage systems is hindered by the need to recopy data that becomes dirty during the migration process, leading to slowdowns and inefficiencies, as data is frequently fetched and written, with current technologies struggling to manage large data chunks effectively.
Innovation Solution
A computer-implemented method that determines extents to migrate, divides them into strides, and prioritizes their copying based on cache utilization and IO statistics, intercepts IO commands to prevent data from becoming dirty, and dynamically adjusts the migration queue to minimize recopying, using a migration manager to optimize the process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is frequently fetched and written during migration, then data accessibility is improved, but migration speed deteriorates due to recopying overhead
Solution Approach 1:
The system intercepts IO commands before they can modify migrated data, preventing data from becoming dirty in advance. This preliminary action avoids the need for recopying operations that would otherwise be required to maintain data consistency, thereby preserving migration speed while still allowing data accessibility.
Solution Approach 2:
The migration manager applies a counter-action by blocking IO commands that would write to migrated extents. This preliminary anti-action prevents the harmful effect of data modification during migration, eliminating the recopying overhead that would result from such modifications and maintaining high migration speed.
2Productivity
If the migration queue is dynamically adjusted to prioritize certain strides, then migration efficiency is improved, but system complexity increases
Solution Approach 1:
The migration manager continuously monitors IO command patterns and cache utilization, using this feedback to dynamically adjust the migration queue priorities. Strides that are less likely to be accessed are prioritized for migration, optimizing migration efficiency while the feedback mechanism manages the complexity through adaptive rather than static control.
Solution Approach 2:
The migration queue is made dynamic rather than static, allowing priorities to change based on real-time system conditions. The system adapts its migration strategy by adjusting queue order based on observed IO patterns, improving efficiency while the dynamic nature of the system handles the complexity through flexibility rather than rigid structures.
3Productivity
If IO commands are intercepted to prevent data from becoming dirty, then migration speed is improved, but data access flexibility deteriorates
Solution Approach 1:
The migration manager acts as an intermediary between IO commands and migrated data, selectively blocking only those commands that would modify data during migration. This intermediary approach preserves migration speed by preventing dirty data while maintaining data access flexibility by allowing read operations and other non-modifying accesses to proceed normally.
Data Source
AI summary
Disclosed is a computer-implemented method to increase the speed of extent migration. The method comprises determining a plurality of extents to migrate. The extents are stored in a source tier and the source tier is one tier of a tiered storage system. The method also comprises forming a first queue, wherein the first queue contains the plurality of extents to be migrated. The method further includes selecting a first extent to copy to a target tier. The copying comprises dividing the extent into a plurality of strides, forming a second queue containing the strides, selecting and copying the first stride, and removing the first stride from the second queue. The method further comprises changing a mapping table to point to the target tier, removing the first extent from the first queue, and releasing the first area of the first tier.