Virtual Storage Pool Capacity Normalization via Dynamic Data Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems using data striping, the unbalanced distribution of new data across virtualized storage pools becomes a problem when new devices are added, leading to restricted data writing as older devices approach full utilization, causing inefficiencies in data management.
Innovation Solution
A method is implemented to collect utilization statistics across storage pool members, classify them as source, target, or neutral based on their utilization levels, and iteratively move data from source to target members to normalize the pool, using a random selection algorithm to maintain balanced data striping across all members.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data striping is used across storage pool members, then data writing performance is improved, but when older devices approach full utilization, new data becomes restricted to new members only, causing unbalanced distribution
Solution Approach 1:
The patent implements dynamic data migration between storage pool members based on real-time utilization monitoring. When new members are added, the system automatically migrates data from older, fuller members to new, emptier members to maintain balanced distribution. This dynamic adjustment resolves the contradiction by adapting the data distribution pattern over time rather than using a static striping pattern that becomes unbalanced as members reach different utilization levels.
Solution Approach 2:
The system continuously monitors storage utilization levels across all pool members and uses this feedback to trigger data migration operations. When utilization imbalance is detected (older members approaching full capacity while new members have available space), the system responds by migrating data to restore balance. This feedback mechanism ensures that data distribution balance is maintained despite the initial contradiction caused by incremental member addition.
2Quantity of substance
If new empty devices are added to an existing storage pool, then storage capacity is expanded, but data distribution becomes unbalanced with older devices approaching 100% utilization
Solution Approach 1:
The patent performs preliminary data migration operations immediately when new storage members are added to the pool. Rather than allowing data to become concentrated on older members, the system proactively migrates data from older, fuller members to the newly added members before the imbalance becomes severe. This preliminary action prevents the utilization uniformity problem from developing in the first place.
Solution Approach 2:
The system changes the data distribution parameters by dynamically adjusting which members receive new data blocks based on current utilization levels. When new members are added, the striping pattern is recalculated to include these members, and existing data is remapped to achieve more uniform utilization across all members. This parameter change approach allows the system to maintain utilization uniformity while expanding storage capacity.
3Manufacturing precision
If data is restricted to new less-utilized members only, then older members maintain low utilization, but overall data writing efficiency decreases
Solution Approach 1:
The patent implements a self-service data migration mechanism where the storage system automatically monitors its own utilization levels and triggers data migration operations without external intervention. When utilization imbalance is detected, the system autonomously migrates data from older members to newer members, ensuring that all members can continue to participate in data writing operations. This self-service approach maintains both utilization control and writing efficiency by preventing the scenario where data is restricted to only a subset of members.
Solution Approach 2:
The system dynamically adjusts data distribution across members based on real-time utilization monitoring. Rather than restricting data writing to only new members or using a static pattern, the system continuously adapts the distribution pattern to maintain balanced utilization. This dynamic approach ensures that data writing efficiency is maintained by utilizing all available members while keeping utilization levels balanced through automatic migration when needed.
Data Source
AI summary
A system for normalizing capacity utilization within virtual storage pools includes collecting utilization statistics across individual storage devices that are members of a virtualized storage pool. Using the pool utilization level statistics, pool members may be assigned a score and, according to that score, may be marked as source, target or neutral members. Based on the scores of each pool member, data may be migrated among pool members, in particular, from source members to target members. The process may be iterative such that the statistics and scores may be used only to perform a fraction of the data movement that would be needed to normalize the storage pool before the scores are then recalculated. In this way, the system may dynamically adapt to changing conditions within the pool such as device additions, removals and inefficient striping of new data.