Striped Data Rebalancing via Parallel Chunk Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for rebalancing striped data across multiple storage devices are inefficient, requiring serial data movement and temporary storage, leading to prolonged processing times, especially when adding or removing storage devices.
Innovation Solution
The method involves allocating contiguous storage spaces into chunks, where the number of chunks is the least common multiple of specific values, and then distributing these chunks across logical stripes, allowing for parallel data movement and reduced data transfer, thereby optimizing the rebalancing process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional relayout techniques are used to add stripes to striped local volume, then data is moved serially through temporary storage devices, but the process takes excessively long time (2.5 hours for 1200GB)
Solution Approach 1:
The patent segments the relayout process into multiple parallel operations by dividing data into different stripe sets that can be moved simultaneously across multiple storage devices. Instead of moving all data serially through temporary storage, the system performs segmented parallel transfers, reducing the overall relayout time from hours to minutes while maintaining data integrity through coordinated chunk allocation and verification.
2Reliability
If conventional relayout techniques are used, then temporary storage devices are required to hold data during transfer, but this increases device complexity and storage overhead
Solution Approach 1:
The patent uses a metadata structure (chunk allocation table) as an intermediary to coordinate data movement between source and destination storage devices. Instead of requiring physical temporary storage devices, the system uses the metadata layer to track and manage parallel data transfers, eliminating the need for additional hardware while maintaining reliable rebalancing operations.
3Reliability
If conventional relayout techniques are used, then all striped data must be moved serially, but this reduces productivity and increases operational time by 94%-95%
Solution Approach 1:
The patent implements continuous parallel data movement by maintaining multiple active transfer streams simultaneously. While conventional methods complete one serial transfer at a time, this system continuously moves multiple stripe sets in parallel across available storage devices, maximizing throughput and reducing operational time by eliminating idle periods and maintaining constant productive action throughout the rebalancing process.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosed computer-implemented method for rebalancing striped information across multiple storage devices may include (1) allocating "Y" contiguous storage spaces on "Y" physical devices, (2) dividing, at the computing device, the "Y" continuous storage spaces into "N" subvolumes, (3) allocating the "N" subvolumes to "X" logical stripes, (4) allocating each of the first "Y" logical stripes to a respective physical stripe in each of the "Y" physical stripes, (5) distributing remaining subvolumes to respective logical stripes, and (6) allocating the respective subvolumes across the "Y" physical stripes. Various other methods, systems, and computer-readable media are also disclosed.