Erasure-Coded Storage Re-Striping With Minimal Data Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Increasing the stripe width in erasure encoded storage systems is resource-intensive, as conventional methods require copying entire data containers to new storage locations, making it inefficient to add storage nodes and improve capacity.
Innovation Solution
The method involves determining a tile reconfiguration that minimizes the number of data blocks to be rearranged during re-striping by calculating optimal tile configurations using the CP transformation, allowing for efficient re-striping without rearranging the maximum number of data blocks, and utilizing a centralized metadata service for node assignments and storage mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional re-striping is performed by copying the entire data container to another storage location with new encoding parameters, then the stripe width can be increased to improve storage efficiency, but the process becomes very resource expensive and time-consuming
Solution Approach 1:
The patent divides the data container into tiles of data blocks and processes them independently. Instead of copying the entire data container, the system segments it into manageable tiles that can be re-striped separately, reducing the overall resource overhead and time required for re-striping operations.
Solution Approach 2:
The patent pre-calculates optimal tile configurations and identifies which data blocks can remain in their original positions before performing re-striping. By determining the reconfiguration plan in advance and keeping maximum data blocks unchanged, the system minimizes data movement and reduces the resource cost of re-striping.
2Quantity of substance
If the stripe width is increased to improve storage efficiency when adding storage nodes, then storage capacity and efficiency are improved, but the resource overhead for re-striping increases significantly
Solution Approach 1:
The patent extracts and identifies the specific data blocks that need to be moved during re-striping, rather than moving all data blocks. By taking out only the necessary data blocks for reconfiguration and leaving the maximum number of blocks in their original positions, the system reduces the amount of data movement and associated resource overhead.
Solution Approach 2:
The patent changes the stripe width parameter to increase storage capacity while simultaneously optimizing the re-striping process by calculating configurations that minimize data movement. The system adjusts encoding parameters and tile configurations to achieve higher storage efficiency with reduced resource overhead.
3Adaptability or versatility
If data blocks are rearranged during re-striping to achieve desired tile width, then the storage configuration is optimized, but the number of data blocks that need to be moved increases
Solution Approach 1:
The patent performs preliminary calculation to determine the optimal tile configuration and identifies which data blocks can remain in their original positions. By planning the reconfiguration in advance and maximizing the number of blocks that stay in place, the system minimizes data movement time while achieving the desired tile width configuration.
Solution Approach 2:
The patent creates a mapping or plan of the new tile configuration without immediately moving all data blocks. It identifies the optimal arrangement and determines which blocks need to be moved versus which can remain in place, effectively using a conceptual copy of the configuration plan to guide the minimal necessary data movement.
Data Source
AI summary
Aspects of the present disclosure describe methods and systems for optimized re-striping in an erasure encoded storage. In one exemplary aspect, a method may receive a request to re-stripe a plurality of data blocks arranged as a tile in the erasure encoded storage, wherein the request comprises a desired tile width. The method may identify (1) a number of data blocks in the tile and (2) a width of the tile. The method may determine a maximum number of data blocks that do not need to be rearranged when reconfiguring the tile to the desired tile width. Furthermore, the method may determine a tile reconfiguration with the desired tile width that does not rearrange the maximum number of the data blocks of the tile, and may re-stripe the tile in accordance with the tile reconfiguration.