HashTag Codes for Reduced Repair Bandwidth in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems with a large number of nodes, the repair bandwidth is high due to wider stripes of erasure codes, leading to poor performance during repair processes.
Innovation Solution
Implementing an enhanced HashTag Code that optimizes repair bandwidth by logically dividing computing nodes into disjunctive subsets, creating partitions, and generating error correction codes based on specific conditions to minimize the number of nodes required for repair.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If erasure codes are used with a large number of computing nodes to ensure data redundancy and fault tolerance, then reliability is improved, but repair bandwidth increases and repair performance deteriorates
Solution Approach 1:
The patent divides the computing nodes into multiple disjunctive subsets, where each subset can independently contribute to repairing failed nodes. This segmentation allows the repair process to access data from fewer nodes within each subset, reducing the total repair bandwidth required while maintaining the overall fault tolerance of the distributed storage system
2Quantity of substance
If wider stripes of erasure codes are used across more computing nodes, then data redundancy is improved, but repair bandwidth increases
Solution Approach 1:
By segmenting nodes into disjunctive subsets and creating partitions within each subset, the patent enables data redundancy to be distributed across more nodes while limiting repair operations to only the necessary subsets. This reduces the effective repair bandwidth needed compared to traditional approaches that would access all nodes in the stripe
Solution Approach 2:
The patent introduces a new organizational dimension by creating partitions within disjunctive subsets, allowing the system to achieve high data redundancy through multi-level organization rather than simply increasing the number of nodes in a single stripe, thereby reducing repair bandwidth requirements
Data Source
AI summary
A system can determine to generate error correction codes for a storage system that comprises computing nodes. The system can logically divide the computing nodes into disjunctive subsets. The system can, for each disjunctive subset, create a partition, and identify a group of elements that comprises sub-strips of the disjunctive subset. The system can add a first element from each disjunctive subset of the disjunctive subsets to the partition, and removing the first element from the group of elements. The system can, for each disjunctive subset, add a second element from the group of elements along with elements stored in parity notes of a sub-strip where the second element is located. The system can, in response to determining that a number of elements of the disjunctive subset is at least as large as a number of the computing nodes, create the error correction codes based on the elements.