Locally Recoverable Coding for Distributed Storage Repair
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face high storage costs and low access efficiency due to high storage redundancy in multiple repetitions, and excessive repair bandwidth when using Reed-Solomon (RS) erasure coding, which is not suitable for frequently accessed data.
Innovation Solution
A locally recoverable coding method is introduced, which divides file segments into data blocks, generates global and local coding blocks through XOR operations, reducing the amount of data needed for node repair and improving access efficiency by minimizing repair bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If RS erasure coding is used to reduce storage redundancy, then storage costs are reduced, but access efficiency decreases and repair bandwidth increases
Solution Approach 1:
The patent segments the coding structure into global coding blocks and local coding blocks. Global coding blocks provide overall data protection with reduced redundancy compared to traditional RS codes, while local coding blocks provide rapid repair capability for individual node failures. This segmentation allows the system to achieve both low storage redundancy and high access efficiency by using different coding strategies for different purposes.
Solution Approach 2:
The patent applies local quality by creating local coding blocks that are specifically optimized for repair operations. Each local coding block is associated with a specific group of data blocks and provides dedicated repair capability for that group. This local optimization enables fast repair with minimal bandwidth consumption while the global coding structure maintains overall storage efficiency.
2Loss of substance
If RS erasure coding is used to reduce storage redundancy, then storage efficiency improves, but repair bandwidth increases
Solution Approach 1:
The patent divides the repair process into local and global components. When a node fails, the repair process first uses local coding blocks to reconstruct the failed data with minimal bandwidth consumption. Only if local repair is insufficient does the system escalate to using global coding blocks. This segmented repair strategy dramatically reduces the average repair bandwidth compared to traditional RS codes that require reading from many nodes.
Solution Approach 2:
The patent performs preliminary action by pre-computing and storing local coding blocks alongside the data blocks. These local coding blocks are prepared in advance specifically for repair operations, so when a node fails, the system can immediately use the pre-computed local codes without needing to perform complex global decoding operations, thus reducing repair bandwidth and time.
3Productivity
If multiple repetitions are used to improve access efficiency, then read efficiency increases, but storage redundancy increases
Solution Approach 1:
The patent segments the redundancy strategy into local replication (through local coding blocks) and global erasure coding (through global coding blocks). The local coding blocks provide rapid access for common read operations by keeping related data and codes close together, while the global coding blocks provide overall data protection with minimal redundancy. This segmentation allows the system to achieve high read efficiency without the excessive storage redundancy of traditional multi-replication schemes.
Data Source
AI summary
Disclosed is a data encoding, decoding and recovering method of a distributed storage system for data protection of the distributed storage system. The methods include using local recoverable coding, and calling Reed-Solomon coding on data blocks obtained from divisions of file segments based on coding parameters to generate global coding blocks, locally coding to data blocks and global coding blocks respectively to generate local coding blocks. The methods can also include computing decoded block indices and recovered block indices according to current node state, reading block data of assistant node, and implementing decoding of file segments and recovery of failed blocks. The coding method of the present disclosure can reduce the amount of data that needs to be transmitted when recovering a failed node by increasing local coding blocks and speed up the node recovery speed.