Data Local Recovery with Parity Blocks for Reduced Read Amplification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing local repair solutions for erasure codes lack high fault tolerance capability and efficient local recovery of global parity blocks.
Innovation Solution
A data local recovery method that involves encoding data blocks to generate global and local parity blocks, where the local parity blocks include encoding information of the global parity blocks, allowing recovery of any object block within a data group using remaining readable blocks, and ensuring the sum of local parity blocks is linearly independent of global parity blocks, enabling recovery of up to r+1 lost blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional replication-type data recovery methods are used, then data reliability is improved, but storage space consumption increases
Solution Approach 1:
The patent divides the data into multiple data groups, each containing data blocks, global parity blocks, and local parity blocks. This segmentation allows selective recovery of only the necessary parity information for each group, reducing the overall storage space required compared to traditional replication methods while maintaining data reliability.
Solution Approach 2:
The patent introduces local parity blocks that are specific to each data group, allowing local recovery within each group without requiring global parity blocks. This local quality approach optimizes storage by storing only the necessary parity information for each group rather than replicating data across the entire system.
2Quantity of substance
If erasure code is used to reduce storage space, then storage space consumption is reduced, but read amplification during data reconstruction increases
Solution Approach 1:
The patent segments the parity information into local parity blocks for each data group, allowing recovery operations to read only the necessary local parity blocks rather than requiring reads across the entire distributed storage system. This segmentation significantly reduces read amplification during data reconstruction while maintaining the space efficiency of erasure codes.
Solution Approach 2:
The patent introduces local parity blocks as intermediary elements that mediate between data blocks and global parity blocks. These local parity blocks enable efficient local recovery operations, acting as intermediaries that reduce the need for extensive reading operations across the distributed storage system.
3Ease of repair
If existing local repair solution is used, then local recovery capability is improved, but fault tolerance capability for global parity blocks remains insufficient
Solution Approach 1:
The patent designs the local parity blocks to serve multiple functions: they enable local recovery for data blocks within their respective groups, and simultaneously provide the capability to recover global parity blocks. This multi-functionality achieves both ease of repair and high fault tolerance capability, as the same local parity blocks contribute to both local and global recovery operations.
Solution Approach 2:
The patent merges the functions of local recovery and global parity recovery into a unified system where local parity blocks serve both purposes. By combining these functions, the system achieves both ease of repair for local data blocks and enhanced fault tolerance capability for global parity blocks without requiring separate recovery mechanisms.
Data Source
AI summary
Embodiments of the present application provide a data local recovery method, a device and a storage medium. A global parity block and a local parity block in any data group of a data set are obtained by encoding a data block, and when any data block within the data group is lost, the data block can be recovered according to remaining data blocks, the global and local parity blocks. When the local parity block is lost, the local parity block can be recovered according to the data block within the data group. The local parity block includes encoding information of the global parity block, and when the global parity block is lost, the global parity block can be recovered according to the data block and the local parity block.