Parity Block Coding with Local Reconstruction for Faster Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage solutions in distributed systems face inefficiencies in data recovery due to the significant traffic and time consumption required to recover lost parity blocks, as they necessitate the acquisition of multiple original data blocks.
Innovation Solution
A parity block generation method that determines a global coding matrix for a plurality of original data blocks and local coding matrices for divided local domains, establishing a derivation relationship between global and local reconstruction parity blocks, allowing recovery using unlost parity blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If lost parity blocks are recovered by acquiring original data blocks, then data recoverability is ensured, but traffic consumption and time consumption increase significantly
Solution Approach 1:
The patent creates local reconstruction parity blocks as copies/derivatives of global parity blocks. These local parity blocks are generated by performing XOR operations on original data blocks within the same local domain, creating a redundant copy that can be used for recovery without needing to access all original data blocks. This copying mechanism enables faster recovery by using locally available parity blocks instead of requiring widespread data block acquisition.
Solution Approach 2:
The patent divides the distributed storage system into multiple local domains, with each domain containing a subset of original data blocks and corresponding local reconstruction parity blocks. This segmentation allows recovery operations to be performed locally within each domain using only the local parity blocks and local data blocks, rather than requiring system-wide access to all original data blocks. The segmentation reduces the scope and time of recovery operations.
2Reliability
If lost parity blocks are recovered by acquiring original data blocks, then data integrity is maintained, but network traffic consumption increases
Solution Approach 1:
The patent segments the storage system into local domains with locally-generated reconstruction parity blocks. When parity block recovery is needed, the system only needs to access original data blocks and local reconstruction parity blocks within the same local domain, dramatically reducing the network traffic required compared to acquiring all original data blocks from across the distributed system. This segmentation localizes the recovery traffic to minimal necessary transmissions.
Solution Approach 2:
The local reconstruction parity blocks act as intermediaries between the original data blocks and the recovery process. Instead of directly acquiring all original data blocks for recovery, the system uses these intermediary local parity blocks as a intermediate step, reducing the amount of data that needs to be transmitted over the network. The local parity blocks serve as local proxies that enable recovery with minimal network involvement.
3Reliability
If global parity blocks are generated for all original data blocks, then comprehensive data protection is achieved, but the complexity of parity block management increases
Solution Approach 1:
The patent segments global parity block management into multiple local reconstruction parity blocks, each managed independently within its local domain. This segmentation reduces management complexity by allowing each local domain to handle its own parity blocks separately, rather than requiring centralized management of all global parity blocks. The segmentation divides the complex global management task into simpler, distributed local management tasks.
Solution Approach 2:
The patent performs preliminary generation of local reconstruction parity blocks during the normal data storage operation, before any data loss occurs. These local parity blocks are pre-computed and stored alongside the original data blocks in each local domain. This preliminary action ensures that when recovery is needed, the parity blocks are already available and prepared, eliminating the need for complex real-time computation or coordination during recovery operations.
Data Source
AI summary
A parity block generation method and apparatus are disclosed. The method comprises: for a plurality of original data blocks for which parity blocks are to be generated, determining a global coding matrix for the plurality of original data blocks; for each local domain that is obtained by dividing the plurality of original data blocks, determining a local coding matrix of the local domain on the basis of the global coding matrix, wherein each local domain comprises original data blocks that belong to the local domain; and generating, on the basis of the global coding matrix, global parity blocks for the plurality of original data blocks, and generating, on the basis of the local coding matrix of each local domain, a local reconstruction parity block of each local domain, such that the global parity blocks have a derivation relationship with the local reconstruction parity block of each local domain.