Erasure-Coded Block ID Encoding for Verified Data Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing erasure coded storage systems cannot verify block identification information during reconstruction operations, leading to potential data corruption and incorrect block usage, as block IDs are not encoded into the erasure correcting code.
Innovation Solution
Incorporating block identification information into the erasure correcting code, specifically using Reed-Solomon codes, to ensure that block IDs are unique and verifiable at both read and reconstruction times, providing end-to-end protection by verifying the correct blocks are used in the reconstruction process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If block IDs are excluded from XOR operations in erasure coded storage systems, then the encoding process is simpler and faster, but data integrity cannot be verified during reconstruction operations
Solution Approach 1:
The patent merges block identification information with data blocks by encoding both together using erasure correcting codes. The block ID is XORed along with data blocks to generate parity blocks, creating a unified encoded structure where identification and data are inseparably linked. This allows verification of block identity during reconstruction without separate verification steps.
Solution Approach 2:
The patent performs preliminary encoding of block identification information together with data blocks before storage. By pre-encoding the block ID into the parity blocks along with the data, the system prepares verification capability in advance, eliminating the need for separate verification operations during reconstruction and maintaining encoding efficiency.
2Reliability
If block IDs are included in XOR operations, then data integrity can be verified during reconstruction, but parity block IDs may coincide with data block IDs causing identification conflicts
Solution Approach 1:
The patent applies local quality by treating block identification information and data blocks differently within the encoding process. The block ID is encoded with specific properties (unique identification capability) while data blocks have their own properties. The encoding scheme ensures that the resulting parity block IDs maintain the unique identification property by design, preventing conflicts between parity and data block IDs.
3Quantity of substance
If traditional erasure coding is used without block ID encoding, then storage capacity is maximized, but the system cannot detect or prevent incorrect block usage during reconstruction
Solution Approach 1:
The patent implements feedback by encoding block identification information into the parity blocks. During reconstruction, the system can verify whether the retrieved blocks have the correct IDs by checking against the encoded identification information in the parity blocks. This feedback mechanism detects incorrect block usage without requiring additional storage overhead beyond the erasure code structure.
Data Source
AI summary
A data storage system includes a plurality of data blocks. A set of data blocks are protected by an erasure correcting code and each of the data blocks in the set of data blocks includes block identification information. The data storage system includes a processor and logic integrated with the processor, executable by the processor, or integrated with and executable by the processor. The logic is configured to verify the block identification information for each of the data blocks in the set of data blocks at the time of read and, as part of reconstructing a data block, reconstruct the block identification information for the reconstructed data block, and verify the block identification information.