Erasure Code Decoding Modes for Distributed Data Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional erasure codes, such as Reed-Solomon, are not well-suited for distributed data storage due to unpredictable performance during erasure operations, particularly in non-systematic modes, which affects data integrity and availability in modern data storage environments like IoT and cloud storage.
Innovation Solution
The development of a high-performance erasure code, OPTFEC, which combines systematic and non-systematic modes with a Mojette transform for dual-mode operation, enabling efficient encoding and decoding by dividing the decoding phase into preamble and stable phases, utilizing arithmetic operations for error correction, and optimizing CPU cycles through aligned parity projections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Reed-Solomon systematic code is used for non-erasure operation, then optimal performance is achieved, but performance suffers severely when erasure is present
Solution Approach 1:
The patent implements a dynamic code switching mechanism that adapts the erasure code type based on operational conditions. The system transitions between Reed-Solomon systematic code for non-erasure operations and Mojette transform code for erasure operations, optimizing performance for each specific scenario while maintaining data integrity
Solution Approach 2:
The patent changes the fundamental parameter of code type selection based on erasure presence detection. By monitoring whether erasures are present in the data storage system, the system dynamically adjusts the encoding/decoding parameters to use the most appropriate code, thereby resolving the performance-reliability contradiction
2Reliability
If Mojette transform non-systematic code is used for erasure operation, then data reconstruction is enabled, but performance is not optimal during non-erasure operation
Solution Approach 1:
The system dynamically selects between Mojette transform and Reed-Solomon code based on whether erasures are detected. This dynamic adaptation ensures that Mojette's superior reconstruction capability is utilized only when needed, while Reed-Solomon handles routine operations with optimal performance
3Reliability
If data is replicated over several DC sites, then data availability is improved, but time gaps between copies are created and extra work is multiplied
Solution Approach 1:
The patent implements efficient data copying mechanisms through erasure coding that allows partial reconstruction. Instead of requiring complete replication of data across multiple data centers, the system can reconstruct missing data from partial copies, reducing the time and resources needed for data synchronization while maintaining availability
Data Source
AI summary
A method of redundantly encoding data includes formatting the data into columns and rows, and generating first and second sets of projections of the data using an encoding transform. For each set of projections generated, an encoding parameter of the encoding transform is set to a different value. The first and second sets of projections are stored as the encoded data. A decoding method reads settings including an indication of a number of data fragments. The number of data fragments is compared to a number of projections in a first set of projections of the encoded data in order to determine whether to use a first or a second decoding mode. The encoded data is then decoded according to the selected decoding mode and the result is outputted.