Erasure Coding Parameters for Repair-Free Data Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for predicting the reliability of erasure-coded data require costly repair processes, which involve significant computational overhead and network traffic, and are not always necessary, especially for data that is only relevant for a short period.
Innovation Solution
A new data storage technique using a continuous-time Markov chain model that calculates the Mean Time to Data Loss (MTTDL) without the need for repair, allowing for sufficient erasure coding redundancy to be established based on parameters N and K to ensure data reliability for a designated retention period.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data repair is performed to maintain reliability, then data reliability is improved, but computational overhead and network traffic increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-calculating the Mean Time to DataLoss (MTDLD) using a continuous-time Markov chain model before actual data loss occurs. This allows the system to predict reliability metrics and determine appropriate erasure coding parameters in advance, eliminating the need for costly repair operations after failures occur. The MTDLD calculation provides proactive reliability assessment without requiring actual repair actions.
2Reliability
If data repair is performed to maintain reliability, then data reliability is improved, but network traffic increases significantly
Solution Approach 1:
The system performs preliminary reliability assessment through MTDLD calculation using Markov chain models, determining the appropriate number of redundant fragments (N-K) before failures occur. This preliminary determination of erasure coding parameters prevents the need for subsequent repair operations that would generate network traffic, as the redundancy is already optimally configured in advance.
3Duration of action of stationary object
If erasure coding redundancy is increased to extend data retention period, then data retention period is improved, but storage overhead increases
Solution Approach 1:
The patent applies parameter changes by using the MTDLD calculation to determine the optimal erasure coding parameters (N and K) that balance data retention period requirements with storage overhead. The continuous-time Markov chain model allows dynamic adjustment of these parameters based on the desired retention period, finding the minimum necessary redundancy (N-K) to achieve the target MTDLD without excessive storage overhead.
4Duration of action of stationary object
If erasure coding parameters are optimized for long retention, then data retention period is improved, but computational complexity increases
Solution Approach 1:
The patent extracts the complex reliability calculation from the data storage and retrieval operations by using a continuous-time Markov chain model that runs independently during system initialization or configuration. This separation allows the MTDLD calculation to determine optimal erasure coding parameters without adding computational complexity to the actual data operations, as the parameter optimization is performed once in advance rather than continuously during data access.
Data Source
AI summary
An improved data storage technique achieves a desired level of reliability by providing sufficient redundancy in erasure coded data to maintain the data, without repair, for a prescribed period of time. The improved technique employs a newly devised, continuous-time Markov chain model. The model can be applied in computerized systems to establish erasure coding parameters for storing and reliably maintaining data for a designated period of time, without any need to repair the data to reestablish an original or previous level of erasure coding redundancy.