Cross-Data-Center FEC Recovery for Lower Storage Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data durability methods in distributed systems require significant storage overhead due to replication of identical data across multiple data centers, leading to increased costs and inefficiencies.
Innovation Solution
Implementing data reconstruction techniques using forward error correction (FEC) codes to generate parity blocks, allowing failed information blocks to be reconstructed from unique data blocks distributed across multiple data centers, reducing the need for duplicate storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional replication models are used to store identical data across multiple data centers, then data availability and reliability are improved, but storage overhead and costs increase significantly
Solution Approach 1:
The patent segments data into unique information blocks distributed across multiple data centers, replacing traditional replication of identical data copies. Each data center stores different portions or representations of the data, allowing reconstruction through coordination between centers rather than maintaining full duplicate copies locally.
Solution Approach 2:
The patent creates a multi-functional system where data blocks can serve multiple purposes: they can be read locally at their storing data center, transmitted to other data centers for reconstruction, or used to regenerate failed blocks through coordination protocols. This universal utility reduces the need for dedicated duplicate storage.
2Quantity of substance
If data blocks are distributed across multiple data centers using forward error correction codes, then storage footprint is reduced, but system complexity increases due to reconstruction protocols
Solution Approach 1:
The patent uses forward error correction codes to create redundant information blocks that can be copied and distributed across data centers. These coded blocks contain embedded redundancy that enables mathematical reconstruction of original data without requiring exact duplicate copies, reducing storage footprint while maintaining recovery capability.
Solution Approach 2:
The patent introduces coordination protocols and metadata structures as intermediaries that manage the distribution and reconstruction processes. These intermediaries handle the complexity of coordinating between data centers, tracking block locations, and orchestrating reconstruction operations, thereby managing system complexity in a structured manner.
3Ease of repair
If identical data is replicated across data centers, then failure recovery is simplified, but storage costs and resource inefficiency increase
Solution Approach 1:
The patent changes the fundamental parameter of data representation from identical copies to unique coded blocks with mathematical relationships. This parameter change allows failure recovery through computational reconstruction rather than simple copy retrieval, reducing resource consumption while maintaining recovery capability through algorithmic processes.
Data Source
AI summary
Disclosed examples include writing first information blocks in first storage; transmitting second information blocks to second storage, the first and second information blocks including data blocks and parity blocks; determining whether a failed information block can be reconstructed based on available ones of the first information blocks in the first storage; and writing a reconstructed information block in the first storage to replace the failed information block.