Homed Data Encoding With Cross-Center Error-Correcting Chunks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face challenges in ensuring data availability and integrity, particularly when an entire data center fails, as they often require redundant storage that can be inefficient in terms of space and management.
Innovation Solution
An encoding system that stores data chunks at 'home' data centers and error-correcting code chunks at other centers, allowing for reconstruction of damaged data even if a complete data center fails, while using less than double the storage capacity by distributing data across multiple centers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored redundantly across multiple data centers, then data availability and integrity are improved, but storage capacity requirements increase
Solution Approach 1:
The patent segments data into distinct blocks (e.g., Block A and Block B) and stores them at different home data centers. Each block is independently managed with its own error-detecting codes, enabling granular fault tolerance without requiring complete redundancy of entire datasets across all data centers.
Solution Approach 2:
The patent changes the parameter of data representation by encoding data blocks using error-detecting codes (such as CRC). This transformation allows the system to detect and handle errors in individual blocks without requiring full redundancy of all data, thereby improving reliability while controlling storage requirements.
2Reliability
If error-correcting codes are used to protect against complete data center failure, then data integrity is improved, but the amount of extra data stored increases
Solution Approach 1:
The patent divides data into blocks and applies error-detecting codes to each block separately. This segmentation allows the system to use minimal redundancy per block rather than requiring extensive error-correcting codes for the entire dataset, reducing the total amount of extra storage needed while maintaining integrity protection.
Solution Approach 2:
The patent introduces an intermediary mechanism where a receiving data center generates error-detecting codes for data blocks it receives. These codes serve as intermediaries that enable error detection and reconstruction without requiring the storing of complete redundant copies of all data blocks at every data center.
3Speed
If data is homed at specific data centers for local access efficiency, then access speed is improved, but fault tolerance against home data center failure is reduced
Solution Approach 1:
The patent segments data into blocks distributed across multiple home data centers. If a home data center fails, only the specific data blocks stored there are affected, while other blocks remain accessible at their respective home data centers. This segmentation limits the impact of failures and enables partial data recovery without requiring complete data replication for fault tolerance.
Solution Approach 2:
The patent applies local quality by allowing each data center to independently manage and generate error-detecting codes for its locally stored data blocks. This local approach enables each data center to optimize for both access speed (by keeping data locally) and fault tolerance (by having independent error detection capabilities) without requiring centralized redundancy management.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for skewed orthogonal coding techniques. In one aspect, a method includes receiving a first block of data chunks and a second block of data chunks. Data from the first block is specified to be stored at a first group of storage nodes, and data from the second block is specified to be stored at the second group of storage nodes. The first block of data chunks are stored at the first group of storage nodes and the second block of data chunks are stored at the second group of storage nodes. A block of code chunks is stored at the third group of storage nodes, wherein each code chunk is generated using an error-correcting code and corresponding data chunks from the first and second blocks.