Orthogonal Erasure Coding for Local Data Center Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face challenges in balancing reliability and cost, particularly in managing storage node failures across multiple data centers, where communication between centers is costly and time-consuming, and there is a need to reduce communication and storage space requirements while maintaining data integrity.
Innovation Solution
A computer-implemented method using linear error-correcting codes to generate and allocate row and column code chunks across storage nodes, allowing for localized reconstruction of damaged data within a data center without inter-data center communication, utilizing maximum distance separable (MDS) codes and error-detecting codes to identify and correct errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored redundantly across multiple data centers, then data reliability is improved, but communication cost and time increase when reconstructing damaged data
Solution Approach 1:
The patent segments the error correction functionality into two independent codes: row code stored locally at each data center and column code distributed across data centers. This segmentation allows local reconstruction using row code without requiring communication with other data centers, thereby reducing communication time while maintaining reliability through the distributed column code structure.
Solution Approach 2:
The patent introduces a two-dimensional storage structure where data is organized in rows and columns, with row code chunks stored locally and column code chunks distributed across data centers. This dimensional organization enables local error correction within a data center using row code, eliminating the need for inter-data-center communication for routine error recovery, while column code provides cross-data-center redundancy.
2Reliability
If error-correcting codes are used to correct errors, then data reliability is improved, but storage space requirements increase
Solution Approach 1:
The patent applies partial error correction by using row code locally within each data center to correct errors without requiring full distributed reconstruction. This partial action approach corrects the most common failure scenarios (single data center failures) using only local storage space, while the column code provides additional protection only when necessary, thereby optimizing the trade-off between storage overhead and reliability.
3Reliability
If communication between storage nodes is increased to reconstruct damaged chunks, then data integrity is maintained, but communication resources and costs increase
Solution Approach 1:
The patent performs preliminary error correction using row code chunks that are pre-stored locally at each data center. This preliminary action enables immediate local reconstruction of damaged data without triggering inter-data-center communication, thereby maintaining data integrity while minimizing communication resource usage. The column code chunks serve as a backup correction mechanism only when local correction is insufficient.
Data Source
AI summary
Methods, systems, and apparatus, including computer program products, for orthogonal coding for data storage. In one aspect, a method includes receiving a block of data comprising m rows and n columns of data chunks. For each row in the block of data, (c-n) columns of error-correcting row code chunks are generated using a first linear error-correcting code in systematic form and the particular row's data chunks. For each column in the block of data and for each generated column, particular column and (r-m) error-correcting column code chunks for the particular column are allocated to a distinct group of storage nodes, wherein: the column code chunks are generated using a second linear error-correcting code in systematic form and the particular column's data chunks or row code chunks; m and n are greater than one; and c is greater than n and r is greater than m.