Erasure Code Data Storage Across Heterogeneous Media
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise organizations face inefficiencies in data storage due to the limitations of traditional storage solutions, including excessive storage space consumption by 1+N replica configurations and the inability to utilize heterogeneous storage media, which can lead to unacceptable downtimes and performance issues in cloud storage systems.
Innovation Solution
The implementation of a computer-implemented method that divides data into chunks, generates parity bits using an erasure code, and splits these chunks and parity bits across heterogeneous storage media targets, optimizing storage consumption and reliability while reducing redundancy levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional 1+N replica configurations are used for high durability, then data reliability is improved, but storage space consumption increases excessively
Solution Approach 1:
The patent changes the redundancy parameter from traditional 1+N replication to erasure code with configurable redundancy factor (e.g., 1.5x or less). This allows maintaining the same data reliability through mathematical encoding rather than simple copying, significantly reducing storage space consumption while preserving durability.
Solution Approach 2:
The patent applies different redundancy levels to different data blocks within the same storage system. By dividing data into chunks and applying erasure codes selectively, it optimizes storage efficiency for each block while maintaining overall system reliability, rather than uniformly applying 1+N replication across all data.
2Reliability
If hard-RAID or soft-RAID storage solutions are used on homogeneous media devices, then storage reliability is improved, but the ability to consume heterogeneous media devices is lost
Solution Approach 1:
The patent creates a universal storage system that can work with multiple types of storage media (heterogeneous devices) through a common erasure code framework. The system can allocate data and parity blocks across different device types (SSD, HDD, cloud storage) based on their characteristics, making the storage solution adaptable to various hardware configurations while maintaining reliability.
Solution Approach 2:
The patent segments data into multiple blocks and distributes them across heterogeneous storage devices. By dividing data into chunks and applying erasure codes, it enables flexible allocation across different media types, allowing the system to consume diverse hardware resources effectively while preserving data integrity through distributed redundancy.
3Productivity
If cloud storage is used as backend target storage for deduplication storage system, then storage scalability is improved, but storage availability deteriorates due to weak/ eventual consistency model and occasional downtimes
Solution Approach 1:
The patent generates and stores parity blocks in advance before actual data loss occurs. By pre-computing erasure code parity blocks and storing them alongside data blocks in the cloud, the system cushions against potential cloud storage failures or downtimes, ensuring data can be reconstructed even if cloud storage becomes temporarily unavailable.
Solution Approach 2:
The patent performs data encoding and parity generation as preliminary actions before data is written to cloud storage. This preliminary processing ensures that even if cloud storage experiences eventual consistency delays or downtime, the locally stored parity blocks enable immediate data reconstruction without waiting for cloud storage recovery.
Data Source
AI summary
The disclosed computer-implemented method for storing data may include (i) identifying a request to backup a set of data according to a storage-as-a-service configuration that stores data into cloud storage, (ii) dividing the set of data to be backed up into chunks of data, (iii) dividing, for each of the chunks of data, a respective chunk into blocks of data, (iv) generating, for each of the chunks of data, at least one block of parity bits for the respective chunk of data based on applying an erasure code to the blocks of data, and (v) splitting, during backing up the chunks of data according to the storage-as-a-service configuration, the respective blocks of data and the generated block of parity bits into heterogeneous storage media targets such that the storage-as-a-service configuration benefits from the heterogeneous storage media targets. Various other methods, systems, and computer-readable media are also disclosed.