Erasure-Coded Storage Aggregation for Capacity-Reliability Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data centers face inefficiencies due to the need for significant reserve storage capacity and redundant data storage, which reduces operating efficiency and increases energy and financial costs, as well as the requirement for additional hardware infrastructure.
Innovation Solution
Implementing erasure coded storage aggregation techniques that break down data files into fragments and store them across multiple nodes, allowing for efficient data redundancy without the need for additional hardware, enabling computation nodes to provide storage capacity while maintaining data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If duplicate copies of data are stored in storage nodes to guard against data loss, then data reliability is improved, but storage capacity is reduced
Solution Approach 1:
The patent divides data into multiple fragments using erasure coding algorithms, distributing these fragments across different nodes. This segmentation allows the system to maintain data reliability while optimizing storage capacity utilization, as only a threshold number of fragments are needed to reconstruct the original data, unlike traditional duplicate copying which requires full redundant copies.
Solution Approach 2:
The patent changes the storage parameter from storing complete duplicate copies to storing erasure-coded fragments. This parameter change in the storage methodology enables more efficient use of storage capacity while maintaining the same level of data reliability, as erasure coding provides redundancy through mathematical relationships rather than physical duplication.
2Adaptability or versatility
If reserve storage capacity is set aside for computation operations, then computation flexibility is improved, but operating efficiency is reduced
Solution Approach 1:
The patent enables storage nodes to serve multiple functions: they can store third-party data using erasure coding while simultaneously maintaining their own data buffers for computation operations. This multi-functionality eliminates the need for separate reserve storage capacity, as the same storage infrastructure supports both data redundancy and computation buffering needs.
Solution Approach 2:
The patent merges the functions of data storage with redundancy protection and computation buffering into a unified storage system. By combining these previously separate functions, the system improves operating efficiency while maintaining the flexibility needed for computation operations, as storage nodes can dynamically allocate capacity based on needs.
3Quantity of substance
If additional hardware infrastructure is built to increase storage capacity, then storage capacity is improved, but energy consumption increases
Solution Approach 1:
The patent enables existing storage nodes to provide additional storage capacity for third-party data by utilizing erasure coding techniques. This self-service approach allows the infrastructure to serve multiple purposes without requiring additional hardware, thereby avoiding the increased energy consumption that would result from building new data centers or expanding existing hardware infrastructure.
Data Source
AI summary
Embodiments of erasure coded storage aggregation are disclosed. The erasure coded storage aggregation includes storing a data file as erasure coded fragments in a plurality of nodes of one or more data centers. The erasure coded storage aggregation further includes monitoring an access frequency of the data file. Based on the comparison between the access frequency and a predetermined threshold, the data file is either reconstructed from the erasure coded fragments and stored in a storage node or retained as erasure coded fragments in the plurality of nodes of the one or more data centers.