Erasure-Coded Storage Aggregation for Data Center Capacity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data centers face inefficiencies due to the need for reserve storage capacity and redundant data storage, leading to reduced operating efficiency, increased energy consumption, and the requirement for additional hardware infrastructure to ensure data integrity against hardware and software failures.
Innovation Solution
Implementing erasure coded storage aggregation techniques that break down data files into fragments and store them across multiple nodes, allowing for efficient data redundancy without the need for additional servers or hardware, while enabling computation nodes to provide storage capacity for third-party data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If duplicate copies of data are stored in storage nodes to guard against data loss, then data reliability is improved, but storage capacity is reduced
Solution Approach 1:
The patent divides data into multiple fragments using erasure coding algorithms, distributing these fragments across different nodes. This segmentation allows the system to maintain data reliability while improving storage efficiency, as only a subset of fragments is needed to reconstruct the original data, unlike traditional duplicate copying which requires full replicas.
Solution Approach 2:
The patent changes the storage parameter from storing complete duplicate copies to storing erasure-coded fragments. This parameter change enables the system to achieve the same reliability level with less storage capacity, as erasure coding provides mathematical guarantees that any k fragments out of n can reconstruct the original data, where n > k.
2Reliability
If reserve storage capacity is set aside for computation operations, then computation reliability is improved, but operating efficiency is reduced
Solution Approach 1:
The patent enables storage nodes to serve multiple functions: they can store third-party data using erasure coding while simultaneously maintaining their ability to buffer their own data for computation operations. This multi-functionality eliminates the need for dedicated reserve storage capacity, as the same storage infrastructure supports both data redundancy and computation buffering needs.
Solution Approach 2:
The patent merges the functions of data storage, data redundancy, and computation buffering into a unified storage system. By combining these previously separate functions, the system eliminates idle reserve capacity and improves operating efficiency while maintaining the reliability needed for computation operations.
3Quantity of substance
If additional servers and hardware infrastructure are built to increase storage capacity, then storage capacity is improved, but energy consumption increases
Solution Approach 1:
The patent uses virtual copying through erasure coding instead of physical duplication of hardware. By creating mathematical fragments of data that can be reconstructed, the system achieves increased storage capacity without physically adding more servers or hardware infrastructure, thereby avoiding the associated energy consumption increase.
Data Source
AI summary
Embodiments of erasure coded storage aggregation are disclosed. The erasure coded storage aggregation includes storing a data file as erasure coded fragments in a plurality of nodes of one or more data centers. The erasure coded storage aggregation further includes monitoring an access frequency of the data file. Based on the comparison between the access frequency and a predetermined threshold, the data file is either reconstructed from the erasure coded fragments and stored in a storage node or retained as erasure coded fragments in the plurality of nodes of the one or more data centers.