Distributed Storage Data Amplification Reduction via Erasure Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face data amplification issues when performing I/O operations across geographically distinct sites, leading to increased network traffic and costs due to the need to replicate data across multiple nodes, especially when using RAID 1 and erasure coding techniques.
Innovation Solution
Configuring a leader node at one site and a proxy node at another site to reduce data amplification by optimizing the distribution of write requests and using erasure coding, such as RAID 1 and RAID 5, to minimize the amount of data transmitted over wide area networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated across multiple geographically distinct sites for fault tolerance, then reliability is improved, but network traffic and data transmission costs increase
Solution Approach 1:
The system segments data into data blocks and applies different replication strategies to different sites. Primary sites receive full data replication while secondary sites receive only erasure coded data, dividing the network traffic burden across multiple sites with different roles.
Solution Approach 2:
Different sites are assigned different qualities of data storage based on their roles. Primary sites store complete data blocks for full functionality, while secondary sites store erasure coded data blocks that provide fault tolerance but require less bandwidth for updates.
2Reliability
If full data replication is performed across all sites, then fault tolerance is improved, but the amount of data transmitted over wide area networks increases
Solution Approach 1:
The system changes the parameter of data representation by applying erasure coding transformations. Instead of transmitting identical data blocks to all sites, it transmits transformed erasure coded blocks to secondary sites, reducing the quantity of data that needs to be transmitted while maintaining fault tolerance.
Solution Approach 2:
The system creates different types of copies for different sites. Primary sites receive exact copies of data blocks, while secondary sites receive erasure coded copies that can reconstruct the original data but require less transmission bandwidth and provide distributed fault tolerance.
3Loss of energy
If erasure coding is applied to reduce data transmission, then network traffic is reduced, but device complexity increases
Solution Approach 1:
The system segments the erasure coding function across multiple nodes rather than implementing it centrally. Each node performs local encoding operations on data blocks, distributing the computational complexity across the distributed system and avoiding a single point of complexity.
4Reliability
If multiple copies of data are stored across distributed nodes, then fault tolerance is improved, but the number of write requests increases
Solution Approach 1:
Different nodes are assigned different roles in the write operation process. Primary nodes handle the full write operation including receiving client requests and performing encoding, while secondary nodes only perform local storage of erasure coded blocks, reducing the number of write operations they must process.
Data Source
AI summary
Component objects of a virtual disk are backed by first storage nodes, which are at a primary site, and replicated in second storage nodes, which are at a secondary site. Responsive to a write command issued to the virtual disk, a coordinating node at the primary site issues a first write operation to a first replication module running therein, and transmits a second write operation to a second replication module running in a coordinating node at the secondary site. The first replication module, responsive to the first write operation, instructs the first storage nodes backing first and second component objects of the virtual disk to write the data. The second replication module, responsive to the second write operation, instructs the second storage nodes backing third and fourth component objects, which are replicas of the first and second component objects, respectively, to write the data.