Updating Erasure-Coded Protection Sets in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Erasure-coded storage systems face high capacity overheads due to degraded protection sets in geographically distributed storage environments, particularly when data chunks are deleted, leading to unreasonably high overheads and increased inter-zone network traffic during updates.
Innovation Solution
The system employs a resource-efficient method to update degraded data protection sets by using XOR operations and Galois Field arithmetic, allowing for minimal inter-zone network traffic updates by maintaining only one zone's data protection set updated, while the other zone's data protection set is adjusted using transformation chunks to re-protect the system without requiring inter-zone updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional erasure-coded protection sets are updated in geographically distributed storage environments, then data protection is maintained, but inter-zone network traffic increases significantly and capacity overheads become unreasonably high
Solution Approach 1:
The protection set update process is segmented by zone, allowing each zone to independently update its local protection set using only local data chunks. This eliminates the need for inter-zone data transfer during protection set updates, thereby reducing inter-zone network traffic while maintaining data protection across the geographically distributed storage system.
Solution Approach 2:
Each zone in the geographically distributed storage system maintains its own local protection set with locally computed parity chunks. This local quality approach ensures that protection operations are performed independently within each zone using local resources, avoiding the need to transfer data between zones and reducing network traffic overhead.
2Reliability
If traditional erasure-coded protection sets are updated in geographically distributed storage environments, then data protection is maintained, but capacity overheads increase unreasonably
Solution Approach 1:
The protection set is segmented into zone-specific protection sets, where each zone maintains its own protection set with locally computed parity information. This segmentation eliminates redundant storage of protection data across zones, reducing overall capacity overheads while maintaining equivalent data protection levels through distributed redundancy.
3Reliability
If complete protection set updates are performed across all zones, then data protection is ensured, but system resources and time are consumed excessively
Solution Approach 1:
The protection set update operation is divided into independent zone-level operations. Each zone can update its protection set autonomously without waiting for other zones, enabling parallel execution of update operations. This segmentation dramatically reduces the total update time while ensuring data protection across the entire geographically distributed system.
Solution Approach 2:
Each zone pre-computes its local protection set using only locally available data chunks before any data deletion or modification occurs. This preliminary action within each zone eliminates the need for coordinated inter-zone communication during updates, reducing both time and resource consumption while maintaining data protection integrity.
Data Source
AI summary
In one or more embodiments described herein, system, method, and/or computer program products that facilitate updating degraded data protection sets in a geographically distributed storage system. According to an embodiment, a method can comprise receiving, by a system comprising a processor and a memory, a request to update a protection set for a first coded chunk in response to detecting deletion of a primary data chunk, wherein the first coded chunk and the primary data chunk are stored in a geographically distributed data storage system. The method can comprise generating, by the system, a transformation data chunk utilizing a secondary data chunk stored in the geographically distributed data storage system. The method can further comprise generating, by the system, a second coded chunk having an updated protection set, wherein the updated protection set is generated utilizing the transformation data chunk.