Demoted Chunks for Distributed Storage Traffic Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based data storage systems face inefficiencies in network traffic due to the need for remote chunk replication during recovery and deletion processes, particularly when using the XOR technique for remote replication, which increases capacity overhead and network traffic.
Innovation Solution
Implementing demoted chunks, where replicated data chunks are saved instead of deleted after creating XOR chunks, allowing for reduced network traffic by using locally available demoted copies for recovery and deletion, and reporting occupied space as available to manage storage efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If replicated data chunks are deleted after creating XOR chunks, then storage capacity is optimized, but network traffic increases during recovery and deletion operations
Solution Approach 1:
The patent performs preliminary action by saving replicated data chunks as demoted chunks after creating XOR chunks, rather than deleting them immediately. This preliminary retention of data allows the system to avoid future network traffic during recovery and deletion operations, as the demoted chunks can be reused locally when needed.
Solution Approach 2:
The patent implements a selective discarding and recovering strategy where replicated chunks are transformed into demoted chunks that can be recovered and reused. Instead of permanently deleting replicated chunks, the system recovers them in a demoted state, allowing them to serve as local copies during recovery and deletion operations, thereby reducing network traffic.
2Reliability
If remote chunk replication is performed during recovery and deletion, then data protection is maintained, but network traffic and capacity overhead increase
Solution Approach 1:
The patent introduces demoted chunks as an intermediary between fully replicated chunks and deleted chunks. These demoted chunks serve as local intermediaries that can satisfy recovery and deletion operations without requiring remote replication, thereby reducing network traffic while maintaining data protection through the XOR chunk mechanism.
Solution Approach 2:
The system performs preliminary action by creating and retaining demoted chunks before recovery or deletion operations are needed. This advance preparation ensures that when recovery or deletion is required, local demoted copies are available, eliminating the need for immediate remote replication and reducing network traffic.
3Loss of energy
If demoted chunks are retained instead of deleted, then network traffic is reduced, but storage space is occupied
Solution Approach 1:
The patent applies discarding and recovering by transitioning replicated chunks to demoted chunks that occupy storage space but can be discarded (deleted) when no longer needed. The system recovers value from these demoted chunks by using them to reduce network traffic during operations, and can discard them when storage space needs to be freed, balancing storage usage against network traffic reduction benefits.
Data Source
AI summary
One embodiment is related to a method for implementing a demoted chunk, comprising: replicating a first chunk from a first zone of a cloud storage system to a third zone of the cloud storage system; replicating a second chunk from a second zone of the cloud storage system to the third zone; creating an exclusive or (XOR) chunk at the third zone based at least in part on the replicated first chunk and the replicated second chunk; and marking the replicated first chunk and the replicated second chunk as demoted first chunk and demoted second chunk, respectively, at the third zone.