Logical Compaction of Degraded Chunks in Geographically Diverse Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage techniques face inefficiencies in reducing storage resource consumption, particularly when dealing with partially filled or degraded data chunks in geographically diverse storage systems, as they often require transferring and storing entire chunks, which wastes network and storage resources.
Innovation Solution
The method involves logically compacting degraded chunks by representing relevant data fragments differently, allowing for convolution with other chunks to create a new, smaller convolved chunk that consumes less storage space and reduces network resource usage, using techniques like XOR operations and chunk tables to manage and deconvolve the data efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire degraded chunks are transferred and stored in geographically diverse locations, then data redundancy and reliability are improved, but storage resource consumption and network resource usage increase
Solution Approach 1:
The degraded chunk is segmented into relevant data fragments and non-relevant data. Only the relevant fragments are transferred and stored in geographically diverse locations, while non-relevant data is discarded. This segmentation approach maintains data redundancy and reliability while significantly reducing storage resource consumption compared to transferring entire chunks.
Solution Approach 2:
The method extracts only the essential relevant data fragments from degraded chunks that are necessary for data recovery. By taking out and storing only these critical fragments in geographically diverse locations, the system achieves adequate redundancy without the overhead of storing complete degraded chunks, thus reducing storage resource consumption.
2Loss of information
If entire degraded chunks are transferred over the network, then data completeness is improved, but network resource usage and transfer time increase
Solution Approach 1:
The network transfer process is optimized by segmenting the degraded chunk into relevant data fragments. Only these essential fragments are transmitted over the network to geographically diverse storage locations, which reduces network transfer time and resource usage while maintaining sufficient data completeness for recovery operations.
Solution Approach 2:
Instead of transferring complete degraded chunks, the method applies partial action by transmitting only the necessary relevant data fragments. This partial transfer approach achieves adequate data completeness for recovery purposes while significantly reducing network transfer time and resource consumption.
3Reliability
If degraded chunks are stored in their original form, then data recovery capability is maintained, but storage space efficiency decreases
Solution Approach 1:
Degraded chunks are segmented into relevant data fragments and non-relevant data. The system stores only the relevant fragments in geographically diverse locations, maintaining full data recovery capability while eliminating waste of storage space on non-relevant data portions.
Solution Approach 2:
The method discards non-relevant data from degraded chunks that cannot contribute to recovery. By discarding this redundant information and storing only the essential relevant fragments, the system maintains data recovery capability while achieving superior storage space efficiency.
Data Source
AI summary
Generating, in a geographically diverse storage system, a degraded convolved chunk that consumes less storage space than a convolved chunk that is not a degraded chunk is disclosed. The degraded convolved chunk can be generated at a third zone of the storage system and be based on a compressed representation of a first chunk from a first zone of the storage system and a second chunk from a second zone of the storage system, wherein the first chunk is a degraded chunk that comprises at least one non-relevant chunk fragment. In an embodiment, the compressed representation can be generated at the first zone. In another embodiment the compressed representation can be generated at the third zone. In an aspect, mapping data corresponding a logical compression of first chunk fragments to physical storage locations of the first chunk fragments can be employed in data operations of the storage system.