Merkle Tree Cloud Storage Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for transmitting backup objects over networks require the transmission of all data blocks, leading to inefficiencies, especially when dealing with large files, as they do not effectively utilize existing data chunks stored in deduplicating block stores.
Innovation Solution
The use of Merkle Tree representations to identify and transmit only changed blocks, along with corresponding Merkle nodes, to a deduplicating block store, leveraging the hierarchical nature of Merkle Trees to reduce the number of messages required and optimize bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data blocks are transmitted for backup objects, then complete data redundancy is achieved, but network bandwidth is wasted and transmission time increases
Solution Approach 1:
The patent extracts only the changed blocks from the source object by comparing Merkle trees, rather than transmitting all blocks. This is achieved by identifying divergent paths in the Merkle tree structure and extracting only those specific blocks that differ between the source and destination objects, thereby eliminating redundant data transmission while maintaining complete data redundancy.
Solution Approach 2:
The patent segments the data transmission task by dividing the object into blocks and further organizing them into a Merkle tree structure. This hierarchical segmentation allows the system to identify and transmit only the specific changed blocks rather than treating the entire object as a single unit, optimizing network bandwidth usage while ensuring complete reconstruction capability.
2Reliability
If all data blocks are transmitted for backup objects, then complete data redundancy is achieved, but transmission time increases
Solution Approach 1:
The patent extracts only the changed blocks from the source object by comparing Merkle trees, rather than transmitting all blocks. This is achieved by identifying divergent paths in the Merkle tree structure and extracting only those specific blocks that differ between the source and destination objects, thereby eliminating redundant data transmission while maintaining complete data redundancy.
Solution Approach 2:
The patent performs preliminary action by constructing Merkle trees for both the source object and previously stored objects before transmission. This pre-computation of hash structures enables rapid identification of changed blocks, allowing the system to prepare the differential data in advance and significantly reducing the actual transmission time while ensuring complete data redundancy.
3Loss of energy
If Merkle Tree representations are used to transmit only changed blocks, then network bandwidth is optimized, but message complexity increases
Solution Approach 1:
The patent uses copying by transmitting Merkle node representations that reference existing blocks in the destination store. Instead of transmitting redundant block data, the system copies references to blocks that already exist at the destination, allowing reconstruction of the complete object through a combination of transmitted changed blocks and existing referenced blocks, thereby optimizing bandwidth while managing message complexity through efficient referencing.
4Productivity
If Merkle Tree representations are used to transmit only changed blocks, then transmission efficiency is improved, but computational overhead increases
Solution Approach 1:
The patent segments the data transmission task by dividing the object into blocks and further organizing them into a Merkle tree structure. This hierarchical segmentation allows the system to identify and transmit only the specific changed blocks rather than treating the entire object as a single unit, optimizing network bandwidth usage while ensuring complete reconstruction capability.
Solution Approach 2:
The patent uses copying by transmitting Merkle node representations that reference existing blocks in the destination store. Instead of transmitting redundant block data, the system copies references to blocks that already exist at the destination, allowing reconstruction of the complete object through a combination of transmitted changed blocks and existing referenced blocks, thereby optimizing bandwidth while managing message complexity through efficient referencing.
Data Source
AI summary
Efficient cloud storage systems, methods, and media are provided herein. Exemplary methods may include locating a Merkle tree of a stored object on a deduplicating block store, comparing an object at a source location to the Merkle tree of the stored object, determining changed blocks for the object at a source location, and transmitting a message across a network to the deduplicating block store, the message including the change blocks and Merkle nodes that correspond to the change blocks.