Snapshot Tree Data Replication via Chunk Identifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication methods are inefficient and time-consuming, especially when dealing with large volumes of data, as they often involve copying entire blocks and can result in duplicative data storage.
Innovation Solution
The use of a tree data structure, such as a snapshot tree, to organize and replicate file system data by identifying and sending only new or modified chunk identifiers, allowing the receiving site to request and replicate only the necessary data chunks, thereby avoiding duplication and reducing the replication time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is copied from one site to another using traditional methods, then data replication is achieved, but the process takes a long period of time and consumes excessive network bandwidth
Solution Approach 1:
The patent divides data into smaller chunks and uses a tree data structure (snapshot tree) to organize these chunks. By segmenting the data replication process into individual chunk operations rather than copying entire files or blocks, the system can identify and replicate only the necessary portions of data, significantly improving replication speed and reducing time consumption.
Solution Approach 2:
The patent extracts only the new or modified chunks from the source data by comparing chunk identifiers in the snapshot tree with previously replicated data. This extraction approach allows the system to send only the necessary data chunks to the destination site, eliminating the need to transfer redundant data and thereby reducing both time and network bandwidth consumption.
2Quantity of substance
If traditional data copying methods are used, then data is replicated to the destination site, but duplicative data is stored consuming additional storage space
Solution Approach 1:
The patent performs preliminary comparison of chunk identifiers before actual data transfer. By using the snapshot tree structure to identify which chunks are new or modified, the system can determine in advance exactly what data needs to be replicated. This preliminary identification prevents the destination site from storing duplicative data that already exists, optimizing storage space utilization.
3Loss of energy
If chunk identifiers are used to identify new data, then network bandwidth is conserved and duplication is reduced, but the complexity of the replication system increases
Solution Approach 1:
The patent introduces a snapshot tree data structure as an intermediary mechanism to manage chunk identifiers. This tree structure serves as a mediator between the source and destination sites, enabling efficient identification of new chunks through identifier comparison. While this adds some complexity, it provides a systematic and organized approach that scales well and reduces overall system complexity compared to alternative methods of tracking and comparing large volumes of data chunks.
Data Source
AI summary
A data identifier for each data portion of a first group of different data portions of a first version of data is determined. The first version of the data is represented in a tree structure that references the determined data identifiers. A second version of the data is represented in a second tree structure using at least a portion of elements of the first tree structure of the first version. The second tree structure references one or more data identifiers of a portion of the second version of the data that is different from the first version of the data. The one or more data identifiers of the portion of the second version of the data that is different from the first version of the data are identified and sent. A response indicating which of the data portions corresponding to the sent one or more data identifiers are requested to be provided for replication is received.