Incremental Data Replication via Snapshot Hash Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data replication techniques are inefficient for enterprises with high data replication requirements, as they often involve full data replication rather than incremental changes, leading to increased processing and transfer volumes.
Innovation Solution
A process for efficiently detecting changes between different snapshots of a file system by comparing directory entry hashes and identifying namespace changes, allowing for incremental replication of only the detected changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full data replication is performed, then data availability and reliability are ensured, but processing volume and data transfer volume increase significantly
Solution Approach 1:
The patent extracts only the changed portions of data (incremental changes) from the complete data set and replicates only those extracted changes to the destination storage system, rather than replicating the entire data set. This is achieved through change detection mechanisms that identify modified files, directories, or data blocks between snapshots.
Solution Approach 2:
The patent segments the data replication process into change detection, change identification, and incremental replication stages. By dividing the data into individual files, directories, or blocks and tracking changes at these granular levels, the system replicates only the necessary segments that have changed, reducing overall data transfer volume while maintaining data reliability.
2Reliability
If full data replication is performed, then data consistency is maintained, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary actions by creating snapshots of the source data at specific points in time and maintaining change logs that record modifications between snapshots. This preliminary tracking of changes enables the system to quickly identify what needs to be replicated without performing time-consuming full data comparisons during the replication process itself.
Solution Approach 2:
The patent applies partial action by performing replication only on the portion of data that has changed (incremental replication) rather than replicating the entire data set. This partial replication approach maintains data consistency while significantly reducing processing time and computational resource requirements compared to full replication.
3Productivity
If change detection techniques are implemented, then data transfer efficiency improves, but detection precision and namespace change identification become more challenging
Solution Approach 1:
The patent introduces intermediaries in the form of snapshots and change logs that mediate between the source data and the replication process. These intermediaries capture the state of data at specific points in time and record changes systematically, enabling accurate detection of file-level, directory-level, and block-level changes including namespace modifications without compromising detection precision.
Solution Approach 2:
The patent adds temporal dimensionality to change detection by incorporating timestamps and snapshot versions into the detection process. By tracking changes across multiple time points and maintaining versioned snapshots, the system achieves high detection accuracy for various types of changes including file modifications, directory renames, and block alterations, while improving replication efficiency.
Data Source
AI summary
The described technology is generally directed towards efficient detection of data changes between different data representations, to enable replication of the detected data changes. A process is presented that compares different “snapshot” representations of data, wherein the data may have changed between the different snapshots. The process can detect changed files a well as namespace changes that occur when a file or directory is deleted, added, or moved to a different location.