Cross-File Differential Content Synchronization via Patch Mechanism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In enterprise environments, remote file synchronization over networks often faces bandwidth issues due to the transfer of entire files, leading to latency and inefficiency, especially when dealing with incremental changes, which affects user experience and costs for mobile users with limited resources.
Innovation Solution
The Cross-File Differential Content Synchronization System (CDCSS) transfers only the differences in data content by using a patch mechanism that instructs recipients to build modified content based on existing data, employing client-side indexing and content-defined chunking to reduce computational and I/O overhead, and optimize data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire files are transferred for synchronization, then data consistency is ensured, but bandwidth consumption increases and transfer latency increases
Solution Approach 1:
The patent divides files into multiple chunks and computes content-defined chunking hashes to identify only the portions of files that have changed. Instead of transferring entire files, the system segments the transfer into minimal necessary chunks, reducing bandwidth consumption while maintaining data consistency through selective synchronization of only modified segments.
2Reliability
If entire files are transferred for synchronization, then data consistency is ensured, but transfer time increases
Solution Approach 1:
By segmenting files into content-defined chunks and identifying only modified portions through hash comparison, the system transfers only the necessary segments rather than entire files. This segmentation approach dramatically reduces transfer time while ensuring data consistency through the use of content-defined chunking hashes to verify integrity of transferred portions.
3Loss of energy
If content-defined chunking is used to reduce data transfer, then bandwidth consumption decreases, but computational overhead increases
Solution Approach 1:
The system performs content-defined chunking and hash computation in advance during file synchronization operations. By pre-computing chunk hashes and maintaining indexes of file contents locally, the system reduces the computational burden during actual transfer operations. This preliminary action allows for efficient identification of changed portions without excessive computational overhead during critical transfer moments.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems and techniques for synchronizing content, such as a file, between one or more clients and one or more servers are provided. Example embodiments provide a Cross- File Differential Content Synchronization System (CDCSS) to synchronize files between remote systems in a nearly instantaneous manner without necessitating the transfer of the entire contents of a file. These components cooperate to transfer only the differences in data content wherever possible to a recipient system by providing a patch mechanism that instructs the recipient to build the modified or new data content based upon content it already has access to. In addition, in one embodiment the CDCSS synchronization solution provides a client-based index, rather than a server-based index, for tracking patches and files, which reduces the amount of computational and I/O overhead on the server and which does not require a negotiation between a client and a server to provide an update.