File Update via Rolling Checksum Block Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for updating large files are inefficient, as they often require downloading the entire updated container file, even if only minor changes have been made, leading to significant time consumption, especially in scenarios like application virtualization and file synchronization across multiple computers.
Innovation Solution
A method that involves breaking down files into variable-sized blocks, calculating rolling checksums, and comparing hash values to identify and download only the blocks that have changed, allowing for efficient updating by assembling the new file on the client computer using information from a server.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire updated container file is downloaded to replace the old version, then the file is updated to the current version, but the downloading time becomes very large even for small changes
Solution Approach 1:
The patent divides the container file into multiple fixed-size blocks and calculates a rolling checksum for each block. By comparing checksums between the local file and update file, the system identifies only the specific blocks that need to be downloaded, rather than downloading the entire file. This segmentation approach directly resolves the contradiction by maintaining update completeness while dramatically reducing download time for partial updates.
Solution Approach 2:
The patent extracts only the necessary portions (specific blocks) from the update file that contain actual changes, rather than transferring the entire updated file. The rolling checksum mechanism enables precise identification and extraction of only those blocks that differ between versions, eliminating unnecessary data transfer while ensuring the file is fully updated.
2Measurement precision
If rolling checksums are calculated for the entire file using segments, then block comparison accuracy is improved, but the computational complexity increases
Solution Approach 1:
The patent divides the file into fixed-size blocks and calculates rolling checksums independently for each block rather than for the entire file. This segmentation reduces computational complexity by processing smaller units in parallel, while still achieving high accuracy through the mathematical properties of rolling checksums that ensure precise block identification.
Solution Approach 2:
The patent uses rolling checksums as a parameter transformation that converts large blocks of data into small fixed-size hash values. This parameter change enables efficient comparison with minimal computational overhead while maintaining high precision in identifying matching blocks, resolving the contradiction between accuracy and complexity.
3Ease of manufacture
If fixed-size blocks are used for file segmentation, then the block comparison process is simplified, but the precision in identifying changed regions may be reduced
Solution Approach 1:
The patent applies rolling checksums as a parameter transformation that operates on fixed-size blocks to produce precise identification of changed regions. The mathematical properties of rolling checksums ensure that even small changes within a fixed block are detected, maintaining high precision while preserving the simplicity of fixed-size block processing.
Solution Approach 2:
The patent introduces rolling checksums as an intermediary mechanism between fixed-size blocks and change detection. The checksum acts as a mediator that preserves the simplicity of fixed-block processing while enabling precise identification of changed regions through its mathematical properties that detect any variation within the block.
Data Source
AI summary
A method for updating and formatting a local file is described. An update file, having a newer version of the local file, is accessed from a server over a network and a list of new hash values corresponding to data blocks of the update file is retrieved. Divisions between the data blocks are located when a value of a rolling checksum reaches a particular value, the data blocks being of variable size. New hash values of the update file are compared with local hash values of the local file. For each of the new hash values not having a matching local hash value, the corresponding data block is downloaded from the server and written to a new file. For each of the new hash values having a matching local hash value, the corresponding block is copied from the local file to the new file.