Hash-Based Delta Compression for Smaller Patch Files
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional delta compression systems consume significant compute power, storage bandwidth, and storage capacity, leading to increased operational costs and reduced efficiency in data storage and transmission.
Innovation Solution
A computer-implemented method for delta compression that involves hashing portions of source and target data files, generating hashing tables, and creating patch files with instructions to efficiently determine and store differences between files, using hashing functions like djb2 and CRC32 to optimize compression and decompression processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional delta compression mechanisms are used, then data differences can be identified and stored, but significant compute power, storage bandwidth, and storage capacity are consumed
Solution Approach 1:
The patent divides the data file into multiple blocks and processes each block independently through hashing and comparison operations. This segmentation allows parallel processing of different blocks, reducing overall compute power consumption while maintaining accurate difference identification across the entire file.
Solution Approach 2:
The patent introduces hash values as an intermediary mechanism to represent data blocks. Instead of directly comparing large amounts of raw data, the system computes hash values for each block and compares these compact representations. This intermediary approach significantly reduces compute power requirements while preserving the ability to accurately identify data differences.
2Quantity of substance
If conventional delta compression mechanisms are used, then data compression can be achieved, but storage bandwidth and storage capacity are significantly consumed
Solution Approach 1:
The patent extracts only the essential difference information between data blocks by comparing hash values and generating minimal patch data. Instead of storing or transmitting entire data blocks or comprehensive difference sets, the system extracts and stores only the necessary patch instructions, significantly reducing storage bandwidth consumption and improving storage capacity utilization.
3Measurement precision
If detailed comparison of data portions is performed, then accurate delta differences can be determined, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary hashing of data blocks before detailed comparison. By pre-computing hash values for each block and organizing them in a structured format, the system prepares the data in advance for efficient comparison. This preliminary action reduces the time required for actual delta difference determination while maintaining accuracy.
Solution Approach 2:
The patent replaces direct mechanical comparison of data blocks with hash-based comparison. Instead of byte-by-byte comparison operations, the system uses cryptographic hash functions to generate compact representations that can be compared much faster. This substitution maintains detection accuracy while dramatically reducing processing time.
Data Source
AI summary
Delta compression method, system and computer program product. Portions of source and target data files are hashed using a hashing function. A target data file is compared against the source data file to determine at least one delta difference between the files. A source data file hashing table is generated. The table includes hashed portions of the source and target data files stored in corresponding source file offset locations and corresponding target file offset locations, respectively. Portions of the source and target files are compared using corresponding source and target file offset locations. At least one common sequence of characters in the portions of the source and target files is determined based on the comparison. A patch file is generated based on the determined sequence of characters.