Hash-Based Delta Encoding for Faster Large-Block Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The XDelta encoding algorithm's byte-by-byte matching process leads to a heavier workload and longer time for data compression, resulting in relatively low data compression efficiency due to larger reference blocks.
Innovation Solution
The method involves performing a hash operation on target sub-blocks to obtain key values, querying hash tables for matching reference data, and generating encoding sequences with matching lengths and offsets to narrow the matching range, thereby reducing compression time and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If byte-by-byte matching is performed in XDelta encoding, then matching accuracy is improved, but data compression time increases and efficiency decreases
Solution Approach 1:
The patent divides the target block into multiple sub-blocks and processes each sub-block independently through hash operations. This segmentation allows parallel processing and reduces the overall matching time while maintaining accuracy through systematic verification of each segment against the reference block.
Solution Approach 2:
The patent performs hash operations on sub-blocks before actual matching to pre-identify potential matching positions. By calculating hash values in advance and comparing them with reference block hashes, the system narrows down the search space before performing detailed byte-by-byte verification, thus reducing total compression time.
2Productivity
If larger reference blocks are used, then compression ratio is improved, but matching workload increases and processing speed decreases
Solution Approach 1:
The patent segments both the target block and reference block into sub-blocks of manageable size. This allows the system to handle large reference blocks by processing them in smaller chunks through hash operations, reducing the computational complexity of matching while still achieving good compression ratios through comprehensive sub-block comparison.
Solution Approach 2:
The patent changes the parameter of block processing by introducing sub-block division and hash-based filtering. Instead of directly comparing entire large blocks, the system transforms the problem into comparing hash values of smaller sub-blocks, which reduces the effective matching workload while preserving the ability to achieve high compression ratios.
3Manufacturing precision
If comprehensive matching is performed across entire blocks, then encoding accuracy is improved, but system performance deteriorates
Solution Approach 1:
The patent performs preliminary hash operations on all sub-blocks before actual matching. This preliminary action creates a filtered set of potential matches based on hash value comparison, which then guides the detailed matching process. This two-stage approach ensures encoding accuracy through thorough verification while improving system performance by eliminating obviously non-matching segments early.
Solution Approach 2:
By dividing blocks into sub-blocks and processing them through hash operations, the patent enables systematic verification of encoding accuracy at the sub-block level while maintaining overall system performance through efficient parallel processing and reduced computational overhead compared to monolithic block matching.
Data Source
AI summary
An encoding method and apparatus is described. In an encoding method, when a first target sub-block in a target block is obtained, a hash operation is first performed on the first target sub-block. Then, a first hash table is queried for a corresponding hash value according to an operation result, and a corresponding location in a reference block is found according to the hash value obtained by means of query, that is, first reference data is found. The first piece of target data in the first target sub-block is matched with the first reference data, and second target data in the target block is matched with second reference data in the reference block. In this way, an approximate location is predetermined, so that a range in which matching needs to be performed is narrowed, a data compression time is reduced, and data compression efficiency is improved.