DEFLATE Match Selection for Better Huffman Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing DEFLATE algorithm for lossless data compression does not optimize the selection of matching strings for replacement, leading to suboptimal compression rates due to fixed sliding windows and sequential matching order, which can result in inefficient Huffman encoding.
Innovation Solution
The proposed method modifies the DEFLATE algorithm by selecting the length of a matching string based on a distance threshold and window size dependent on the string length, allowing for more flexible matching windows and incorporating a distance factor into the match selection process, and using separate Huffman encoding for different distance sets to improve compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed sliding window size is used in DEFLATE compression, then the algorithm structure is simple and easy to implement, but the compression rate is suboptimal due to inability to adapt to different string lengths
Solution Approach 1:
The patent applies dynamics by making the sliding window size variable rather than fixed. The window size dynamically adapts based on the length of the matching string being compressed, allowing the algorithm to optimize for different data patterns while maintaining a relatively simple overall structure.
Solution Approach 2:
The invention changes the parameter of window size from a constant value to a variable that depends on the matching string length. This parameter change enables the algorithm to achieve better compression rates by adapting to different data characteristics without requiring complete algorithmic redesign.
2Productivity
If sequential matching order is used in DEFLATE compression, then the implementation is straightforward, but the compression efficiency is reduced due to suboptimal match selection
Solution Approach 1:
The patent changes the selection criteria parameter from purely sequential to a combination of distance and length factors. By modifying the selection parameter to consider both how far back a match occurs and how long the match is, the algorithm achieves better compression efficiency while maintaining reasonable implementation complexity.
3Loss of information
If longest match selection is used without distance consideration, then the matching coverage is maximized, but the Huffman encoding efficiency is reduced due to larger distances requiring more bits
Solution Approach 1:
The invention changes the selection parameter to balance two factors: match length and distance. By incorporating distance as a selection criterion alongside length, the algorithm selects matches that are both sufficiently long for good compression and sufficiently close to minimize the bits required for distance encoding in Huffman coding.
Solution Approach 2:
The patent applies local quality by making the match selection criterion location-dependent. Different regions of the data stream may benefit from different balances between length and distance, and the algorithm adapts locally to optimize for each specific context rather than applying a single global criterion.
4Loss of information
If separate Huffman encoding for different distance sets is implemented, then the bit representation is optimized, but the encoding complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the distance space into different sets or ranges, each with its own optimized Huffman encoding. This segmentation allows the encoder to use more efficient bit representations for different distance categories, reducing overall bit consumption while keeping each individual encoding table manageable in size.
Data Source
AI summary
A compression algorithm replaces duplicative strings with a copy pair indicating a location and length of a preceding identical string that is within a window from the duplicative string. Rather than a replacing a longest matching string within a window from a given point with a copy pair, the longest matching string may be used provide it is at least two bytes larger than the next longest matching string or is at a distance that is less than some multiple of a distance to the next longest matching string. In another aspect, the length of the window in which a matching string may be found is dependent on a length of the matching string. In yet another aspect, rather than labeling each literal and copy pair to indicate what it is, strings of non-duplicative literals are represented by a label and a length of the string.