Deflate Compression With Adaptive Huffman Trees and Lower Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The deflate compression algorithm faces trade-offs between using a fixed Huffman coding tree, which may not accurately represent data probabilities, and generating an optimal tree, which requires additional space to transmit, leading to suboptimal compression ratios due to the static nature of the coding tree.
Innovation Solution
Implementing a variable context dynamic encoder that processes the output of a first compression algorithm, such as LZSS, to generate a tree of non-overlapping bit-sequences where sequence length is inversely proportional to the likelihood of the symbol, eliminating the need to transmit unused symbols and statistics, and using an adaptive context Huffman coding algorithm that evolves over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed Huffman coding tree is used, then the compression process is simpler and faster, but the compression ratio is lower because the tree may not accurately represent the actual data probabilities
Solution Approach 1:
The patent applies dynamics by transitioning from a static fixed Huffman coding tree to a dynamic adaptive Huffman coding tree that evolves during compression. The tree structure is updated incrementally as data is processed, allowing it to adapt to the actual probability distribution of the input data while maintaining reasonable compression speed through efficient update mechanisms.
Solution Approach 2:
The patent changes the parameter of the Huffman coding tree from fixed to adaptive. By allowing the tree structure parameters to change dynamically based on observed data frequencies, the system achieves better compression ratios while managing the complexity of parameter updates through systematic approaches.
2Loss of substance
If an optimal Huffman coding tree is generated, then the compression ratio improves, but additional space is required to transmit the tree definition
Solution Approach 1:
The patent extracts only the essential elements needed for compression from the full Huffman tree definition. By identifying and transmitting only the critical tree structure information rather than complete tree definitions, it reduces transmission overhead while maintaining the benefits of adaptive coding for improved compression ratios.
Solution Approach 2:
The patent applies partial action by transmitting a simplified or partial representation of the Huffman tree rather than the complete tree structure. This partial transmission is sufficient to enable the decoder to reconstruct or infer the necessary coding information, reducing the amount of data that needs to be transmitted while still achieving good compression ratios.
3Device complexity
If a static Huffman coding tree is used, then the encoding process is simpler, but the compression efficiency is reduced due to inability to adapt to data characteristics
Solution Approach 1:
The patent transforms the static encoding process into a dynamic one where the Huffman tree adapts to data characteristics during compression. This dynamic approach improves compression efficiency by matching the code structure to the actual data distribution while managing complexity through systematic update procedures.
Solution Approach 2:
The adaptive Huffman coding system performs self-service by automatically adjusting its own tree structure based on the data it processes. The encoder monitors data frequencies and updates the tree accordingly, eliminating the need for external intervention or complex pre-processing, thereby achieving better compression efficiency with manageable complexity.
Data Source
AI summary
A method and apparatus for compressing data is described. In one embodiment, a processor receives one or more strings of data to be compressed. Duplicate strings are replaced with pointers using a first compression algorithm. An output of the first compression algorithm is processed with a second compression algorithm using a variable context dynamic encoder to generate a tree of non-overlapping bit-sequences where the length of each sequence is being inversely proportional of the likelihood of that symbol needing to be encoded. Unused symbols are not generated on the tree.