Huffman Code Length Limiting Without Tree Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional canonical Huffman encoding (CHE) algorithms for data compression do not guarantee a maximum code length, leading to repeated cycles and reduced compression throughput, especially with emerging standards like Xpress and Brotli that require more frequent code length limitations due to increased numbers of symbols.
Innovation Solution
The proposed solution involves alternative techniques for length limiting compression codes that do not require reconstructing the entire encoding tree, allowing for efficient length limitation without frequency adjustments, thereby improving compression throughput and enabling more efficient accelerator implementations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If conventional canonical Huffman encoding is used without code length guarantee, then compression ratio is improved, but compression throughput deteriorates due to repeated encoding cycles
Solution Approach 1:
The patent pre-calculates and stores the cumulative frequency values and code length information during the initial Huffman tree construction phase. This preliminary action allows the encoder to determine code lengths for all symbols in advance, avoiding the need for repeated encoding cycles and frequency adjustments, thus maintaining both high compression ratio and throughput
Solution Approach 2:
The patent creates a separate data structure (lookup table) that stores pre-computed code length information as a copy of the original frequency data. This copying approach allows the encoder to quickly retrieve code lengths without reprocessing the original frequency information, eliminating repeated encoding cycles while preserving compression efficiency
2Manufacturing precision
If code length limitation is enforced by repeating CHE process with frequency adjustment, then code length bound is guaranteed, but latency increases by thousands of cycles
Solution Approach 1:
The patent performs code length validation and adjustment during the initial tree construction phase rather than after encoding. By preliminarily determining which symbols exceed the maximum code length and adjusting their frequencies in advance, the patent guarantees code length bounds without requiring repeated encoding passes, reducing latency from thousands of cycles to a single pass
Solution Approach 2:
The patent introduces an intermediary data structure (cumulative frequency table with code length information) that mediates between the original frequency data and the final encoding process. This intermediary structure allows the system to enforce code length constraints without directly repeating the entire CHE process, as the intermediary already contains pre-computed information about code lengths and required adjustments
3Adaptability or versatility
If emerging compression standards with increased symbol counts are adopted, then compression capability is enhanced, but code length limitation frequency increases compounding performance penalties
Solution Approach 1:
The patent pre-computes code lengths for all symbols during the initial Huffman tree construction, regardless of the total symbol count. This preliminary computation of code lengths allows the system to handle emerging standards with increased symbol counts (such as Brotli with 704 symbols) without requiring repeated encoding cycles, as the code length information is already available in the lookup table
Solution Approach 2:
The patent changes the approach from iterative frequency adjustment to a single-pass parameter computation method. By computing code lengths as a function of cumulative frequencies in advance and storing them in a lookup table, the system can adapt to different symbol counts in emerging standards without changing the fundamental encoding process, maintaining throughput while supporting enhanced compression capabilities
Data Source
AI summary
A processing device includes compression circuitry to encode an input stream with an encoding that translates multiple symbols of fixed length into multiple codes of variable length between one and a maximum length, to generate a compressed stream. The compression circuitry is to: determine at least a first symbol of the multiple symbols having a first code that exceeds the maximum length; identify a short code of the multiple codes that is to be lengthened to provide an increased encoding capacity for the at least the first symbol; generate multiple code-length converted values including to increase the length of the short code to the maximum length and decrease, to the maximum length, a length of the first code of the at least the first symbol; and generate, with use of the set of code-length converted values, the compressed stream at the output terminal.