Hardware Data Compression With Pre-Huffman Match Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data compression methods using dynamic Huffman code tables in DEFLATE-style compression are inefficient due to sequential processing, which delays Huffman encoding until histograms are fully generated, leading to increased compression time.
Innovation Solution
A hardware data compressor that constructs 'dynamic-prime' Huffman code tables after an initial fraction of the input block is scanned, allowing parallel processing with LZ77 compression and incremental sorting of the symbol list, thereby hiding Huffman coding time behind LZ77 compression time and reducing overall compression time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If dynamic Huffman code tables are constructed after full histogram generation in sequential processing, then Huffman encoding accuracy is improved, but compression time increases
Solution Approach 1:
The patent applies preliminary action by constructing 'dynamic-prime' Huffman code tables after scanning only an initial fraction of the input block, before the complete histogram is generated. This allows Huffman encoding to begin in parallel with the remaining LZ77 compression work, rather than waiting for sequential completion. The preliminary Huffman tables provide sufficiently accurate encoding while enabling time overlap, thus reducing total compression time without significantly sacrificing encoding accuracy.
Solution Approach 2:
The patent implements continuity of useful action by overlapping Huffman encoding operations with ongoing LZ77 compression and incremental sorting. Instead of sequential processing where Huffman encoding waits for complete histogram generation, the system maintains continuous productive work by performing Huffman encoding on already-scanned portions while the scanner continues processing the remainder of the input block. This eliminates idle waiting time and keeps all processing units continuously utilized.
2Measurement precision
If Huffman encoding is performed sequentially after LZ77 compression, then encoding quality is improved, but overall processing speed deteriorates
Solution Approach 1:
The patent performs preliminary Huffman encoding on the portion of the input block that has already been scanned and processed by LZ77, before the entire block is scanned. This preliminary encoding action occurs in parallel with the continuation of LZ77 compression on the remaining unscanned portions, thereby improving processing speed without compromising the quality of encoding for the portions processed.
Solution Approach 2:
The patent transitions from sequential single-dimension processing to parallel multi-dimensional processing by overlapping three operations simultaneously: LZ77 compression scanning, Huffman code table construction, and Huffman encoding. This dimensional change from sequential to parallel execution across multiple processing dimensions enables both high encoding quality and improved processing speed.
3Measurement precision
If complete histogram generation is waited for before Huffman coding, then coding accuracy is improved, but compression time increases
Solution Approach 1:
The patent applies preliminary action by generating Huffman code tables after only an initial fraction of the input block has been scanned and its histogram generated, rather than waiting for the complete input block histogram. This preliminary Huffman table generation provides sufficiently accurate coding while enabling parallel processing with the remainder of the compression pipeline, thus reducing compression time without significantly degrading coding accuracy.
Solution Approach 2:
The patent performs partial histogram generation and partial Huffman encoding on the initially scanned fraction of the input block, rather than waiting for complete processing. This partial action is sufficient to produce useful compressed output in parallel, and the remaining portions are processed subsequently, achieving a balance between accuracy and time efficiency.
Data Source
Figure 1~2A
Figure 2B~3
Figure 4
AI summary
A hardware data compressor for compressing a block of characters. A first hardware engine finds a match of a string of characters at a current location in the block with an earlier occurrence in the block of the string of characters, determines a first size that is a total number of bits into which the matched string would be Huffman encoded as literals using a Huffman code table, calculates a back pointer from the current location to the earlier occurrence of the string, and determines a second size that is a number of bits into which the back pointer would be Huffman encoded using the Huffman code table. A second hardware engine Huffman encodes the matched string if the first size is less than the second size and otherwise Huffman encodes the back pointer.