Hardware Data Compressor with Concurrent Huffman Sorting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data compression methods using dynamic Huffman code tables in DEFLATE-style compression are inefficient due to sequential processing, which delays Huffman encoding until histograms are fully generated, and sorting frequencies is time-consuming, adding to the overall compression time.
Innovation Solution
A hardware data compressor that constructs dynamic-prime Huffman code tables after an initial portion of the input block is scanned and uses them in parallel with ongoing scanning, and sorts frequencies incrementally concurrently with input block scanning to hide sorting time behind scanning time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If sequential processing is used to generate histograms and sort frequencies before Huffman encoding, then processing simplicity is maintained, but total compression time increases
Solution Approach 1:
The patent applies preliminary action by constructing dynamic-prime Huffman code tables after scanning only an initial portion of the input block, before the complete histogram is available. This allows Huffman encoding to begin in parallel with the remaining scanning process, rather than waiting for sequential completion of histogram generation and sorting.
Solution Approach 2:
The patent implements continuity of useful action by overlapping three processes: LZ77 compression scanning, Huffman code table construction, and frequency sorting. These operations run concurrently rather than sequentially, keeping hardware resources continuously utilized and eliminating idle waiting time between processing stages.
2Measurement precision
If complete histogram generation and frequency sorting are performed before Huffman encoding, then encoding accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies partial action by constructing Huffman code tables using only a portion of the input data (initial scan portion) rather than requiring the complete histogram. This partial construction provides sufficient accuracy for effective compression while enabling parallel processing with the remainder of the scanning process.
Solution Approach 2:
The patent implements dynamics by using dynamic-prime Huffman code tables that are constructed and updated during the scanning process rather than being static. The code tables adapt as more data becomes available, allowing the system to balance between early encoding benefits and final encoding accuracy.
3Productivity
If parallel processing is used to overlap Huffman coding with LZ77 compression, then compression speed is improved, but device complexity increases
Solution Approach 1:
The patent applies merging by combining multiple processing functions into unified hardware structures. The same scanning logic serves both LZ77 compression and histogram generation, and the sorting hardware simultaneously prepares data for Huffman encoding, reducing overall device complexity despite parallel operation.
Solution Approach 2:
The patent implements multi-functionality by designing hardware components that perform multiple roles. The scanning engine generates both compression data and frequency statistics, while the sorting mechanism prepares data for both immediate encoding and final histogram completion, allowing parallel processing without proportionally increasing hardware complexity.
4Loss of time
If frequency sorting is performed incrementally concurrent with scanning, then sorting time is hidden behind scanning time, but processing complexity increases
Solution Approach 1:
The patent applies preliminary action by performing incremental frequency sorting during the scanning process rather than waiting for complete data collection. This preliminary sorting prepares frequency data for early Huffman code table construction while the scanning continues, effectively hiding sorting time within the scanning duration.
Solution Approach 2:
The patent implements continuity of useful action by maintaining continuous sorting operations throughout the scanning process. The sorting hardware operates continuously alongside scanning, ensuring that frequency data is always ready for Huffman encoding without creating idle time or processing gaps.
Data Source
AI summary
A hardware data compressor includes a first hardware engine that scans an input block of characters to produce a stream of tokens, the stream of tokens comprising replacement back pointers to matched strings of characters of the input block and non-replaced characters of the input block. The hardware data compressor also includes a second hardware engine that receives the stream of tokens and maintains a sorted list of symbols associated with the tokens. The hardware data compressor also includes the second hardware engine concurrently maintains the sorted list of symbols by frequency of occurrence as the first hardware engine produces the tokens of the stream.