Hardware Data Compressor with Dynamic Hash Selection by Block Type
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data compression methods using dynamic Huffman code tables in DEFLATE-style compression are inefficient due to sequential processing, which delays Huffman encoding until histograms are fully generated, leading to increased compression time.
Innovation Solution
A hardware data compressor that constructs 'dynamic-prime' Huffman code tables after scanning only an initial fraction of the input block, allowing parallel processing with LZ77 compression, and incrementally sorts the symbol list concurrently with scanning to hide sorting time behind compression time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sequential processing is used to generate histograms before Huffman encoding, then Huffman code tables can be constructed accurately, but compression time increases
Solution Approach 1:
The patent applies preliminary action by constructing initial Huffman code tables (dynamic-prime tables) before the entire histogram is complete. The system scans only an initial fraction of the input block to generate preliminary frequency data, constructs Huffman tables from this partial data, and begins encoding immediately. This allows compression to start earlier without waiting for complete histogram generation, thereby reducing overall compression time while maintaining acceptable code table accuracy for the majority of the data.
2Productivity
If dynamic Huffman code tables are constructed after full histogram generation, then optimal compression ratios are achieved, but processing efficiency decreases
Solution Approach 1:
The patent implements continuity of useful action by overlapping multiple operations that would traditionally be sequential. Specifically, the LZ77 compression engine continues scanning the input block while the Huffman encoding engine simultaneously encodes data using the dynamic-prime tables constructed from the initial fraction. This parallel execution eliminates idle time and keeps both engines productive concurrently, significantly improving overall processing efficiency without sacrificing compression quality.
3Measurement precision
If sorting is performed after histogram generation, then symbol lists are properly ordered for Huffman encoding, but sorting time adds to overall compression time
Solution Approach 1:
The patent applies preliminary action by performing incremental sorting during the histogram generation phase rather than as a separate post-processing step. As frequency data is collected from the initial fraction scan, the symbol list is sorted in preparation for immediate Huffman table construction. This eliminates the need for a separate sorting phase after complete histogram generation, as the sorting is already accomplished with the partial data available earlier in the process.
Data Source
Figure 1~2A
Figure 2B~3
Figure 4
AI summary
A hardware data compressor that compresses an input block of characters by replacing strings of characters in the input block with back pointers to matching strings earlier in the input block. A hash table is used in searching for the matching strings in the input block. A plurality of hash index generators each employs a different hashing algorithm on an initial portion of the strings of characters to be replaced to generate a respective index. The hardware data compressor also includes an indication of a type of the input block of characters. A selector selects the index generated by of one of the plurality hash index generators to index into the hash table based on the type of the input block.