Hardware Data Compression With Pre-Huffman Literal vs Back-Pointer Choice

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional data compression methods using dynamic Huffman code tables in DEFLATE-style compression are inefficient due to sequential processing, which delays Huffman encoding until histograms are fully generated, and sorting frequencies is time-consuming, adding to the overall compression time.

Innovation Solution

A hardware data compressor that constructs dynamic-prime Huffman code tables after an initial portion of the input block is scanned and uses them in parallel with ongoing scanning, and sorts frequencies incrementally concurrently with scanning to hide sorting time behind scanning time, reducing overall compression time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If dynamic Huffman code tables are constructed after full histogram generation, then compression accuracy is improved, but compression time increases

Engineering Contradiction:

Improvecompression accuracyVSAvoidcompression time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies preliminary action by constructing initial Huffman code tables before the entire histogram is complete. The system generates code tables based on partial frequency data from the initial portion of the input block, allowing compression to proceed in parallel with ongoing scanning and histogram generation, rather than waiting for complete frequency data

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements continuity of useful action by overlapping three processes simultaneously: (1) scanning the input block and generating the remaining histogram, (2) constructing Huffman code tables based on initial frequency data, and (3) performing Huffman encoding. This parallel execution eliminates idle time and maintains continuous productive work throughout the compression process

Inventive Principle:

Principle #20Continuity of useful action

2Productivity

If frequencies are sorted after histogram generation, then encoding efficiency is improved, but total processing time increases

Engineering Contradiction:

Improveencoding efficiencyVSAvoidtotal processing time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by performing frequency sorting on the initial portion of frequency data before the complete histogram is available. This preliminary sorting enables code table construction to proceed earlier, and the sorting operation is overlapped with ongoing histogram generation, so the sorting time does not add to the critical path

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements dynamics by making the Huffman code tables adaptive and updateable throughout the compression process. The system can reconstruct code tables as new frequency data becomes available, allowing the encoding to adapt to the actual data distribution while maintaining parallel processing of sorting, table construction, and encoding operations

Inventive Principle:

Principle #15Dynamics

3Device complexity

If sequential processing is used for LZ77 compression and Huffman encoding, then implementation simplicity is maintained, but compression throughput decreases

Engineering Contradiction:

Improveimplementation simplicityVSAvoidcompression throughput

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The patent applies segmentation by dividing the compression process into independent parallel segments: (1) LZ77 scanning and matching, (2) histogram generation, (3) Huffman code table construction, and (4) Huffman encoding. Each segment can operate independently and simultaneously on different portions of the data, increasing throughput without requiring complex inter-segment coordination

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent merges previously sequential operations into a unified parallel process. The scanning, histogram generation, code table construction, and encoding operations that were traditionally performed sequentially are merged into overlapping parallel processes that share resources and data flows, thereby increasing throughput while maintaining manageable system complexity

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS9509336B1Hardware data compressor that pre-huffman encodes to decide whether to huffman encode a matched string or a back pointer thereto

Publication Date: 2016.11.29 VIA ALLIANCE SEMICON CO LTD

AI summary

A hardware data compressor for compressing a block of characters. A first hardware engine finds a match of a string of characters at a current location in the block with an earlier occurrence in the block of the string of characters, determines a first size that is a total number of bits into which the matched string would be Huffman encoded as literals using a Huffman code table, calculates a back pointer from the current location to the earlier occurrence of the string, and determines a second size that is a number of bits into which the back pointer would be Huffman encoded using the Huffman code table. A second hardware engine Huffman encodes the matched string if the first size is less than the second size and otherwise Huffman encodes the back pointer.