Hardware DEFLATE Header Generation with History Buffer Replay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The DEFLATE compression algorithm requires two passes through the data, one to compute statistics and another for actual compression, which is inefficient and increases latency and CPU load, especially in hardware compression systems where resources are limited.
Innovation Solution
Implementing a hardware compressor with multiple modes of operation, including a one-descriptor mode that performs both passes simultaneously using a history buffer to store and reuse input data, reducing the need for intermediate tokens and minimizing area footprint, and a two-descriptor mode that stores code tables and headers in memory for larger data sets, allowing for efficient Huffman code and DEFLATE header generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If two-pass DEFLATE compression is used to compute Huffman codes based on block statistics, then compression efficiency is improved, but processing time and CPU load increase
Solution Approach 1:
The patent performs the statistics computation and Huffman code generation in advance during a first pass through the data, storing the results in a history buffer. This preliminary action allows the second pass to use pre-computed codes without recalculating, thereby maintaining compression efficiency while reducing the computational burden during actual compression operations.
Solution Approach 2:
The compression process is divided into two distinct passes: a first pass that computes statistics and generates Huffman codes, and a second pass that performs actual compression using the pre-computed codes. This segmentation allows the computationally intensive statistics computation to be separated from the compression operation, enabling optimization of each phase independently.
2Productivity
If two-pass DEFLATE compression is implemented to optimize Huffman codes, then compression ratio is improved, but device complexity increases
Solution Approach 1:
The patent creates a copy of the input data in a history buffer during the first pass, allowing the second pass to operate on this copy while the original data remains available for reference. This copying mechanism enables the two-pass algorithm to function without requiring complex in-place transformations, simplifying the overall device architecture while maintaining high compression ratios.
3Productivity
If hardware compressor is used to perform Huffman table and header calculations, then CPU load is reduced, but area footprint increases
Solution Approach 1:
The patent extracts the computationally intensive Huffman table generation and DEFLATE header calculation functions from the main CPU and implements them in dedicated hardware circuitry. This extraction transfers the computational burden to specialized hardware components, significantly reducing CPU load while the modular design keeps the additional hardware footprint manageable.
Solution Approach 2:
The history buffer in the hardware compressor serves multiple functions: it stores input data for the first pass, holds computed statistics, and provides data for the second pass compression. This multi-functionality reduces the need for separate dedicated storage components, thereby minimizing the overall area footprint while enabling CPU offloading.
Data Source
AI summary
An embodiment of an integrated circuit may comprise a hardware compressor to compress data, the hardware compressor including circuitry to store input data in a history buffer, compute one or more code tables based on the input data, and compute a compression stream header based on the computed one or more code tables. Other embodiments are disclosed and claimed.