Hardware Compression Header Generation for Single-Pass DEFLATE
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression technologies, such as the DEFLATE algorithm, require multiple passes through the data for efficient compression, leading to increased computational overhead and latency.
Innovation Solution
Implementing a hardware compressor with integrated Huffman table generation capabilities, allowing for single-pass data compression using modes like one-descriptor, two-descriptor, statistics-input, no-header, complete-tree, and hybrid-compress modes, reducing CPU load and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If DEFLATE algorithm uses optimized Huffman codes for each block, then compression ratio is improved, but computational overhead and latency increase due to requiring two passes through the data
Solution Approach 1:
The patent pre-computes Huffman codes using a training set before actual compression. This preliminary action creates a library of pre-optimized Huffman codes that can be directly applied during compression without requiring statistical analysis of the actual data block, thereby eliminating the two-pass requirement and reducing latency while maintaining good compression ratios
Solution Approach 2:
The system pre-generates DEFLATE block headers during the training phase. These pre-computed headers are stored and reused during compression, eliminating the need to compute headers during the actual compression process, thus reducing computational overhead and latency
2Productivity
If DEFLATE algorithm computes Huffman codes dynamically based on block statistics, then compression efficiency is improved, but device complexity increases due to requiring statistical analysis and code generation
Solution Approach 1:
The patent performs all statistical analysis and Huffman code generation during an offline training phase using a representative training set. This preliminary computation creates a lookup table of pre-optimized codes that are stored for reuse. During actual compression, the system simply retrieves pre-computed codes based on data characteristics, dramatically reducing computational complexity while maintaining compression efficiency
Solution Approach 2:
The system creates a copy of optimized Huffman codes derived from training data and stores them in a lookup table. Instead of重新 computing codes for each compression task, the system copies and applies appropriate pre-computed codes, reducing computational complexity while preserving compression efficiency
Data Source
Figure 1~2A
Figure 2B
Figure 2C
AI summary
An embodiment of an integrated circuit may comprise a hardware compressor to compress data, the hardware compressor including circuitry to store input data in a history buffer, compute one or more code tables based on the input data, and compute a compression stream header based on the computed one or more code tables. Other embodiments are disclosed and claimed.