Compression Hash Processor with FIFO Hash-Chain Collision Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware implementations of data compression algorithms like GZIP, based on DEFLATE and Huffman coding, face complexity in achieving high bandwidth due to the need for a search history buffer and compare length functions, which can be inefficient in generating hash values for variable-length strings.
Innovation Solution
The development of an adaptive rate compression hash processor that uses a hash function to map variable-length strings to fixed values, employing a hash table and hash chain to quickly index data sets, and a circuit design with multiple memory banks and FIFOs to manage hash collisions, allowing for high-throughput compression with minimal overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a search history buffer and compare length function are used to implement compression, then compression capability is achieved, but device complexity increases
Solution Approach 1:
The patent replaces the traditional mechanical search history buffer and compare length function with a hash-based system. Instead of physically searching through history buffers, the invention uses hash functions to compute indices and directly access relevant data, substituting mechanical search operations with mathematical hashing operations that are computationally more efficient and hardware-friendly.
Solution Approach 2:
The invention changes the parameter representation by using fixed-size hash values (e.g., 16-bit indices) instead of variable-length string comparisons. This parameter transformation allows the system to work with fixed-width data structures in hardware, simplifying the computational logic and reducing circuit complexity while maintaining compression effectiveness.
2Speed
If variable-length strings are hashed to fixed values, then search efficiency improves, but hash collision management becomes complex
Solution Approach 1:
The patent segments the hash table into multiple banks (e.g., 8 banks) to distribute hash collisions across different memory structures. Each bank handles a portion of the hash space, and collision resolution is managed independently in each bank using FIFO structures. This segmentation reduces the complexity of collision management in any single bank while maintaining overall search efficiency.
Solution Approach 2:
The invention introduces FIFO (First-In-First-Out) structures as intermediary elements between the hash function and the actual data storage. These FIFOs act as buffers that manage hash collisions by temporarily holding collided entries and resolving them in a systematic manner, simplifying the overall collision management mechanism.
3Productivity
If multiple memory banks and FIFOs are used to manage hash collisions, then throughput increases, but manufacturing complexity increases
Solution Approach 1:
The patent designs the multiple memory banks and FIFO structures to perform multiple functions: they serve as both hash tables for indexing and as collision resolution mechanisms. The same hardware structures are used for both high-speed access and collision management, reducing the need for separate dedicated components and simplifying the manufacturing process.
Solution Approach 2:
The invention implements a nested structure where FIFOs are embedded within or closely integrated with the memory banks. This nesting allows the system to achieve high throughput through pipelined operations while maintaining a compact hardware footprint that is more manageable in terms of manufacturing and layout design.
Data Source
AI summary
An input file is processed according to hash algorithm that references sets of literals to preceding sets of literals to facilitate copy-offset command generation. Preceding instances are identified by generating a hash of the literal set and looking up a corresponding entry in a hash table. The hash table may be accessed by placing look-up requests in a FIFO buffer. When the FIFO buffer is full, generation of the hash chain is suspended until it is no longer full. When repeated literals are found, generation of the hash chain is likewise suspended. The hash chain is used to generate a command file, such as according to the LZ algorithm. Runs of consecutive literals are replaced by a run-length command. The command file may then be encoded using Huffman encoding.