Reconfigurable String Processor Architecture for LZ77 Block Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The DEFLATE compressed data format, which uses the LZ77 algorithm and Huffman coding, faces limitations in block size and distance limitations, particularly for uncompressible blocks and duplicated strings within a 32K byte window, affecting data compression efficiency.
Innovation Solution
The proposed solution involves a re-configurable hardware architecture that includes a history buffer, bit-extracting logic, and a re-configurable controller to efficiently decompress and compress data using LZ77 tables and Huffman coding, with components capable of re-tasking, combining, or splitting to optimize data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a fixed block size is used in DEFLATE compression, then the hardware design is simplified, but compression efficiency deteriorates for uncompressible blocks larger than 65,535 bytes
Solution Approach 1:
The patent implements dynamic block sizing by allowing the block size to be variable rather than fixed. The system can adjust block sizes adaptively based on the compressibility of the data, enabling efficient handling of both small compressible blocks and large uncompressible blocks by switching between different block size configurations as needed.
Solution Approach 2:
The patent segments the compression process into multiple independent stages including separate logic for literal matching, length-distance pair generation, and Huffman encoding. This modular segmentation allows each component to be optimized independently and enables parallel processing of different blocks, thereby improving overall compression efficiency without significantly increasing hardware complexity.
2Quantity of substance
If a 32K byte sliding window is used for LZ77 matching, then memory requirements are reduced, but the ability to reference duplicated strings deteriorates when they occur beyond this window
Solution Approach 1:
The patent extends the reference capability beyond the traditional 32K byte window by introducing a hierarchical memory structure with multiple levels. The system uses a combination of a small fast history buffer for recent data and a larger slower memory for historical data, effectively creating a multi-dimensional addressing space that allows reference to duplicated strings across much larger distances while maintaining reasonable memory requirements at each level.
Solution Approach 2:
The patent implements nested memory structures where a small history buffer is nested within a larger sliding window memory, which in turn is nested within an even larger historical data storage. This nested architecture allows the system to maintain multiple levels of context simultaneously, enabling reference to duplicated strings at various distance scales without requiring the entire address space to be accessible at once.
3Manufacturing precision
If separate Huffman code trees are used for each block, then compression accuracy is improved, but processing time increases due to repeated tree construction
Solution Approach 1:
The patent performs preliminary analysis of the data characteristics at the beginning of each block to predict the optimal Huffman code tree structure before actual compression. By pre-calculating frequency distributions and pre-building candidate code trees based on initial scanning, the system avoids repeated full tree constructions during the main compression pass, thereby maintaining high compression accuracy while reducing processing time.
Solution Approach 2:
The patent merges the Huffman encoding process with the LZ77 matching process by integrating the frequency analysis and code generation into the same hardware pipeline. This combining of operations allows the system to accumulate frequency statistics and generate code trees in parallel with the compression process, eliminating sequential overhead and improving overall processing efficiency while maintaining block-independent code trees for accuracy.
4Adaptability or versatility
If a scalable processor architecture is implemented for multiple string processing applications, then versatility is improved, but device complexity increases
Solution Approach 1:
The patent designs a universal processor architecture that can execute multiple string processing algorithms including LZ77 compression, LZS compression, and various checksum calculations through a single unified hardware platform. The system uses configurable processing elements and parameterizable memory structures that can be dynamically reconfigured to support different algorithms, providing multi-functionality without requiring separate dedicated hardware for each application.
Solution Approach 2:
The patent achieves versatility through parameter changes rather than structural changes. The same base architecture supports different string processing algorithms by modifying operational parameters such as window size, block size, matching length, and encoding schemes. This parameter-based configurability allows a single hardware design to adapt to multiple applications while maintaining relatively simple underlying structure, thereby improving versatility without proportionally increasing complexity.
Data Source
AI summary
Systems, methods, and apparatus for a scalable processor architecture for variety of string processing application are described. In one such apparatus, n input first in, first out (FIFO) buffer stores an input stream. A plurality of memory banks store data from the input stream. A re-configurable controller processes the input stream. And an output FIFO buffer stores the processed input stream.