Byte Stream Compression Using Mirrored Prefix Code Histories
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing stream-based compression methods are inefficient in reducing bandwidth consumption and responsiveness due to the need to transmit prefix tables and lack of accurate matching in history buffers.
Innovation Solution
The system employs an encoder that uses an arbitrarily sized circular history buffer with an elaborate hashing scheme to compress byte streams by replacing unmatched data with context-specific prefix codes, allowing the decoder to mirror the encoder's prefix code computation and eliminate the need for transmitting prefix tables.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional stream-based compression methods are used, then data can be compressed, but the number of bits required for encoding increases due to the need to transmit prefix tables
Solution Approach 1:
The decoder mirrors the encoder's prefix code computation by maintaining identical history buffers and hashing schemes, allowing it to generate the same prefix codes locally without receiving them in the transmitted data stream. This eliminates the need for separate prefix table transmission while ensuring both encoder and decoder use identical coding schemes.
Solution Approach 2:
The patent extracts and eliminates the prefix table transmission component from the compression system. By having the decoder independently compute prefix codes through mirroring the encoder's history buffer and hashing algorithm, the system removes the overhead of transmitting prefix tables, reducing total bit requirements.
2Productivity
If simple hashing schemes are used for history buffer matching, then processing speed is maintained, but matching accuracy decreases
Solution Approach 1:
The patent segments the hashing process into multiple stages with increasing specificity. It begins with a fast initial hash lookup for quick rejection of non-matching entries, then applies more elaborate hashing schemes to promising candidates. This multi-level segmentation maintains high processing speed for obvious non-matches while achieving high accuracy for potential matches.
Solution Approach 2:
The patent applies different hashing qualities to different situations. For the majority of history buffer entries that are unlikely to match, a simple fast hashing scheme is used. For entries that show promise based on initial matching criteria, a more elaborate and accurate hashing scheme is applied. This local differentiation optimizes both speed and accuracy where needed.
3Quantity of substance
If fixed-size history buffers are used, then memory usage is controlled, but compression ratio decreases due to inability to capture long-range dependencies
Solution Approach 1:
The patent implements a dynamic history buffer that can effectively access an arbitrarily large portion of previous data. Through clever use of hashing functions that operate on the entire history buffer without requiring it to be kept in contiguous memory, the system achieves O(1) access time regardless of buffer size, allowing very large or even unbounded history buffers without proportionally increasing memory requirements.
Solution Approach 2:
The patent replaces the mechanical constraint of fixed buffer size with a hashing-based access mechanism. Instead of being limited by physical buffer boundaries, the system uses hash functions to directly access any previous byte position in the history, effectively substituting the mechanical buffer structure with a logical hashing structure that can handle arbitrarily large histories with constant memory access time.
Data Source
AI summary
Systems and methods for stream-based compression include an encoder of a first device that receives an input stream of bytes including a first byte preceded by one or more second bytes. The encoder may determine to identify a prefix code for the first byte. The encoder may select a prefix code table using the one or more second bytes. The encoder may identify, from the selected prefix code table, the prefix code of the first byte. The encoder may generate an output stream of bytes by replacing the first byte in the input stream with the prefix code of the first byte. The encoder may transmit the output stream from the encoder of the first device to a decoder of a second device. The output stream may have a fewer number of bits than the input stream.