Variable-Length Copy Pointer Encoding for Higher Data Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression algorithms, such as SLDC, face limitations in achieving optimal compression efficiency, particularly in handling consecutive data units and displacement counts, which affects the overall compression ratio and storage efficiency.
Innovation Solution
The implementation of variable length code tables, specifically designed using Huffman coding techniques, to generate copy pointer and literal symbols by determining match counts and displacement counts, allowing for more efficient encoding and decoding of data streams, thereby improving compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fixed-length encoding is used for data symbols, then the encoding structure is simple, but the compression ratio is poor
Solution Approach 1:
The patent applies variable-length encoding where the number of bits used to represent data symbols varies based on the match count. Specifically, the match count field uses 1-4 bits depending on the value range, and the displacement field uses 10-14 bits depending on the displacement magnitude. This dynamic parameter adjustment achieves better compression ratios compared to fixed-length encoding while maintaining manageable complexity through structured field definitions.
2Loss of substance
If variable length code tables are implemented, then the compression efficiency is improved, but the encoding complexity increases
Solution Approach 1:
The patent segments the encoded data into distinct fields with specific functions: a match count field (1-4 bits) indicating the number of matching consecutive bytes, and a displacement field (10-14 bits) indicating the distance to the match in the history buffer. This segmentation allows the decoder to systematically process each field according to its bit-length specifications, managing the complexity of variable-length encoding through structured field definitions.
3Measurement precision
If more bits are used for match count and displacement fields, then the precision of data representation is improved, but the compressed data size increases
Solution Approach 1:
The patent dynamically adjusts the bit allocation for match count and displacement fields based on the actual data characteristics. The match count field uses 1 bit for values 0-1, 2 bits for values 2-3, 3 bits for values 4-7, and 4 bits for values 8-15. The displacement field uses 10 bits for displacements 0-1023, 11 bits for 1024-2047, 12 bits for 2048-4095, and 14 bits for 4096-16383. This adaptive parameter adjustment ensures sufficient precision for representing match positions and lengths while minimizing the overall bit requirements.
Data Source
AI summary
Provided are a computer program product, system, method, and data structure for using variable length code tables to compress an input data stream to a compressed output data stream. If a number of consecutive data units in the input data stream match the number of consecutive data units in a history buffer of previously received data units in the input data stream, a copy pointer symbol is generated indicating the copy pointer symbol referencing previously received data units in the history buffer and indicating a location of a start of the consecutive data units in the history buffer. A determination is made of an encoding of the number. Indication is made in the generated copy pointer symbol the determined encoding of the number and the copy pointer symbol is outputted in the compressed output data stream.