Variable-Size Entropy Coding for Better Compression of Mixed Symbols
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing entropy-based data compression methods are inadequate for handling symbols of variable sizes, as they assume all symbols have the same size, leading to suboptimal compression and decompression efficiency.
Innovation Solution
A method and system that sample data to establish relative frequencies of symbols of different sizes, generate variable-length codewords based on these frequencies and sizes, and compress data using these codewords, including metadata for decompression, allowing for efficient compression and decompression of symbols with varying sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing entropy-based data compression methods are used, then data compression is achieved, but compression efficiency is suboptimal when handling symbols of variable sizes
Solution Approach 1:
The patent applies dynamics by making the symbol size parameter variable rather than fixed. The compression system dynamically adapts to handle symbols of different sizes (e.g., 8-bit, 16-bit, 32-bit, 64-bit) by adjusting the encoding process based on the actual symbol size in the input data, thereby improving compression efficiency for variable-sized symbols while maintaining adaptability.
2Device complexity
If fixed-size symbol assumption is made, then simpler encoding process is achieved, but decompression accuracy deteriorates for variable-sized symbols
Solution Approach 1:
The patent applies preliminary action by embedding size metadata in the compressed data structure before decompression occurs. The compressed representation includes information about the original symbol size, which is used during decompression to accurately reconstruct variable-sized symbols. This preliminary inclusion of size information ensures decompression accuracy without significantly increasing encoding process complexity.
3Productivity
If variable-length codewords are generated based on symbol frequencies, then compression ratio is improved, but decompression complexity increases
Solution Approach 1:
The patent applies the intermediary principle by introducing a codebook or lookup table as an intermediate structure that maps variable-length codewords back to their corresponding symbols. During decompression, the stored size metadata acts as an intermediary that guides the reconstruction process, allowing the system to achieve high compression ratios through variable-length encoding while managing decompression complexity through structured lookup mechanisms.
Data Source
AI summary
Methods, devices and systems for data compression and decompression are disclosed. A collection of data is obtained. The collection of data is sampled to establish, for a plurality of different symbol sizes, relative frequencies of symbols of the respective sizes in the collection of data. A code is generated to contain variable-length codewords by entropy encoding sampled symbols in the collection of data based on a metric which reflects the relative frequencies of the sampled symbols as well as their sizes. Symbols in the collection of data are compressed into compressed representations using the generated code, wherein the compressed representation of a symbol comprises a codeword which represents the symbol as well as metadata for decompressing the compressed representation.