Anchor-Symbol Bitmap Compression for Low-Latency Data Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods face challenges in achieving high compression ratios while minimizing computational resources and latency, particularly in applications where frequent symbols dominate the data set.
Innovation Solution
A method that sets a symbol anchor value with the highest occurrence probability, generating a bit-map to differentiate anchor and non-anchor symbols, and compressing only the non-anchor symbols, along with the bit-map, to create compressed data, which can be efficiently decompressed by reversing the process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If traditional compression methods (Huffman, Lempel-Ziv) are used, then compression capability is provided, but computing resources and latency increase
Solution Approach 1:
The patent segments the data stream into anchor symbols (most frequent symbol) and non-anchor symbols. A bit-map is generated to mark positions of non-anchor symbols, and only these non-anchor symbols are stored in the compressed data. This segmentation allows the system to exploit the high frequency of anchor symbols to achieve compression without requiring complex computational resources for encoding and decoding.
Solution Approach 2:
The patent extracts and removes the anchor symbol (most frequent symbol) from the data stream, representing its positions through a compact bit-map instead of storing the actual symbol values. Only non-anchor symbols are retained in the compressed data. This extraction approach significantly reduces data size while minimizing computational overhead, as the decompression process simply needs to reconstruct anchor symbols at marked positions.
2Loss of substance
If compression ratio is increased, then data size is reduced, but computational overhead increases
Solution Approach 1:
The patent employs a self-service mechanism where the most frequent symbol (anchor symbol) automatically serves as the reference for compression. The system identifies the anchor symbol through simple frequency counting and uses it to generate the bit-map. During decompression, the anchor symbol itself serves to reconstruct the original data at marked positions. This self-service approach achieves high compression ratios with minimal computational overhead, as no complex external algorithms are required.
3Measurement precision
If lossless compression is implemented, then data accuracy is maintained, but processing time increases
Solution Approach 1:
The patent inverts the traditional compression approach by not compressing the frequent symbol but rather compressing the positions where the frequent symbol does NOT appear. The bit-map stores positions of non-anchor symbols, and during decompression, the system iterates through the bit-map and reconstructs anchor symbols at positions where bit-map bits are zero. This inverted approach maintains lossless compression while significantly reducing processing time, as the decompression process is straightforward and requires minimal computational steps.
Data Source
AI summary
An apparatus includes an input interface and compression circuitry. The input interface is configured to receive input source data. The compression circuitry in configured to set a symbol anchor value, having a highest occurrence probability among the symbol values in the input source data, to generate a bit-map by (i) for every symbol in the input source data whose symbol value is the anchor value, setting a respective bit in the bit-map to a first binary value, and (ii) for every symbol in the source data whose symbol value differs from the anchor value, setting the respective bit in the bit-map to a second binary value, and to generate compressed data including (i) the bit-map and (ii) the symbols whose symbol values differ from the symbol anchor value.