Bidirectional Context Modeling for Fractional-Bit Compression Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional entropy encoding compression algorithms face inefficiencies due to the need for fractional bit compression and the wastage of residual information from unmatched symbols, which are distributed throughout the compressed data stream.
Innovation Solution
A bidirectional context model is employed, using three-symbol contexts and escape tokens to create a tree structure for adaptive encoding and decoding, allowing for efficient encoding and decoding processes by buffering and strategically placing literal values within the compressed data stream.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional entropy encoding compression algorithms are used, then compression is achieved through statistical modeling, but fractional bit compression efficiency is lost due to whole-bit boundaries
Solution Approach 1:
The patent buffers unmatched symbols (literals) during the compression process and processes them in batches after the main compression is complete. This preliminary buffering allows the system to collect residual information from multiple symbols and compress them together, recovering the fractional bit efficiency that would be lost if each symbol were compressed individually to whole-bit boundaries.
2Reliability
If escape tokens are used for unmatched symbols, then literal values are preserved, but residual information from these symbols is wasted
Solution Approach 1:
The patent intentionally delays the compression of unmatched symbols (literals) until after the main compression process. During this delay, the residual information from these symbols is buffered and accumulated. After the probability model has been fully updated by the matched symbols, the buffered literals are compressed using the finalized model, thereby recovering the residual information that would otherwise be wasted.
3Adaptability or versatility
If adaptive probability models are updated continuously, then local correlations are captured, but compression opportunities are lost when symbols are scattered throughout the stream
Solution Approach 1:
The patent segments the compression process into two distinct phases: a first phase where matched symbols are compressed and the probability model is updated adaptively, and a second phase where buffered unmatched symbols are compressed using the finalized model. This segmentation allows the system to maintain adaptive modeling for local correlations while collecting all residual information from unmatched symbols for batch processing, thereby improving overall compression performance.
Data Source
AI summary
Techniques for improving encoding and decoding data are described herein. According to one embodiment, in response to a symbol retrieved from an input stream having a sequence of symbols, it is determined whether a current context can encode the retrieved symbol. A code representing an escape down token in the current context is emitted to a code stream if the current context cannot encode the retrieved symbol. A code representing the retrieved symbol in the current context is emitted to the code stream if the current context can encode the retrieved symbol. Optionally, the current context is escaped to a context having a higher order by emitting a code representing an escape up token in the code stream, where a higher order context is a child context to a lower order context as a parent context.