Non-Binary Context Mixing Compression for Higher Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Context mixing compression algorithms, despite achieving high compression ratios, are slow due to their binary nature and complex neural computations, limiting their bandwidth in applications like NAND flash memory systems.
Innovation Solution
Implementing a non-binary context mixing compressor/decompressor that uses an ensemble of context models to generate predictions, which are then mixed using a maximum entropy classifier or neural network, allowing for faster and more efficient data processing by training the classifier with techniques like stochastic gradient descent.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If binary context mixing compression algorithms are used, then high compression ratios are achieved, but processing speed becomes slow
Solution Approach 1:
The patent changes the fundamental parameter of the compression algorithm from binary (bit-by-bit processing) to non-binary (symbol-based processing with alphabet size > 2). This allows processing multiple bits simultaneously while maintaining compression effectiveness, thereby improving processing speed without sacrificing compression ratio
Solution Approach 2:
The patent replaces the traditional neural network mixer with a maximum entropy classifier that uses closed-form mathematical solutions instead of iterative neural network computations. This substitution eliminates the need for complex forward-propagation and backpropagation operations, significantly reducing computational overhead while maintaining prediction accuracy
2Measurement precision
If neural network mixing is used in context mixing algorithms, then prediction accuracy improves, but computational complexity increases
Solution Approach 1:
The patent substitutes the neural network mixing mechanism with a maximum entropy classifier that relies on closed-form mathematical solutions. This replacement eliminates iterative training and inference computations, reducing computational complexity from O(n*d) neural network operations to O(d) closed-form calculations, where d is the feature dimension
Solution Approach 2:
The patent changes the computational approach from learning-based neural network parameters to information-theoretic maximum entropy parameters. By using closed-form solutions based on entropy maximization principles, the system achieves comparable prediction accuracy without the computational burden of neural network training and inference
3Manufacturing precision
If context mixing algorithms process one bit at a time, then compression accuracy is maintained, but throughput decreases
Solution Approach 1:
The patent changes the processing granularity from 1-bit to multi-bit symbols by increasing the alphabet size beyond 2. This allows the arithmetic encoder to process multiple bits simultaneously in a single encoding operation, directly increasing throughput while maintaining compression accuracy through the same context mixing prediction framework
Solution Approach 2:
The patent transitions from unary (1-bit) processing to higher-dimensional symbol processing by expanding the alphabet size. This dimensional expansion enables parallel processing of multiple bits within each symbol, effectively increasing throughput without compromising the precision of compression through the maintained context mixing approach
Data Source
AI summary
A technique for non-binary context mixing in a compressor includes generating, by a plurality of context models, model predictions regarding a value of a next symbol to be encoded. A mixer generates a set of final predictions from the model predictions. An arithmetic encoder generates compressed data based on received input symbols and the set of final predictions. The received input symbols belong to an alphabet having a size greater than two and the mixer generates a feature matrix from the model predictions and trains a classifier that generates the set of final predictions.