Local Entropy Encoding for Neural Data Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data compression systems are inefficient in representing data, often using a global code symbol probability distribution that does not accurately reflect the frequency of occurrence of code symbols, leading to suboptimal compression and increased storage and transmission requirements.
Innovation Solution
The system employs an encoder neural network to generate a representation of data as an ordered collection of code symbols, which are then partitioned into subsets and entropy encoded using local code symbol probability distributions learned from training data or determined directly from the data, allowing for custom distributions that better represent the data's frequency, resulting in more efficient compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a global code symbol probability distribution is used for entropy encoding, then the encoding process is simple and fast, but the compression efficiency is suboptimal because it does not accurately reflect the frequency of occurrence of code symbols in different regions
Solution Approach 1:
The patent divides the image data into multiple partitions or blocks, and for each partition, it determines a local code symbol probability distribution based on the actual frequency of occurrence of code symbols in that specific partition. This segmentation allows the encoding system to adapt to local variations in data characteristics, significantly improving compression efficiency compared to using a single global probability distribution.
Solution Approach 2:
The patent implements local quality by creating different code symbol probability distributions for different regions (partitions) of the input data. Each partition receives a probability distribution that is specifically tailored to its local characteristics, allowing the entropy encoder to optimize compression for each region according to its unique statistical properties.
2Productivity
If custom code symbol probability distributions are determined from training data, then compression efficiency improves, but the system complexity and training requirements increase
Solution Approach 1:
The patent employs a preliminary training phase where the system processes training data to learn and determine code symbol probability distributions for various partitions. This preliminary action allows the system to pre-compute and store probability distribution models that can be efficiently applied during actual compression operations, separating the complex learning process from the operational phase.
Solution Approach 2:
The patent creates a model or representation of the data's statistical properties through training, storing the learned code symbol probability distributions as reusable templates. During compression, these pre-learned distributions are copied and applied to corresponding partitions of the input data, avoiding the need to recompute distributions from scratch for each compression task.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for compressing and decompressing data. In one aspect, an encoder neural network processes data to generate an output including a representation of the data as an ordered collection of code symbols. The ordered collection of code symbols is entropy encoded using one or more code symbol probability distributions. A compressed representation of the data is determined based on the entropy encoded representation of the collection of code symbols and data indicating the code symbol probability distributions used to entropy encode the collection of code symbols. In another aspect, a compressed representation of the data is decoded to determine the collection of code symbols representing the data. A reconstruction of the data is determined by processing the collection of code symbols by a decoder neural network.