Neural Network Latent Representation Segmentation for Video Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current lossy image and video compression techniques face challenges in achieving high compression ratios while maintaining acceptable visual quality, particularly due to propagation of errors in predictive frames and inefficiencies in storing temporally correlated data.
Innovation Solution
A method involving a trained neural network for encoding and decoding images or videos, where the latent representation is divided into sub-latent representations, entropy encoded, and transmitted as a bitstream, allowing for efficient compression and decompression by leveraging spatial and temporal correlations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If lossless compression is used to preserve all original information, then image or video quality is maintained, but data quantity reduction is limited
Solution Approach 1:
The patent introduces a probabilistic model as an intermediary between the original image and the compressed representation. Instead of directly storing pixel values or simple differences, the system models the probability distribution of pixel values and stores parameters of this distribution. This intermediary probabilistic representation allows for significant data reduction while preserving essential visual information through the statistical properties of the data.
2Quantity of substance
If lossy compression is used to reduce data quantity, then transmission efficiency improves, but visual quality deteriorates due to information loss
Solution Approach 1:
The patent changes the parameters being stored from raw pixel values to parameters describing probability distributions (such as mean and variance). This parameter transformation allows the system to capture the essential characteristics of image regions without storing every pixel value. By modeling the statistical properties rather than individual values, the system achieves efficient compression while maintaining visual quality through the learned probability distributions.
3Quantity of substance
If predictive frames are used to exploit temporal redundancy, then compression ratio increases, but error propagation occurs affecting subsequent frames
Solution Approach 1:
The patent segments the image or video data into multiple independent probability distribution models, one for each spatial location or region. Each segment is modeled independently with its own probability parameters, which are then compressed separately. This segmentation approach allows error isolation, where corruption or loss in one segment's data does not propagate to other segments, thereby improving reliability while maintaining compression efficiency.
Data Source
AI summary
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation, wherein the latent representation has a probability distribution described by at least one probability parameter; dividing the latent representation into a plurality of sub-latent representations, wherein each sub-latent representation has a sub-probability distribution described by at least one sub-probability parameter; entropy encoding the plurality of sub-latent representations using the plurality of at least one sub-probability parameters to produce a bitstream; transmitting the bitstream to a second computer system; entropy decoding the bitstream using the plurality of at least one sub-probability parameters to retrieve the plurality of sub-latent representations and combining the plurality of sub-latent representations to retrieve the latent representation; and decoding the latent representation using a second trained neural network to produce an output image, wherein the output image is an approximation of the input image.