Neural Video Compression With Alias-Free Latent Upsampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning-based video compression techniques introduce flickering artifacts due to the violation of equivariance and locality properties during low-bitrate compression, leading to degraded video quality.
Innovation Solution
The technique involves upsampling the latent space representation of video content before quantization to preserve equivariance and locality, using patch-wise factorization to minimize artifacting, and encoding the upscaled latent space representation into a compressed form.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine-learning-based compression is used to reduce bitrate, then compression efficiency is improved, but flickering artifacts and aliasing occur due to violation of equivariance and locality properties
Solution Approach 1:
The patent applies segmentation by dividing the video content into patches and processing them independently through patch-wise factorization. This allows the model to maintain locality properties by treating each patch separately, preventing the propagation of artifacts across the entire video frame while still achieving compression efficiency through the latent space representation.
Solution Approach 2:
The patent transforms the video content into a latent space representation, effectively moving the data to another dimension. This latent space transformation enables the model to capture essential features while discarding redundant information, achieving compression without the harmful artifacts that occur in traditional spatial domain compression.
2Quantity of substance
If compression bitrate is reduced to improve storage and transmission efficiency, then compression ratio is improved, but video quality deteriorates due to increased artifacts
Solution Approach 1:
The patent extracts only the essential information from the video content by encoding it into a compact latent space representation. This extraction process removes redundant information while preserving the core visual features, enabling high compression ratios without significant quality loss and eliminating the need for high bitrates.
Solution Approach 2:
The patent changes the representation parameters by transforming video data from the pixel domain to a latent space domain. This parameter transformation allows for efficient compression by representing visual information in a more compact and artifact-free manner, achieving both small data size and high quality simultaneously.
3Productivity
If traditional compression models are used to achieve high compression ratios, then compression efficiency is improved, but aliasing artifacts appear in the reconstructed video
Solution Approach 1:
The patent replaces traditional mechanical compression systems with a neural network-based latent space encoding system. This substitution eliminates the aliasing artifacts inherent in conventional approaches by using learned representations that naturally preserve equivariance and locality properties, achieving high compression ratios without artifact generation.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques and apparatus for encoding content using a neural network. An example method generally includes encoding video content into a latent space representation through an encoder implemented by a first machine learning model. A code is generated by upsampling the latent space representation of the video content. A prior is calculated based on a conditional probability of obtaining the upsampled latent space representation conditioned by the latent space representation of the video content. A compressed version of the video content is generated based on a probabilistic model implemented by a second machine learning model, the generated code, and the calculated prior, and the compressed version of the video content is output for transmission.