Frequency-Decomposed Latent Compression for Scalable ROI Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing end-to-end neural network-based image and video compression methods lack flexibility in adapting the quality of reconstructed images and videos, particularly in local regions of interest, and do not efficiently utilize spatial redundancy reduction techniques.
Innovation Solution
A method involving frequency decomposition of images into multiple latent representations, which are grouped and encoded separately, allowing for progressive decoding and quality scalability by transmitting individual latent representations in separate bitstreams, and enabling localized quality enhancement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If end-to-end neural network-based compression methods are used, then compression efficiency is improved, but flexibility in adapting quality and spatial redundancy reduction capability deteriorates
Solution Approach 1:
The patent segments the frequency domain into multiple sub-bands (low frequency and high frequency components) and processes them independently through separate neural network encoders. This segmentation enables selective compression of different frequency components, allowing quality adaptation in specific regions while maintaining overall compression efficiency. The frequency-subband-level independence provides the flexibility needed to adapt quality without sacrificing compression performance.
Solution Approach 2:
The patent implements local quality enhancement by allowing different quality levels for different spatial regions. Through frequency decomposition, the system can apply higher quality reconstruction to specific regions of interest while using lower quality for other areas. The independent encoding of frequency sub-bands enables this localized quality control, resolving the contradiction between compression efficiency and quality adaptability.
2Productivity
If end-to-end neural network-based compression methods are used, then compression efficiency is improved, but capability to reduce spatial redundancy deteriorates
Solution Approach 1:
The patent segments the image into multiple frequency sub-bands and processes each sub-band independently through separate neural network encoders. This segmentation enables the system to target and reduce spatial redundancy in specific frequency ranges, improving overall redundancy reduction capability while maintaining compression efficiency. The independent processing of frequency components allows for more effective spatial redundancy removal compared to monolithic end-to-end approaches.
3Adaptability or versatility
If frequency decomposition is applied, then quality scalability and progressive decoding are enabled, but computational complexity increases
Solution Approach 1:
The patent segments frequencies into sub-bands that can be processed independently, enabling quality scalability. The frequency decomposition is performed at a manageable level, and the independent encoding of sub-bands allows for progressive decoding without requiring excessive computational resources. This segmented approach to frequency processing enables quality scalability while keeping computational complexity within acceptable limits.
Data Source
AI summary
In one implementation, we propose an end-to-end image video compression method that decomposes the spatial frequencies of the input content into a partitioned latent representation. Decomposed frequencies in the latent space are analyzed and grouped into separate latent representation or separate tensors, each tensor being jointly optimized to be decoded independently one from another. Therefore, the decoder can independently decode the tensors in a scalable manner to progressively reconstruct the input. This method enables quality scalability by progressively transmitting individual latent representations of decomposed frequency data, separated in the produced latent space. Furthermore, the quality scalability of region of interest (ROI) is enabled by which the decoder takes only corresponding latent representations in the enhancement tensors as input together with latent representations already delivered to the decoder.