Perceptual Image Coding Using AC Energy Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image and video coding standards face challenges in accurately modeling human visual perception of distortion, as peak signal-to-noise ratio (PSNR) does not accurately represent human sensitivity to distortion, and structural similarity metrics are computationally complex to calculate, leading to inefficiencies in coding decisions.
Innovation Solution
The method employs perceptual quality measurements based on AC energy to adjust quantization step sizes and segment images/video based on AC energy, using modified distortion weighting factors and AC energy-based segmentation to improve coding efficiency and visual quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If structural similarity metrics are used to measure distortion, then measurement precision of human perception is improved, but device complexity increases due to computational burden
Solution Approach 1:
The patent changes the parameter used for distortion measurement from complex structural similarity metrics to a simplified perceptual metric based on block variance and gradient magnitude. This new parameter maintains correlation with human perception while being computationally efficient, directly resolving the contradiction between measurement precision and device complexity
Solution Approach 2:
The patent employs a lightweight, computationally inexpensive distortion metric that can be calculated quickly and discarded after use. This 'cheap' metric replaces expensive structural similarity calculations, achieving acceptable perceptual quality assessment without the computational burden of complex metrics
2Manufacturing precision
If quantization parameter is adjusted based on perceptual metrics, then manufacturing precision of visual quality is improved, but productivity decreases due to multiple passes required
Solution Approach 1:
The patent calculates perceptual characteristics (variance and gradient magnitude) during the initial encoding pass and uses these pre-computed values to guide quantization parameter adjustment in subsequent passes. This preliminary action avoids the need for exhaustive searches and multiple iterative passes, maintaining visual quality while improving encoding efficiency
Solution Approach 2:
The encoding system uses its own pre-computed perceptual characteristics to automatically adjust quantization parameters without requiring external quality assessment or multiple evaluation passes. The encoder serves its own quality optimization needs using internally available information, eliminating the productivity penalty of external quality measurement loops
3Loss of information
If AC energy-based segmentation is applied, then loss of information is reduced by preserving perceptually important details, but device complexity increases due to additional processing steps
Solution Approach 1:
The patent applies AC energy-based segmentation to identify regions with different perceptual importance and applies different quantization strategies to different regions. High AC energy regions (containing important details) receive finer quantization while low AC energy regions receive coarser quantization, reducing overall information loss with minimal additional processing complexity
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Methods and devices for perceptual image and video coding. A method for reconstructing a quantization parameter for a current block includes predicting the quantization parameter, wherein predicting the quantization parameter is based upon a previous quantization parameter used for a neighboring block in the image, an estimated AC energy of the neighboring block, and an estimated AC energy of the current block; decoding from the bitstream a quantization parameter residual; and deriving the quantization parameter from the predicted quantization parameter and the quantization parameter residual. The method may include reconstructing the pixels of the current block, including dequantizing decoded quantized transform domain coefficients based on the quantization parameter.