Neural Codec Simulated Predictors for Image Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for improving the performance of standard codecs using artificial intelligence (AI) face challenges in learning from the neural network outputs, as the deterioration of images during compression by standard codecs is not discernible.
Innovation Solution
A neural codec is introduced, comprising a first simulated predictor for predicting blocks within a current frame based on neighbor blocks, a second simulated predictor for predicting blocks using reference blocks from adjacent frames, and a selection network for choosing the best predicted block based on prediction mode.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a standard codec is used for image compression, then compression performance is achieved, but the deterioration of image quality is not discernible and learning of neural network is not possible
Solution Approach 1:
The patent introduces a simulated predictor as an intermediary component that generates predicted blocks mimicking the output of a standard codec. This simulated predictor acts as a mediator between the neural network and the standard codec, providing discernible prediction errors that enable learning while maintaining compatibility with standard codec operations. The simulated predictor translates the non-discriminable output of standard codecs into a form that reveals quality deterioration patterns for neural network training.
2Manufacturing precision
If a neural network is added to improve standard codec performance, then subjective image quality may be improved, but the network parameters of the standard codec cannot be altered
Solution Approach 1:
The patent segments the codec system into two independent parts: a neural network component for quality enhancement and a standard codec component for compression. The simulated predictor is designed as a separate module that does not modify the standard codec's internal parameters but works in parallel to provide learnable predictions. This segmentation allows the neural network to be trained and optimized independently while the standard codec maintains its original parameter set unchanged.
Solution Approach 2:
The simulated predictor serves as an intermediary that bridges the neural network and standard codec without requiring parameter changes in the standard codec. It generates predicted blocks that enable the neural network to learn compression artifacts and quality degradation patterns, while the standard codec continues to operate with its original parameters unchanged.
3Manufacturing precision
If post-processing is applied to improve standard codec output, then performance is improved, but the result is used as input and learning is not possible
Solution Approach 1:
The simulated predictor performs preliminary action by generating predicted blocks before the actual compression process. These predicted blocks reveal the expected compression artifacts and quality deterioration in advance, enabling the neural network to learn from these patterns. This preliminary prediction step creates a learnable signal that would otherwise be hidden in the standard codec's output, facilitating neural network training without requiring complex post-processing analysis.
Data Source
AI summary
A neural codec includes a first simulated predictor for predicting a first block corresponding to a target block within a current frame, wherein the first block is predicted in accordance with the target block of the to-be-predicted current frame and pixels of neighbor blocks adjacent to the target block being input to the first simulated predictor, a second simulated predictor for predicting a second block corresponding to a target block using a reference block of a frame determined based on a prediction mode, wherein the second block is predicted in accordance with the reference block of a reference frame adjacent to the current frame and the target block being input to the second simulated predictor, and a selection network configured to select, based on the prediction mode, one of the first block and the second block as a predicted block.