Neural Network Video Coding for Image Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies, such as HEVC, face challenges in improving subjective quality and coding efficiency due to limitations in image restoration methods, particularly in handling residual, prediction, and reconstructed data.
Innovation Solution
The integration of Neural Networks (NNs), specifically Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), and Recurrent Neural Networks (RNNs), into the video coding system to process residual, prediction, and reconstructed data, enabling improved image restoration and coding efficiency by enhancing the processing of chroma and luma components and deriving sub-blocks for more efficient data handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional image restoration methods are used in HEVC, then the coding system maintains simplicity, but subjective quality and coding efficiency cannot be improved
Solution Approach 1:
The patent replaces traditional mechanical/image processing restoration methods with a neural network-based system. The neural network processes residual, prediction, and reconstructed data through learned patterns rather than fixed algorithms, enabling superior quality improvement while managing complexity through integrated design
Solution Approach 2:
The patent combines multiple data types (residual data, prediction data, reconstructed data) as composite inputs to the neural network, and integrates the neural network output with traditional video coding components to create a hybrid system that achieves both quality improvement and controlled complexity
2Productivity
If more data types are processed by the neural network, then coding efficiency improves, but computational complexity increases
Solution Approach 1:
The patent applies neural network processing selectively to specific data types and coding scenarios rather than universally to all data. The system determines which combinations of residual, prediction, and reconstructed data to process, applying neural network enhancement only where it provides meaningful coding efficiency gains while limiting unnecessary computational overhead
Data Source
AI summary
Method and apparatus of video encoding video coding for a video encoder or decoder using Neural Network (NN) are disclosed. According to one method, input data or a video bitstream are received for blocks in one or more pictures, which comprise one or more colour components. The residual data, prediction data, reconstructed data, filtered-reconstructed data or a combination thereof is derived for one or more blocks of said one or more pictures. A target signal corresponding to one or more of the about signal types is processed using a NN (Neural Network) and the input of the NN or an output of the NN comprises two or more colour components. According to another method, A target signal corresponding to one or more of the about signal types is processed using a NN and the input of the NN or an output of the NN comprises two or more colour components.