Video Coding Image Filtering With Constrained Hybrid Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies, such as HEVC and VVC, face limitations in achieving superior coding efficiency despite advancements, necessitating improved methods to enhance compression while maintaining video quality.
Innovation Solution
A hybrid training framework for neural networks is employed, combining offline training on diverse datasets with online training tailored to specific video content, utilizing a hybrid training framework that refines neural network layers with constraints to improve video coding efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional video coding standards (HEVC, VVC) are used, then coding efficiency is improved compared to previous standards, but superior coding efficiency cannot be achieved despite additional coding tools
Solution Approach 1:
The patent replaces traditional mechanical video coding tools (prediction methods, transform, quantization) with a neural network-based system. The neural network learns optimal coding parameters and transformations directly from data, substituting the fixed mechanical processing pipelines of HEVC/VVC with an adaptive intelligent system that can achieve superior coding efficiency while maintaining video quality.
2Adaptability or versatility
If neural networks are trained offline on diverse datasets, then generalization capability is improved, but adaptation to specific video content is limited
Solution Approach 1:
The patent implements dynamic training where the neural network transitions from static offline training to dynamic online training. The network is initially trained offline on diverse datasets to learn general video coding patterns, then continuously refined online on specific video content to adapt to content-specific characteristics. This dynamic two-stage training approach enables the system to balance generalization capability with content-specific performance optimization.
3Adaptability or versatility
If online training is performed without constraints, then adaptation to specific content is improved, but computational complexity increases
Solution Approach 1:
The patent applies local quality by constraining online training to specific subsets of neural network parameters rather than training all parameters. Different regions of the network are updated with different levels of freedom - some parameters are kept fixed from offline training while others are allowed to adapt online. This localized parameter update strategy enables content adaptation while controlling computational complexity by focusing training resources only where needed.
Data Source
AI summary
A method and an apparatus for hybrid training of neural networks for video coding are provided. The method includes: obtaining, in an offline training stage, an offline trained network by training a neural network offline; and refining, in an online training stage, a plurality of neural network layers with constraint on a plurality of parameters of the plurality of neural network layers, where the plurality of neural network layers may include at least one neural network layer in the offline trained network or in a simple neural network connected to the offline trained network.