CNN Synthesized View Quality Enhancement for 3D Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D video coding techniques fail to effectively address distortions in synthesized views, leading to poor image quality due to ignored distortions in depth generation and view synthesis, resulting in artifacts and reduced coding efficiency.
Innovation Solution
A Convolutional Neural Network (CNN) based system for synthesized view quality enhancement is introduced, which formulates distortion elimination as an image restoration task, incorporating learned CNN models into 3D video codecs to improve view synthesis performance and reduce artifacts by considering geometric and compression distortions, and implements post-processing at the decoder side to remediate mixed distortions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing 3D video coding techniques are used, then data volume is reduced through depth-based synthesis, but image quality deteriorates due to unaddressed distortions in synthesized views
Solution Approach 1:
The patent replaces traditional mechanical image processing methods with deep learning-based neural networks to model and remove distortions. The neural network learns complex distortion patterns from training data and applies them to restore synthesized views, achieving superior image quality compared to conventional iterative optimization approaches.
Solution Approach 2:
The patent changes the optimization parameters by using learned neural network parameters instead of traditional hand-crafted filters. The neural network parameters are trained to specifically model distortion characteristics, allowing adaptive removal of geometric and compression distortions while preserving image quality.
2Manufacturing precision
If distortion removal is performed using traditional methods, then image quality improves slightly, but computational complexity increases significantly
Solution Approach 1:
The patent substitutes computationally intensive traditional iterative optimization methods with a pre-trained neural network that performs distortion removal in a single forward pass. This substitution dramatically reduces computational complexity while maintaining or improving image quality through learned distortion models.
Solution Approach 2:
The neural network model is pre-trained offline on extensive training data containing various distortion types. This preliminary training allows the model to capture complex distortion patterns without requiring computationally expensive real-time optimization, enabling fast distortion removal during actual video coding operations.
3Productivity
If CNN-based distortion removal is applied, then image quality and coding efficiency improve, but system complexity increases
Solution Approach 1:
The neural network model serves multiple functions: it models geometric distortions, removes compression artifacts, and adapts to different video content and distortion types through its universal training data. This multi-functionality consolidates what would otherwise require multiple separate processing stages into a single integrated operation, improving coding efficiency.
Solution Approach 2:
The patent uses a learned representation of distortion patterns from training data to create a digital copy of the distortion model. This copied knowledge is stored in the neural network weights, allowing the system to remove distortions by applying the learned pattern rather than performing complex real-time analysis, thus reducing operational complexity.
Data Source
AI summary
Systems and methods which provide Convolutional Neural Network (CNN) based synthesized view quality enhancement for video coding are described. Embodiments may comprise an encoder configured for CNN based synthesized view quality enhancement configured to provide improved coding efficiency while maintaining synthesized view quality. Additionally or alternatively, embodiments may comprise a virtual viewpoint generator configured for CNN based synthesized view quality enhancement configured provide post-processing of the synthesized view at the decoder side to reduce the artifacts. CNN based synthesized view quality enhancement may, for example, be provided for 3D video coding to improve its coding efficiency, which can be utilized in 3D scenarios, such as 3DTV.