Backward View Synthesis Prediction for Multiview Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding systems face challenges in efficiently encoding and decoding multiview video content, particularly in rendering a large number of views required for advanced 3D displays, which leads to high bitrate demands and computational complexity.
Innovation Solution
The implementation of a backward view synthesis prediction mechanism that uses co-located depth/disparity information to synthesize views, allowing for block-based, low complexity in-loop view synthesis prediction, enabling efficient encoding and decoding of multiview video by deriving disparity information and using it to generate prediction blocks for texture blocks in a reference area.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large number of views are rendered for advanced 3D displays, then the quality and versatility of 3D video content is improved, but the bitrate requirements and computational complexity increase significantly
Solution Approach 1:
The video content is segmented into multiple views that are encoded separately but can be selectively combined at the decoder. Instead of transmitting all views at full quality, the patent segments the multiview content into a base layer (essential views) and enhancement layers (additional views), allowing flexible reconstruction of different numbers of views based on bitrate availability.
Solution Approach 2:
The patent employs scalable video coding where video parameters such as resolution, frame rate, and quality level can be adjusted by selecting different layers from the bitstream. This allows the same encoded content to be adapted to different bitrate requirements while maintaining the ability to render multiple views when sufficient bandwidth is available.
2Adaptability or versatility
If a large number of views are rendered for advanced 3D displays, then the quality and versatility of 3D video content is improved, but the computational complexity increases significantly
Solution Approach 1:
View synthesis prediction is performed in advance during the encoding process and the results are stored in reference buffers. When decoding, the decoder can directly use these pre-computed prediction blocks rather than performing complex view synthesis calculations in real-time, significantly reducing the computational burden at the display device.
Solution Approach 2:
The patent uses reference picture buffers to store pre-processed view synthesis data that can be copied and reused for multiple decoding operations. Instead of recalculating view synthesis for each view, the decoder copies reference blocks from the buffer, reducing computational complexity while maintaining the ability to generate multiple views.
3Ease of manufacture
If conventional video coding systems are used for multiview video, then implementation is straightforward, but coding efficiency is insufficient for high-quality 3D content
Solution Approach 1:
The patent extends conventional H.264/AVC video coding tools to handle multiview content by adding view synthesis prediction functionality that works alongside existing prediction mechanisms. The same coding framework processes both conventional single-view and multiview content, maintaining implementation simplicity while significantly improving coding efficiency through the additional view synthesis layers.
Data Source
Figure 1~3
Figure 4~7b
Figure 5
AI summary
There is disclosed a method, apparatus and computer program product in which a view synthesis prediction utilizes a depth map, disparity information or other ranging information of a co-located reference frame in another view. A first uncompressed texture block of a first texture picture representing a first view and ranging information associated with the first uncompressed texture block are obtained. Disparity information with reference to a second view is derived on the basis of the ranging information. Reference samples of the second view are obtained on the basis of the disparity information. A view synthesis prediction is performed by using the reference samples to obtain a reference block for encoding the first uncompressed texture block.