Multi-View Video Atlas Coding for Lower Decoder Pixel Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing immersive video systems face challenges in providing high-quality immersion due to the inefficiencies in data transmission and processing of multi-view videos, particularly in devices with limited resources, where intermediate views are synthesized from decoded views but require optimized patch arrangements and transformations to reduce processing complexity.
Innovation Solution
A method for coding and decoding multi-view videos that involves determining and applying transformations such as oversampling, sub-sampling, and pixel value modifications to patches within atlases to optimize pixel occupancy and compression costs, with syntax elements indicating transformations to be applied during decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If patches are extracted from multiple views and arranged in atlases for intermediate view synthesis, then the quality of immersion is improved, but the number of pixels to be processed by the decoder increases
Solution Approach 1:
The patent segments the multi-view video data by extracting only specific patches from original views and arranging them in atlases, rather than processing complete views. This segmentation reduces the total number of pixels that need to be decoded while maintaining the quality of intermediate view synthesis through strategic patch selection and arrangement.
Solution Approach 2:
The patent extracts only the necessary patches from the original multi-view video data that are required for intermediate view synthesis. By taking out only the essential information (patches) rather than processing complete views, the decoder handles fewer pixels while still achieving high-quality immersive video reconstruction.
2Productivity
If transformations such as oversampling and pixel value modifications are applied to patches in atlases, then compression efficiency is improved, but the decoding complexity increases
Solution Approach 1:
The patent applies transformations (oversampling, sub-sampling, pixel value modifications) to patches during the encoding phase before they are arranged in atlases. By performing these transformations in advance during encoding, the decoder receives pre-processed data that requires simpler processing, thus improving compression efficiency while minimizing decoding complexity.
3Measurement precision
If all original views are transmitted to the decoder, then the visual quality is maintained, but the data transmission cost increases
Solution Approach 1:
The patent extracts only the necessary patches from the original multi-view video data that are required for intermediate view synthesis, rather than transmitting complete views. This extraction approach maintains visual quality by preserving essential scene information while significantly reducing the data transmission cost by eliminating redundant pixel data.
Solution Approach 2:
The patent uses patches from coded views to represent and reconstruct information from uncoded views through intermediate view synthesis. Instead of transmitting all original views, the system creates synthetic copies of the necessary view information using view synthesis algorithms, reducing transmission requirements while maintaining visual quality.
Data Source
AI summary
Methods for coding and decoding a coded data stream representative of a multi-view video. The coded data stream includes data representative of at least one atlas, which corresponds to an image having at least one patch. The at least one patch corresponds to a set of pixels extracted from at least one component of a view of the multi-view video, the view not being coded in the coded data stream. The method for decoding includes decoding, from the coded data stream, the at least one atlas, including decoding the at least one patch, determining, for the decoded patch, whether a transformation, and which transformation, must be applied to the decoded patch, the transformation belonging to the group including at least one oversampling of the patch or a modification of the pixel values of the patch, and applying the determined transformation to the decoded patch.