Immersive Video Encoding Atlas Foreground Background Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for encoding and decoding omnidirectional images in virtual reality systems, particularly those supporting 6 Degrees of Freedom (DoF), face inefficiencies due to the lack of separate processing for foreground and background elements, leading to redundant data and reduced rendering flexibility.
Innovation Solution
The method involves generating an atlas from viewpoint images, encoding metadata to distinguish between background and foreground, and merging background images spatially or temporally to remove redundant data, with metadata indicating the merge type and processing status.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If background images are processed separately from foreground images, then encoding efficiency is improved and redundant data is removed, but device complexity increases due to separate processing requirements
Solution Approach 1:
The patent divides the image processing into separate foreground and background components. Background images are extracted and processed independently from foreground images, allowing for specialized optimization of each component. This segmentation enables the background to be merged across multiple viewpoints and time points while foreground objects maintain their individual characteristics, thereby improving encoding efficiency without requiring complete reprocessing of all image data.
Solution Approach 2:
The patent merges multiple background images from different viewpoints and time points into a single consolidated background representation. This merging process removes redundant data that would otherwise be repeated across multiple images, significantly reducing the overall data volume requiring encoding and transmission, thus improving encoding efficiency.
2Loss of substance
If multiple background images are merged spatially and temporally, then redundant data is removed, but loss of information may occur about the original viewpoint images
Solution Approach 1:
The patent extracts background information from viewpoint images separately from foreground information. By taking out the background component, it can be processed and merged independently without affecting the preservation of foreground details. This extraction allows redundant background data to be removed while maintaining all necessary foreground information from the original viewpoint images.
Solution Approach 2:
The patent applies different processing qualities to different parts of the image. Background regions undergo merging and compression with reduced detail, while foreground regions maintain their original quality and detail. This local quality approach ensures that information loss in merged backgrounds does not impact the overall image quality, as the critical foreground elements remain intact.
3Adaptability or versatility
If foreground and background are processed separately, then rendering flexibility is improved, but metadata complexity increases
Solution Approach 1:
The patent implements dynamic processing where foreground and background can be handled differently based on rendering requirements. The separate processing allows the system to adaptively choose to render only background, only foreground, or both together, providing flexibility for different application scenarios such as virtual reality, augmented reality, or video conferencing, while the metadata structure supports these dynamic rendering choices.
Data Source
AI summary
An image encoding method according to the present disclosure may include generating an atlas based on a plurality of viewpoint images; encoding the atlas; and encoding metadata for the atlas. In this case, the metadata may include first information showing whether a viewpoint image is a background image.