3D Video Stream Compression Using Occlusion Map
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for compressing three-dimensional video signals, particularly those using the frame-packing arrangement and MVC standard, face issues with spatial resolution, high computational cost, and inefficient coding efficiency, especially when dealing with multiple viewpoints, and struggle with synthesizing high-quality intermediate viewpoints due to occlusions and depth estimate errors.
Innovation Solution
The method involves generating and reconstructing a three-dimensional video stream using an occlusion map, where the occlusion map is used to reorganize occluded pixels for more efficient compression, allowing for implicit representation and decoding without explicit transmission, and utilizing standard encoders for compression and decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If frame-packing arrangement is used to represent two views, then backward compatibility with 2D systems is maintained, but spatial resolution is sacrificed and intermediate viewpoints cannot be generated
Solution Approach 1:
The patent segments the video signal into multiple independent view components (first view, second view, third view, etc.) rather than combining them into a single frame-packed signal. Each view can be independently processed and transmitted, allowing receivers to selectively reconstruct any desired viewpoint without being constrained by fixed frame-packing arrangements.
Solution Approach 2:
The patent transitions from 2D frame-packed representation to 3D volumetric representation by introducing depth information and multiple viewpoints. This allows the system to generate intermediate viewpoints through view synthesis algorithms that interpolate between discrete views along the viewing angle dimension, effectively adding a temporal dimension to viewpoint generation.
2Adaptability or versatility
If MVC standard is used to represent multiple viewpoints, then more than two views can be represented, but coding efficiency is inadequate and computational cost is very high
Solution Approach 1:
The patent uses view synthesis to generate intermediate views by copying and interpolating information from existing reference views rather than encoding each view independently. The synthesis algorithm creates virtual views by warping and blending pixels from adjacent real views, significantly reducing the amount of data that needs to be transmitted while maintaining visual quality.
Solution Approach 2:
The patent performs preliminary view synthesis and occlusion detection during the encoding phase to pre-process the multi-view signal. By anticipating which areas will be occluded in intermediate views and preparing compensation data in advance, the system reduces the computational burden during real-time decoding and viewing.
3Adaptability or versatility
If view synthesis is performed to generate intermediate viewpoints, then new viewpoints can be synthesized, but occluded areas produce images of non-optimal quality
Solution Approach 1:
The patent introduces an occlusion map as an intermediary data structure that identifies and marks occluded pixel regions during view synthesis. This map acts as a guide for the decoding process, allowing the receiver to know which areas require special handling and to apply appropriate inpainting or interpolation techniques to restore image quality in previously occluded regions.
Solution Approach 2:
The patent implements a feedback mechanism where occlusion information detected during encoding is transmitted to the decoder, which then uses this feedback to adjust its reconstruction process. The decoder receives guidance about which areas were occluded and applies targeted repair algorithms, creating a closed-loop system that continuously improves image quality based on actual occlusion patterns.
4Manufacturing precision
If number of views and depth maps is increased to reduce occlusions, then quality of synthesized image is improved, but transmission band occupation increases
Solution Approach 1:
The patent applies partial action by transmitting only the necessary subset of views and occlusion information required for high-quality synthesis rather than transmitting all possible view combinations. The system selectively sends depth maps and occlusion data only for views that will actually be used as references during decoding, avoiding the transmission overhead of redundant data.
Data Source
Figure 1
Figure 1bis
Figure 1ter
AI summary
A method is described for generating a three-dimensional video stream by starting from a sequence of video images, said sequence comprising a first view (V0), at least one second view (Vi) of a scene, as well as a depth map (D0) of said first view (Vo), or a disparity map of said at least one second view (V1) with respect to the first view (V0), the method comprising, for one image, the following steps: obtaining at least one occlusion image (O1) comprising the occluded pixels of said second view (V1) by starting from said depth map (D0) or from said disparity map; generating a compacted occlusion image (OC1) by spatially repositioning said occluded pixels of said at least one occlusion image (O1), so as to move said pixels closer to one another; said three-dimensional video stream comprising, for one image, said first view (V0), said depth map (D0), or said disparity map, and said at least one compacted occlusion image (OC1).