Tapestry Encoding for Multi-View Image Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D content encoding and decoding systems face challenges in efficiently processing and transmitting multiple views, leading to increased computational complexity and bit requirements, especially in applications like autostereoscopic displays and free-viewpoint TV, where occluded regions need to be revealed without escalating the number of views and bits to be sent.
Innovation Solution
The system employs a tapestry encoding method that generates a tapestry image and coordinates map from multiple input views, allowing for the derivation of desired views by encoding and decoding this information, which reduces the need to send multiple views separately and minimizes computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple views are provided in their entirety for imaging and video applications, then occluded regions can be revealed, but the number of bits and computational complexity increase
Solution Approach 1:
The patent extracts only the essential information needed to represent the scene - specifically depth maps and selected texture images - rather than transmitting complete multi-view images. This extraction approach reveals occluded regions through depth information while minimizing the data volume that needs to be processed and transmitted.
Solution Approach 2:
The patent applies different quality levels to different parts of the scene representation. Depth maps are processed at reduced resolution compared to full-view images, and texture images are selectively applied only where needed. This local quality approach maintains sufficient detail for revealing occluded regions while reducing overall computational complexity.
2Loss of information
If the number of views is increased to reveal occluded regions, then scene information completeness improves, but the number of bits to be transmitted increases
Solution Approach 1:
The patent extracts depth information from multi-view images to create depth maps, which are then used to synthesize views that reveal occluded regions. This extraction method provides complete scene information without needing to transmit multiple full-resolution views, dramatically reducing the number of bits required.
Solution Approach 2:
The patent creates synthetic views by copying and warping texture images based on depth map information. Instead of transmitting multiple captured views, the system generates additional views through image processing, reducing bit requirements while maintaining scene information completeness.
3Manufacturing precision
If multiple views are transmitted separately, then view quality is maintained, but transmission efficiency decreases
Solution Approach 1:
The patent merges multiple views into a unified representation consisting of depth maps and selected texture images. This combined representation maintains the quality needed to generate high-quality views while improving transmission efficiency by reducing the total data volume compared to transmitting separate full-resolution views.
Solution Approach 2:
The patent uses texture copying and warping techniques to generate multiple views from a single set of source images and depth maps. This approach maintains view quality by preserving original texture details while significantly improving transmission efficiency by transmitting only the source images once rather than multiple times.
Data Source
Figure 1~2
Figure 3~4
Figure 5A~6
AI summary
Representation and coding of multi-view images using tapestry encoding are described. A tapestry comprises information on a tapestry image and a world coordinates map associated with the tapestry image, each of which may contain information from one or more perspectives of a scene. Perspective images of a scene can be generated from the tapestry image and the world coordinates map.