Multiple Focal Plane Coding From Texture and Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The video plus depth (V+D) format faces limitations in dynamic range and quality of depth signals, bit allocation, and flexibility for varying user viewpoints, hindering its use in advanced 3D applications.
Innovation Solution
A multi-format approach using depth blending to decompose depth maps into component depth maps (CDMs) and multiple focal planes (MFPs), allowing flexible bit allocation and quality adjustment based on depth ranges, with a subset of these components being transmitted and reconstructed to support varying viewpoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If video coding methods are used for coding depth data, then coding efficiency is improved, but coding quality of depth signals deteriorates
Solution Approach 1:
The patent segments the monolithic depth signal coding problem into separate coding paths for video and depth components. By dividing the coding process and applying specialized methods to each component type, the system achieves both coding efficiency for video and preserved quality for depth signals, resolving the contradiction between productivity and measurement precision.
Solution Approach 2:
The patent applies different coding strategies to different signal components based on their specific requirements. Video data receives coding optimized for visual quality and compression efficiency, while depth data receives coding that preserves measurement precision. This localized approach to quality allows simultaneous optimization for both productivity and measurement precision in their respective domains.
2Productivity
If bits are allocated between video and depth components in V+D format, then overall compression is improved, but flexibility for different content purposes deteriorates
Solution Approach 1:
The patent implements dynamic bit allocation that can adapt to different content types and application requirements. Rather than fixed allocation, the system can adjust the distribution of bits between video and depth components based on the specific content characteristics and intended use, maintaining both compression efficiency and flexibility for various content purposes.
Solution Approach 2:
The patent enables changing allocation parameters between video and depth components based on content requirements. By making the bit allocation a variable parameter that can be adjusted according to different content purposes (e.g., 3D reconstruction vs. stereoscopic viewing), the system achieves both good compression and adaptability to different applications.
3Measurement precision
If depth maps are coded with high accuracy, then 3D reconstruction quality is improved, but data transmission requirements deteriorate
Solution Approach 1:
The patent extracts and transmits only the essential depth information needed for 3D reconstruction rather than transmitting complete high-precision depth maps. By taking out only the critical depth components and parameters, the system achieves good 3D reconstruction quality while significantly reducing data transmission requirements.
Solution Approach 2:
The patent applies partial coding to depth maps, transmitting a subset of depth information that is sufficient for acceptable 3D reconstruction quality. Rather than coding the entire depth map at full resolution, the system uses partial action on the most important depth components, reducing data transmission requirements while maintaining adequate reconstruction quality.
Data Source
AI summary
System and methods are provided for using multi-format representation and coding of visual information. The system accesses an image data that comprises a texture data and a depth map; decomposes the depth map into a plurality of component depth maps (CDMs); and generates multiple focal planes (MFPs) comprising a plurality of focal planes. Each respective focal plane is based on the texture data and a respective CDM of the plurality of CDMs. The system selects a data subset including one or more of: (a) the texture data, (b) the depth map, (c) the plurality of CDMs, or (d) the plurality of focal planes; generates encoded data based on the selected data subset; and transmits, over a communication network, the encoded data to a client device to cause the client device to: generate for display or for further processing an image based on the encoded data.