3D Scene Data Signals With View-Dependent Rendering Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing three-dimensional scene representations, such as immersive video, suffer from high data rates, processing complexity, and limited flexibility, leading to suboptimal image quality and user experience.
Innovation Solution
Incorporating a view dependency indication, such as a Lambertian indication, into the data signal to represent the degree of variation in visual properties as a function of viewing direction, allowing for improved rendering and image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If volumetric representations or light field representations are used to provide three-dimensional scene information, then view image synthesis quality is improved, but data rate and processing complexity increase
Solution Approach 1:
The patent segments the scene representation into multiple view images captured from different camera positions, rather than using a single volumetric or light field representation. Each view image is processed and transmitted separately, allowing the system to achieve high synthesis quality through multi-view consistency while reducing the processing complexity of individual components.
Solution Approach 2:
The patent applies different processing and transmission strategies to different regions of the scene based on their view dependency characteristics. Regions with high view dependency (such as specular surfaces) are handled differently from regions with low view dependency, optimizing the balance between synthesis quality and processing complexity for each local area.
2Adaptability or versatility
If multi-view with depth representation is used to represent the scene, then scene representation capability is improved, but data rate increases
Solution Approach 1:
The patent extracts and transmits only the essential view dependency information alongside the multi-view images, rather than transmitting complete volumetric or light field data. This extraction approach maintains the scene representation capability for various view positions while significantly reducing the overall data rate by omitting redundant information.
Solution Approach 2:
The patent uses a limited set of view images captured from specific camera positions, rather than capturing all possible views. By strategically selecting a partial set of views that covers the essential scene characteristics, the system achieves adequate scene representation capability while minimizing the data rate requirement.
3Manufacturing precision
If view dependency indication is added to the data signal, then rendering accuracy is improved, but data rate increases
Solution Approach 1:
The patent encodes view dependency information as additional parameters in the data signal, such as metadata indicating the degree of view dependency for different scene regions. These parameter changes enable the rendering system to accurately adapt to different viewing directions while the efficient encoding maintains acceptable data rates by using compact parameter representations.
Data Source
AI summary
An apparatus generates a data signal comprising three dimensional image data providing a representation of a three dimensional scene. The three dimensional image data includes at least one image providing visual data for the scene. The data signal further comprises a view dependency indication for an image region of the image where the view dependency indication is indicative of a degree of variation of one or more visual properties for scene points of the image region as a function of viewing direction. A rendering apparatus comprises a receiver (201) receiving the data signal and a renderer (203) generates a view image of the scene from a view pose from the three dimensional image data in dependence on the view dependency indication. Specifically, blending of contributions from different points of the scene to a given pixel of the view image may be dependent on the view dependency indication. The view dependency indication may be indicative of view dependency for scene points represented by the image region.