Volumetric Video Encoding for Specular Reflection Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3DoF video formats fail to handle specular reflections and other complex light effects, leading to replication of reflected objects and inadequate rendering of 3D scenes with Lambertian surfaces, causing viewer frustration and dizziness due to incomplete head translations.
Innovation Solution
A method for encoding 3D scenes by generating depth, color, and reflectance atlases, incorporating Bidirectional Reflectance Distribution Function (BRDF) parameters and lists of reflected color patches, enabling ray-tracing-based rendering to handle complex light effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing 3DoF video formats are used to render 3D scenes, then encoding and decoding is simpler, but specular reflections and complex light effects are not handled correctly, causing replication of reflected objects and inadequate rendering
Solution Approach 1:
The patent segments the 3D scene into multiple patches, where each patch can be independently processed for reflectance properties. This allows complex light effects to be handled patch-by-patch, improving rendering accuracy without overwhelming the encoding system with monolithic complexity.
Solution Approach 2:
The patent introduces BRDF parameters and reflectance information as additional encoding parameters. By changing the parameter set to include physical light reflection properties, the system achieves accurate specular reflection rendering while maintaining manageable encoding complexity through efficient parameter representation.
2Ease of operation
If 3DoF video is used instead of volumetric video, then the system is simpler, but users experience dizziness and frustration due to inability to translate head position and experience parallax
Solution Approach 1:
The patent transitions from 2D video frames to 3D volumetric representation by encoding depth information and 3D scene geometry. This dimensional upgrade enables users to translate their viewpoint in three-dimensional space and experience genuine parallax effects, dramatically improving user freedom and comfort.
Solution Approach 2:
The patent performs preliminary encoding of complete 3D scene information including geometry, texture, and reflectance properties during the encoding phase. This preliminary action stores all necessary spatial information, allowing the decoder to efficiently render any viewpoint without real-time complex computations, thus enabling user freedom without proportional increase in system complexity.
3Reliability
If reflectance patches and BRDF parameters are encoded for each patch, then complex light effects are rendered accurately, but data stream size increases
Solution Approach 1:
The patent applies local quality by encoding reflectance and BRDF parameters selectively for each patch based on its specific optical properties. Patches with simple diffuse reflection use minimal parameters, while patches with complex specular properties use full BRDF parameter sets. This localized approach achieves accurate light effect rendering while minimizing overall data stream size.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables realistic rendering of 3D scenes with specular reflections, enhancing immersion and preventing dizziness by allowing head translations and parallax experiences.
Implementation Method 1
generating first information encoding parameters of a Bidirectional Reflectance Distribution Function model of a reflection of the light on the reflectance patch
Data Source
AI summary
Methods, devices and data streams are proposed to encode, transport and decode 3D volumetric videos. The embodiments encompass the signaling of non-Lambertian patches together with their light reflection properties, so as to enable a ray-tracing based rendering engine to synthesize visually realistic virtual views with respect to light effects. A retro-compatible solution enabling conventional rendering engines to render such described 3D scene but without the advanced light effects is also described.