Volumetric Video Material Segmentation for Realistic Light Reflection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing volumetric video rendering technologies fail to accurately model the interaction of light with different materials, leading to unrealistic reflections and reduced perceived quality in virtual and augmented reality environments.
Innovation Solution
A system and method for texturing volumetric video by generating metadata that associates material groups with light properties, using neural networks and decision algorithms to assign material descriptors to texels, and encoding these descriptors for realistic light reflection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If material segmentation is performed to accurately model light-material interactions, then rendering realism is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent applies segmentation by dividing the volumetric video content into distinct material regions (e.g., skin, metal, fabric) and assigning different light interaction properties to each segment. This allows realistic rendering by treating each material type separately while managing computational complexity through region-based processing rather than pixel-by-pixel analysis.
Solution Approach 2:
The patent implements local quality by assigning specific material properties (reflectivity, absorption, scattering coefficients) to different spatial regions of the volumetric video based on their material composition. This enables localized accurate light modeling where needed (e.g., highly reflective metal surfaces) while using simplified models for other regions, balancing realism with computational efficiency.
2Measurement precision
If detailed material properties are assigned to each texel, then light reflection accuracy is improved, but data storage requirements increase
Solution Approach 1:
The patent merges multiple texels with identical or similar material properties into single material groups or clusters. Instead of storing detailed material properties for every individual texel, the system groups adjacent texels sharing the same material characteristics and stores a single set of properties for the group, significantly reducing data storage requirements while maintaining light reflection accuracy.
Solution Approach 2:
The patent creates a universal material property framework where a limited set of material descriptors (reflectivity, roughness, color) can be applied across multiple texels and material groups. This multi-functional approach allows the same property set to serve various material types, reducing the overall data storage burden while maintaining precision for each specific material instance.
Data Source
AI summary
A method for generating metadata to accompany a volumetric video for texels in an object in a volumetric video. The method comprises steps of: inputting a 2D representation of the object; identifying areas in the representation that have the same one or more properties with respect to light; and generating input material groups, where all texels in each input material group have the same properties with respect to light. There exists a correspondence between at least part of the input representation and at least part of the object in the volumetric video, so that there is a correspondence between texels in the representation and texels in the object so that output material groups can be generated from the input material groups and the properties with respect to light can be stored with the volumetric video as metadata.