Multi-View Rendering of Non-Diffuse Objects Under Bandwidth Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for compressing and transmitting multiple views in immersive video fail to accurately render non-diffuse pixels due to limited transmission rates and computational constraints, leading to incorrect reflectance in alternate views.
Innovation Solution
A method that adjusts the amount of data transmitted for non-diffuse pixels based on pixel and transmission rates, selecting and encoding these pixels to maintain overall view quality, using techniques such as entropy coding and reflectance models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If 2D images with depth maps are used to represent 3D scenes, then bandwidth consumption is reduced, but rendering accuracy for non-diffuse objects deteriorates
Solution Approach 1:
The patent segments the 3D scene representation into multiple components: 2D images, depth maps, and virtual light sources. This segmentation allows each component to be optimized independently - 2D images provide base visual information, depth maps add spatial data, and virtual light sources enable accurate lighting calculations for non-diffuse objects, thereby resolving the contradiction between bandwidth efficiency and rendering accuracy.
Solution Approach 2:
The patent introduces virtual light sources as intermediary elements that mediate between the compressed 2D image data and the final rendered output. These virtual light sources carry lighting information that enables accurate rendering of non-diffuse objects without requiring transmission of complete 3D geometric and material data, thus achieving both bandwidth reduction and rendering accuracy.
2Manufacturing precision
If complete 3D scene data is transmitted, then rendering accuracy is improved, but bandwidth consumption increases
Solution Approach 1:
The patent creates virtual copies of light sources in the virtual environment that replicate the lighting behavior of real-world sources. These virtual light source copies enable accurate lighting rendering without transmitting actual 3D scene geometry and material properties, achieving rendering accuracy while minimizing bandwidth consumption through selective data transmission.
Solution Approach 2:
The patent changes the representation parameters from complete 3D geometric and material data to a simplified set comprising 2D images, depth maps, and virtual light source parameters. This parameter transformation maintains essential rendering information while significantly reducing data transmission requirements, resolving the bandwidth-accuracy contradiction.
3Device complexity
If 2D images are used for rendering, then device complexity is reduced, but the ability to render non-diffuse objects deteriorates
Solution Approach 1:
The patent performs preliminary extraction and transmission of lighting-relevant information (depth maps and virtual light source data) before rendering. This preliminary action enables the rendering system to accurately handle non-diffuse objects by having the necessary lighting and spatial information pre-prepared, without requiring complex real-time calculations or complete 3D scene data during rendering.
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
A method implemented by a server (1SVR) for transmitting data used by a device (1CTL) for rendering multiple views (Vref, Vi) of a single scene, said multiple views comprising at least one basic view (Vref) and at least one additional view (Vi), said method comprising steps of: - generating (SS10) a stream (STR) by encoding said views (Vref, Vi); - selecting (1SS30) non-diffuse pixels (SNDP) among detected non-diffuse pixels (NDP); - generating (1SS40) data (1DND) representative of the selected non-diffuse pixels (SNDP; and - transmitting (SS50) said stream (STR) and said data (1DND) to said device, wherein a size of said data is a function of a pixel rate (PR) of said device and/or of a transmission rate (TR) between said server and said device.