3D Model View-Dependent Surface Irradiance Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image reconstruction techniques fail to accurately capture and reproduce viewpoint-dependent lighting effects such as specular highlights and reflections, resulting in low-quality, unrealistic images with latency, which hinders immersive experiences in extended-reality environments.
Innovation Solution
A method involving the creation of a 3D model from visible-light images captured from multiple viewpoints, dividing 3D points into groups based on surface normals, materials, and textures, and storing information about different surface irradiances to generate high-fidelity images from novel viewpoints in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing image reconstruction techniques are used, then image generation can be performed, but the images fail to accurately capture and reproduce viewpoint-dependent lighting effects such as specular highlights and reflections, resulting in low quality and unrealistic images
Solution Approach 1:
The patent segments the image reconstruction process into multiple components: capturing images from multiple viewpoints, creating a 3D model with color information, dividing 3D points into groups based on surface properties, and storing view-dependent surface irradiance information separately. This segmentation allows each component to be optimized independently, improving both image quality and accuracy of lighting effects.
Solution Approach 2:
The patent transitions from 2D image processing to 3D modeling by creating a three-dimensional model from multi-viewpoint images. This dimensional change enables accurate representation of surface geometry and view-dependent lighting effects, as the 3D model preserves spatial relationships and surface normals that are essential for reproducing specular highlights and reflections from novel viewpoints.
2Manufacturing precision
If high-fidelity image reconstruction is pursued, then image quality improves, but the process requires more computational resources and time, leading to considerable latency
Solution Approach 1:
The patent performs preliminary actions by pre-capturing images from multiple viewpoints and pre-processing them to create a 3D model with stored color and surface irradiance information. This pre-processing work is done offline or in advance, so that when real-time image reconstruction is needed, the system can quickly retrieve and render using the pre-computed 3D model, significantly reducing latency while maintaining high fidelity.
Solution Approach 2:
The patent applies local quality by dividing 3D points into groups based on their surface properties (surface normals, materials, textures) and storing view-dependent irradiance information specifically for regions where it is most needed. This selective approach focuses computational resources on capturing detailed lighting effects in critical areas while reducing processing requirements in less demanding regions, balancing quality and speed.
3Device complexity
If existing image reconstruction models are used, then the process can be simplified, but the models lack the required resolution and capability to represent realistic visual details of the real-world environment
Solution Approach 1:
The patent creates a universal 3D model that serves multiple functions: it stores geometric information, color information, surface normal data, and view-dependent irradiance information. This multi-functional model can be used for various applications including image reconstruction from novel viewpoints, virtual reality, augmented reality, and 3D visualization, eliminating the need for separate specialized models for each application while maintaining high resolution and detail representation.
Data Source
AI summary
A computer-implemented method including: receiving visible-light images captured from viewpoints using visible-light camera(s); creating 3D model of real-world environment, wherein 3D model stores colour information pertaining to 3D points on surfaces of real objects (204); dividing 3D points into groups of 3D points, based on at least one of: whether surface normal of 3D points in group lie within predefined threshold angle from each other, differences in materials of real objects, differences in textures of surfaces of real objects; for group of 3D points, determining at least two of visible-light images in which group of 3D points is captured from different viewpoints, wherein said images are representative of different surface irradiances of group of 3D points; and storing, in 3D model, information indicative of different surface irradiances.