Light Source Localization via Virtual Reference Viewpoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In mixed reality applications, dynamically changing lighting environments in 3D scenes pose a challenge for realistic virtual object lighting, particularly on mobile devices where real-time rendering of complex light sources is time-consuming and requires simplification.
Innovation Solution
The method involves determining virtual reference viewpoints in a 3D scene, obtaining maps of registered real and virtual shadows, and using these to estimate light source parameters, allowing for efficient lighting parameter determination by pre-computing shadow maps for known camera trajectories and updating them as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If complex indoor dynamic scene light sources are modeled accurately, then lighting realism is improved, but rendering time increases significantly
Solution Approach 1:
The patent pre-computes shadow maps from multiple virtual reference viewpoints before runtime. These pre-computed shadow maps are stored and reused during rendering, eliminating the need to calculate complex shadow geometry in real-time. This preliminary action allows the system to maintain high lighting realism with dynamic light sources while meeting real-time rendering constraints on mobile devices.
2Measurement precision
If shadow maps are frequently recalculated to match camera movement, then lighting accuracy is improved, but computational burden increases
Solution Approach 1:
The patent segments the camera's field of view into multiple discrete virtual reference viewpoints. Shadow maps are computed once for each viewpoint and stored. During rendering, the system selects the pre-computed shadow map corresponding to the current camera viewpoint, avoiding frequent recalculations. This segmentation approach maintains lighting accuracy by having viewpoint-specific shadow maps while dramatically reducing computational time by eliminating continuous shadow map regeneration.
3Adaptability or versatility
If complete shadow map creation is performed for all viewpoints, then lighting coverage is improved, but memory usage and processing load increase
Solution Approach 1:
The patent computes shadow maps for aĉé set of discrete virtual reference viewpoints rather than continuously for all possible camera positions. This partial action provides sufficient lighting coverage for typical camera movements while keeping memory usage and processing load manageable. The system balances comprehensive lighting coverage with resource constraints by selecting a representative subset of viewpoints.
Data Source
AI summary
A method and an apparatus for processing a 3D scene are presented. Techniques are disclosed for determining light source locations, including tracking a current viewpoint of a camera capturing object(s) in a 3D scene and determining a reference viewpoint relative to the current viewpoint of the camera. According to aspects, a light source location is determined by obtaining a registered map of real cast shadows of the object(s) from an input image captured by the camera, registered with respect to the reference viewpoint. Then, for candidates of light sources, obtaining respective maps of virtual shadows of the object(s) created with respect to the reference viewpoint, and determining the location of the light source based on the candidates of light sources with respective maps of virtual shadows that match the registered map of real cast shadows.