Nadir Shadow Light Source Estimation in 3D AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for estimating light sources in 3D scenes are complex, resource-consuming, and time-consuming, particularly when used in mixed or augmented reality applications, as they require prior knowledge of the camera pose and 3D scene geometry.
Innovation Solution
A method that captures a nadir view of a 3D scene using a camera positioned on an object acting as a support, detects shadows cast by the object, and determines the direction of real light sources using information about the object's height or 3D model, allowing for light source estimation without prior knowledge of the scene.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If prior methods for estimating light sources are used, then light source direction can be determined, but the process becomes complex and resource-consuming due to requiring camera pose and 3D scene geometry
Solution Approach 1:
The patent extracts and utilizes only the essential element needed for light source estimation - the shadow cast by the camera support object - while discarding the complex preprocessing requirements of camera pose estimation and 3D scene geometry modeling. This selective extraction maintains measurement precision while significantly reducing processing complexity.
Solution Approach 2:
The camera support object serves dual purposes: it supports the camera physically and simultaneously provides a shadow that encodes light source information. This self-service approach eliminates the need for separate calibration objects or complex scene modeling, reducing device complexity while maintaining the ability to determine light source direction accurately.
2Measurement precision
If prior methods for estimating light sources are used, then light source direction can be determined, but the process becomes time-consuming due to preprocessing requirements
Solution Approach 1:
The camera is positioned on a support object before capturing the image, which preliminarily establishes the geometric relationship between the camera, support object, and light source. This preliminary positioning enables direct light source estimation from the shadow without requiring time-consuming post-capture preprocessing steps for camera pose and scene geometry.
Solution Approach 2:
The method extracts only the shadow information from the captured image, ignoring other complex scene elements. This selective extraction reduces processing time by focusing computational resources solely on analyzing the shadow geometry rather than performing comprehensive scene understanding and camera pose estimation.
3Productivity
If simple shadow detection is used, then processing becomes faster and simpler, but prior knowledge of camera pose and 3D geometry is required
Solution Approach 1:
The camera support object provides its own geometric information through its shadow. By knowing the physical dimensions of the support object (which is typically a standard camera tripod or mount), the system can directly compute light source direction from the shadow without requiring external 3D scene geometry data or complex camera pose calibration, thus maintaining processing efficiency while eliminating information loss.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach simplifies light source estimation, reducing computational resources and time, while enabling realistic rendering of virtual objects in augmented and mixed reality applications by accurately determining light source directions and intensities.
Implementation Method 1
obtaining an image of at least a nadir view of the 3D scene, captured by at least one camera
Data Source
AI summary
A method for processing a 3D scene, and corresponding device, system and computer program are disclosed. In an example embodiment, the disclosed method includes: obtaining an image comprising at least a nadir view of a 3D scene, captured by at least one camera; detecting, in the image, at least one shadow cast by at least one object of the 3D scene acting as a support for the at least one camera; and determining a direction of at least one real light source from the at least one detected shadow and at least information representative of the object.