3D Lighting Reconstruction for Accurate XR Object Placement

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing techniques for lighting virtual content in XR environments inaccurately reflect real objects' positions, sizes, and orientations, are unreliable due to static scanning, and inconsistently model lighting, leading to suboptimal photorealism.

Innovation Solution

An imaging system using a visible-light camera, depth sensor, and pose-tracking means to reconstruct a three-dimensional lighting model of the real-world environment, capturing data from various poses to create accurate sample lighting data for virtual objects in real-time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If lighting information is produced considering all real objects to be at infinity, then the processing complexity is reduced, but the lighting accuracy deteriorates causing distorted reflections and incorrect positions

Engineering Contradiction:

Improveprocessing complexityVSAvoidlighting accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent segments the real-world environment into multiple depth regions (near, mid, far) and processes lighting information differently for each region. Objects in the near field are processed with full 3D spatial information while distant objects use simplified infinity assumptions, resolving the contradiction between processing complexity and lighting accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different lighting processing qualities to different spatial regions. Local lighting information is captured and applied specifically to virtual objects in corresponding depth regions, ensuring high accuracy for nearby objects while reducing processing load for distant objects.

Inventive Principle:

Principle #3Local quality

2Ease of operation

If lighting information is captured from a fixed location, then the equipment control is simplified, but the lighting reliability deteriorates due to location-dependent lighting variations

Engineering Contradiction:

Improveequipment controlVSAvoidlighting reliability

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The system uses the XR device's own motion tracking capabilities to automatically capture lighting information from multiple positions as the user moves. No manual intervention is needed - the system self-services by continuously updating lighting models based on the device's changing position and orientation.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent transitions from static fixed-location lighting capture to dynamic multi-position lighting capture. The lighting model is continuously updated as the XR device moves through the environment, adapting to different viewing angles and positions automatically.

Inventive Principle:

Principle #15Dynamics

3Extent of automation

If manual scanning is performed to capture lighting information, then the equipment automation is reduced, but the lighting precision improves through controlled data collection

Engineering Contradiction:

Improvemanual interventionVSAvoidlighting precision

Core Design Contradiction:

Extent of automationVSMeasurement precision

Solution Approach 1:

The patent replaces manual mechanical scanning operations with automated sensor-based lighting capture. The XR device's cameras and sensors automatically capture lighting information from multiple positions as the user naturally moves, eliminating the need for manual scanning while maintaining or improving precision.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Adaptability or versatility

If lighting information is modelled for distant virtual content, then the adaptability is improved, but the lighting consistency deteriorates due to inaccurate illumination from wrong light sources

Engineering Contradiction:

Improvevirtual content placement flexibilityVSAvoidlighting consistency

Core Design Contradiction:

Adaptability or versatilityVSStability of the object's composition

Solution Approach 1:

The patent adds a depth dimension to lighting information storage and retrieval. Lighting data is organized by depth regions, allowing the system to select appropriate lighting models based on the virtual object's distance from the user. This ensures consistent lighting by matching the lighting model to the correct spatial region.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentEP4134917B1Imaging systems and methods for facilitating local lighting

Publication Date: 2025.12.31 VARJO TECH OY

Figure 1~2
Figure 3
Figure 4

AI summary

Disclosed is an imaging system (100, 202) comprising visible-light camera(s) (102), depth sensor(s) (104), pose-tracking means (106), and server(s) (108) configured to: control visible-light camera(s) and depth sensor(s) to capture visible-light images and depth images of real-world environment (300), respectively, whilst processing pose-tracking data to determine poses of visible-light camera(s) and depth sensor(s); reconstruct three-dimensional lighting model of real-world environment representative of lighting in different regions of real-world environment; receive, from client application (206), request message comprising information indicative of location in real-world environment where virtual object(s) is to be placed; utilise three-dimensional lighting model to create sample lighting data for said location, wherein sample lighting data is representative of lighting at given location in real-world environment; and provide client application with sample lighting data.