XR Lighting Model Rendering From Keyframe and Depth Images

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing image processing methods for extended reality (XR) scenarios fail to consider lighting intensity and color, leading to poor reproduction of virtual objects and a diminished immersive experience.

Innovation Solution

Capture environmental keyframe and depth images using an XR device, determine a lighting model based on these images, and render the XR object based on the lighting model.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If extended reality objects are displayed based only on camera viewpoint without considering lighting, then device complexity is reduced, but rendering quality and immersive experience deteriorate

Engineering Contradiction:

Improverendering system complexityVSAvoidrendering quality

Core Design Contradiction:

Device complexityVSManufacturing precision

Solution Approach 1:

The system performs preliminary action by capturing environmental keyframe images and depth images before rendering extended reality objects. A lighting model is determined in advance based on these images, analyzing lighting intensity, color, and direction in the real environment. This pre-computed lighting model is then applied during rendering to achieve photorealistic lighting effects without adding complexity to the real-time rendering pipeline.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The lighting model serves as an intermediary between the captured environmental images and the extended reality objects. The lighting model extracts lighting parameters (intensity, color, direction) from the keyframe images and applies them to illuminate the virtual objects, mediating the interaction between real environment capture and virtual object rendering to achieve seamless integration.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If lighting models are determined based on environmental images, then immersive experience is improved, but processing time and computational resources increase

Engineering Contradiction:

Improveimmersive experience qualityVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system extracts only the essential lighting parameters (lighting intensity, color, and direction) from the environmental keyframe images using a lighting model. Instead of processing entire high-resolution images, the lighting model extracts and stores only the critical lighting characteristics, significantly reducing processing time and computational resource requirements while maintaining high immersive experience quality.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The lighting model transforms complex image data into simplified lighting parameters (intensity, color, direction). This parameter transformation reduces the data complexity from full image processing to essential lighting attributes, enabling fast computation and real-time application during extended reality rendering without compromising the quality of lighting reproduction.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20260024276A1Image processing method and apparatus, device, and medium

Publication Date: 2026.01.22 BEIJING ZITIAO NETWORK TECH CO LTD

AI summary

Embodiments of the present disclosure relate to an image processing method and apparatus, a device, and a medium, where the method includes: obtaining an environmental keyframe image and a corresponding depth image captured by an extended reality device; determining a lighting model based on the environmental keyframe image and the depth image; and rendering an extended reality object to be rendered based on the lighting model.