AR Virtual Content Mixing with Lighting and Camera Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality technology struggles to create natural lighting effects for 3D virtual content, especially in indoor environments with limited lighting, and faces challenges in real-time adjustments due to camera parameters, leading to unnatural mixing with real objects.
Innovation Solution
A virtual content-mixing apparatus and method that utilize lighting and camera physical-modeling data to simulate actual lighting conditions, incorporating parameters like lens distortion, color balance, and white balance, and creating a lighting environment map using mirror ball images or RGB and depth sensors to render virtual content in a more natural and stable manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If 3D virtual content is provided in augmented reality, then virtual objects can be overlaid on real environment, but the lighting application to 3D virtual content becomes unnatural and easily recognizable as not real
Solution Approach 1:
The patent applies parameter changes by adjusting lighting parameters (intensity, direction, color temperature) of virtual content based on analysis of real environment lighting captured by the camera. The system dynamically modifies lighting parameters to match the captured environment, making virtual objects blend naturally with real surroundings.
Solution Approach 2:
The system uses feedback by continuously analyzing the captured real environment image to extract lighting information, then applying this information to adjust the lighting of virtual content in real-time. This closed-loop approach ensures virtual objects consistently match the actual lighting conditions of the environment.
2Ease of operation
If augmented reality is used in mobile environment, then portability is improved, but camera-related parameter variations (exposure, ISO, noise) cause color differences between actual objects and 3D virtual content
Solution Approach 1:
The system applies self-service by automatically capturing camera parameters (exposure, ISO, white balance) along with the environment image, then using these parameters to adjust virtual content rendering without requiring manual calibration. The mobile device itself provides the necessary data for color correction.
Solution Approach 2:
The patent changes rendering parameters by adjusting virtual content color properties based on captured camera parameters. When camera exposure or white balance changes, the system dynamically modifies virtual object parameters to maintain color consistency between real and virtual elements.
3Adaptability or versatility
If real-time parameter adjustment is implemented for camera changes, then adaptability to different camera settings is improved, but computational complexity increases
Solution Approach 1:
The system applies preliminary action by pre-processing the captured environment image to extract lighting characteristics and camera parameters before virtual content rendering. This preparation step stores lighting direction, intensity, and color information in a format ready for quick application to virtual objects during rendering.
4Illumination intensity
If lighting effects are applied to virtual content, then realism is improved, but processing requirements increase making it difficult to operate on low-specification devices
Solution Approach 1:
The patent applies local quality by selectively applying lighting effects only to relevant portions of virtual content based on their spatial relationship with light sources in the captured environment. Instead of uniformly processing all virtual objects, the system applies lighting calculations only where necessary, reducing overall computational load.
Data Source
AI summary
Disclosed herein are a virtual content-mixing method for augmented reality and an apparatus for the same. The virtual content-mixing method includes generating lighting physical-modeling data based on actual lighting information for outputting virtual content, generating camera physical-modeling data by acquiring a plurality of parameters corresponding to a camera, and mixing the virtual content with an image that is input through an RGB camera, based on the lighting physical-modeling data and the camera physical-modeling data.