AR Relighting Using Hybrid Virtual and Physical Lighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR devices struggle with inefficient lighting estimation, leading to unrealistic rendering effects and reduced user immersion due to limited processing power and battery drain, especially in mobile devices, necessitating improved relighting methods for enhanced realism.
Innovation Solution
A method for AR content rendering that combines virtual and physical lighting information to achieve hybrid relighting effects, allowing for realistic integration of virtual objects into real-world scenes, with partial processing delegated to external devices to optimize resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If lighting estimation is performed to achieve realistic AR rendering, then rendering quality is improved, but processing power requirements increase and battery drain accelerates
Solution Approach 1:
The patent segments the lighting estimation process into two distinct phases: a first lighting estimation performed by an external processing device to generate initial lighting information, and a second lighting estimation performed by the AR device using the pre-processed lighting information. This segmentation divides the computationally intensive task between devices, reducing the processing burden and battery consumption of the AR device while maintaining rendering quality.
2Productivity
If lighting estimation is performed in real-time to maintain immersion, then user experience is improved, but processing complexity increases
Solution Approach 1:
The patent applies preliminary action by performing the first lighting estimation in advance before the AR rendering process. The external processing device generates lighting information that is then reused during real-time AR rendering, eliminating the need for repeated full lighting estimations and reducing processing complexity during actual use.
Solution Approach 2:
The patent introduces lighting information as an intermediary element that mediates between the complex lighting estimation process and the real-time rendering requirements. This intermediary data structure allows the system to achieve real-time performance by reusing pre-processed lighting information rather than performing complete lighting estimations during real-time rendering.
3Manufacturing precision
If comprehensive lighting information is processed to achieve accurate relighting, then realism is improved, but device resources are overwhelmed
Solution Approach 1:
The patent extracts the computationally intensive first lighting estimation process from the AR device and performs it on an external processing device. This extraction removes the heavy processing burden from the AR device while preserving the comprehensive lighting analysis, allowing accurate relighting without overwhelming device resources.
Data Source
AI summary
A method for performing augmented reality (AR) content rendering, includes: obtaining virtual relighting effects based on virtual lighting information representative of at least one virtual light source; obtaining hybrid relighting effects based on the virtual lighting information and based on physical lighting information representative of at least one physical light source; and generating an AR video picture by aggregating a real-world scene relighted based on the virtual relighting effects and at least one virtual object relighted based on the hybrid relighting effects; and rendering the AR video picture.