Mobile AR Ray Tracing Between Real Objects With Texture Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AR experiences on mobile devices lack realism due to limited resources, and existing ray tracing techniques consume too much processing power, making them inefficient for real-time video augmentation.
Innovation Solution
A mobile device-based AR system uses ray tracing to generate realistic reflections by applying a ray tracing process to a 3D model of objects in real-time video feeds, storing textures of objects across frames, and modifying visual properties based on optical effects, even when objects are not visible.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If ray tracing techniques are applied to mobile devices for real-time AR experiences, then the realism of AR experiences is improved, but the processing power consumption increases significantly
Solution Approach 1:
The system performs preliminary actions by capturing and storing texture information of objects in advance before they are needed for AR rendering. This allows the complex ray tracing calculations to be performed offline or with reduced computational load during real-time operation, resolving the contradiction between realism and processing power consumption.
Solution Approach 2:
The AR experience is segmented into multiple components: texture capture and storage handled separately from real-time rendering. The system divides the complex ray tracing process into manageable segments, processing different aspects of realism at different times and computational levels, thereby reducing instantaneous processing power requirements.
2Manufacturing precision
If ray tracing is applied to mobile devices, then the visual quality of AR experiences is improved, but the device resources become insufficient
Solution Approach 1:
Instead of performing computationally intensive ray tracing calculations in real-time on the mobile device, the system creates a simplified representation or copy of the texture information and uses this for rendering. This copying approach maintains visual quality while significantly reducing the computational resources required during AR operation.
Solution Approach 2:
Complex texture analysis and ray tracing preparation are performed as preliminary actions before actual AR rendering. By pre-processing and storing essential visual information, the system maintains high visual quality without requiring excessive device resources during real-time operation.
3Manufacturing precision
If real-time video augmentation is performed with ray tracing, then the realism of object interactions is improved, but the processing time increases
Solution Approach 1:
The system performs preliminary texture capture and storage of object information before real-time AR rendering is needed. This preliminary action eliminates the need for time-consuming ray tracing calculations during real-time operation, maintaining realism while reducing processing time to acceptable levels for real-time video augmentation.
Solution Approach 2:
The processing timeline is segmented into offline texture capture phase and online rendering phase. By separating these temporal phases, the system achieves realistic object interactions without the processing time delays that would occur if all calculations were performed in real-time.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the realism of AR experiences on mobile devices, reducing resource consumption and providing a seamless, efficient, and realistic visualization of virtual objects in real-world scenes.
Implementation Method 1
estimate an optical effect on a portion of the first object relative to a second object
Implementation Method 2
estimate an optical effect on a portion of the first object relative to a second object
Data Source
AI summary
Aspects of the present disclosure involve a system for performing ray tracing between augmented reality (AR) and real-world objects. The system accesses, by the mobile device, a video depicting a first object. The system obtains, by the mobile device, a three-dimensional (3D) model of the first object. The system applies, by the mobile device, a ray tracing process to the 3D model of the first object to estimate an optical effect on a portion of the first object relative to a second object that is depicted in the video. The system modifies a visual property of the portion of the first object based on the optical effect relative to the second object.