AR Reflection Mapping for Realistic Virtual Object Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual objects in augmented and mixed reality environments lack accurate rendering of reflections on reflective surfaces, leading to artificial and non-photo-realistic effects, compromising user experience.
Innovation Solution
A system and method for rendering reflections on virtual objects in electronic devices by determining reflective surfaces, estimating their type, and applying a generated reflection map using semantic and depth cues, considering material properties and shapes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If virtual objects are overlaid onto the physical environment in AR/MR space, then the virtual objects can be integrated with the physical environment, but the reflections of virtual objects on reflective surfaces cannot be accurately rendered, making the virtual objects look artificial
Solution Approach 1:
The system performs preliminary actions by detecting reflective surfaces and generating reflection maps before rendering the virtual object. The reflective surface detection identifies surfaces with high reflectivity in advance, and the reflection map is pre-computed based on the virtual object's properties and the detected surface characteristics, enabling accurate reflection rendering when the virtual object is displayed.
Solution Approach 2:
The patent introduces an intermediary reflection map as a mediator between the virtual object and the reflective surface. This reflection map contains pre-computed reflection information that bridges the virtual object's rendering with the physical reflective surface's characteristics, allowing the virtual object to reflect accurately on real surfaces without requiring complex real-time rendering calculations.
2Reliability
If virtual objects are placed directly in front of reflective surfaces, then the positioning is simple, but the reflection is incorrect and the virtual objects do not appear photo-realistic
Solution Approach 1:
The system implements feedback by continuously detecting the reflective surface properties and adjusting the reflection map accordingly. When a virtual object is placed near a reflective surface, the system detects the surface's reflectivity characteristics and uses this feedback information to generate an accurate reflection map, ensuring the virtual object appears realistic in the reflection.
Solution Approach 2:
The patent applies parameter changes by modifying the reflection map based on the virtual object's position relative to the reflective surface. The system adjusts reflection parameters such as intensity, orientation, and distortion according to the object's placement, ensuring accurate and photo-realistic reflections regardless of where the virtual object is positioned in the AR/MR environment.
Data Source
AI summary
A method for rendering a reflection for a virtual object in an electronic device is provided. The method includes obtaining one or more red, green, and blue (RGB) frames and a depth-map associated with an augmented reality (AR) scene visualized in the electronic device for a user, determining one or more reflective surfaces in the AR scene based on the obtained RGB frames and the depth-map, identifying a type of the determined one or more reflective surfaces in the AR scene, generating a reflection map for the determined one or more reflective surfaces based on a placement of the virtual object, the identified type of the determined one or more reflective surfaces, and applying the generated reflection map on the determined one or more reflective surfaces to render the reflection for the virtual object in the electronic device.