AR Virtual Object Surface Reflection Using Dual Camera Environment Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality technologies limit the sense of reality by only using a limited real image captured by the user's front-facing camera, failing to project the user and surrounding objects onto virtual objects, resulting in a disconnect between the virtual and real environments.
Innovation Solution
An image processing method and device that acquire and render environment textures from both the front and rear cameras, allowing the user's perspective and surrounding environment to be reflected on virtual objects, enhancing the realism of augmented reality images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If only a limited real image captured by the user's front-facing camera is used for environment mapping, then the device complexity is reduced, but the sense of reality for the augmented reality image deteriorates
Solution Approach 1:
The environment mapping function is segmented into multiple directional components. Instead of using a single front-facing camera, the system divides the environment capture into front direction (front-facing camera) and rear direction (rear-facing camera), with each camera capturing environment textures from its respective direction to create a more complete reflective environment.
Solution Approach 2:
The system transitions from two-dimensional front-facing environment capture to three-dimensional omnidirectional environment capture by incorporating rear-facing camera data. This adds the rear spatial dimension to the environment mapping, enabling virtual objects to reflect both front and rear surroundings realistically.
2Device complexity
If only a rear real image captured by the back-facing camera is used as environment texture, then the device complexity is reduced, but the sense of reality deteriorates because the user cannot be projected onto the virtual object surface
Solution Approach 1:
The system merges the environment textures from both the front-facing camera and rear-facing camera into a unified environment map. This combination allows the virtual object's reflective surface to display both the rear environment (captured by back-facing camera) and the front environment including the user (captured by front-facing camera), creating a realistic reflection that matches real-world expectations.
3Speed
If environment mapping uses only a limited real image in front of the user, then the processing speed is improved, but the sense of reality deteriorates
Solution Approach 1:
The system performs preliminary action by capturing environment textures from both front and rear cameras in advance before rendering the augmented reality scene. This pre-capture approach allows the system to prepare comprehensive environment maps that include all necessary reflective information, enabling faster real-time rendering without compromising the completeness of the reflective environment.
Data Source
AI summary
In accordance with an aspect of the present disclosure, there is provided a method for processing image performed by image processing device for rendering an augmented reality image. The method comprises, acquiring a first real image corresponding to a first direction from the image processing device, acquiring a second real image corresponding to a second direction, different from the first direction, from the image processing device, determining reflectivity of a surface of a virtual object in the augmented reality image; and rendering, according to the reflectivity, the surface of the virtual object by using an environment texture, the environment texture generated based on the first real image and the second real image.