Mixed Reality Object Virtualization via Extraction and Replacement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mixed reality technologies face challenges in seamlessly integrating virtual elements with real-world environments, often resulting in overly virtual or distracting presentations that fail to effectively ground users in reality.
Innovation Solution
The system virtualizes target objects within a mixed reality presentation by overlaying extraction and replacement virtual objects onto real-world environments, allowing virtual elements to interact and behave differently, while maintaining the appearance of real objects existing in the environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual elements are integrated into mixed reality presentations, then the virtuality and engagement of the presentation is improved, but the realism and grounding in the real world deteriorates
Solution Approach 1:
The patent extracts the target object from the real-world environment and replaces it with a virtual object. The system identifies a target object in the real environment, extracts it by overlaying an extraction object that fills in the background, and then presents a virtual object in its place. This allows virtual elements to be integrated without permanently altering the real environment, maintaining realism while enabling virtuality.
Solution Approach 2:
The patent applies virtualization locally to specific target objects rather than transforming the entire environment. The extraction object and virtual object are selectively applied only to identified target objects, leaving the rest of the real-world environment unchanged. This localized approach maintains overall realism while providing virtuality where needed.
2Adaptability or versatility
If extraction objects are overlaid to remove target objects, then the integration of virtual elements is improved, but the visual coherence and detection of the real environment deteriorates
Solution Approach 1:
The patent uses an extraction object as an intermediary between the real environment and the virtual object. The extraction object fills in the background area where the target object was located, creating a seamless transition. This intermediary element masks the boundary between real and virtual, maintaining visual coherence while enabling virtual element integration.
Solution Approach 2:
The extraction object is configured to match the visual characteristics of the surrounding real environment, including color, texture, and lighting. By adapting the visual properties of the extraction object to blend with the background, the system maintains visual coherence and makes the transition to virtual objects imperceptible.
Data Source
AI summary
An exemplary mixed reality presentation system accesses video data representative of a real-world environment proximate to a mixed reality player device that is configured to present, to a user of the mixed reality player device, a mixed reality presentation. In the mixed reality presentation, one or more virtual objects included within a virtual domain associated with the real-world environment are presented together with real objects included within the real-world environment. The mixed reality presentation system identifies, based on the accessed video data, a target object among the real objects included within the real-world environment. The mixed reality presentation system extracts the target object from the mixed reality presentation and replaces, within the mixed reality presentation, the target object with a virtual object included within the virtual domain. The virtual object implements a representation of the target object. Corresponding methods are also disclosed.