Virtual Representation of External User Interface Devices in VR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional artificial reality systems hinder user interaction with external devices, such as smartphones, due to blocked field of view and insufficient resolution of passthrough optics, preventing seamless interfacing within virtual environments.
Innovation Solution
The system tracks user interface devices using computer vision, headset tracking cameras, or other methods and generates a virtual representation within the virtual environment, allowing users to interact with these devices through virtual objects that mirror the real-world interface, enabling interaction without removing the headset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If passthrough optics are used to view external devices, then users can see real-world devices, but the resolution is insufficient and interaction is hindered
Solution Approach 1:
The patent creates a virtual copy of the external user interface device within the augmented reality environment. This virtual representation replicates the device's appearance and interface elements, allowing users to interact with the copied version rather than the physical device. The virtual copy can be manipulated, selected, and controlled through gesture recognition, solving both the resolution limitation and interaction difficulty of passthrough optics.
2Adaptability or versatility
If the field of view is blocked by the headset, then immersive virtual experience is achieved, but user interaction with external devices is prevented
Solution Approach 1:
The patent introduces a virtual representation as an intermediary between the user and the physical external device. This mediator allows the user to interact with the external device's functionality while remaining immersed in the virtual environment. The virtual object serves as a proxy that translates physical device interactions into virtual space, enabling seamless interaction without breaking immersion.
Solution Approach 2:
The patent transitions the interaction from physical space to virtual space by creating a three-dimensional virtual representation of the external device within the augmented reality environment. This dimensional transformation allows users to access and manipulate device functions through spatial gestures in the virtual space while maintaining the immersive headset experience.
3Ease of operation
If users remove the headset to interact with devices, then full interaction capability is restored, but immersion is broken
Solution Approach 1:
The patent creates a virtual copy of the external user interface device within the augmented reality environment. This virtual representation replicates the device's appearance and interface elements, allowing users to interact with the copied version rather than the physical device. The virtual copy can be manipulated, selected, and controlled through gesture recognition, solving both the resolution limitation and interaction difficulty of passthrough optics.
Data Source
AI summary
The disclosed computer-implemented method may include communicatively coupling a user interface device to a virtual reality device, capturing inputs from a user on the user interface device, displaying, on the virtual reality device, a virtual representation of the captured inputs from the user, and mirroring, on the virtual reality device, content presented on the user interface device. Various other methods, systems, and computer-readable media are also disclosed.