VR Gaze-Triggered Camera for Input Device Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies fail to effectively mirror 2D media data from source devices into 3D virtual environments, particularly in Virtual Reality (VR) settings, where user interaction with input devices like keyboards and mice is not seamlessly integrated within the VR environment.
Innovation Solution
A method and system that utilize a head-mounted VR display device with processors and a non-transitory computer-readable storage device to present virtual 3D surfaces, receive and display streaming media data, detect user gaze direction, and activate a camera to capture user interaction with input devices, projecting this interaction as a 2D video within the VR environment, while integrating multiple camera streams and applying image processing algorithms to track motion and direction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If 2D media data is mirrored from source devices to VR environment, then media data visibility is improved, but integration with 3D virtual environment is insufficient
Solution Approach 1:
The patent projects 2D media data onto 3D virtual surfaces within the VR environment, transforming flat 2D content into a three-dimensional spatial context. This allows media data to be displayed on virtual walls, floors, or other 3D surfaces, enabling seamless integration between 2D content and the 3D virtual environment while maintaining full visibility of the media data.
2Loss of information
If user interaction with input devices is captured and displayed in VR, then interaction visibility is improved, but system complexity increases
Solution Approach 1:
The patent introduces a camera as an intermediary device that captures user interactions with physical input devices (keyboard, mouse, etc.) and transmits this visual information to the VR environment. This intermediary approach allows the system to display interaction visibility without requiring direct integration between the input devices and the VR system, thereby managing complexity while achieving full interaction visibility.
3Measurement precision
If camera captures user interaction continuously, then interaction monitoring is improved, but energy consumption increases
Solution Approach 1:
The patent implements conditional camera activation based on user gaze direction. The camera captures user interactions only when the user is actively looking at the input device, as detected by gaze tracking sensors. This periodic activation rather than continuous operation maintains precise interaction monitoring capability while significantly reducing energy consumption by keeping the camera dormant during periods when interaction capture is not needed.
Data Source
AI summary
A method and a system for interfacing at least one computerized source device from within a Virtual Reality (VR) environment, produced by a head mounted VR display device, said method comprises the following steps:presenting a virtual 3D surface within the VR environment;receiving streaming media data from at least one source device as streaming video;displaying the said streaming video within the VR environment as a 2D video projection over the said at least one virtual 3D surface;identifying a condition wherein the user directs his/her gaze towards a designated direction;commanding a camera to produce a video stream, capturing the user's interaction with an input device of the at least one source device; anddisplaying said streaming video from said camera within the VR environment, showing the user's interaction with the said input device.