Virtual Input Device Mapping for Spatially Aware VR Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual reality systems struggle to effectively integrate physical input devices like keyboards and mice into the virtual environment, leading to user disorientation and difficulty in providing inputs during immersive experiences.
Innovation Solution
A system and method for porting physical objects, such as keyboards and mice, into virtual reality by detecting their location and orientation in the physical space, rendering a corresponding virtual model, and providing spatial feedback on user interactions, allowing seamless integration and input capabilities during virtual reality mode.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a user wears a head mounted display (HMD) for virtual reality immersion, then the immersive experience is improved, but the user becomes disoriented and loses spatial awareness of physical input devices
Solution Approach 1:
The patent creates a virtual model that copies the physical input device's appearance, position, and orientation in the virtual reality environment. This virtual representation allows users to locate and interact with input devices while immersed in VR, resolving the disorientation problem by providing visual feedback of the physical device's location without breaking immersion.
Solution Approach 2:
The virtual model acts as an intermediary between the physical input device and the user's perception in VR. Instead of directly seeing the physical device (which would require removing the HMD), the user interacts with the virtual representation that mediates the connection, maintaining immersion while providing spatial awareness.
2Adaptability or versatility
If the physical view is completely occluded during virtual reality mode, then the virtual reality immersion is improved, but the user cannot locate physical input devices
Solution Approach 1:
The system creates a virtual copy of the physical input device that appears in the correct position and orientation within the virtual reality environment. This copy preserves the spatial information of the physical device while allowing complete occlusion of the physical view, thus maintaining immersion without losing location information.
Solution Approach 2:
The patent transitions the physical input device's representation from the physical dimension to the virtual dimension. By rendering the virtual model in the VR space, the system provides location information in a different dimensional context, allowing users to locate devices through virtual space rather than physical line-of-sight.
3Device complexity
If traditional virtual reality input methods are used without physical device tracking, then the system complexity is reduced, but the ease of operation and user experience deteriorate
Solution Approach 1:
The system uses a multi-functional approach where the imaging sensor serves both to capture the physical environment and to track physical input devices. The same sensor array and image processing pipeline are used for both general VR functionality and specific input device detection, avoiding the need for separate tracking hardware and reducing overall system complexity.
Solution Approach 2:
The system uses its existing imaging sensors and processing capabilities to automatically detect and track physical input devices without requiring additional specialized sensors or complex external tracking infrastructure. The HMD's own resources are leveraged to provide enhanced input functionality, reducing system complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed herein are related to a system and a method for porting a physical object in a physical space into a virtual reality. In one approach, the method includes detecting an input device in a physical space relative to a user of the input device. In one approach, the method includes presenting, by a display device to the user, a virtual model of the detected input device in a virtual space at a location and an orientation. The location and the orientation of the virtual model in the virtual space may correspond to a location and an orientation of the input device in the physical space relative to the user. In one approach, the method includes visually providing relative to the virtual model in the virtual space, through the display device, spatial feedback on the user's interaction with the input device in the physical space.