Tactile Interaction in Virtual Environments Using Real Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality (VR) and mixed reality (MR) technologies lack the ability to provide tactile interactions between users and virtual objects, limiting the immersive experience by not allowing users to physically interact with real-world objects that correspond to virtual content.
Innovation Solution
The system receives data from sensors to track the position and orientation of real objects and users, allowing it to determine interactions and render corresponding virtual objects on a display, enabling users to interact with virtual content by manipulating real objects, which can change the virtual environment based on these interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional VR/MR technologies are used, then virtual environments can be displayed, but tactile interaction with virtual objects is not enabled
Solution Approach 1:
The patent introduces real physical objects as intermediaries between users and virtual environments. These physical objects serve as mediators that users can touch and manipulate, while their corresponding virtual representations respond to the same interactions. This mediator approach enables tactile interaction without requiring complex haptic feedback devices, as the physical object itself becomes the interface.
Solution Approach 2:
The system creates virtual copies of physical objects that replicate the interaction properties of their real-world counterparts. When a user interacts with a physical object (grasping, rotating, moving), the virtual copy performs the same action in the virtual environment. This copying mechanism allows tactile interaction to be transferred from the physical to the virtual domain without direct haptic connection.
2Reliability
If real objects are tracked and rendered as virtual objects, then immersive interaction is enhanced, but measurement and detection difficulty increases
Solution Approach 1:
The system employs sensors with multi-functionality to handle various tracking requirements. A single sensor system can detect position, orientation, and interaction forces simultaneously, eliminating the need for multiple specialized sensors. This universal approach simplifies the detection system while maintaining high tracking precision for immersive experiences.
Solution Approach 2:
The system implements continuous feedback loops where sensor data is processed in real-time to update virtual object positions and responses. This feedback mechanism allows the system to adapt to user interactions dynamically, maintaining accurate tracking even as objects move and change orientation. The feedback loop compensates for measurement uncertainties and enhances tracking reliability.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
Tactile virtual reality (VR) and/or mixed reality (MR) experiences are described. Techniques described herein include receiving data from a sensor and accessing a position and an orientation of a real object that is physically present in a real scene. Furthermore, techniques described herein include identifying the real object based at least in part on the position and the orientation of the real object and causing a graphical element corresponding to the real object to be rendered on a display of a VR and/or MR display device. The graphical element can be determined based at least in part on a VR and/or MR application. The techniques described herein include determining an interaction with the real object and causing a functionality associated with the graphical element to be performed in the VR or MR environment rendered via the VR and/or MR display device, respectively.