VR Overlay Hand Tracking for Realistic Multi-User Simulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing simulators lack the ability to provide a realistic interaction experience between multiple users, as they only indicate control changes without showing the movement of controls, and require access to the simulation software's code for integration of user interactions.
Innovation Solution
An overlay technology using VR headsets and hand tracking devices, such as LeapMotion, to replicate user movements and interactions across multiple simulations without modifying the simulation software, utilizing APIs and communication interfaces to synchronize user inputs and outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing simulators are used for multi-user interaction, then the simulation software can be kept simple and certified, but the interaction realism is poor as only control changes are indicated without showing actual movements
Solution Approach 1:
The patent introduces an overlay system as an intermediary layer between the simulation software and the user. This overlay captures hand movements using tracking devices and displays them in the VR environment, enabling realistic interaction visualization without modifying the certified simulation software core. The overlay acts as a mediator that adds interaction realism while preserving the original simulation's certification status.
Solution Approach 2:
The patent creates a visual copy of the user's hand movements by tracking real hand positions and rendering virtual hands that mirror these movements in the overlay. This copying approach allows the simulation to display realistic hand movements without requiring the simulation software itself to be modified, thus maintaining certification while improving interaction realism.
2Ease of operation
If access to simulation software code is required for integrating user interactions, then the integration can be precise and seamless, but the complexity and cost of certification increases
Solution Approach 1:
The patent segments the interaction system into two independent parts: the original simulation software (which remains certified and unchanged) and the overlay system (which handles interaction tracking and visualization). This segmentation allows the overlay to be developed and updated separately without triggering recertification of the core simulation software, reducing complexity and cost.
Solution Approach 2:
The overlay system serves as an intermediary that interfaces with the simulation software through established APIs without requiring access to or modification of the simulation's core code. This approach enables precise interaction integration while avoiding the certification complexity that would arise from modifying the simulation software itself.
3Productivity
If multiple users interact in separate simulations, then each user's simulation can be optimized independently, but the sense of presence and cooperation is reduced
Solution Approach 1:
The patent merges the visual representations of multiple users' hands into a shared VR environment through the overlay system. While each user's simulation remains independently optimized, the overlay combines hand tracking data from multiple users and displays their respective hands in a common spatial context, preserving the sense of presence and cooperation.
Solution Approach 2:
The patent adds a new dimensional layer to the simulation experience by introducing the overlay that operates parallel to the individual simulations. This additional dimension allows users to see each other's hands and movements without interfering with the independent optimization of each simulation environment.
Data Source
AI summary
Method providing interactive simulation comprising a first computer displaying first simulation interacting with a first user, the first computer is connected to a first head mounted display, HMD, and a first interactive control, a second computer displaying a second simulation interacting with a second user, the second computer is connected to a second HMD and a second interactive control, wherein the first computer is connected to the second computer via a network.