VR Training State Synchronization with Dynamic Object Importance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR training systems face challenges in achieving high tracking accuracy and reducing latency while maintaining a low cost, particularly in synchronizing virtual and real-world site states for enhanced training realism, especially with limited computing resources and network communication requirements.
Innovation Solution
A training system that synchronizes a virtual reality site state and a real-world site state by dynamically assigning importance to tracked objects using a sensor data processor, which includes modules for helmet pose tracking, trainee ego-pose tracking, object pose tracking, and scene graph updates, utilizing IMU, RGBD cameras, and LiDAR to optimize tracking based on object importance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high precision tracking system (e.g., OptiTrack) is used, then tracking accuracy is improved, but cost and device complexity increase
Solution Approach 1:
The patent uses virtual avatars as copies of real objects and trainees in the VR environment. These avatars replicate the visual and spatial characteristics of real objects, allowing the system to track and synchronize VR object positions with real object positions without requiring complex physical tracking infrastructure for every object.
Solution Approach 2:
The patent replaces complex mechanical tracking systems (like OptiTrack with cameras and markers) with a VR-based virtual tracking system. Instead of physically tracking real objects with multiple cameras, the system tracks virtual avatars within the VR environment and synchronizes their positions with real objects, substituting mechanical tracking with computational geometry and coordinate transformation.
2Loss of time
If server-based computation is used for tracking, then tracking latency is reduced, but network communication requirements increase
Solution Approach 1:
The patent implements self-service tracking by processing sensor data locally on the wearable device rather than relying on server-based computation. The system uses onboard processors to estimate poses of trainees and objects in real-time, eliminating the need for continuous network communication with a server while maintaining low latency performance.
Solution Approach 2:
The patent introduces local processing units (wearable computers, mobile devices) as intermediaries between sensors and the tracking system. These intermediaries perform pose estimation and object tracking computations locally, acting as a buffer that eliminates direct server dependency and reduces network communication overhead while maintaining real-time performance.
3Device complexity
If embedded/wearable computer is used, then network communication is reduced, but number of trackable objects and tracking speed are limited
Solution Approach 1:
The patent segments the tracking system into multiple independent tracking modules, each responsible for tracking specific objects or trainees. This modular approach allows the wearable device to handle multiple tracking tasks simultaneously by distributing computational workload across separate processing units, increasing the total number of trackable objects without overwhelming the local processor.
Solution Approach 2:
The patent applies partial action by selectively tracking only the most relevant objects and trainees based on current task requirements and spatial proximity. Instead of attempting to track all possible objects in the environment, the system prioritizes tracking based on interaction relevance, allowing the wearable device to maintain high tracking speed for critical objects while reducing computational load for less important objects.
Data Source
AI summary
Disclosed are a training system and method for synchronizing a virtual reality site state and a real-world site state and performing tracking optimization based on dynamic object importance. The training system for synchronizing a virtual reality (VR) site state and a real-world site state and performing tracking optimization based on dynamic object importance includes a sensor data processor configured to estimate the pose and state of a real object from sensor data and to update the state of an object within a VR site based on the pose and state of the real object.