AR HMD Gesture Switching 2D to 3D Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) and virtual reality (VR) head-mounted displays (HMDs) lack intuitive methods for seamlessly switching between 2D and 3D modes during computer simulations, such as games, especially when interacting with real-world environments.
Innovation Solution
A system that includes a processor configured to receive user-generated hand gestures in free space, allowing users to switch the presentation of computer simulations from a traditional display to an HMD, with options for object-based or image-based streaming, pausing the simulation if needed, and using eye-tracking to conserve bandwidth by streaming only objects in the direction of gaze.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If object-based streaming is used to improve immersion in 3D mode, then bandwidth consumption increases
Solution Approach 1:
The system applies different streaming qualities to different spatial regions: high-quality object-based streaming is directed only toward the user's gaze direction while other regions receive lower priority or no streaming. This localized quality adjustment maintains immersion where needed while conserving bandwidth in less critical areas.
Solution Approach 2:
Instead of streaming all objects in the virtual environment, the system selectively streams only those objects that fall within the user's field of view or gaze direction. This partial action approach provides sufficient immersion for the current viewing context without the excessive bandwidth consumption of complete environment streaming.
2Ease of operation
If seamless switching between 2D and 3D modes is implemented, then system complexity increases
Solution Approach 1:
The system dynamically adjusts rendering parameters, streaming configurations, and presentation modes based on real-time detection of user gestures and gaze direction. This dynamic adaptation enables seamless transitions between 2D and 3D modes without requiring complex manual switching mechanisms, as the system automatically responds to intuitive user inputs.
Solution Approach 2:
The system continuously monitors user gestures and gaze patterns, using this feedback to automatically determine when to switch between 2D and 3D modes. This closed-loop feedback mechanism simplifies the user interface by replacing complex switching controls with natural, intuitive gestures that the system interprets and responds to in real-time.
3Reliability
If all objects are streamed to HMD for 3D presentation, then bandwidth usage increases
Solution Approach 1:
The system directs high-quality object streaming only toward the user's gaze direction and immediate field of view, while reducing or eliminating streaming to peripheral and less relevant areas. This localized quality distribution maintains visual completeness where the user is actually looking while conserving bandwidth in less critical viewing zones.
Solution Approach 2:
The system streams only the subset of objects that are currently visible or likely to be viewed by the user, rather than streaming all objects in the virtual environment. This partial streaming approach provides sufficient visual completeness for the active viewing context without the excessive bandwidth consumption of comprehensive object streaming.
Data Source
AI summary
A computer game can be viewed on a TV through an augmented reality (AR) head-mounted display (HMD). The wearer can make a gesture such as a first to cause the HMD to present 2D rectangles and 3D objects representing locations in the room that the game can be “presented” in. The wearer may then open his hand and “cast” toward one of the room locations, and the game is transferred to the HMD for presentation in the associated rectangle (in 2D) or 3D object (in 3D).