Mixed Reality Headset Gesture Control and Real-Time Blending
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed reality systems face issues such as user disorientation due to excessive clutter, difficulty in focusing on important elements, and virtual reality sickness, primarily caused by skew or lack of synchronization between real and virtual environments, and the need for external controllers that detract from the immersive experience.
Innovation Solution
A mixed reality system that uses spatially separated image capture devices on a headset to create a 3D environment, allowing users to select and interact with real-world objects and data using hand gestures, with a processor configured to generate control signals for data manipulation and blend real-world elements into the virtual environment in real-time, thereby enhancing user immersion and awareness of their surroundings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a completely virtual reality environment is used to display multiple data sources, then information display capability is improved, but user disorientation and virtual reality sickness increase
Solution Approach 1:
The patent merges virtual reality and augmented reality approaches by displaying virtual data sources in a three-dimensional virtual environment while simultaneously overlaying captured images of the real-world environment. This combination allows users to access multiple data sources without complete immersion in virtuality, reducing disorientation and sickness while maintaining information display capability.
Solution Approach 2:
The patent uses spatial positioning and three-dimensional placement of both virtual data sources and real-world image elements. By distributing information across multiple dimensions in space rather than on flat screens, the system maintains comprehensive information display while providing depth cues that help users maintain spatial awareness and reduce disorientation.
2Object-affected harmful factors
If augmented reality systems blend real-world images with virtual environment, then user orientation is improved, but visual clutter and difficulty in focusing on important elements increase
Solution Approach 1:
The patent segments the display environment into distinct virtual data sources positioned at different locations and the real-world captured images. This segmentation allows users to mentally separate and focus on specific data sources of interest while the real-world background provides orientation context, reducing visual clutter through spatial organization.
Solution Approach 2:
The patent applies different visual qualities and characteristics to different regions of the display. Virtual data sources can be highlighted, animated, or positioned prominently in specific areas, while the real-world background provides contextual orientation. This local differentiation helps users focus on important elements without being overwhelmed by the entire blended environment.
3Ease of operation
If external controllers are used to interact with data in virtual environment, then interaction capability is improved, but immersion experience deteriorates
Solution Approach 1:
The patent replaces mechanical external controllers with voice recognition and gesture-based interaction systems. Users can issue voice commands or make hand gestures to select, manipulate, and interact with virtual data sources, maintaining full immersion in the three-dimensional environment while achieving effective interaction capability without physical controller interference.
4Object-affected harmful factors
If real-time image blending is implemented, then user immersion is improved, but processing complexity and synchronization requirements increase
Solution Approach 1:
The patent performs preliminary processing of captured real-world images by pre-aligning them with the virtual environment framework and pre-positioning them in the three-dimensional space. This preliminary action reduces the computational burden during real-time operation, as the system only needs to blend and render rather than perform complex alignment and spatial mapping from scratch, reducing processing complexity while maintaining immersion.
Data Source
Figure 1~2
Figure 3~4
AI summary
A system for displaying multiple sources of data within a mixed reality environment, comprising a headset with a visor which is placed over a user's eyes in use. The visor includes a screen and the system further comprises a processor for generating a three-dimensional virtual environment to be displayed on the screen, and within which the data is arranged for viewing by the user. The system includes one or more cameras for capturing real world images in the vicinity of the user, and provides a facility for enabling selected objects from the captured images to be blended and included in the 3D virtual environment displayed on the screen. An extension of the system provides the facility for tailoring a mixed reality environment for a given application.