Mixed Reality Headset Obstruction Removal via External Camera Blending
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing head-mounted display systems for vehicles, such as aircraft, struggle to seamlessly integrate real-world and virtual data, particularly when the user's field of view is obstructed by internal components, limiting the ability to view external environments effectively.
Innovation Solution
A headset system with a processor that generates a three-dimensional mixed reality environment by blending image data from internal and external environments, allowing users to selectively remove physical obstructions, such as control panels or walls, to create a transparent or translucent view of the external environment, using image capture devices and hand gesture recognition for control inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a helmet-mounted display with transparent visor and internal screen is used to superimpose virtual information on the real world field of view, then the user can access critical data (speed, weapons, target), but the user's view of the real world environment is partially obstructed by the visor and screen assembly
Solution Approach 1:
The patent introduces an intermediary system consisting of external cameras and a processing unit that captures the real world environment from outside the vehicle and presents it to the user through the HUD. This mediator bypasses the physical obstruction caused by the visor and internal screen, allowing the user to see through the canopy as if it were transparent while still receiving all virtual information overlays.
Solution Approach 2:
The system creates a visual copy of the external environment using cameras mounted on the vehicle exterior. These camera feeds are processed and displayed on the HUD screen, providing the user with an accurate representation of the outside world that is not blocked by the physical canopy structure.
2Measurement precision
If the HUD displays a complete mixed reality environment including all physical obstructions, then the user has accurate spatial awareness, but the user cannot see through obstructions like control panels or walls to view important external elements
Solution Approach 1:
The system applies different transparency qualities to different regions of the HUD display based on user interaction. When the user selects a specific obstruction (control panel, wall, or other structure), only that local region becomes transparent or translucent, allowing viewing of external elements through that specific area while maintaining the integrity and visibility of all other spatial references in the mixed reality environment.
Solution Approach 2:
The transparency of physical obstructions in the mixed reality environment is made dynamic rather than static. The system allows the user to interactively control which obstructions become transparent at any given time, enabling flexible adaptation of the display to match the user's current operational needs and attention focus.
3Ease of operation
If traditional control panels and physical interfaces are used in the vehicle interior, then the user has tactile control options, but these physical components create obstructions that block the user's view of the external environment
Solution Approach 1:
The system uses the HUD interface as an intermediary that overlays virtual controls and information directly in the user's field of view. This eliminates the need for physical control panels that obstruct the view, as all control functions are presented and operated through the transparent display interface, maintaining ease of operation without sacrificing external visibility.
Data Source
AI summary
Apparatus for selectively displaying an operational area comprising an internal and an external environment separated from another by at least one physical obstruction, the apparatus comprising a headset including a screen for placing over a user's eyes, the system further comprising a processor configured to generate a three-dimensional virtual environment, and an image capture device for capturing images of the internal environment, said processor being configured to blend image data representative thereof into said three dimensional virtual environment to create a mixed reality environment including a representation of said at least one physical obstruction, the processor configured to receive image data representative of said external environment and to remove at least a portion of said physical obstruction from said mixed reality environment displayed on said screen, and blend said image data of said external environment into said mixed reality environment wherein said physical obstruction appears to be transparent.