Chromakeyed Tactile Touch-Sensitive Surface for Mixed Reality Text Readability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mixed-reality simulators face challenges in displaying high-resolution real text on controls or touchscreen displays due to low resolution capture by head-mounted display cameras, leading to readability issues in immersive training environments.
Innovation Solution
An interactive mixed masking system utilizing a chromakeyed tactile touch-sensitive surface and touch-sensitive input circuit, which registers and processes touch inputs to create a composite mixed reality scene by isolating and preserving specific pixels, allowing high-resolution virtual text to be overlaid on real-world views in a head-mounted display.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If video camera feed is used to display real text in HMD apparatus, then real-world view is captured, but resolution is reduced and text readability deteriorates
Solution Approach 1:
The patent uses a camera to capture the real text in the environment and processes the captured video images to create a high-resolution representation of the text. This copying approach allows the system to overcome the resolution limitations of direct camera capture by processing and enhancing the captured image data.
Solution Approach 2:
The patent introduces an intermediary processing system that acts as a mediator between the camera capture and the final display. The system processes the captured video frames to extract and enhance text information, then overlays this enhanced text on the pass-through view, effectively mediating between the low-resolution capture and the high-resolution display requirement.
2Ease of operation
If camera pass-through is used to show real controls, then real-world visibility is achieved, but text readability and interaction precision worsen
Solution Approach 1:
The patent applies local quality enhancement by specifically targeting and enhancing the text regions in the captured video frames. Instead of uniformly processing the entire image, the system identifies text areas and applies enhanced processing to these specific regions, maintaining high resolution for text while preserving the overall pass-through view quality.
Solution Approach 2:
The system creates a copied and enhanced version of the text from the captured video frames. This copied text is then overlaid on the pass-through view, providing a high-resolution representation that complements the real-world visibility while improving text readability.
3Measurement precision
If virtual text is overlaid on real controls, then text resolution is improved, but alignment accuracy and interaction reliability may deteriorate
Solution Approach 1:
The patent employs feedback mechanisms to continuously monitor and adjust the alignment between the overlaid virtual text and the real controls. The system processes video frames to detect the position and orientation of real controls, then uses this feedback information to accurately position and align the virtual text overlay, ensuring consistent alignment accuracy.
Solution Approach 2:
The patent replaces traditional mechanical alignment systems with image processing and computer vision techniques. Instead of using physical alignment mechanisms, the system uses digital image processing to detect, track, and align virtual text with real controls, achieving accurate alignment through software-based methods.
Data Source
AI summary
A system comprising a device including a chromakeyed tactile touch-sensitive surface (TTSS) and touch-sensitive input (TSI) circuit to sense touch applied to the TTSS by a hand. A processor registers a location of a chromakey mask, the device, and a virtual portal location within a virtual reality (VR) scene including virtual pixels of virtual user-interactive simulator (VUIS) controls. The processor detects and isolates, within the mask, first pixels having a non-chromakey color overlapping the chromakey mask and second pixels matching a color of the chromakey mask. The first pixels are preserved to form a pass-through image. A composite mixed reality scene is formed with the VR scene and preserved pixels registered to the VR scene. Those VUIS controls corresponding to the preserved pixels are omitted from the VR scene and remaining virtual pixels of the VUIS controls overlap and link to the TSI circuit.