Gesture-Controlled HMD Search via Image Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users of transmission-type head mounted display apparatuses face difficulties in searching for information about objects they are unsure of, as existing techniques require prior knowledge of keywords associated with the objects, making it hard to obtain desired information efficiently.
Innovation Solution
The apparatus includes an image display unit, a gesture detection unit, and a display controller that displays images and related information based on predefined gestures, allowing users to interact with displayed objects and receive relevant information without needing to know specific keywords, with options for text or voice input and varying display modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If users input keywords to search for objects, then search accuracy is improved, but ease of operation deteriorates because users must know specific keywords in advance
Solution Approach 1:
The patent introduces an image recognition system as an intermediary between the user and the search database. Instead of requiring users to input keywords, the system captures images of objects, automatically recognizes them through image processing, and retrieves related information. This mediator eliminates the need for users to know specific keywords while maintaining search accuracy.
Solution Approach 2:
The patent replaces the mechanical keyboard input system with an optical image capture system. Users simply point the device at objects to be searched, and the system automatically processes the visual information. This substitution of input methodology dramatically improves ease of operation while maintaining search effectiveness.
2Loss of information
If multiple search attempts are made to find information about unknown objects, then information completeness is improved, but loss of time increases
Solution Approach 1:
The patent performs preliminary image recognition and object identification automatically before the user needs to search for information. By pre-processing the visual data and identifying the object in advance, the system eliminates the need for multiple iterative search attempts, thereby reducing time loss while ensuring information completeness.
Solution Approach 2:
The system performs self-service by automatically capturing images, recognizing objects, and retrieving information without requiring user intervention or multiple search attempts. The device serves itself by autonomously completing the entire search process from image capture to information retrieval, significantly reducing time loss.
3Device complexity
If traditional button-based control is used in transmission-type HMD, then device complexity is reduced, but ease of operation deteriorates due to limited interaction methods
Solution Approach 1:
The patent implements a multi-functional control system that combines image capture, gesture recognition, and voice processing capabilities within the HMD. This universal control interface can handle various interaction scenarios (pointing, gesturing, speaking) through a unified system architecture, improving ease of operation without significantly increasing perceived device complexity.
Solution Approach 2:
The patent introduces dynamic control methods including gesture recognition and voice commands that adapt to user needs in real-time. The system can switch between different interaction modes (image-based, gesture-based, voice-based) dynamically, providing flexible and intuitive control that enhances ease of operation while maintaining manageable device complexity through software-based solutions.
Data Source
AI summary
A transmission-type head mounted display apparatus including an image display unit configured to transmit an external scene and display an image of a display object to be viewed along with the external scene, a gesture detection unit configured to detect a predefined gesture for the displayed image of the display object, and a display controller configured to perform at least one of display of the image of the display object in a display mode corresponding to the gesture having been detected and display of related information that corresponds to the gesture having been detected and that is associated with the display object.