AR Headset Input via Foot and Hand Gesture Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of head-mounted displays (HMDs) with cameras for augmented reality (AR) applications faces challenges in user input efficiency, as traditional input devices like keyboards or mice complicate the operation, reducing the benefits of AR technology in understanding real-space situations.
Innovation Solution
An image processing device that includes feature data storage, environment map generation, superimposition display data management, and operating object recognition to facilitate user input by overlaying information on the visual field, allowing users to select items by positioning their feet or hands within the AR environment, and dynamically adjusting the display based on head tilt and rotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional input devices like keyboards or mice are used with HMDs, then input functionality is provided, but user input efficiency deteriorates and operation complexity increases
Solution Approach 1:
The patent extracts the input function from traditional external devices (keyboards, mice) and integrates it directly into the HMD system through sensor-based detection. The sensor unit detects head movements, eye movements, or hand gestures, allowing users to interact with the AR environment using natural body movements rather than external input devices, thereby improving ease of operation while reducing overall system complexity
Solution Approach 2:
The HMD system is designed to perform multiple functions: displaying AR information, detecting user intentions through sensors, and executing operations based on detected movements. This multi-functionality eliminates the need for separate input devices, as the HMD itself serves as both display and input interface, resolving the contradiction between providing input functionality and maintaining operational simplicity
2Loss of information
If AR technology is used to overlay information on visual field, then understanding of real space is improved, but visibility of superimposed information may deteriorate when information density is high
Solution Approach 1:
The patent applies local quality by selectively displaying AR information only at locations relevant to the user's current focus or task, rather than uniformly distributing information across the entire visual field. The control unit determines which superimposed information to display based on sensor detection results and environmental context, ensuring that information is presented in optimal locations for visibility and comprehension, thus preventing information overload while maintaining spatial understanding
Solution Approach 2:
The system performs preliminary actions by pre-processing and organizing information before superimposition, selecting only the most relevant data to display based on the user's current context. This prevents information overload by filtering and prioritizing content in advance, ensuring that superimposed information remains visible and meaningful without overwhelming the user's visual field
Data Source
AI summary
There is provided an image processing device including a superimposition display position determining unit which determines a position of an object having a predetermined flat surface or curved surface out of an object imaged in an input image based on an environment map, a superimposition display image generating unit which generates a superimposition display image by setting superimposition display data at the position of the object determined by the superimposition display position determining unit, an image superimposing unit which superimposes the superimposition display image on a visual field of a user, an operating object recognizing unit which recognizes an operating object imaged in the input image, and a process executing unit which executes a process corresponding to an item selected based on a position of the operating object recognized by the operating object recognizing unit.