Head-Mounted Display for Discreet Palm-Based AR Object Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MR systems face challenges in enabling intuitive operations of AR objects in public or narrow spaces without disturbing others or requiring large gestures.
Innovation Solution
A head-mounted display (HMD) that allows users to directly operate AR objects on an operation screen displayed on or in front of their palm, using hand gestures to select and interact with AR objects without needing large movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gesture-based operation is used to operate AR objects, then operation intuitiveness is improved, but it disturbs others in public places and requires large movements that are difficult in narrow places
Solution Approach 1:
The operation interface is extracted from the traditional large gesture space and concentrated onto the user's hand, specifically the palm area. The HMD displays an operation screen that maps virtual buttons and controls directly onto the user's palm, allowing operations to be performed with minimal hand movement while maintaining intuitiveness through direct visual feedback on the hand surface.
Solution Approach 2:
The HMD display system acts as an intermediary between the user's hand gestures and the AR object operations. Instead of requiring direct large gestures in space, the system mediates by projecting an operation interface onto the palm, translating small hand movements into precise control commands for AR objects, thus reducing spatial requirements while maintaining operational intuitiveness.
2Ease of operation
If gesture-based operation is used to operate AR objects, then operation intuitiveness is improved, but it requires large movements that are difficult in narrow places
Solution Approach 1:
The operation interface is extracted from the traditional large gesture space and concentrated onto the user's hand, specifically the palm area. The HMD displays an operation screen that maps virtual buttons and controls directly onto the user's palm, allowing operations to be performed with minimal hand movement while maintaining intuitiveness through direct visual feedback on the hand surface.
Solution Approach 2:
The operation interface is shifted from three-dimensional spatial gestures to a two-dimensional surface (the palm). By projecting the operation screen onto the hand surface, the system converts spatial movement requirements into surface-based interactions, dramatically reducing the movement range needed while preserving operational intuitiveness through direct visual-tactile correspondence.
3Length of moving object
If projection-based operation interface is displayed on the palm, then operation space requirement is reduced, but it requires precise display positioning and alignment
Solution Approach 1:
The system uses the user's own hand as the display surface, eliminating the need for external projection screens or precise external positioning. The HMD tracks the hand's position and orientation in real-time, dynamically adjusting the operation screen placement to match the hand's movement, thereby converting a precision positioning problem into a self-adapting system that automatically aligns with the user's anatomy.
Solution Approach 2:
The operation interface is made dynamic by continuously tracking the hand's position and orientation. Rather than requiring precise static positioning, the system adapts the display in real-time to follow hand movements, maintaining alignment between the virtual controls and the physical hand surface throughout the interaction, thus reducing precision requirements while preserving operational accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The purpose of the present invention is to improve the intuitive operability of a head-mounted display used for an MR system for displaying a real space and a virtual object that are superimposed on one another. In order to attain the purpose, the head-mounted display displays an AR object in a real space so as to form an MR space. The head-mounted display comprises: a camera that captures an image of the real space and acquires captured video; a distance-measuring camera that measures a distance from a real object in the real space; and a controller. The controller comprises: captured object processing for recognizing the real object from the captured video; AR object processing for obtaining the AR object and assigning a position, which includes a distance in the real space, to the AR object, and displayed video generation processing for generating displayed video in the MR space while reflecting the perspective of the real object and the AR object in the video. The controller is further provided with: processing for detecting an operation-screen display object from the captured video; and processing for displaying an MR-space operation screen on the operation-screen display object. Video on the operation screen includes the AR object in the MR space.