Head-Mounted Display MR Palm Interface for AR Object Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MR systems face challenges in providing intuitive operations for AR objects in public or narrow spaces, as methods like projecting operation inputs on the palm are not effective for seamless interaction in mixed reality environments.
Innovation Solution
A head-mounted display (HMD) equipped with a camera, distance-measuring camera, and controller processes real-space images to generate an MR space, allowing intuitive operation through an operation screen displayed on the user's palm or other body parts, enabling selection and interaction with AR objects without large movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gesture-based operation is used in public places, then operation intuitiveness is improved, but disturbance to others increases
Solution Approach 1:
The operation interface is extracted from the physical gesture space and projected onto a virtual operation screen in the MR space. Users interact with AR objects through the virtual operation screen rather than through large physical gestures, eliminating disturbance to others while maintaining operation intuitiveness through direct manipulation of virtual controls.
Solution Approach 2:
The operation interface transitions from three-dimensional physical gesture space to a two-dimensional virtual operation screen in the MR space. This dimensional transformation allows users to perform operations intuitively through the virtual interface without requiring large physical movements that would disturb others in public spaces.
2Ease of operation
If large gesture movements are used for operation, then operation intuitiveness is improved, but usability in narrow spaces deteriorates
Solution Approach 1:
The operation interface moves from physical gesture space to a virtual operation screen in the MR space. This allows users to perform intuitive operations on AR objects through the virtual interface without requiring large physical gestures, making the system usable in narrow spaces where large movements are restricted.
Solution Approach 2:
A virtual operation screen is created as a copy of the physical control interface in the MR space. This virtual copy allows users to interact with AR objects intuitively through the displayed controls without needing physical space for large gestures, enabling operation in confined areas.
3Device complexity
If button projection on palm is used for operation input, then device simplicity is improved, but operation intuitiveness in MR space deteriorates
Solution Approach 1:
The operation interface transitions from a two-dimensional projection on the palm to a virtual operation screen in the three-dimensional MR space. This allows the operation screen to be positioned and oriented to match the MR space context, improving intuitiveness while maintaining the simplicity of using the camera and display without additional physical controls.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables direct operation of AR objects on an operation screen, facilitating intuitive interaction in MR spaces without the need for large movements, suitable for public or confined areas.
Implementation Method 1
a camera that captures an image of the real space and acquires captured video
Implementation Method 2
a distance-measuring camera that measures a distance from a real object in the real space
Data Source
AI summary
A head-mounted display comprises: a camera that captures an image of the real space and acquires captured video; a distance-measuring camera that measures a distance from a real object in the real space; and a controller. The controller comprises: captured object processing for recognizing the real object from the captured video; AR object processing for obtaining the AR object and assigning a position, which includes a distance in the real space, to the AR object, and displayed video generation processing for generating displayed video in the MR space while reflecting the perspective of the real object and the AR object in the video. The controller is further provided with: processing for detecting an operation-screen display object from the captured video; and processing for displaying an MR-space operation screen on the operation-screen display object. Video on the operation screen includes the AR object in the MR space.