Head-Mounted Display With Palm-Based MR Operation Screen
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MR systems face challenges in enabling intuitive operations for AR objects in public or narrow spaces, as methods like projecting operation inputs on the palm are not effective for seamless interaction.
Innovation Solution
A head-mounted display system with a camera, distance-measuring camera, and controller processes real and AR objects to generate an MR space, allowing operation screens to be displayed on the user's palm for intuitive interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If operation input is projected on the palm as disclosed in Patent Document 1, then operation input can be recognized, but intuitive operation in MR space cannot be attained
Solution Approach 1:
The patent creates a virtual copy of the MR space operation screen and displays it on the user's palm. This virtual operation screen contains copied AR objects from the MR space, allowing users to interact with familiar virtual interfaces rather than abstract hand gestures, thereby achieving intuitive operation while maintaining operational responsiveness
Solution Approach 2:
The patent introduces an operation screen as an intermediary interface between the user and AR objects in MR space. Instead of directly manipulating AR objects or using abstract gestures, the operation screen serves as a mediating layer that translates user interactions into meaningful operations on remote AR objects, resolving the contradiction between intuitiveness and responsiveness
2Ease of operation
If gesture operation is used in narrow places, then operation can be performed, but large movement becomes difficult
Solution Approach 1:
The patent transitions the operation interface from physical space to a different dimension by projecting the operation screen onto the user's palm. This allows the operation interface to exist in a virtual dimension independent of physical movement constraints, enabling full operation capability without requiring large physical movement ranges in narrow spaces
3Ease of operation
If button projection on palm is used, then operation input can be specified, but intuitive operation in MR space cannot be attained
Solution Approach 1:
The operation screen serves multiple functions simultaneously: it displays AR objects from MR space, provides interaction interfaces for selecting and operating those objects, and adapts to different operation scenarios. This multi-functionality eliminates the need for separate complex interface systems while achieving operation intuitiveness
Solution Approach 2:
The operation screen is dynamically generated based on the MR space content and user context. AR objects and their corresponding operation interfaces are created on-demand rather than using fixed complex menus, reducing interface complexity while maintaining operation intuitiveness through context-aware display
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables direct operation of AR objects on an operation screen without large movements, facilitating intuitive interaction in MR spaces.
Implementation Method 1
a camera that captures an image of the real space and acquires captured video
Implementation Method 2
a distance-measuring camera that measures a distance from a real object in the real space
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The purpose of the present invention is to improve the intuitive operability of a head-mounted display used for an MR system for displaying a real space and a virtual object that are superimposed on one another. In order to attain the purpose, the head-mounted display displays an AR object in a real space so as to form an MR space. The head-mounted display comprises: a camera that captures an image of the real space and acquires captured video; a distance-measuring camera that measures a distance from a real object in the real space; and a controller. The controller comprises: captured object processing for recognizing the real object from the captured video; AR object processing for obtaining the AR object and assigning a position, which includes a distance in the real space, to the AR object, and displayed video generation processing for generating displayed video in the MR space while reflecting the perspective of the real object and the AR object in the video. The controller is further provided with: processing for detecting an operation-screen display object from the captured video; and processing for displaying an MR-space operation screen on the operation-screen display object. Video on the operation screen includes the AR object in the MR space.