Mixed Reality Mobile Terminal Visibility and UI Mode Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In mixed reality (MR) experiences, mobile terminals are either superimposed with virtual objects, making them difficult to visually recognize, or displayed separately, which can impede immersivity and intuitive operations.
Innovation Solution
An information processing apparatus that determines whether a mobile terminal is visually recognizable and switches between a first operation mode where the user directly operates the terminal and a second mode where a virtual user interface is displayed, allowing seamless application operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a virtual mobile terminal is superimposed on the real mobile terminal in MR, then the user can visually recognize the operation performed on the mobile terminal, but the mobile terminal cannot be visually recognized when virtual objects are superimposed on it
Solution Approach 1:
The patent creates a virtual mobile terminal that copies the appearance and functionality of the real mobile terminal. This virtual copy is displayed in the MR environment to convey operation information to the user, while the real terminal continues to function as the control input device. The virtual terminal serves as a visual representation that does not interfere with the actual operation capability.
Solution Approach 2:
The patent separates the functions of visual display and operational control by dividing the interface into two components: the virtual mobile terminal for displaying operation information and the real mobile terminal for receiving user input. This segmentation allows each component to perform its specialized function without compromising the other.
2Ease of operation
If the virtual mobile terminal and real mobile terminal are displayed separately, then the mobile terminal can be visually recognized, but this will impede immersivity
Solution Approach 1:
The patent merges the virtual mobile terminal with the real mobile terminal by superimposing the virtual representation onto the physical device. This creates a unified visual experience where the virtual and real terminals occupy the same spatial location in the user's field of view, maintaining immersivity while ensuring the real terminal remains visually recognizable.
Solution Approach 2:
The virtual mobile terminal acts as an intermediary layer between the user and the real mobile terminal. It provides visual feedback about operations without blocking the view of the real device, serving as a transparent mediator that enhances information delivery while preserving the visual recognition of the physical terminal.
3Loss of information
If a virtual mobile terminal is superimposed on the real mobile terminal, then operation information can be displayed, but intuitive operations become difficult due to superimposing error
Solution Approach 1:
The system continuously tracks the position and orientation of the real mobile terminal and uses this feedback to dynamically adjust the position of the virtual mobile terminal in the MR environment. This real-time feedback mechanism ensures that the virtual representation remains accurately aligned with the physical device, compensating for any movement or positioning errors.
Solution Approach 2:
The patent adjusts the parameters of the virtual mobile terminal (position, orientation, scale) based on the detected parameters of the real mobile terminal. By dynamically changing these parameters in response to the physical device's state, the system maintains accurate superimposition despite variations in the real terminal's position or orientation.
Data Source
AI summary
An information processing apparatus generates an image to be displayed on a display device that is mountable to a head of a user, decides an operation mode of an application, and accepts an input operation from a mobile terminal that the user is capable of holding and operating, and perform an application operation on the basis of the operation mode and the input operation. The information processing apparatus decides at least one of a first operation mode, in which the application operation is performed by the user operating the mobile terminal, and a second operation mode, in which a virtual user interface (UI) relating to an operation is displayed on the display device and the application operation is performed by the user operating the virtual UI, as the operation mode.