Mixed Reality Viewpoint Control for Accurate Virtual Object Designation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In mixed reality systems, users wearing head-mounted displays (HMDs) struggle to accurately designate virtual objects on shared images displayed on other devices, such as tablets, due to independent viewpoint control issues.
Innovation Solution
A system that allows for easy and accurate designation of virtual objects by enabling independent viewpoint control on tablet devices, using a configuration that includes HMDs, scene data management servers, and tablet terminal devices, which allow for switching between MR and VR modes, enabling users to select and highlight virtual components within the shared mixed reality space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the HMD user changes the position and orientation of the viewpoint independently, then the HMD user can freely control their view in the mixed reality space, but the tablet user who is sharing the image cannot control the viewpoint and the display screen changes independently making it difficult to accurately designate virtual objects
Solution Approach 1:
The system dynamically switches between two operational modes: MR mode where the tablet displays the live mixed reality view from the HMD's viewpoint, and VR mode where the tablet provides independent viewpoint control. This dynamic mode switching allows the system to adapt to different operational needs, resolving the contradiction between following HMD viewpoint changes and enabling independent tablet viewpoint control.
Solution Approach 2:
The functionality is segmented into two distinct modes (MR mode and VR mode) that can be independently selected. In MR mode, the tablet passively displays the HMD viewpoint. In VR mode, the tablet actively controls the viewpoint. This segmentation allows each mode to optimize for its specific purpose without compromising the other.
2Adaptability or versatility
If the tablet displays the mixed reality image shared from the HMD, then multiple users can simultaneously share and view the mixed reality space, but the tablet user cannot independently control the viewpoint position and orientation
Solution Approach 1:
The tablet device is designed to perform multiple functions through mode switching: it can act as a passive display for collaborative viewing in MR mode, or as an active control device for independent viewpoint manipulation in VR mode. This multi-functionality allows a single device to serve both collaborative and individual operational needs.
3Loss of information
If the display screen of the tablet changes independently of the tablet user's intention, then the tablet reflects real-time changes in the HMD viewpoint, but it makes it difficult to accurately designate a desired component
Solution Approach 1:
The system provides a preliminary anti-action by introducing VR mode as a counter-measure to the independent viewpoint changes in MR mode. When accurate designation is needed, the user switches to VR mode where the tablet stabilizes the viewpoint, preventing the independent changes that would otherwise make designation difficult.
Data Source
Figure 1~2
Figure 3A~3D
Figure 4
AI summary
An image of a virtual object is generated in accordance with a position and orientation of a first display device. In a case where a first mode is set, the generated image of the virtual object is displayed on a first display device and second display device. In a case where a second mode different from the first mode is set, a still image of the virtual object is displayed on the second display device. A portion of the virtual object in the still image is selected based on an instruction input by a user observing the second display device during display of the still image of the virtual object. An image of the virtual object displayed on the second display device is updated based on the selected portion of the virtual object.