Mixed Reality Object Linking for Shared Multi-User Editing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality (MR) systems do not allow multiple users to simultaneously perform operations such as disposing or replacing virtual objects in a shared real space, and they lack the ability to share real spaces among users.
Innovation Solution
A mixed reality display device and method that connects multiple MR display devices worn by users, allowing them to share reality videos and virtual objects, recognize real objects, link virtual objects to these objects based on user operations, and synchronize and edit these objects across devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a portable information terminal is added to the HMD to enable sharing of MR video, then multiple persons can view the MR experience, but the ability to perform operations such as disposing or replacing virtual objects is lost for non-HMD users
Solution Approach 1:
The patent enables the portable information terminal to perform multiple functions: it serves both as a display device for viewing MR video and as a control device for operating virtual objects. The terminal runs an application that provides both viewing capabilities and operation capabilities, allowing non-HMD users to fully interact with the MR space without requiring additional hardware.
Solution Approach 2:
The patent merges the viewing function and operation function into a single portable information terminal system. Instead of separating these functions into different devices, the terminal integrates both capabilities through a unified application interface, allowing users to switch between viewing MR video and performing operations on virtual objects seamlessly.
2Adaptability or versatility
If only the HMD user can operate virtual objects, then the system remains simple to control, but other experiencing persons cannot participate in object manipulation
Solution Approach 1:
The patent segments the operation capabilities by assigning different functions to different input methods. The HMD handles operations through gaze detection and voice commands, while the portable terminal handles operations through touch interface and physical buttons. This segmentation allows multiple users to operate virtual objects simultaneously without creating control conflicts.
Solution Approach 2:
The portable information terminal acts as an intermediary device that bridges the gap between non-HMD users and the virtual object manipulation capability. It provides a familiar touch interface for operating virtual objects and transmits these operations to the MR system, enabling indirect but effective control without requiring users to wear HMDs.
3Adaptability or versatility
If multiple devices are connected to enable sharing and collaboration, then real spaces can be shared among users, but the system complexity increases
Solution Approach 1:
The patent creates a virtual copy of the real space environment that can be accessed and manipulated by multiple users through their respective devices. Each user's device receives and displays a synchronized version of the shared MR video, allowing them to view and interact with the same virtual objects in the shared space without requiring direct peer-to-peer device connections.
Data Source
AI summary
The MR display device includes a controller and a display unit. The controller recognizes a reality object from a reality video imaged by a camera and links a predetermined virtual object to the reality object based on an operation of an experiencing person. The display unit displays a video of the linked virtual object. The controller acquires a recognition result of a reality object generated by the other MR display device and data of the linked virtual object, via a communication unit, synthesizes the acquired data and the own data, and displays a video obtained by the synthesis on the display unit. In addition, the controller allows an edit of the virtual object linked to the reality object on the video displayed on the display unit, based on an operation of the experiencing person.