Mixed Reality Display Sharing for Synchronized Virtual Object Editing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality (MR) systems do not allow multiple users to simultaneously perform operations such as disposing or replacing virtual objects in a shared real space, and they fail to share real spaces among experiencing persons.
Innovation Solution
A mixed reality display device and method that connects multiple MR display devices worn by users, allowing them to share reality videos and virtual objects, recognize reality objects, link virtual objects to these objects based on user operations, and edit these objects in a synchronized manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a portable information terminal is combined with HMD to share MR experience, then multiple persons can view the same MR content, but the users cannot perform operations of disposing or replacing virtual objects in the real space
Solution Approach 1:
The patent enables the HMD to serve multiple functions: it acts as both a display device for MR content and a control device for manipulating virtual objects. The controller in the HMD receives operations from the portable terminal and executes them to dispose or replace virtual objects, allowing one device to fulfill multiple roles that were previously distributed across separate devices.
2Adaptability or versatility
If HMD shares its real space with other users, then collaborative MR experience is enabled, but other users cannot share their own real spaces with the HMD user
Solution Approach 1:
The patent merges multiple real space videos from different HMD users into a single synthesized shared space video. The controller acquires real space videos from other HMDs via communication units, synthesizes them with the local real space video, and displays the combined result. This allows multiple users' real spaces to be merged into a common collaborative environment.
3Adaptability or versatility
If multiple HMDs are connected to share MR experience, then real spaces can be shared, but the system complexity increases
Solution Approach 1:
The patent introduces a communication unit as an intermediary that facilitates data exchange between multiple HMDs. This communication unit handles the transmission and reception of real space videos and virtual object data between devices, simplifying the complexity of direct peer-to-peer connections by providing a standardized communication interface.
4Productivity
If the HMD processes all virtual object operations locally, then response time is fast, but other users cannot participate in manipulating virtual objects
Solution Approach 1:
The patent implements a feedback mechanism where the HMD sends operation results back to the portable terminal and other users. When the controller executes a virtual object operation, it transmits the result to other HMDs via communication units, allowing all users to see and respond to the same operations in real-time, creating a collaborative feedback loop.
Data Source
AI summary
The MR display device includes a controller and a display unit. The controller recognizes a reality object from a reality video imaged by a camera and links a predetermined virtual object to the reality object based on an operation of an experiencing person. The display unit displays a video of the linked virtual object. The controller acquires a recognition result of a reality object generated by the other MR display device and data of the linked virtual object, via a communication unit, synthesizes the acquired data and the own data, and displays a video obtained by the synthesis on the display unit. In addition, the controller allows an edit of the virtual object linked to the reality object on the video displayed on the display unit, based on an operation of the experiencing person.