Mixed Reality Scene Model for Remote Collaboration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current remote collaboration systems in virtual reality lack seamless integration of real and virtual scenes, requiring manual effort for scene configuration and object identification, which hinders immersive and accurate collaboration between users in different physical locations.
Innovation Solution
The system configures a real scene in a virtual reality environment (VRE) and a remote VRE with a Mixed Reality (MR) scene model, automatically calibrating the remote MR scene to correspond in three-dimensional space, and updates both VREs with changes, using sensor data to recreate the real scene as a triangular mesh and identifying objects from a known object set for enhanced accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual scene configuration and object identification methods are used in remote collaboration systems, then users can establish virtual reality environments for collaboration, but the process requires substantial manual effort and time for scanning and digitizing scenes
Solution Approach 1:
The system performs preliminary actions by automatically scanning the real scene, generating the MR scene model, and pre-configuring both the real scene VRE and remote VRE before collaboration begins. This eliminates the need for manual scene configuration during the collaboration process, reducing both operational effort and time loss.
Solution Approach 2:
The system enables self-service by automatically performing scene scanning, model generation, and VRE configuration without requiring manual user intervention. The automated calibration process further allows the system to self-adjust spatial correspondences, freeing users from time-consuming manual setup tasks.
2Measurement precision
If automated calibration of remote MR scene is implemented to correspond in three-dimensional space, then spatial accuracy between real and remote scenes is improved, but system complexity increases
Solution Approach 1:
The system introduces an intermediary MR scene model that serves as a bridge between the real scene and the remote VRE. This intermediary model facilitates automated calibration by providing a common reference framework, enabling accurate spatial correspondence without requiring direct complex interactions between the real and virtual environments.
Solution Approach 2:
The automated calibration process utilizes parameter changes in the MR scene model, adjusting spatial transformation parameters to achieve accurate correspondence between the real scene and remote VRE. By systematically modifying these parameters, the system achieves high measurement precision while managing complexity through algorithmic automation.
3Reliability
If continuous updates to VREs are provided based on changes in real and remote scenes, then immersive collaboration experience is enhanced, but data transmission and processing requirements increase
Solution Approach 1:
The system implements periodic action by updating the VREs at scheduled intervals or triggered by specific events rather than continuously. This approach maintains reliable collaboration experience by ensuring timely updates while reducing data transmission energy consumption by avoiding constant data exchange between locations.
Data Source
AI summary
Apparatus and associated methods relate to immersive collaboration based on configuring a real scene VRE operable from a real scene and a remote VRE operable remote from the real scene with an MR scene model of the real scene, creating an MR scene in each of the real scene VRE and remote VRE based on augmenting the MR scene model with an object model, calibrating the remote MR scene to correspond in three-dimensional space with the real scene MR scene model, and automatically providing immersive collaboration based on the MR scene in the remote VRE and updating the real scene VRE with changes to the remote VRE. In an illustrative example, the MR scene model of the real scene may be determined as a function of sensor data scanned from the real scene. In some embodiments, the MR scene model may be augmented with an object model identified from the real scene. The object model identified from the real scene may be, for example, selected from a known object set based on matching sensor data scanned from the real scene with an object from a known object set. In some embodiments, the remote MR scene may be calibrated based on applying a three-dimensional transform calculated as a function of the real MR scene and remote MR scene geometries. Some designs may recreate a subset of the real scene in the remote VRE and update the real scene VRE with changes to the remote VRE. Various embodiments may advantageously provide seamless multimedia collaboration based on updates to the remote VRE in response to physical changes to the real scene, and updating the real scene VRE in response to changes in the remote VRE.