3D Representation Merging for Wider XR Views Without Content Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems and techniques are inadequate for providing an immersive view of recorded content by effectively merging different 3D representations, particularly in extended reality environments, to expand the field of view beyond the limitations of captured images or videos.
Innovation Solution
A method and system that identifies regions of interest in recorded images or videos and merges them with a persistent 3D model, allowing for the expansion of the view to 180° or greater, incorporating scene properties and updating the representation over time to provide an immersive 3D environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If a captured image or video is used to represent an environment, then the content is authentic and accurate to the recording, but the field of view is limited and cannot provide an immersive 360° view
Solution Approach 1:
The patent merges a persistent 3D model (providing wide FOV) with a captured video representation (providing authentic content) by identifying corresponding regions and combining them. The captured video is projected onto the persistent model, creating a hybrid representation that maintains both the expanded field of view and the integrity of the captured content through region-based merging.
2Adaptability or versatility
If a persistent 3D model is used to expand the field of view, then the immersive view is achieved, but the captured content may be replaced or distorted
Solution Approach 1:
The patent applies local quality by treating different regions of the persistent model differently based on their correspondence to captured content. Regions that correspond to captured video areas maintain high fidelity to the original recording, while non-corresponding regions use the persistent model data to provide expanded FOV. This region-based differentiation ensures both immersive capability and content authenticity.
3Area of stationary object
If the entire captured video is merged with the persistent model, then the field of view is expanded, but processing complexity increases
Solution Approach 1:
The patent extracts only the relevant portions of the captured video that correspond to regions in the persistent model, rather than merging the entire video. By identifying and extracting only the corresponding regions for merging, the system reduces processing complexity while still achieving field of view expansion through the selective combination of captured content and persistent model data.
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods for providing a view of a three-dimensional (3D) representation that is generated by merging 3D representations based on identified regions of interest and location. For example, a process may include obtaining a first 3D representation of a physical environment and a second 3D representation that was generated based on frames of image data of an area of the physical environment. The process may further include identifying a region of interest associated with the second 3D representation and identifying a portion of the first 3D representation based on the first area depicted in the image data and the identified region of interest. The process may further include generating a merged 3D representation by combining the identified portion of the first 3D representation with a portion of the second 3D representation, and presenting a view of the merged 3D representation.