Volumetric Representation Generation for Mixed Reality Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack the capability to capture a three-dimensional (3D) representation of an environment that includes both physical and computer-generated objects, only capturing two-dimensional (2D) images or video streams.
Innovation Solution
An electronic device generates a volumetric representation of a capture region by obtaining depth information and disambiguating points from a 3D point cloud, using this information to create a 3D model that includes both physical and computer-generated objects within a defined capture region.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a system captures only 2D images or video streams, then the capture mechanism is simple and straightforward, but the system lacks the capability to capture 3D representations of environments
Solution Approach 1:
The patent transitions from 2D image capture to 3D volumetric capture by introducing depth information and spatial coordinates. The system captures images at multiple depths and combines them to create volumetric representations, adding the depth dimension to traditional 2D photography. This allows the system to capture three-dimensional representations of both physical and virtual objects within a mixed reality environment.
Solution Approach 2:
The patent introduces depth maps and point cloud data as intermediary representations between the captured images and the final volumetric output. These intermediaries facilitate the transformation from 2D images to 3D volumetric representations by providing spatial information and depth relationships, enabling the system to reconstruct three-dimensional structures from two-dimensional inputs.
2Loss of information
If the system captures both physical and computer-generated objects in 3D, then the representation capability is enhanced, but the complexity of processing and generating volumetric data increases
Solution Approach 1:
The patent segments the capture space into multiple depth planes and captures images at each depth level separately. This segmentation allows the system to process physical and virtual objects at different depths independently, reducing the overall processing complexity. By dividing the volumetric capture task into multiple 2D image captures at different depths, the system can manage the complexity of representing both physical and computer-generated objects more effectively.
Solution Approach 2:
The patent creates point cloud representations and depth maps as simplified copies or approximations of the full volumetric data. These intermediate representations capture the essential spatial and depth information without requiring complete volumetric processing of all pixels. The point clouds serve as lightweight proxies that preserve the 3D structure while reducing computational complexity for further processing and rendering.
3Measurement precision
If the system generates detailed volumetric representations, then the spatial accuracy and depth information are improved, but the data processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary processing by capturing depth information and generating point cloud representations during the image capture phase itself. Rather than computing full volumetric representations after capturing all images, the system prepares depth maps and spatial data in advance, organizing the data into structured formats that facilitate faster subsequent processing. This preliminary organization of depth and spatial information reduces the computational burden during the final volumetric reconstruction stage.
Data Source
AI summary
A method includes displaying, on a display, a representation of a physical environment and a computer-generated object. The method includes generating a three-dimensional (3D) point cloud associated with the representation of the physical environment. The method includes obtaining depth information characterizing the physical environment. The method includes obtaining a capture event associated with a capture region within the representation of the physical environment. The capture region includes a portion of the computer-generated object. The method includes, in response to obtaining the capture event, disambiguating a group of points from the 3D point cloud, and generating, based on a function of the depth information and the group of points, a volumetric representation of the capture region. The group of points satisfies a spatial threshold with respect to the capture region. The volumetric representation includes a volumetric representation of the portion of the computer-generated object.