Embedded Multi-View 3D Media Capture With Reduced Data Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating three-dimensional digital media representations require significant additional data, such as dense depth maps or high processing times, limiting efficiency and transfer rates, and traditional formats offer limited interactive experiences.
Innovation Solution
A method for generating multi-view interactive digital media representations (MIDMRs) by combining convex and concave motion capture, allowing embedded views and interactive selection, with enhancement algorithms for stabilization and interpolation, and using IMU data for angle estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If dense depth maps or optical flow maps are used to describe scene structure, then manufacturing precision of 3D models is improved, but device complexity and data requirements increase significantly
Solution Approach 1:
The patent segments the 3D model creation process into distinct phases: capturing multiple 2D images from different viewpoints, extracting feature points and matching them across images, constructing a sparse 3D point cloud, and finally generating the complete 3D model. This segmentation allows the system to achieve accurate 3D reconstruction without requiring dense depth maps for the entire scene, thereby reducing data complexity while maintaining manufacturing precision.
Solution Approach 2:
The patent transitions from 2D image data to 3D spatial representation by capturing images from multiple viewpoints and using triangulation geometry. Instead of relying on dense depth information in a single view, the system infers 3D structure by combining information from multiple 2D projections, effectively using dimensional transformation to achieve accurate 3D modeling with reduced data requirements.
2Manufacturing precision
If computer generation of polygons or texture mapping is used to create 3D models, then manufacturing precision is improved, but productivity decreases due to high processing times and resources
Solution Approach 1:
The patent performs preliminary actions during the image capture phase by collecting multiple viewpoint images and pre-extracting feature points and descriptors. This preliminary processing organizes the data in a way that facilitates efficient 3D reconstruction later, reducing the computational burden during model generation and thereby improving productivity without sacrificing 3D model quality.
Solution Approach 2:
The patent uses 2D images captured from multiple viewpoints as copies of the 3D scene, and through feature matching and triangulation, reconstructs the 3D model. This approach avoids the need for complex polygon generation and texture mapping by directly deriving 3D geometry from 2D image copies, significantly reducing processing time and resource requirements while maintaining model accuracy.
3Device complexity
If traditional 2D flat image formats are used, then device complexity is reduced, but loss of information increases due to limited viewing angles and interactive capabilities
Solution Approach 1:
The patent merges multiple 2D images captured from different viewpoints into a unified 3D model representation. This combination preserves scene details from all viewing angles simultaneously, allowing users to interactively explore the scene from any perspective. The merged 3D model retains comprehensive scene information while maintaining manageable system complexity through efficient data structures and rendering techniques.
Data Source
AI summary
Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.


