Volumetric Imaging Using Depth Data and SLAM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for forming volumetric images of moving objects require complex, professional imaging apparatuses that are costly and inaccessible to average users, limiting the ability to capture and share high-quality volumetric video using user-held devices.
Innovation Solution
A method using a user-held device to acquire video and depth data, combined with pose data, to process and render moving volumetric images, eliminating the need for additional sensors and equipment by employing depth-from-disparity methods and SLAM techniques, and utilizing deep-learning for segmentation and visual effects to create realistic representations of moving objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If professional imaging apparatus with multiple cameras and sensors is used, then volumetric image quality is improved, but device complexity and cost increase
Solution Approach 1:
The patent combines video data and depth data from a single user-held device into a unified processing pipeline. The segmentation module processes both video and depth data together to generate segmented images that delineate the moving object, merging multiple data streams into a cohesive volumetric reconstruction process.
Solution Approach 2:
The user-held device performs multiple functions: it captures both video and depth data, tracks its own pose through SLAM techniques, and processes this multi-functional data stream to generate volumetric images of moving objects, replacing the need for specialized professional equipment.
2Measurement precision
If multiple synchronized cameras and sensors are used, then volumetric image accuracy is improved, but ease of operation deteriorates
Solution Approach 1:
The system performs self-calibration and self-tracking through SLAM techniques. The user-held device automatically determines its own pose and trajectory without requiring external calibration equipment or operator intervention, enabling average users to operate the system easily while maintaining high accuracy.
Solution Approach 2:
The patent implements preliminary pose estimation and segmentation before final volumetric reconstruction. The segmentation module pre-processes video and depth data to identify object boundaries, and the SLAM system pre-establishes the device's motion trajectory, preparing data in advance for the final accurate volumetric image generation.
3Measurement precision
If complex processing algorithms are used, then volumetric image quality is improved, but processing time increases
Solution Approach 1:
The patent divides the complex processing task into distinct modules: a segmentation module that separates the moving object from the background using both video and depth data, and a volumetric reconstruction module that generates the 3D model. This segmentation of processing tasks enables parallel computation and optimizes processing efficiency while maintaining high image quality.
Data Source
AI summary
A method for generating a moving volumetric image of a moving object from data recorded by a user-held device comprising: acquiring, from the user-held device, video and depth data of the moving object, and pose data; and communicating the acquired data to a computing module. Then, processing the video data to extract images that are segmented to form segmented images; passing the segmented images, depth data and pose data through a processing module to form a sequence of volumetric meshes defining the outer surface of the moving object; rendering the sequence of volumetric meshes with a visual effect at least partly determined from the video data to form a rendered moving volumetric image; and communicating the rendered moving volumetric image to at least one device including the user-held device. Then, displaying, at a display of the at least one device, the rendered moving volumetric image.