3D Model Generation Excluding Moving Objects via Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating 3D models of physical environments based on images and depth information are inaccurate and inefficient when there are moving objects, such as people or animals, in the captured image information.
Innovation Solution
The implementation uses depth data and an object mask generated from RGB image data to exclude moving objects, such as people or animals, from the 3D model. This is achieved through segmentation or masking techniques, including the use of machine learning models like neural networks to generate segmentation masks that identify objects associated with motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If moving objects are included in 3D model generation using depth sensors and RGB cameras, then the model captures complete environmental information, but the model accuracy and realism deteriorate due to distortion and noise from moving objects
Solution Approach 1:
The patent applies segmentation by generating a mask that divides the image into movable objects and stationary objects. The mask is created by comparing depth information across multiple frames to identify regions with significant depth changes, which correspond to moving objects. This segmentation allows the system to selectively exclude moving objects from 3D model generation while preserving stationary objects, thereby maintaining model accuracy without losing complete environmental information.
2Measurement precision
If segmentation masks are generated using machine learning models to identify moving objects, then the exclusion of moving objects becomes more accurate, but the device complexity increases
Solution Approach 1:
The patent replaces complex machine learning models with a simpler depth-difference-based segmentation approach. Instead of using computationally intensive neural networks to identify moving objects, the system compares depth values from consecutive frames and identifies moving objects based on threshold-based depth changes. This substitution maintains adequate object identification accuracy while significantly reducing device complexity and computational requirements.
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that determines generates a three-dimensional (3D) model based on depth data and a segmentation mask. For example, an example process may include obtaining depth data including depth values for pixels of a first image, obtaining a segmentation mask associated with a second image, the segmentation mask identifying a portion of the second image associated with an object, and generating a 3D model based on the depth data and the mask.