3D Mesh Reconstruction Masking Moving Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mixed-reality systems face challenges in accurately reconstructing geometric surfaces in environments with moving objects, leading to inefficiencies and inaccuracies due to blurring effects like ghosting, as existing techniques are resource-intensive and based on false assumptions about depth image accuracy.
Innovation Solution
The system identifies moving objects by assigning motion state identifiers to pixels in depth maps, excluding dynamic object information from the 3D mesh reconstruction process, thereby reducing resource consumption and eliminating blurring artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional techniques are used to reconstruct geometric surfaces by fusing depth maps, then the 3D mesh can be generated, but blurring effects and ghosting artifacts appear for moving objects
Solution Approach 1:
The system performs preliminary action by detecting moving objects and masking their regions in depth maps before the fusion process. This pre-processing step prevents moving objects from contributing to the 3D mesh reconstruction, thereby eliminating ghosting artifacts while maintaining accurate reconstruction of static environments.
Solution Approach 2:
The invention extracts and removes moving object data from the depth maps through masking operations. By taking out the problematic moving object information before fusion, the system avoids introducing blurring and ghosting effects into the final 3D mesh, while preserving the accuracy of static object reconstruction.
2Manufacturing precision
If post-processing operations are performed to clean up ghosting effects, then the 3D mesh quality improves, but computational resources are consumed
Solution Approach 1:
Instead of performing resource-intensive post-processing operations to remove ghosting effects, the system applies preliminary action by masking moving objects before the fusion process. This approach eliminates the need for subsequent cleanup operations, significantly reducing computational resource consumption while achieving the same quality improvement.
Solution Approach 2:
The invention converts the harmful presence of moving objects into a beneficial filtering mechanism. By detecting and masking moving objects, the system transforms what would be a source of artifacts into a guide for selective data inclusion, thereby improving 3D mesh quality without requiring additional post-processing computational resources.
3Quantity of substance
If all pixel data is included in the 3D mesh reconstruction, then complete environmental representation is achieved, but moving objects introduce inaccuracies
Solution Approach 1:
The system applies local quality by differentiating the treatment of different pixel regions. Static object pixels are included in the 3D mesh reconstruction for complete environmental representation, while moving object pixels are masked out to maintain measurement precision. This localized selective inclusion achieves both completeness for static elements and accuracy by excluding moving elements.
Solution Approach 2:
The invention segments the depth map data into static and moving object regions through object detection and masking. This segmentation allows the system to process different pixel populations differently: including static pixels for comprehensive environmental representation while excluding moving pixels to preserve depth measurement precision, thereby resolving the contradiction between completeness and accuracy.
Data Source
AI summary
Optimizations are provided for reconstructing geometric surfaces for an environment that includes moving objects. Multiple depth maps for the environment are created, where some of the depth maps correspond to different perspectives of the environment. A motion state identifier is assigned to at least some pixels in at least some of the depth maps corresponding to moving objects in the environment. A composite 3D mesh is built using at least some of the multiple depth maps, by incorporating pixel information from the depth maps, while omitting pixel information identified by the motion state identifiers as being associated with moving objects.