3D Object Extraction Using Multi-View Triangle Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for three-dimensional object detection in real-world environments lack semantic information, leading to inefficiencies in instance segmentation and computational expense, particularly when dealing with large visual scenes.
Innovation Solution
An automated object instance extraction process that segments a two-dimensional scene from multiple viewpoints, mapping and aggregating triangles to extract three-dimensional objects efficiently without relying on semantic information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If instance segmentation algorithms are used for 3D object detection, then object detection accuracy is improved, but computational cost increases significantly
Solution Approach 1:
The patent segments the 3D point cloud into multiple 2D projections (front, back, left, right views) and performs instance segmentation on each 2D view independently. This divides the complex 3D segmentation task into simpler 2D tasks, reducing computational complexity while maintaining detection accuracy through multi-view aggregation.
Solution Approach 2:
The patent transforms the 3D object detection problem into multiple 2D instance segmentation problems by creating orthogonal projections of the point cloud. This dimensionality reduction allows standard 2D instance segmentation algorithms to be applied efficiently, avoiding the need for computationally expensive 3D-specific algorithms while still achieving accurate 3D object detection through view aggregation.
2Reliability
If 3D instance segmentation is performed on large visual scenes, then complete object detection is achieved, but processing time increases
Solution Approach 1:
The patent divides the large visual scene into multiple 2D projection views and processes each view independently through instance segmentation. This parallelizable segmentation approach reduces processing time compared to sequential 3D processing, while the aggregation of results from all views ensures complete object detection across the entire scene.
Solution Approach 2:
The patent performs instance segmentation on multiple 2D projections (more than the minimal single view), which exceeds the bare minimum required for detection. This excessive action of processing multiple views ensures complete object detection even for occluded objects, while the computational efficiency of 2D processing keeps overall processing time acceptable.
3Productivity
If 2D instance segmentation is used for 3D object detection, then computational efficiency is improved, but occlusion problems occur
Solution Approach 1:
The patent creates multiple 2D projections (front, back, left, right views) of the 3D point cloud and performs instance segmentation on each projection. Objects that are occluded in one view may be visible in another view, allowing the system to reconstruct complete 3D object information by aggregating detection results across all views, thus overcoming occlusion limitations.
Solution Approach 2:
By transforming the single 3D view into multiple 2D orthogonal projections, the patent enables observation of occluded objects from different dimensional perspectives. This multi-dimensional 2D approach maintains computational efficiency of 2D processing while recovering information lost to occlusion in any single view through result aggregation.
Data Source
AI summary
A method for extracting a 3D object from a 3D environment, including segmenting an initial 2D scene obtained from an initial viewpoint to identify a plurality of object instances; mapping a selected object instance to an initial subset of triangles underlying the selected object instance; generating additional viewpoints around the selected object instance, and for each additional viewpoint: generating a further 2D scene from the additional viewpoint; segmenting the further 2D scene, thereby identifying candidate object instances; identifying a given candidate object instance as best matching the selected object instance, thereby obtaining a best matching candidate object instance; identifying an additional subset of triangles underlying the best matching candidate object instance; aggregating the additional subset of triangles to the initial subset of triangles; and outputting the aggregated subset of the triangles.