3D Scene Integration Using Semantic Point Cloud Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The lack of comprehensive scene understanding limits the capability for complex tasks in applications such as robotics.
Innovation Solution
A computer-implemented method and pipeline that derive object classifications and geometric points from pixel-wise semantic segmentation of multiple image frames, merge these points based on a mutually closest geometric point metric, and control the movement of autonomous objects to achieve tasks using merged geometric points.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pixel-wise semantic segmentation is performed on multiple image frames to derive object classifications and geometric points, then scene understanding accuracy is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent applies segmentation by dividing the scene into distinct object instances with unique identifiers. Each detected object is segmented as a separate entity with its own geometric points and classifications, enabling independent processing and merging of object data across multiple frames while maintaining overall scene understanding accuracy.
Solution Approach 2:
The patent merges geometric points from multiple image frames by matching object classifications and applying mutually closest geometric point metrics. This combining approach integrates information across frames to improve scene understanding accuracy while optimizing computational efficiency through consolidated object representations.
2Measurement precision
If geometric points from multiple image frames are merged using object classifications and mutually closest geometric point metric, then scene integration accuracy is improved, but processing speed decreases
Solution Approach 1:
The patent performs preliminary actions by pre-processing image frames to extract object classifications and geometric points before the merging operation. This preparation includes identifying and labeling objects in advance, which streamlines the subsequent merging process and reduces computational burden during real-time processing.
Solution Approach 2:
The patent applies parameter changes by using object classifications as key parameters for matching and merging geometric points. By changing the approach from raw coordinate matching to classification-based merging with mutually closest point metrics, the system achieves both accuracy and efficiency in scene integration.
3Adaptability or versatility
If comprehensive semantic segmentation is performed to enable complex tasks in robotics, then task capability is improved, but system resource consumption increases
Solution Approach 1:
The patent extracts only the essential information needed for complex robotic tasks by performing semantic segmentation to identify object classifications and geometric points. Rather than processing complete image data, the system extracts and processes only the relevant semantic features, reducing resource consumption while maintaining task capability.
Solution Approach 2:
The patent creates a universal scene understanding framework that can support multiple complex tasks in robotics. By merging geometric points and object classifications into a unified 3D scene representation, the system enables versatile task execution including navigation, manipulation, and interaction with various objects, maximizing adaptability across different robotic applications.
Data Source
AI summary
A computer-implemented method is provided. The aspects include deriving, by one or more processors from a pixel-wise semantic segmentation of at least two image frames, object classifications representing one or more objects in the at least two image frames. The aspects further include deriving, by the one or more processors, geometric points representing the one or more objects in the at least two image frames. The aspects also include merging, by the one or more processors, the geometric points based on the object classifications that match and a mutually closest geometric point metric to obtain merged geometric points for each of the one or more objects. The aspects additionally include controlling movement of an autonomous object to achieve a task responsive to at least one of the one or more objects represented by the merged geometric points.