Occlusion-Gated Sensor Fusion for Accurate Object Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sensor fusion systems face challenges in handling occlusions where data from one sensor is fused with data from another sensor, leading to mixing of data from occluded objects, which can introduce noise and reduce accuracy in perception tasks.
Innovation Solution
An occlusion resolving gated mechanism is implemented to diminish sensor data from occluded objects by aligning sensor data to a common coordinate frame, binning points based on distance, and using machine learning models to deemphasize points that do not match segmented objects, thereby reducing noise and improving perception accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If sensor data from multiple sensors is fused into a single model, then the completeness of environmental representation is improved, but noise and reduced accuracy occur due to mixing of data from occluded and foreground objects
Solution Approach 1:
The patent segments sensor data into foreground and background components using depth information and occlusion detection. By dividing the point cloud data into distinct segments corresponding to different depth ranges and occlusion states, the system can process and fuse data from visible and occluded objects separately, preventing noise mixing while maintaining complete environmental representation.
Solution Approach 2:
The patent introduces depth information and occlusion masks as intermediary elements between raw sensor data and the fused environmental model. These intermediaries enable the system to distinguish between foreground and background objects, selectively fuse data based on occlusion status, and maintain accuracy while achieving complete environmental coverage.
2Device complexity
If sensor data fusion is performed without occlusion handling, then processing complexity is reduced, but noise increases due to mixing of occluded and foreground object data
Solution Approach 1:
The patent performs preliminary occlusion detection and depth-based segmentation before the sensor data fusion process. By pre-identifying occluded regions and separating foreground/background points using depth information, the system eliminates noise at an early stage, allowing subsequent fusion operations to proceed with reduced complexity and without introducing additional noise.
Solution Approach 2:
The patent applies different processing qualities to different regions of the sensor data based on occlusion status. Visible regions receive standard fusion processing, while occluded regions undergo specialized handling with adjusted weighting and filtering. This local differentiation reduces overall noise while maintaining manageable processing complexity through region-specific optimization.
Data Source
AI summary
Techniques and systems are provided for processing sensor data. For instance a process can include obtaining first sensor data of an environment, wherein the first sensor data includes a representation of a first object occluding a second object, obtaining second sensor data of the environment, wherein the second sensor data includes points associated with the first object and points associated with the second object, generating estimated segment data from the first sensor data, wherein the estimated segment data includes a first segment corresponding to the first object; matching points associated with the first object to the first segment, and deemphasizing points associated with the second object based on matching the points associated with the first object to the first segment.