Sensor Fusion Logic for 3D Object Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated systems for object detection, identification, and localization in environments, such as robots and autonomous vehicles, face inefficiencies due to incorrect reconciliation of detections from multiple sensors, failure to recognize objects within cells, and incorrect localization when merging detection data in two-dimensional space.
Innovation Solution
A system that includes a compute device with a sensor fusion logic unit, which selectively applies weights to data from multiple sensors like cameras and lidar based on context and sensor attributes, and applies user-defined rules to merge bounding boxes, determining degrees of belief for object representation, and performs object localization using depth information and belief functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If detection data from multiple sensors is merged in two-dimensional space, then processing complexity is reduced, but localization accuracy deteriorates
Solution Approach 1:
The patent transitions from two-dimensional merging to three-dimensional merging by incorporating depth information from sensors such as LIDAR and stereo cameras. The sensor fusion logic unit merges detection data in 3D space using bounding volumes (e.g., 3D bounding boxes) that include depth dimensions, thereby maintaining localization accuracy while accounting for the additional computational complexity through efficient data structures and algorithms.
2Reliability
If bounding box proposals from multiple sensors are reconciled by rejection, then false positives are reduced, but detection reliability deteriorates
Solution Approach 1:
Instead of rejecting bounding box proposals, the patent merges detection data from multiple sensors by combining bounding boxes into unified bounding volumes in three-dimensional space. The sensor fusion logic unit integrates detections from different sensors (e.g., camera, LIDAR, radar) by spatially aligning them and merging overlapping regions, which preserves true positives while filtering false positives through multi-sensor consensus rather than simple rejection.
3Productivity
If object recognition is performed within cells, then detection speed is improved, but recognition accuracy deteriorates
Solution Approach 1:
The patent segments the environment into discrete cells or grid positions in three-dimensional space, with each cell containing detection data from multiple sensors. The sensor fusion logic unit performs recognition within each cell by aggregating sensor data spatially, maintaining detection speed through localized processing while improving recognition accuracy by combining multi-sensor information within each segmented cell rather than processing the entire scene uniformly.
Data Source
AI summary
Technologies for performing sensor fusion include a compute device. The compute device includes circuitry configured to obtain detection data indicative of objects detected by each of multiple sensors of a host system. The detection data includes camera detection data indicative of a two or three dimensional image of detected objects and lidar detection data indicative of depths of detected objects. The circuitry is also configured to merge the detection data from the multiple sensors to define final bounding shapes for the objects.