Multi-Sensor Object Tracking With Unified Detection for Stable Association
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately tracking objects due to discrepancies between sensor data from different types, leading to jittery or flickering object representations and increased computing bandwidth and memory usage, especially when dealing with erroneous depth measurements from sensors like depth cameras.
Innovation Solution
The implementation of a system that aggregates data from multiple sensor types, such as lidar, vision, and radar, into a multi-channel image, which is then input into a machine-learning model to generate a unified estimated object detection, reducing the need for direct comparison of object detections and stabilizing track associations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If object detections from multiple sensor types are directly compared and processed, then tracking accuracy may be maintained, but computing bandwidth and memory usage increase significantly
Solution Approach 1:
The patent extracts only the essential tracking information (position, velocity, acceleration, heading) from sensor data and processes this extracted data through the ML model, rather than processing complete object detections from all sensors. This extraction approach maintains tracking accuracy while significantly reducing computing bandwidth and memory requirements.
Solution Approach 2:
Instead of comparing object detections from multiple sensors and then processing them, the patent inverts the approach by having the ML model directly process the raw sensor data to generate tracking information. This eliminates the need for direct comparison of multiple detection results and reduces computational complexity.
2Stability of the object's composition
If sensor data from multiple types is processed to reduce discrepancies, then object representation stability improves, but computing resources are consumed
Solution Approach 1:
The patent changes the parameters being processed from complete object detections (which include many varying attributes) to specific tracking parameters (position, velocity, acceleration, heading) that are more stable and essential. The ML model is trained to output these specific parameters directly, reducing the need to process and reconcile multiple detection attributes while improving representation stability.
Solution Approach 2:
The patent replaces the mechanical approach of directly comparing and reconciling detections from multiple sensors with an ML-based system that learns to process sensor data and generate stable tracking information. This substitution allows the system to handle sensor discrepancies more efficiently without consuming excessive computing resources.
3Adaptability or versatility
If depth camera data is used for object detection, then perception coverage is improved, but erroneous depth measurements cause large variations in tracking
Solution Approach 1:
The patent introduces an ML model as an intermediary between the depth camera data and the tracking output. The ML model processes the depth data along with data from other sensors and learns to compensate for depth measurement errors, producing more accurate and stable tracking information while maintaining the perception coverage benefits of using depth cameras.
Solution Approach 2:
The system uses feedback from multiple sensor types to correct erroneous depth measurements. The ML model receives data from lidar, radar, and other sensors alongside depth camera data, and uses this multi-sensor feedback to identify and correct depth measurement errors, maintaining both perception coverage and measurement precision.
Data Source
AI summary
Tracking a current and/or previous position, velocity, acceleration, and/or heading of an object using sensor data may comprise determining whether to associate a current object detection generated from recently received (e.g., current) sensor data with a previous object detection generated from formerly received sensor data. In other words, a track may identify that an object detected in former sensor data is the same object detected in current sensor data. However, multiple types of sensor data may be used to detect objects and some objects may not be detected by different sensor types or may be detected differently, which may confound attempts to track an object. An ML model may be trained to receive outputs associated with different sensor types and/or a track associated with an object, and determine a data structure comprising a region of interest, object classification, and/or a pose associated with the object.