3D Object Tracking With Joint Detection and Trajectory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently and accurately detecting and tracking objects in a surrounding environment, particularly for autonomous vehicles, due to the complexity of processing sensor data from various sources and the need for improved object association across multiple frames.
Innovation Solution
A machine-learned model comprising one or more first neural networks for object detection and second neural networks for object tracking is jointly trained using backpropagation, combined with a flow network and a trajectory linear program to optimize object trajectories, leveraging sensor data from LIDAR and RGB inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional separate systems are used for object detection and tracking, then system complexity is reduced, but object association accuracy across multiple frames deteriorates
Solution Approach 1:
The patent combines separate object detection and tracking systems into a unified machine-learned model that processes sensor data end-to-end. The model integrates detection networks and tracking networks into a single architecture with shared features, enabling joint optimization of detection and tracking while maintaining association accuracy across multiple frames.
2Measurement precision
If end-to-end machine-learned models are used for joint detection and tracking, then object association accuracy improves, but computational resources required increase
Solution Approach 1:
The unified model is divided into distinct functional components: detection networks for identifying objects in individual frames, tracking networks for associating objects across frames, and a linear program for optimizing trajectories. This segmentation allows selective processing and efficient resource utilization while maintaining end-to-end training benefits.
Solution Approach 2:
The model employs a linear program to optimize trajectory parameters by adjusting matching scores and association probabilities. This parameter optimization approach reduces computational burden compared to exhaustive search methods while preserving high association accuracy through mathematically optimal trajectory selection.
3Measurement precision
If complex processing techniques are applied to sensor data, then detection accuracy improves, but processing speed deteriorates
Solution Approach 1:
The detection networks extract features and generate object proposals in advance before the tracking and trajectory optimization stages. This preliminary processing organizes data into structured formats with pre-computed features, reducing the computational load on subsequent tracking and association operations while maintaining detection accuracy.
Data Source
AI summary
Systems and methods for detecting and tracking objects are provided. In one example, a computer-implemented method includes receiving sensor data from one or more sensors. The method includes inputting the sensor data to one or more machine-learned models including one or more first neural networks configured to detect one or more objects based at least in part on the sensor data and one or more second neural networks configured to track the one or more objects over a sequence of sensor data. The method includes generating, as an output of the one or more first neural networks, a 3D bounding box and detection score for a plurality of object detections. The method includes generating, as an output of the one or more second neural networks, a matching score associated with pairs of object detections. The method includes determining a trajectory for each object detection.