3D Object Tracking With Joint Detection and Trajectory Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently and accurately detecting and tracking objects in a surrounding environment, particularly for autonomous vehicles, due to the complexity of processing sensor data and the need for improved object detection and association techniques.
Innovation Solution
A machine-learned model is employed that includes jointly trained first neural networks for object detection and second neural networks for object tracking, utilizing a flow network and a trajectory linear program to optimize object trajectories, leveraging sensor data from LIDAR and RGB cameras, and applying deep learning for improved accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional object detection and tracking systems are used, then object detection can be performed, but computational resources are excessive and processing efficiency is low
Solution Approach 1:
The patent combines object detection and object tracking into a single unified neural network model. The detector component and tracker component are jointly trained through backpropagation, allowing the system to perform both functions simultaneously with shared computational resources, thereby improving processing efficiency while reducing overall computational burden compared to separate systems.
Solution Approach 2:
The unified neural network model serves multiple functions: it performs both object detection (identifying objects in sensor data) and object tracking (associating objects across sequential sensor data). This multi-functional approach eliminates the need for separate specialized systems, optimizing resource utilization while maintaining high processing efficiency.
2Measurement precision
If separate detection and tracking systems are used, then each function can be optimized independently, but the overall system complexity increases
Solution Approach 1:
By merging detection and tracking into a single unified neural network, the patent reduces system complexity while maintaining precision. The shared architecture and joint training process create synergies between detection and tracking functions, allowing both to achieve high accuracy without the overhead of managing separate independent systems.
3Measurement precision
If conventional object association methods are used, then object matching can be performed, but tracking accuracy over sequential data is insufficient
Solution Approach 1:
The unified neural network employs feedback through backpropagation, where tracking errors are propagated back to adjust detection parameters and vice versa. This continuous feedback loop during joint training enables the system to learn optimal detection and tracking strategies together, significantly improving tracking accuracy and object association reliability over sequential sensor data.
Solution Approach 2:
The system performs preliminary detection to generate candidate objects, then uses the tracker component to associate these candidates across sequential frames. The flow network component pre-processes potential associations before final trajectory determination, enabling accurate tracking by preparing and refining candidate matches in advance.
Data Source
AI summary
Systems and methods for detecting and tracking objects are provided. In one example, a computer-implemented method includes receiving sensor data from one or more sensors. The method includes inputting the sensor data to one or more machine-learned models including one or more first neural networks configured to detect one or more objects based at least in part on the sensor data and one or more second neural networks configured to track the one or more objects over a sequence of sensor data. The method includes generating, as an output of the one or more first neural networks, a 3D bounding box and detection score for a plurality of object detections. The method includes generating, as an output of the one or more second neural networks, a matching score associated with pairs of object detections. The method includes determining a trajectory for each object detection.