Joint 2D and 3D Object Tracking for Noisy Position Estimates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object tracking solutions for autonomous and semi-autonomous systems often rely heavily on 3D-centric methods, leading to inaccuracies and object misses due to the lack of integration of 2D image features, especially when 3D position estimates from DNNs are noisy.
Innovation Solution
A joint 2D and 3D object tracking approach that simultaneously tracks objects in both 2D image space and 3D world space, using a framework that includes state prediction, association, and update operations utilizing both 2D and 3D features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D-centric tracking methods are used, then 3D position estimation is achieved, but tracking accuracy deteriorates due to noisy 3D position estimates from DNNs
Solution Approach 1:
The patent merges 2D image space tracking and 3D world space tracking into a unified joint tracking framework. The tracker simultaneously maintains and updates both 2D bounding box positions and 3D object positions, using fusion operations to combine measurements from both spaces. This integration allows the system to leverage the complementary strengths of both 2D visual features and 3D spatial reasoning, resolving the contradiction by combining multiple measurement sources to achieve both accurate 3D estimation and reliable tracking.
Solution Approach 2:
The tracking system is designed with multi-functionality to handle both 2D image-based detection and 3D world-based positioning within a single unified framework. The tracker can operate in both 2D and 3D modes and automatically switches or fuses between them based on the situation, making the system universally applicable to various tracking scenarios while maintaining high accuracy and reliability.
2Productivity
If 3D-only tracking is used, then computational efficiency is improved, but tracking accuracy deteriorates due to lack of 2D image features
Solution Approach 1:
The patent segments the tracking process into distinct 2D and 3D processing streams that operate somewhat independently but are fused through association operations. The 2D stream handles image-based detection and tracking while the 3D stream handles spatial reasoning and position estimation. This segmentation allows each stream to be optimized for its specific task while maintaining overall computational efficiency, yet achieving high accuracy through the fusion of both streams.
3Device complexity
If 2D image features are not integrated, then system complexity is reduced, but tracking robustness deteriorates leading to object misses and inaccurate signals
Solution Approach 1:
The patent introduces a new dimensional aspect to tracking by simultaneously operating in both 2D image space and 3D world space. Instead of choosing one dimension over the other, the system adds another dimension of operation, creating a multi-dimensional tracking space where 2D visual features and 3D spatial information are integrated. This dimensional expansion increases robustness by providing multiple references for tracking, while the systematic organization of these dimensions keeps complexity manageable.
Data Source
AI summary
In various examples, techniques for multi-dimensional tracking of objects using two-dimensional (2D) sensor data are described. Systems and methods may use first image data to determine a first 2D detected location and a first three-dimensional (3D) detected location of an object. The systems and methods may then determine a 2D estimated location using the first 2D detected location and a 3D estimated location using the first 3D detected location. The systems and methods may use second image data to determine a second 2D detected location and a second 3D detected location of a detected object, and may then determine that the object corresponds to the detected object using the 2D estimated location, the 3D estimated location, the second 2D detected location, and the second 3D detected location. The systems and method then generate, modify, delete, or otherwise update an object track that includes 2D state information and 3D state information.