Camera Object Tracking Using Multi-Model Kalman Filters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing camera-based driver assistance systems face challenges in reliable state estimation of a preceding vehicle, particularly when the vehicle speed is zero, leading to observation problems, and require complex sensor fusion methods to overcome these issues.
Innovation Solution
The use of two extended Kalman filters with different models, combined through an interacting multi-model filter, to provide a robust state estimation, along with optical tracking using the Lukas-Kanade algorithm and a third Kalman filter for improved accuracy and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single extended Kalman filter is used for state estimation, then the system is simple, but the observation problem causes divergence at zero vehicle speed leading to unreliable estimation
Solution Approach 1:
The system divides the single filter into multiple specialized filters (first extended Kalman filter with nonlinear transition matrix, second extended Kalman filter with linear transition matrix, and third extended Kalman filter for optical tracking). Each filter handles specific scenarios, preventing divergence that occurs in a single unified filter when vehicle speed is zero.
Solution Approach 2:
The system changes the transition matrix parameters dynamically by selecting different models (nonlinear vs linear) based on the operating conditions. This allows the filter to adapt to zero-speed scenarios where the standard nonlinear model fails, improving reliability without requiring a completely complex system architecture.
2Reliability
If multiple sensors are fused to determine speed and angle directly, then observation problems are circumvented, but the system complexity and effort increase greatly
Solution Approach 1:
The system replaces physical sensor fusion (mechanically combining multiple sensor inputs) with a mathematical model substitution approach. Instead of fusing data from multiple sensors, the system substitutes different transition matrix models (linear and nonlinear) within the Kalman filter framework to handle different operating conditions, achieving similar reliability without the complexity of multi-sensor integration.
3Measurement precision
If optical tracking is used to track objects over multiple images, then detection accuracy improves, but computing effort increases and tracking may diverge due to background patterns
Solution Approach 1:
The system dynamically adjusts the tracking approach by using a third extended Kalman filter specifically for optical tracking that can be activated or deactivated based on conditions. This allows the system to benefit from optical tracking precision when needed while avoiding excessive computing effort and divergence issues when background patterns interfere.
Solution Approach 2:
The third extended Kalman filter acts as an intermediary between direct object detection and the main state estimation system. It processes optical tracking data and feeds results to the interacting multi-model filter, which weighs the reliability of different sources. This intermediary structure improves precision while managing computing effort through selective processing.
4Reliability
If detection is performed independently for each image, then tracking accuracy is maintained, but computing effort increases compared to optical tracking
Solution Approach 1:
The system merges two approaches by combining independent detection results with optical tracking results through the interacting multi-model filter. The Adaboost cascade method performs independent detection while the Lukas-Kanade algorithm performs optical tracking, and both are integrated through the third extended Kalman filter and IMM filter, achieving reliability without excessive computing effort.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
The invention relates to an object tracking method for a camera-based driver assistance system, which has a camera installed in its own vehicle and oriented in the direction of travel for recording images of a vehicle environment lying in front of the own vehicle in the direction of travel, comprising the steps of detecting an object representing a vehicle driving ahead in an image recorded by the camera and estimating a mean value and a covariance matrix for the current state of the vehicle driving ahead from the position and width of the object in the image using an extended Kalman filter.