Camera-IMU Fusion Tracking for Dynamic Object Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current pose estimation methods for mobile platforms using visual information from cameras have low accuracy due to noise, and existing multi-state constraint Kalman filters (MSCKF) can only estimate the pose of the camera and not a target object in a moving state.
Innovation Solution
A method that combines inertial measurement units (IMUs) with camera frames to predict state vectors for both the camera and target objects, using Kalman filters and image features to update and track the target object's pose, even when partially blocked, by employing a representation point and non-representation points on the target object.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If visual information from camera is used for pose estimation, then the system can operate without additional sensors, but the accuracy of pose estimation deteriorates due to noise
Solution Approach 1:
The patent combines camera visual information with IMU (inertial measurement unit) data to estimate the pose of both the camera and target objects. This fusion of multiple sensor types compensates for the noise in visual information alone, improving measurement precision while accepting increased device complexity.
Solution Approach 2:
The patent introduces an intermediary computational framework that processes both camera and IMU data through a unified state estimation algorithm. This intermediary processing layer integrates the strengths of both sensors and resolves their individual weaknesses, achieving accurate pose estimation for multiple objects simultaneously.
2Measurement precision
If MSCKF is used for pose estimation, then the accuracy of camera pose is improved, but the ability to estimate target object pose deteriorates
Solution Approach 1:
The patent extends the MSCKF algorithm to serve multiple functions: it estimates the pose of the camera platform and simultaneously estimates the pose of target objects in the scene. This multi-functional extension allows a single algorithm to handle both platform navigation and object tracking, improving versatility without sacrificing camera pose accuracy.
Solution Approach 2:
The patent segments the state vector into distinct components: camera pose parameters and target object pose parameters. This segmentation allows the algorithm to independently process and estimate each type of pose while maintaining a unified probabilistic framework, enabling both functions to coexist and be optimized simultaneously.
3Measurement precision
If a representation point is used for target tracking, then the tracking accuracy is improved, but the robustness deteriorates when the point is blocked
Solution Approach 1:
The patent applies local quality by using different types of points (representation points and non-representation points) with different functions on the target object. Representation points provide accurate pose estimation when visible, while non-representation points serve as backup and provide continuity when representation points are occluded, creating a heterogeneous set of features with specialized roles.
Solution Approach 2:
The patent prepares in advance by selecting multiple points on the target object before tracking begins. When occlusion occurs, the system can switch to previously selected alternative points without interruption, cushioning against the harmful effect of blocking and maintaining continuous tracking robustness.
Data Source
AI summary
This disclosure provides a moving object tracking method and apparatus. The method includes: obtaining a current frame captured by a camera; predicting a current state vector of the camera based on an inertial measurement unit IMU and the current frame, to obtain a predicted value of the current state vector of the camera; predicting a current state vector of a target object that is moving in the current frame, to obtain a predicted value of the current state vector of the target object; and updating a Kalman state vector based on a measurement result of an image feature in the current frame. According to technical solutions provided in this disclosure, a target object that is moving in a surrounding environment can be tracked and a pose of the target object can be estimated while a pose of a system can be estimated.