Airborne Target Tracking via Deep Integration of Appearance and Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High visual angles from drone-mounted cameras in airborne-based monitoring scenarios result in small target dimensions and insufficient appearance information, along with unpredictable target types and lens motions affecting tracking reliability.
Innovation Solution
A target tracking method using a deep integration network that combines appearance and motion tracking, employing twin convolutional neural networks for real-time tracking, where appearance and motion information are processed in parallel to derive a final tracking position, integrating hierarchical feature pyramids and frame difference responses to account for lens motions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a horizontal visual angle or low visual angle camera is used for monitoring, then the target dimension and appearance information are sufficient, but the coverage area is limited and deployment is less flexible
Solution Approach 1:
The patent changes the visual angle parameter from horizontal/low angle to high visual angle to achieve larger coverage area and better deployment flexibility, while compensating for the resulting loss of appearance information through deep integration network that fuses appearance and motion features
Solution Approach 2:
The patent adds a temporal dimension by using frame difference images from consecutive frames to supplement the insufficient appearance information from high visual angle shots, enabling the system to maintain tracking accuracy despite the reduced spatial appearance details
2Area of stationary object
If a high visual angle camera is used for airborne-based monitoring, then the coverage area is large and deployment is flexible, but the target dimension becomes small and appearance information is insufficient
Solution Approach 1:
The patent segments the tracking task into two independent streams: appearance tracking using template matching and motion tracking using frame difference images. This segmentation allows each stream to process information optimally without being constrained by the limited appearance quality from high visual angle shots
Solution Approach 2:
The patent merges the appearance tracking results and motion tracking results through a deep integration network that fuses both feature streams. This combining process integrates the complementary strengths of both tracking approaches to achieve reliable target positioning despite insufficient appearance information
3Adaptability or versatility
If the airborne-based platform is deployed flexibly and maneuverably, then the adaptability to various scenarios is improved, but lens motions occur that affect the reliability of target motion features
Solution Approach 1:
The patent introduces frame difference images as an intermediary that captures motion information relative to the previous frame. This intermediary representation helps distinguish true target motion from lens motion artifacts by analyzing changes in the image sequence rather than relying solely on absolute position changes
Solution Approach 2:
The patent replaces traditional mechanical motion estimation methods with a neural network-based motion tracking system that processes frame difference images. This substitution allows the system to learn and adapt to complex motion patterns while filtering out spurious lens motion effects through the deep integration network
Data Source
AI summary
The present disclosure provides a target tracking method and device oriented to airborne-based monitoring scenarios. The method includes: obtaining a video to-be-tracked of the target object in real time; extracting a first frame and a second frame; trimming and capturing the first frame to derive an image for first interest region; trimming and capturing the second frame to derive an image for target template and an image for second interest region; inputting the image for target template and the image for first interest region into an appearance tracker network to derive an appearance tracking position; inputting the image for first interest region and the image for second interest region into a motion tracker network to derive a motion tracking position; and finally inputting the appearance tracking position and the motion tracking position into a deep integration network to derive a final tracking position.