Object Tracking Distillation to Prevent Catastrophic Forgetting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object detection and tracking training sets lack necessary supervisions, leading to inconsistent learning and catastrophic forgetting when jointly trained, resulting in sub-optimal video feature representations and limited object recognition.
Innovation Solution
A student-teacher network framework is employed, where a teacher network is trained on an augmented object detection training set with object tracking supervision, and a student network is trained using a knowledge distillation loss based on the teacher's output, enabling joint training without forgetting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a machine learning model is trained using both object detection training sets and object tracking training sets, then the model can recognize a larger vocabulary of objects, but catastrophic forgetting occurs and the model loses information learned from the object detection training set
Solution Approach 1:
The training process is segmented into two distinct phases: a first training phase using only the object detection training set to learn detection information, and a second training phase using the object tracking training set to learn tracking information. This temporal segmentation prevents catastrophic forgetting by separating the learning of different task types.
Solution Approach 2:
The model learns object detection capabilities in advance during the first training phase before being trained on object tracking. This preliminary action ensures that detection information is firmly established before the tracking training begins, preventing information loss.
2Adaptability or versatility
If object detection and tracking are trained jointly with existing training sets, then both tasks can be performed, but inconsistent learning occurs leading to sub-optimal video feature representations
Solution Approach 1:
The joint training problem is segmented into two sequential phases with different training sets: first phase uses object detection training sets for detection supervision, second phase uses object tracking training sets for tracking supervision. This avoids the inconsistent learning that occurs when both tasks are trained simultaneously with inadequate supervision.
Solution Approach 2:
Object detection features are learned in advance during the first training phase, providing a solid foundation for subsequent tracking training. This preliminary feature learning ensures optimal video feature representations before tracking-specific training begins.
3Reliability
If a model is trained on object tracking data without sufficient detection supervision, then tracking performance improves, but the model lacks the detection supervisions present in object detection training sets
Solution Approach 1:
The training curriculum is segmented so that detection supervision is provided in the first phase, establishing strong detection capabilities. Then tracking training occurs in the second phase, building upon the detection foundation without losing detection supervision benefits.
Solution Approach 2:
Detection supervision is applied in advance during the first training phase, ensuring the model learns robust object detection capabilities before tracking training begins. This preliminary detection learning prevents loss of detection supervision information.
Data Source
AI summary
Systems and methods for object tracking are described. One or more aspects of the systems and methods include receiving a video depicting an object; generating object tracking information for the object using a student network, wherein the student network is trained in a second training phase based on a teacher network using an object tracking training set and a knowledge distillation loss that is based on an output of the student network and the teacher network, and wherein the teacher network is trained in a first training phase using an object detection training set that is augmented with object tracking supervision data; and transmitting the object tracking information in response to receiving the video.