Partially Supervised Online Action Detection via Temporal Proposals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing online action detection systems rely on segment-level annotated data for training, which is time-consuming and costly, hindering their scalability.
Innovation Solution
A partially supervised training model for online action detection using a Temporal Proposal Generator (TPG) and an Online Action Recognizer (OAR) that can be trained with video-level labels only, without pre-annotated segment-level boundary labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If segment-level annotated data is used for training online action detection systems, then detection accuracy is improved, but annotation time and cost increase significantly
Solution Approach 1:
The system performs offline pre-training on segment-level annotated data to learn temporal action patterns, then uses this pre-learned knowledge for online detection without requiring real-time segment-level annotations. This preliminary action allows the system to achieve high detection accuracy while avoiding time-consuming annotations during deployment.
Solution Approach 2:
The system uses video-level annotations (partial supervision) instead of complete segment-level annotations for online training. By applying partial supervision with video-level labels and using techniques like temporal proposal generation and selective search, the system achieves acceptable detection accuracy with significantly reduced annotation requirements.
2Measurement precision
If segment-level annotated data is used for training online action detection systems, then detection accuracy is improved, but scalability is reduced
Solution Approach 1:
The system adopts partial supervision using video-level annotations instead of requiring complete segment-level annotations. This approach dramatically reduces the annotation burden and enables scalable deployment across large video datasets while maintaining acceptable detection accuracy through techniques like temporal proposal generation and online action recognition with video-level labels.
Solution Approach 2:
The system uses automatically generated temporal proposals and online action recognition to reduce dependency on manual segment-level annotations. By leveraging self-generated temporal information from video-level labels and online processing, the system achieves scalability without sacrificing core detection functionality.
3Measurement precision
If offline analysis is used for action detection, then detection accuracy is improved, but real-time detection capability is lost
Solution Approach 1:
The system segments the detection process into offline pre-training phase and online detection phase. During offline pre-training, the system learns temporal action patterns from segment-level annotated data to improve accuracy. During online detection, it processes video frames sequentially in real-time using the learned patterns, achieving both accuracy and real-time performance.
Solution Approach 2:
The system performs preliminary offline training to learn temporal action patterns and features before online deployment. This preliminary action allows the model to achieve high detection accuracy during offline training, then maintains real-time detection capability during online operation by using the pre-learned knowledge without requiring access to future video segments.
Data Source
AI summary
Embodiments described herein provide systems and methods for a partially supervised training model for online action detection. Specifically, the online action detection framework may include two modules that are trained jointly—a Temporal Proposal Generator (TPG) and an Online Action Recognizer (OAR). In the training phase, OAR performs both online per-frame action recognition and start point detection. At the same time, TPG generates class-wise temporal action proposals serving as noisy supervisions for OAR. TPG is then optimized with the video-level annotations. In this way, the online action detection framework can be trained with video-category labels only without pre-annotated segment-level boundary labels.