Iterative Video Activity Detection via Pseudo Temporal Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current weakly supervised temporal activity detection methods fail to accurately mine locating and semantic information from class activation sequences, resulting in low localization accuracy for action detection in videos.
Innovation Solution
A weakly supervised video activity detection method and system based on iterative learning, which extracts spatial-temporal features, constructs a neural network model group, trains the models using class labels and pseudo temporal labels, and selects the model with the highest detection accuracy for action detection, incorporating loss calculations and parameter updates to enhance localization accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If weakly supervised temporal activity detection methods use multi-instance learning or attention mechanism to output class activation sequence, then the method can classify video actions with less manual annotation, but the localization accuracy of actions remains relatively low
Solution Approach 1:
The patent divides the video into multiple clips and processes each clip through the neural network model to generate class activation sequences for each clip. This segmentation allows the model to learn temporal patterns at the clip level while maintaining weak supervision, thereby improving localization accuracy without requiring frame-level annotations.
Solution Approach 2:
The patent transforms the class activation sequence from a one-dimensional temporal representation into a two-dimensional structure by organizing activation scores across multiple clips and action classes. This dimensional transformation enables the model to capture both temporal and semantic relationships, improving localization precision while maintaining annotation efficiency.
2Productivity
If the neural network model is trained only with class labels without temporal information, then the training process is simpler and faster, but the model cannot accurately locate action boundaries in time
Solution Approach 1:
The patent performs preliminary feature extraction on video clips before training the neural network model. By pre-processing the video into clips and extracting initial features, the model can focus on learning temporal patterns during training rather than processing raw video data, thus maintaining training speed while improving temporal localization capability.
Solution Approach 2:
The patent uses the class activation sequences generated by the model as feedback to guide further training and refinement. The activation sequences provide temporal localization signals that are fed back into the model during iterative training, enabling the model to progressively improve its temporal boundary detection without requiring manual temporal annotations.
Data Source
AI summary
The present disclosure relates to a weakly supervised video activity detection method and system based on iterative learning. The method includes: extracting spatial-temporal features of a video that contains actions; constructing a neural network model group; training a first neural network model according to the class label of the video, a class activation sequence output by the first neural network model, and a video feature output by the first neural network model; training the next neural network model according to the class label of the video, a pseudo temporal label output by the current neural network model, a class activation sequence output by the next neural network model, and a video feature output by the next neural network model; and performing action detection on the test video according to the neural network model corresponding to the highest detection accuracy value.