End-to-End Video Action Detection Framework
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video action detection technologies use two independent backbone networks for actor bounding box positioning and action classification, leading to redundant calculations, high complexity, and mutual interference between tasks.
Innovation Solution
An end-to-end framework is introduced, utilizing a unified backbone network, positioning module, and classification module to simultaneously extract actor location features and action category features from video clips, reducing complexity and interference between tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If two independent backbone networks are used for actor bounding box positioning and action classification, then task-specific performance is improved, but device complexity and computational redundancy increase
Solution Approach 1:
The patent merges two independent backbone networks into a single shared backbone network that simultaneously performs both actor bounding box positioning and action classification. This unified architecture shares feature extraction capabilities across both tasks, reducing computational redundancy and device complexity while maintaining task-specific performance through separate task modules that process the shared features independently
2Device complexity
If a unified backbone network is used to replace two independent backbone networks, then device complexity is reduced, but task interference occurs between positioning and classification
Solution Approach 1:
The patent segments the unified backbone network into distinct functional modules: a shared feature extraction backbone that processes video input, and separate task-specific modules for positioning and classification. This segmentation allows the single backbone to serve both tasks without interference by providing isolated processing paths that maintain task independence while sharing computational resources efficiently
3Speed
If 2D image models are used for actor bounding box localization in key frames, then localization speed is improved, but temporal motion representation quality deteriorates
Solution Approach 1:
The patent applies preliminary action by performing actor bounding box localization on key frames first to establish spatial positioning, then uses these localized results as input for subsequent action classification that leverages temporal information from video sequences. This preliminary localization step enables fast initial positioning while the subsequent temporal processing enhances motion representation accuracy
4Measurement precision
If 3D video models are used for action classification, then temporal information extraction is improved, but computational cost increases
Solution Approach 1:
The patent extracts temporal information selectively by using 3D video models only for action classification where temporal context is essential, rather than applying them to all processing stages. The system extracts key temporal features from video sequences for classification while using more computationally efficient 2D processing for other tasks, thereby achieving necessary temporal information extraction with reduced overall computational cost
Data Source
AI summary
The present invention provides a video action detection method and electronic equipment based on an end-to-end framework, which includes a backbone network, a positioning module and a classification module. The method includes: feature extraction of the video clip to be tested by the backbone network, obtaining the video feature map of the video clip, which includes the feature maps of all frames in the video clip; the backbone network extracts the feature map of the key frame from the video feature map and obtains the actor's location features from the feature map of the key frame, and the action category features are obtained from the video feature map; the positioning and classification modules determine the actor's location and action category respectively from the features extracted by the backbone network. This method provided by the present invention has low complexity while achieving better detection performance at the same time.