Action Recognition via Temporal Motion Feature Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 2D convolutional neural networks have low accuracy in action recognition for video data, especially when the order of frames changes, as they fail to effectively capture temporal motion patterns and view variations.
Innovation Solution
The proposed method employs a multi-channel convolutional layer to extract original feature maps, followed by an action information enhancement module to calculate motion information weights and temporal convolution to generate temporal motion feature maps, which are then used for action recognition, incorporating a residual network for improved feature learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D convolutional neural network is used for action recognition, then the recognition process is simple, but the accuracy is low when frame order changes
Solution Approach 1:
The patent transitions from 2D spatial feature extraction to 3D spatiotemporal feature extraction by introducing temporal dimensions. The 3D convolutional neural network applies convolution operations across both spatial dimensions and temporal dimension, enabling the model to capture motion patterns and temporal relationships in video data, thereby improving action recognition accuracy while maintaining reasonable computational complexity
Solution Approach 2:
The patent introduces dynamic temporal modeling through the 3D convolutional layers that adaptively learn temporal patterns. The network dynamically adjusts to different action sequences and temporal variations, making the recognition system robust to frame order changes by learning inherent temporal dynamics rather than relying on fixed frame sequences
2Use of energy by moving object
If conventional 2D convolutional neural network is used, then computational resources are saved, but temporal motion patterns cannot be captured
Solution Approach 1:
The patent adds the temporal dimension to the convolutional operations, transforming 2D spatial convolutions into 3D spatiotemporal convolutions. This allows the network to simultaneously process spatial features and temporal motion patterns, capturing temporal information that would be lost in 2D processing while maintaining efficient computational throughput through optimized 3D convolution implementations
3Measurement precision
If 3D convolutional neural network is used, then action recognition accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the 3D convolutional network into multiple 2D convolutional layers that process different temporal frames separately, then combines their features through temporal pooling or attention mechanisms. This segmentation approach maintains action recognition accuracy by capturing spatiotemporal features while reducing overall model complexity and computational burden compared to a monolithic 3D convolutional network
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An action recognition method. The method comprises: acquiring original feature sub-images of image data of each timing frame in different convolutional channels by means of a multi-channel convolutional layer; respectively taking each timing frame as a target timing frame, calculating a motion information weight of the target timing frame in each convolutional channel according to an original feature sub-image of the target timing frame in each convolutional channel and an original feature sub-image of the next timing frame in each convolutional channel, and acquiring a motion information feature image of the target timing frame in each convolutional channel according to the motion information weight; performing timing convolution on the motion information feature image of the target timing frame in each convolutional channel to obtain a timing motion feature image of the target timing frame in each convolutional channel; and recognizing the action type of a moving object in the image data of the target timing frame according to the timing motion feature image of the target timing frame in each convolutional channel.