Unlock instant, AI-driven research and patent intelligence for your innovation.

A Video Action Recognition Method Based on Spatiotemporal Fusion Features and Attention Mechanism

A technology of time-space fusion and recognition method, applied in character and pattern recognition, computer parts, instruments, etc., can solve problems such as inability to handle sequence problems, and achieve the effect of improving accuracy

Active Publication Date: 2022-03-25
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Although convolutional neural networks can extract spatial features of videos, they cannot handle sequence problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Video Action Recognition Method Based on Spatiotemporal Fusion Features and Attention Mechanism
  • A Video Action Recognition Method Based on Spatiotemporal Fusion Features and Attention Mechanism
  • A Video Action Recognition Method Based on Spatiotemporal Fusion Features and Attention Mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0050] For the convenience of description, the relevant technical terms appearing in the specific implementation manner are explained first:

[0051] LSTM (Long Short-Term Memory): long short-term memory network;

[0052] figure 1 This is the flow chart of the video behavior recognition method based on the spatiotemporal fusion feature and the attention mechanism of the present invention.

[0053] In this embodiment,

[0054] The LSVRC2012 dataset is used for pre-training of the Inception V3 network, and the HMDB-51 and UCF-101 datasets are used for model simulation and validation analysis.

[0055] The HMDB-51 dataset contains 6849 videos, mainly from movie clips, divided into 51 categories, of which 5222 are used as training set, 300 are used as validation set, and 1327 are used as test set.

[0056] The UCF-101 dataset is a video action recognition dataset collected from real life. The video content is all derived from YouTube videos, including 13,320 videos and a total ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a video behavior recognition method based on spatio-temporal fusion features and attention mechanism. The spatio-temporal fusion features of the input video are extracted through the convolutional neural network Inception V3, and then combined with the human visual system on the basis of the spatio-temporal fusion features. The attention mechanism enables the network to automatically assign weights according to the video content, extract the key frames in the video frame sequence, and recognize the behavior from the video as a whole, so as to eliminate the interference of redundant information on the recognition and improve the performance of video behavior recognition. Accuracy.

Description

technical field [0001] The invention belongs to the technical field of behavior recognition, and more particularly, relates to a video behavior recognition method based on spatio-temporal fusion features and attention mechanism. Background technique [0002] Research related to behavior recognition is increasingly widely cited in many application scenarios, such as security monitoring, autonomous driving, video retrieval, etc. Behavior recognition generally refers to identifying the behavior of individuals or groups from video sequences. Often the specific behavior occurs on a continuous sequence of video frames, not just a single video frame. Therefore, the motion information in the video is very important for the recognition of the behavior, and how to effectively characterize the spatiotemporal features in the video is a hotspot in the field of behavior recognition research. [0003] Traditional action recognition relies on handcrafted features extracted from video fram...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06V20/40G06V40/20G06V10/82G06N3/04
CPCG06V40/20G06V20/41G06V20/46G06N3/045
Inventor 徐杰余兴盛纾纬魏号亮
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA