Attention mechanism-based video classification method

A technology of video classification and attention, applied in the field of optical communication, can solve the problems of unfavorable video content recognition, loss of timing information of video features, etc., to achieve the effect of improving the accuracy rate

Inactive Publication Date: 2017-11-10
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF11 Cites 102 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Although this method considers all the video information at each moment, the average method makes the video features lose the timing information, which is not conducive to the identification of video content.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Attention mechanism-based video classification method
  • Attention mechanism-based video classification method
  • Attention mechanism-based video classification method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0041] For the convenience of description, the relevant technical terms appearing in the specific implementation are explained first:

[0042] CNN (Convolutional Neural Network): convolutional neural network;

[0043] LSTM (Long Short-Term Memory): long short-term memory network;

[0044] BPTT (Back Propagation Through Time): time backpropagation algorithm;

[0045] figure 1 It is a flow chart of the video classification method based on the attention mechanism of the present invention.

[0046] In this embodiment, the UCF-101 dataset is downloaded from the CRCV official website as a sample video for training. The UCF-101 dataset contains C=101 category videos, such as ApplyEyeMakeup, ApplyLipstick, ... YoYo, etc., each category corresponds to a Video IDs, as shown in Table 1, are arranged in alphabetical order. For example, the ID of ApplyEyeMakeup is (1,0,0...0), the ID of ApplyLipstick is (0,1,0...0), and the ID of YoYo is (0, 0,0...1), the logo is a C-dimensional vector...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an attention mechanism-based video classification method. The attention mechanism-based video classification method comprises the steps of extracting space characteristic of a video by a convolution neural network (CNN), combining all space characteristics by attention weight, sending all space characteristics to a long- and short-time memory (LSTM) network, extracting time characteristics of the video, and finally, classifying video contents by employing a multi-classification function. An attention mechanism introduced to the LSTM network can be used for simulating an identification function of a human brain, different video contents are distinguished and handled, and the video classification accuracy is effectively improved.

Description

technical field [0001] The invention belongs to the technical field of optical communication, and more specifically relates to a video classification method based on an attention mechanism. Background technique [0002] In recent years, thanks to the powerful feature extraction capabilities of deep learning, breakthroughs have been made in the identification and analysis of video content. The core of video content recognition lies in the extraction of video features. Video features are the physical properties of the video itself, which can reflect the video content from different angles. Karpathy uses the convolutional neural network to extract the spatial features of the video, and expands the convolutional neural network in the time dimension to extract the space-time features of the video. Ji directly extends the two-dimensional convolution kernel to the three-dimensional convolution kernel to capture the motion information between adjacent frames. Combining neurologica...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/08
CPCG06N3/08G06V20/41G06V20/46G06F18/241
Inventor 徐杰何庆强李林科余兴
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap