Motion video classification method and system based on multilevel motion modeling

A video classification, multi-level technology, applied in neural learning methods, character and pattern recognition, biological neural network models, etc., can solve the problems of large video frame jumps, only considering inter-segment motion information, and affecting video classification effects, etc., to achieve The effect of improving expressive ability

Active Publication Date: 2022-05-13
ZHEJIANG LAB
View PDF8 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the problem with these methods is that only the inter-segment motion information is considered, and it is difficult to carry out effective motion modeling due to the large jump of video frames between each segment, which affects the effect of video classification

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Motion video classification method and system based on multilevel motion modeling
  • Motion video classification method and system based on multilevel motion modeling
  • Motion video classification method and system based on multilevel motion modeling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] Specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention.

[0028]The method of the present invention uses the Pytorch framework for experiments, and uses a stochastic gradient descent SGD optimizer with an initial learning rate of 0.01 and a MultiStepLR scheduler. Set training on the Something-Something V1 dataset for 60 iterations, and adjust the learning rate at the 30th, 45th, and 55th iterations. The batch size is 64, the number of video segments , both branches of the network are initialized using the ResNet50 model pre-trained on ImageNet, where the 1D channel-by-channel convolution in each layer is initialized in a manner equivalent to the Temporal Shift operation in the TSM network. Following common settings,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an action video classification method and system based on multilevel motion modeling, which is used for performing multilevel comprehensive modeling on intra-segment and inter-segment motion information and comprises two neural network branches, the extraction module is used for extracting appearance information and inter-segment motion information of a foreground target; and the intra-segment branch processes a difference value of adjacent video frames in each video segment and is used for extracting intra-segment motion information of the foreground target. The frame difference features extracted by the intra-segment branches are used for weighting the inter-segment branch features according to channels, and finally the convolution features of the two branches are fused and jointly input into a classifier for video classification. The method is easy and convenient to implement, the means is flexible, and the remarkable classification effect improvement is achieved on the action video data set.

Description

technical field [0001] The invention relates to the technical field of video classification, in particular to an action video classification method and system based on multi-level motion modeling. Background technique [0002] With the popularization of cameras and the explosion of video applications (such as vibrato, etc.), video accounts for an increasing proportion of network data, and the research on the task of classification of action videos is important in technologies such as intelligent monitoring, automatic driving, and human-computer interaction. The field has important application value. After 2012, video classification methods based on deep learning, especially convolutional neural networks (CNN), have gradually replaced traditional hand-designed features (such as IDT, etc.). There are two main ideas for processing video data: [0003] One is to model continuous video segments, the main methods are 3D convolution, (2+1)D convolution, etc. Among them, (2+1)D c...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06V20/40G06N3/04G06N3/08
CPCG06N3/08G06N3/045
Inventor 卢修生鲍虎军程乐超杨非宋明黎
Owner ZHEJIANG LAB
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap