Video action recognition method and system of multi-level feature fusion model based on hybrid convolution

Abstract

The invention relates to a video action recognition method and system of a multi-level feature fusion model based on hybrid convolution, and belongs to the technical field of computer vision. The method comprises the steps: constructing a hybrid convolution module through employing two-dimensional convolution and separable three-dimensional convolution; performing channel shift operation on each input feature along the time dimension, constructing a time shift module, promoting information flow between adjacent frames, and compensating the defect of capturing the dynamic feature by the two-dimensional convolution operation; exporting multi-level complementary features from different convolutional layers of the backbone network, and performing spatial modulation and time modulation on the multi-level complementary features, so that the features of each level have consistent semantic information in the spatial dimension and have variable visual rhythm clues in the time dimension; according to the method, feature flows from bottom to top and feature flows from top to bottom are constructed, so that the features supplement each other, and the feature flows are processed in parallel to realize multi-level feature fusion; and carrying out model training by utilizing a two-stage training strategy.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and relates to a video action recognition method and system based on a hybrid convolution multi-level feature fusion model. Background technique [0002] The rapid development of the field of artificial intelligence research has prompted human-computer interaction technology to gradually penetrate into people's daily life, and the research on human action recognition derived from it has received extensive attention. In video-based action recognition tasks, traditional methods mainly rely on specific feature design, which has severe domain limitations. In order to overcome the above defects and obtain a more general feature representation, Convolutional Neural Networks (CNN) based on biological visual perception mechanisms have been widely used in the field of action recognition. [0003] The performance of the model on human action recognition is closely related to its ability to represe...

AI Technical Summary

Problems solved by technology

Although pre-defining visual rhythm changes at the input level can significantly improve the model recognition effect, the model complexity increases significantly due to the parameter training involving multiple network branches

Images