Lightweight video action recognition network, method and system based on deep learning

Abstract

The invention discloses a lightweight video action recognition network, method and system based on deep learning, and provides a multi-dimensional module (MDM) for action recognition. The MDM uses a plurality of 2D convolution kernels of different dimensions to reconstruct a 3D convolution kernel, and these modules have high efficiency and recognition accuracy. Specifically, the MDM firstly performs 2D convolution on a video cube from three orthogonal dimensions, and learns appearance and motion features of an action subject in a video in a collaborative manner. Secondly, grouping operation is carried out on the three 2D convolutions along the channel dimension so as to further reduce the parameter calculation amount; and finally, time migration operation is performed on the two collaborative convolutions containing the space-time dimension along the time dimension to effectively obtain long-range and short-range time sequence information in the video. Compared with the calculated amount of the 3D CNN, the method provided by the invention is higher in efficiency, and meets the lightweight demand on the basis of ensuring the recognition performance.

Description

technical field [0001] The invention belongs to the technical field of video recognition, and relates to a video-oriented human action recognition network, method and system, in particular to a lightweight video action recognition network, method and system based on deep learning. [0002] technical background [0003] Deep learning drives progress in video action recognition. Deep learning achieves results far superior to traditional recognition methods by training on massive video datasets. However, considering factors such as device power consumption and storage costs, video-based human motion analysis and recognition applications have high requirements for real-time performance and speed. Therefore, it is of great significance to carry out lightweight design on the human behavior analysis and recognition model. [0004] At present, the mainstream methods of video action recognition can be roughly divided into three categories: [0005] (1) Two-stream approach: Extendin...

AI Technical Summary

Problems solved by technology

[0008] Most of the existing video action recognition models are (2+1)D models or 2D+3D parallel models. Although these methods have achieved certain results, they actually divide the spatiotemporal information into two parts during the learning process. The two independent parts are extracted separately, and the real spatio-temporal information interaction is not realized.

Images