Lightweight video action recognition method and system based on deep learning

Abstract

The invention discloses a lightweight video action recognition method and system based on deep learning, and proposes a multi-dimensional module (MDM) for action recognition. MDM uses multiple 2D convolution kernels of different dimensions to reconstruct 3D convolution kernels. These modules have both high efficiency and recognition accuracy. Specifically, MDM first performs 2D convolutions on a video cube from three orthogonal dimensions to learn the appearance and motion features of action subjects in videos in a synergistic manner. Second, three 2D convolutions are grouped along the channel dimension, aiming to further reduce parameter computation. Finally, a time offset operation is performed along the time dimension for the two co-convolutions that include the spatio-temporal dimension, effectively obtaining long-range and short-range timing information in the video. Compared with the calculation amount of 3D CNN, the method of the present invention has higher efficiency, and meets the requirement of light weight on the basis of ensuring recognition performance.

Description

technical field [0001] The invention belongs to the technical field of video recognition, and relates to a video-oriented human action recognition network, method and system, in particular to a lightweight video action recognition network, method and system based on deep learning. Background technique [0002] Deep learning drives progress in video action recognition. Deep learning achieves results far superior to traditional recognition methods by training on massive video datasets. However, considering factors such as device power consumption and storage costs, video-based human motion analysis and recognition applications have high requirements for real-time performance and speed. Therefore, it is of great significance to carry out lightweight design on the human behavior analysis and recognition model. [0003] At present, the mainstream methods of video action recognition can be roughly divided into three categories: [0004] (1) Two-stream approach: Extending 2D CNN...

AI Technical Summary

Problems solved by technology

[0007] Most of the existing video action recognition models are (2+1)D models or 2D+3D parallel models. Although these methods have achieved certain results, they actually divide the spatiotemporal information into two parts during the learning process. The two independent parts are extracted separately, and the real spatio-temporal information interaction is not realized.

Images