Three-dimensional convolution and Faster RCNN-based video action detection method

Abstract

The invention discloses a three-dimensional convolution and Faster RCNN-based video action detection method. The method comprises the steps of firstly introducing a new model, and encoding a video stream by using a three-dimensional full convolutional network; secondly generating candidate time regions comprising actions based on generated features, and generating a group of candidate frames; andfinally performing classified detection on the candidate frames subjected to different film editing, thereby predicting action types and video action starting and ending time in the video stream, andpredicting spatial position boundary frames of the actions. Compared with an existing method, the method provided by the invention has excellent performance in unpruned data set video time sequence action detection, and can realize action localization in the absence of spatial labeling information.

Description

technical field [0001] The invention belongs to the technical field of image processing, and relates to a video action detection method based on three-dimensional convolution and Faster RCNN. Background technique [0002] With the vigorous development of Internet video media, video content detection and analysis has attracted extensive attention from industry and academia in recent years. Action recognition is an important branch of video content detection and analysis. In the field of computer vision, action recognition has made great progress in both manual features and deep learning features. Action recognition usually boils down to a classification problem, where each action instance in the training phase is pruned from a longer video sequence, and the learned action model is used for either pruned videos (e.g., HMDB51 and UCF101) or untrimmed videos ( For example, action recognition in THUMOS14 and ActivityNet). However, most videos in the real world are unrestricted...

AI Technical Summary

Problems solved by technology

However, S-CNN lacks the ability to predict at fine temporal resolution and locate the precise temporal boundaries of action instances

At the same time, due to the lack of spatial annotation information in the current untrimmed dataset, it is difficult for the current untrimmed dataset to simultaneously locate the spatial bounding box of the action when locating the time boundary of the action.

Images