A video group behavior recognition method based on cascaded transformers

Abstract

The present invention relates to the field of computer vision and deep learning, and in particular to a video group behavior recognition method based on cascaded Transformers. First, a video data set is collected and generated, and the video data set is extracted through a three-dimensional backbone network to extract three-dimensional spatio-temporal features, and a key frame image space is selected. Feature map; preprocess the key frame image space feature map and send it to the human target detection Transformer to output the human target frame in the key frame image; then, map the sub The feature map is combined with the frame feature map around the key frame image to calculate the query / key / value, input the group behavior recognition Transformer, and output the group-level spatio-temporal encoding feature map; finally, the group behavior is classified by the multi-layer perceptron. The invention has the effect of effectively improving the accuracy of group behavior recognition.

Description

technical field [0001] The present invention relates to the field of computer vision and deep learning, in particular to a video group behavior recognition method based on cascade Transformer Background technique [0002] Nowadays, surveillance video has been widely used in social public places and plays an extremely important role in maintaining social public safety. Effective identification of abnormal behaviors and events in surveillance video can better play the role of surveillance video. Group behavior is the most common human behavior in videos. Group behavior recognition can effectively prevent dangerous events by automatically identifying group behavior in videos, and has a wide range of application values. [0003] In natural scenes, video group behavior recognition mainly faces two major challenges. One is that the scene is relatively complex, mainly manifested in the large scale change of the human body, background lighting, mutual occlusion between groups, etc...

AI Technical Summary

Problems solved by technology

Its disadvantage is that when the graph convolutional network extracts the spatial features of a single-frame group, it does not highlight the discriminative individual features in the group, and only performs simple weighted fusion in the video time feature dimension, which cannot extract video timing features well.

Images