Group behavior recognition method based on video clip attention mechanism and interactive relation activity diagram modeling

Abstract

The invention provides a group behavior identification method based on a video clip attention mechanism and interactive relation activity diagram modeling, which is mainly used for solving the problemof group behavior identification precision in a video monitoring scene so as to improve the group behavior identification precision and eliminate a large amount of redundant information in a video. The method comprises the following steps: firstly, extracting key fragments in a video by utilizing a fragment attention mechanism, and extracting spatial and temporal characteristics of the key fragments through a P3D network; secondly, constructing a group activity graph by using a convolution relation mechanism to capture an interaction relation between people, and optimizing the activity graphthrough multiple stages and different types of convolution operations to form dynamic description of a group relation; further, the optimized group relation activity diagram is fused with the originalP3D features through a fusion mechanism, and the purpose is to combine the P3D features of the bottom layer with the group features of the high layer to avoid feature loss; and finally, using a softmax classifier to identify the group behavior according to the fused features, so as to obtain higher group behavior identification precision and effect.

Description

technical field [0001] The invention belongs to the field of group behavior recognition, and in particular relates to a group behavior recognition method based on video segment attention mechanism and interactive relationship activity diagram modeling. Background technique [0002] In recent years, video-based human behavior recognition technology has received extensive attention from the academic community, and it also has very important application prospects in many industrial fields, such as intelligent monitoring, public security, and human-computer interaction. The emergence of convolutional neural networks has greatly promoted the development of tasks such as image classification, image segmentation, and target detection. Many researchers extract complex features from images by building network structures of various depths and widths. The performance of the behavior recognition system depends to a large extent on whether these important information can be effectively ...

AI Technical Summary

Problems solved by technology

[0004] However, in this scheme: (1) the first step is to use the Harris corner tracking algorithm to obtain the spatial information of the moving target in the video image frame. There is a positioning deviation when extracting T-shaped oblique T-shaped corner points; in addition, the Harris algorithm does not have scale invariance, and the detection time is relatively long; (2) The differential recurrent neural network used in the third step is the VGG-16 network Connect with the LSTM network to achieve end-to-end feature extraction; although this can effectively capture the spatiotemporal features between consecutive frames, it cannot extract the dependency relationship between the underlying information, and it is easy to lose the context information; (3) In addition, the scheme uses The method of manual labeling is to label the training samples according to the subject, location and characteristics of the behavior itself, which takes a lot of time and labor costs, and limits the expansion of the model training data set. The data set is mixed with many human factors, which is not conducive to the learning of the network

Images