First-person perspective video interaction behavior identification method based on interaction modeling

Abstract

The invention discloses a first-person visual angle video interaction behavior identification method based on interaction modeling, and proposes to separate a camera wearer from an interactor, learn corresponding static appearance and dynamic motion characteristics respectively, and then explicitly model an interaction relationship between the camera wearer and the interactor. In order to separatethe interactor from the background, a mask is generated by using an attention model, and the learning of the attention model is assisted by using a human body analysis model; a motion module is provided to predict motion information matrixes corresponding to a camera wearer and an interactor respectively, and learning of the motion module is assisted through reconstruction of a next frame. And finally, a dual long-short-term memory module for interactive modeling is proposed, and an interactive relationship is explicitly modeled on the basis of the dual long-short-term memory module. According to the method, the interactive behavior of the first-person perspective can be well described and recognized, and a current optimal recognition result is obtained on a common first-person perspective interactive behavior research data set.

Description

technical field [0001] The invention belongs to the technical field of behavior recognition, and in particular relates to a first-person perspective video interactive behavior recognition method based on interaction modeling. Background technique [0002] At present, the main first-person group behavior recognition methods are divided into two categories. One category uses manually designed motion features such as motion trajectory and optical flow, combined with traditional classifiers such as support vector machines; the other category uses deep learning for feature learning. One class of methods adopts a model similar to third-person video action recognition, using convolutional neural networks and long short-term memory models to directly learn behavioral features from video frames. [0003] A major shortcoming of the prior art described above is that the interaction relationship between the camera wearer and the interactor is not explicitly modeled. Existing technologi...

AI Technical Summary

Problems solved by technology

[0003] The main disadvantage of the above prior art is that it does not explicitly model the interaction between the camera wearer and the interactor

Existing technologies usually directly learn the overall characteristics of the interaction behavior, but the first-person perspective interaction behavior is generated by the interaction between the camera wearer and the interactor, and explicitly modeling this interaction relationship can better describe the interaction Behavior, the existing technology lacks explicit interaction modeling, so the interaction behavior cannot be well described

Images