Human interaction behavior detection method based on self-attention mechanism

Abstract

The invention discloses a human interaction behavior detection method based on a self-attention mechanism, and the method comprises the steps: inputting a preprocessed video clip into a constructed recognition network model, extracting spatial-temporal features through a backbone network, carrying out the dimension reduction of the extracted spatial-temporal features, flattening the spatial-temporal features after the dimension reduction, and carrying out the recognition of human interaction behaviors. And then position coding is carried out to obtain a feature vector containing position information, and the feature vector containing the position information is input into an encoder to obtain a shared feature. The method comprises the following steps: firstly, obtaining instance embedding and interaction relation embedding through two decoders, finally obtaining instance interaction characteristics between each instance and an interaction relation through a similarity decoder, obtaining the similarity between each instance and the interaction relation through a classification operation, and determining the interaction relation to which each instance belongs.

Description

technical field [0001] The present application belongs to the technical field of human interaction behavior detection, and in particular relates to a human behavior interaction detection method based on a self-attention mechanism. Background technique [0002] Understanding human interaction behavior is a very basic task in the field of computer vision, and it is helpful for downstream tasks, such as video surveillance, key event retrieval, overall behavior understanding, and sports analysis. The task of human interaction behavior understanding is to predict human interaction in visual signals, which requires positioning and action category prediction for each person in the scene, as well as finding out the interaction between people. Because the behavior of interaction is relatively abstract and complex, it is often a challenge for computers to complete accurate interaction behavior prediction. [0003] At present, there are two main directions for human interaction behavi...

AI Technical Summary

Problems solved by technology

Because the behavior of interaction is relatively abstract and complex, it is often a big challenge for computers to complete accurate prediction of interactive behavior.

Images