Human behavior identification tag and interactive relationship combined learning method

Abstract

The invention discloses a tag used for human behavior identification and an interactive relationship combined learning method. The method comprises the following steps that: 1) using information including CNN (Convolutional Neural Network) features, HOG (Histogram of Oriented Gradient) features, HOF (Histograms of Optical Flow) features, a distance between people, a head direction and the like to construct an energy function which includes a unitary energy item, a binary energy item, an interactive energy item and a regularization item; 2) using large-interval structured learning to train all model parameters; and 3) carrying out tag and interactive relationship prediction, using an alternant search strategy to solve a complex reasoning problem, and alternately optimizing a tag and an interactive structure in interaction. The method is suitable for images and videos which contain multiple people and multi-behavior categories, and a personal behavior and an interactive behavior between people can be identified.

Description

technical field [0001] The invention belongs to the field of behavior recognition in computer vision, and relates to a human behavior recognition method. The invention judges human interactions while recognizing individual behavior. Background technique [0002] Recognizing human actions in images or videos is a fundamental problem in computer vision, which is crucial in many applications such as motion video analysis, surveillance systems, and video retrieval. In recent work, deep learning has significantly improved the performance of action recognition. However, these works are not suitable for dealing with data containing multi-person interactions. First, they focus on assigning each image an action label, which is not suitable for images containing multiple action categories. Second, they ignore that the interrelationships between people provide important contextual information for recognizing complex human activities like handshakes, fights, and football games. [0...

AI Technical Summary

Problems solved by technology

[0004] In order to overcome the shortcomings of existing human behavior recognition methods that cannot be applied to images of multiple behavior categories, cannot recognize interactive behaviors, and interact as a hidden variable that leads to non-convex training problems, the present invention provides a human behavior recognition method The label and interaction relationship joint learning method, suitable for images containing multiple behavior categories, can jointly learn a new training framework of interaction and individual behavior without using hidden variables, and proposes an effective solution to the corresponding reasoning problem algorithm

Images