Joint learning method of label and interaction relation for human action recognition

Abstract

A label and interactive relationship joint learning method for human behavior recognition, comprising the following steps: 1), using CNN features, HOG features, HOF features, distances between people, head orientation and other information to construct energy functions, Contains unary energy items, binary energy items, interaction energy items, and regularization items; 2) use large-interval structured learning to train all model parameters; 3) label and interaction relationship prediction, use alternate search strategies to solve complex reasoning problems, and iteratively Alternately optimize labels and interaction structures in . The invention is applicable to images and videos containing multiple people and multiple behavior categories, and can identify individual behaviors and interaction behaviors between people at the same time.

Description

technical field [0001] The invention belongs to the field of behavior recognition in computer vision, and relates to a human behavior recognition method. The invention judges human interactions while recognizing individual behavior. Background technique [0002] Recognizing human actions in images or videos is a fundamental problem in computer vision, which is crucial in many applications such as motion video analysis, surveillance systems, and video retrieval. In recent work, deep learning has significantly improved the performance of action recognition. However, these works are not suitable for dealing with data containing multi-person interactions. First, they focus on assigning each image an action label, which is not suitable for images containing multiple action categories. Second, they ignore that the interrelationships between people provide important contextual information for recognizing complex human activities like handshakes, fights, and football games. [0...

AI Technical Summary

Problems solved by technology

[0004] In order to overcome the shortcomings of existing human behavior recognition methods that cannot be applied to images of multiple behavior categories, cannot recognize interactive behaviors, and interact as a hidden variable that leads to non-convex training problems, the present invention provides a human behavior recognition method The label and interaction relationship joint learning method, suitable for images containing multiple behavior categories, can jointly learn a new training framework of interaction and individual behavior without using hidden variables, and proposes an effective solution to the corresponding reasoning problem algorithm

Images