Human skeleton action recognition method based on generalized graph convolution and reinforcement learning

Abstract

The invention provides a human skeleton action recognition method based on generalized graph convolution and reinforcement learning. According to the method, a human skeleton sequence matrix is constructed, a predefined skeleton diagram is constructed, a training set is sent to a generalized graph convolutional network for feature extraction, features are aggregated by using global average pooling, the features are classified by using a full connection layer classifier, and network parameters are updated according to a loss function; based on the trained generalized graph convolutional network, the classifier and the features learned by generalized graph convolution, constructing a feature selection network to adaptively select features useful for recognition in the time dimension, and performing training by using a reinforcement learning method. According to the method, a generalized graph convolutional network is designed for a human skeleton action recognition task and is used for capturing related dependence between any nodes so as to extract richer associated features between the nodes. Meanwhile, a feature selection network is designed and used for selecting features useful for recognition in the time dimension, and therefore more accurate action recognition is achieved.

Description

technical field [0001] The invention belongs to the technical field of video image processing, in particular to a human skeleton action recognition method based on generalized graph convolution and reinforcement learning. Background technique [0002] Human behavior recognition technology has a wide range of applications in video surveillance, video retrieval, and human-computer interaction. Compared with RGB video, human skeleton sequences have excellent properties such as rotation invariance and illumination invariance, so action recognition based on skeleton sequences has significant advantages in complex scenes. Now, with the development of depth sensors and human pose estimation algorithms, it is becoming easier and easier to obtain human skeleton sequences. [0003] Earlier traditional methods mainly designed feature descriptors for human actions or human-object interactions that are general for human skeletons. Generally speaking, such features should be invariant t...

AI Technical Summary

Problems solved by technology

[0008] At present, although the method based on graph convolution has achieved good results, there are still two problems: graph convolution can only extract local associated features and cannot obtain long-distance dependencies between nodes.

Images