Skeleton behavior recognition method based on 3D space-time diagram convolution

Abstract

The invention provides a skeleton behavior recognition method based on 3D space-time diagram convolution, and the method can achieve the spatial modeling and time modeling of skeleton information at the same time, and also can represent the connectivity between space-time information. Meanwhile, the method can obtain excellent recognition accuracy on a large-scale skeleton data set, and is good ingeneralization performance. In the technical scheme of the invention, according to the method, a 2D image convolution Laplace operator and a multi-frame time Laplace operator are combined, a 3D space-time diagram convolutional neural network model is constructed, and updating of a current node in the 3D space-time diagram convolutional neural network model depends on the state of a joint node connected with the current node in a current 2D diagram and is also related to the node state of a corresponding node in a front-back adjacent adjacent 2D diagram; and the method realizes communication of spatial information and time information by combining the related state information in the current 2D graph with the state information of the same nodes in the front and back adjacent 2D graphs, andconstructs 3D graph convolution.

Description

technical field [0001] The invention relates to the technical field of machine vision recognition, in particular to a skeleton behavior recognition method based on 3D space-time graph convolution. Background technique [0002] The skeleton behavior recognition method in the field of machine vision is to use depth cameras, infrared cameras and other sensors to collect the movement data of the target object, and analyze the data, and realize the automatic understanding and behavior analysis of the target object movement with the help of computer. Skeletal behavior recognition technology communicates the underlying video data and high-level action semantic information, so the research on skeletal behavior recognition can be widely used in video surveillance, human-computer interaction, video understanding and other fields. Most of the existing research on skeletal behavior recognition technology is based on recurrent neural network and temporal convolutional network; with the r...

AI Technical Summary

Problems solved by technology

[0003] In order to solve the problem that the existing technology lacks the ability to simultaneously perform spatio-temporal modeling for skeleton information, resulting in unsatisfactory recognition accuracy, the present invention provides a skeleton behavior recognition method based on 3D spatio-temporal graph convolution, which can not only realize skeleton information Simultaneously perform spatial modeling and temporal modeling, and can also represent the connectivity between spatio-temporal information; at the same time, it can achieve excellent recognition accuracy on large skeleton datasets, and has good generalization performance

Images