Human body behavior recognition method and system based on human body skeleton

Abstract

The invention discloses a human body behavior recognition method and system based on a human body skeleton, and the method comprises the steps: obtaining the behavior movement of the human body skeleton and the corresponding skeleton point coordinates, skeleton point inter-frame coordinate differences and skeleton features, and constructing a training set; sequentially training the graph convolution network and the attention mechanism network based on the human body part according to the training set, and constructing a behavior recognition model according to the trained graph convolution network and attention mechanism network; and recognizing the to-be-recognized human skeleton according to the behavior recognition model, and outputting human behavior actions. According to data such as three-dimensional coordinates of human skeleton joint points, coordinate differences between point frames and skeleton features, a graph convolution network is taken as a main body, an attention mechanism network based on human parts is adopted to assist in searching for skeleton points with better distinguishing ability, human behavior actions are classified and recognized, and the recognition precision is improved.

Description

technical field [0001] The invention relates to the technical field of behavior recognition, in particular to a human body behavior recognition method and system based on a human skeleton. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] In recent years, skeleton-based human behavior recognition has become more and more important. Compared with traditional RGB video-based behavior recognition, skeleton-based methods are more adaptable to the background and more robust to lighting conditions. It also has less computation. The skeleton data of a human behavior is mainly a skeleton sequence, each frame in the skeleton sequence contains multiple skeleton points, each skeleton point contains three-dimensional coordinate information, and the three-dimensional coordinates of the skeleton point are directly passed through the multi-modal sensor ( ...

AI Technical Summary

Problems solved by technology

The first category is the method based on convolutional neural network, which mainly regards a skeleton sequence as an image, or changes it into an image through some techniques, and then uses the method of convolutional neural network to perform feature However, the inventor believes that the spatial context correlation of skeleton points on the same frame is not as strong as that of RGB pixels, and the method based on convolutional neural network will lose the original skeleton data to a certain extent. existing spatial information

[0005] The second category is the method based on the cyclic neural network. This method mainly sorts and combines all the skeleton points on a frame through a certain scheme to form a vector, and then uses the cyclic neural network to model the time series and extract Action features; but the inventor believes that the problem with this type of method is that no matter how it is sorted, it is always unable to describe the structure of the human skeleton in space well, that is, the spatial information will also be lost

[0006] Since the human skeleton has a natural graph structure, the method based on the graph convolutional network can be introduced into the behavior recognition based on the skeleton; however, the inventor believes that there are still some problems in this method: first, a graph network cannot iterate a lot Layer, because there is no pooling operation, it may lead to excessive smoothing between each skeleton point; secondly, because it is impossible to use a multi-layer structure to expand the receptive field, it is difficult to generate between two nodes that are far apart on the graph However, human behaviors are likely to be related to nodes that are far away. For example, the action of eating uses both hands and heads, and the hands and heads are far apart on the graph. Therefore, for such actions, based on graph convolutional network The method cannot solve the problem that the skeleton points are far away on the graph and cannot be connected.

Images