Bone action recognition method based on learnable PL-GCN and ECLSTM

Abstract

The invention discloses a skeleton action recognition method based on learnable PL-GCN and ECLSTM, and relates to the field of action recognition. The problems that in the skeleton action recognition process, the feature capture capacity of key frames and significant motion joints is limited, and the similar action classification capacity is weak can be solved. The method comprises the steps that a learnable graph convolutional network (PL-GCN) is provided for the problem that similar action recognition is prone to confusion, and the learnable graph convolutional network (PL-GCN) is used for improving the physical structure of a model; for the problem of weak key frame capture capability, a feature enhanced long and short time memory network (ECLSTM) is provided for enhancing time sequence features; building a skeleton graph by utilizing a graph topological structure of the skeleton sequence data; fusing the spatial features from the image after convolution and the time sequence features extracted by the ECLSTM network; and carrying out average pooling and convolution on the fused features, and then carrying out final feature classification. The method provided by the invention is superior to some current methods in action recognition progress, algorithm complexity and feature extraction capability.

Description

technical field [0001] The invention belongs to the field of action recognition, in particular to a skeleton action recognition method. Background technique [0002] In recent years, skeleton-based human action recognition has developed rapidly in video surveillance, human-computer interaction, and virtual reality, and has attracted widespread attention in the field of computer vision. Skeleton data is a topological representation of human joints and bones. Compared with many methods based on RGB and optical flow, it has a small amount of calculation and strong robustness in the face of complex backgrounds. Speed ​​changes, etc. With the advent of Microsoft's Kinect depth camera and some excellent pose estimation algorithms, more and more researchers have started skeleton-based action recognition. [0003] Early skeletal action recognition usually encoded all body joint positions in each frame as feature vectors or pseudo images for pattern learning, and then fed this imag...

AI Technical Summary

Problems solved by technology

Although the above method uses the long-term memory function of the LSTM network to capture the temporal features of the network, it does not take into account the importance of motion information contained in different image frames. For this, this patent proposes a feature-enhanced ECLSTM network that can capture motion Significant keyframes to efficiently extract temporal features

In addition, the above method directly inputs the bone data into the original information module, and cannot fully extract the spatial features of the bone joint points, because the features of the joint points are closely related to the features of other joint points in the neighborhood, and the connection graph between the bone joint points is a kind of Graph topology

Images