Gesture recognition method based on 3D-CNN and convolutional LSTM

Abstract

The invention discloses a gesture recognition method based on 3D-CNN and convolution LSTM. The method comprises the steps that the length of a video input into 3D-CNN is normalized through a time jitter policy; the normalized video is used as input to be fed to 3D-CNN to study the short-term temporal-spatial features of a gesture; based on the short-term temporal-spatial features extracted by 3D-CNN, the long-term temporal-spatial features of the gesture are studied through a two-layer convolutional LSTM network to eliminate the influence of complex backgrounds on gesture recognition; the dimension of the extracted long-term temporal-spatial features are reduced through a spatial pyramid pooling layer (SPP layer), and at the same time the extracted multi-scale features are fed into the full-connection layer of the network; and finally, after a latter multi-modal fusion method, forecast results without the network are averaged and fused to acquire a final forecast score. According to the invention, by learning the temporal-spatial features of the gesture simultaneously, the short-term temporal-spatial features and the long-term temporal-spatial features are combined through different networks; the network is trained through a batch normalization method; and the efficiency and accuracy of gesture recognition are improved.

Description

technical field [0001] The invention relates to the technical field of machine vision and pattern recognition, in particular to a gesture recognition method based on 3D-CNN and convolutional LSTM. Background technique [0002] Gestures, as a kind of human body language, play a very important role in daily life. In many computer vision application fields such as human-computer interaction, sign language recognition, and virtual reality, gesture recognition technology will undoubtedly have a huge impact. Vision-based gesture recognition aims to recognize and understand meaningful movement information of the human body through machine vision. Recognition remains a very challenging problem. [0003] Traditional gesture recognition methods can be roughly divided into methods based on artificially designed features and traditional machine learning methods. Most of these methods design a feature map that can be used to describe hand movements, and then extract these features fro...

AI Technical Summary

Problems solved by technology

[0005] In order to overcome the shortcomings and deficiencies in the prior art, the present invention provides a gesture recognition method based on 3D-CNN and convolutional LSTM, which solves the problem of only extracting short-term spatio-temporal features or long-term spatio-temporal features in current traditional gesture recognition methods , complex background interference, time-consuming and low recognition rate, etc.

Images