A dynamic gesture recognition method based on Kinect

Abstract

The invention discloses a dynamic gesture recognition method based on Kinect. The method comprises the following steps of acquiring a color image sequence and a depth image sequence of the dynamic gesture by Kinect V2; carrying out the hand detection and segmentation and other pre-processing operations; extracting the spatial features and sequential features of dynamic gesture, outputting space-time characteristic; inputting the outputted space-time characteristic into a simple convolution neural network to extract higher-level space-time characteristic. and classifying by a dynamic gesture classifier; training the dynamic gesture classifiers of color image sequence and depth image sequence respectively, and fusing the output with random forest classifier to get the final dynamic gesture recognition results. The method of the invention provides the dynamic gesture recognition model based on a convolution neural network and a convolution long-short time memory network, the spatial and temporal features of dynamic gestures are processed by these two parts respectively, and the classification results of color image sequence and depth image sequence are fused by stochastic forest classifier, so that the recognition rate of dynamic gestures is greatly improved.

Description

technical field [0001] The invention belongs to the field of computer vision, and more specifically relates to a Kinect-based dynamic gesture recognition method. Background technique [0002] With the continuous development of technologies such as robots and virtual reality, traditional human-computer interaction methods are gradually difficult to meet the needs of natural interaction between humans and computers. Vision-based gesture recognition, as a novel human-computer interaction technology, has received widespread attention from researchers at home and abroad. However, color cameras are limited by the performance of their optical sensors, making it difficult to cope with complex lighting conditions and cluttered backgrounds. Therefore, a depth camera (such as Kinect) with more image information has become an important tool for researchers to study gesture recognition. [0003] Although the Kinect sensor has been successfully applied in face recognition, body tracking...

AI Technical Summary

Problems solved by technology

Because compared with the human body or human face, the human hand has a smaller target on the image, making it more difficult to locate or track, and the human hand has a complex joint structure, and the finger part is prone to self-occlusion during movement, which also makes gesture recognition more vulnerable. The effect of segmentation error, so recognizing gestures in general is still a very challenging problem

Images