Real-time dynamic gesture recognition method based on multi-track matching

Abstract

The invention relates to a real-time dynamic gesture recognition method based on multi-track matching. The method comprises the following steps: copying a video stream is acquired and a video image; segmenting a video image to obtain a hand region image; constructing a convolutional neural network based on a FAST corner detection algorithm; acquiring a positive sample containing all fingertip points, clustering based on the acquired positive sample of the fingertip points and another unprocessed video image to obtain a minimum point set in one frame, matching fingertips in two frames through aglobal nearest neighbor point matching algorithm, and finally performing multi-track classification and dynamic gesture recognition by using an LSTM neural network. According to the method, a CNN andFAST corner detection algorithm is combined, the positions of fingertip points can be rapidly detected, meanwhile, an asymmetric point set matching algorithm and LSTM are fused, high-robustness classification is achieved for dynamic gestures, the gesture recognition efficiency is high, and the recognition effect is good.

Description

technical field [0001] The present invention relates to the technical fields of calculation, estimation and counting, in particular to a real-time dynamic gesture recognition method based on multi-trajectory matching in the field of human-computer interaction and computer vision. Background technique [0002] In the field of human-computer interaction, gesture interaction is one of the most common and important forms of interaction. Gesture recognition is directly related to the accuracy and robustness of gesture interaction, and is a crucial link in gesture interaction. [0003] At present, from the perspective of whether gesture recognition has temporal significance, it can be divided into static gesture recognition and dynamic gesture recognition; static gesture recognition aims to classify gestures in a single frame through the morphological characteristics of gestures, while Dynamic gesture recognition is to classify gestures in a sequence of time. Since in practical a...

AI Technical Summary

Problems solved by technology

[0006] (1) Through 3D convolution, the frames in an interval are trained as a group to achieve the purpose of classifying actions within a period of time. The defect of this method is that it is difficult to recognize long-term gestures;

[0007] (2) After segmenting gestures through color space, track the center of mass of the hand, and use DTW or HMM to predict the trajectory. The disadvantage of this method is that DTW needs to match a large number of templates, which is slow, and HMM cannot cope with complex and long-term trajectories.

Images