Gesture recognition method based on deep neural network and attention mechanism

Abstract

A gesture recognition method based on a deep neural network and an attention mechanism belongs to the field of electronic information. The method comprises the following steps: firstly, introducing an ECA effective channel attention in a double-flow algorithm to enhance the attention of the double-flow algorithm on a gesture key frame, and respectively extracting space and time sequence characteristics in a dynamic gesture by utilizing a space convolutional network and a time convolutional network in the double-flow algorithm; secondly, a gesture frame with the highest attention degree is selected from the spatial stream through ECA, and corresponding hand posture features are extracted through the single-shot multi-frame detector technology; and finally, fusing the hand posture features with human body posture features and gesture time sequence features extracted from double flows, and then classifying and recognizing gestures. According to the method, verification is carried out on a Challarn2013 multi-mode gesture data set, the accuracy rate is 66.23%, and compared with a previous method that double-flow recognition is carried out on the data set only through RGB information, a better gesture recognition effect is obtained.

Description

technical field [0001] The invention belongs to the field of electronic information, and is a gesture recognition method based on a deep neural network and an attention mechanism, through which gesture video data captured by a common camera can be classified into corresponding text meanings. Background technique [0002] Gesture is an important part of human communication, and it is also an important way of human computer interaction (Human Computer Interaction, HCI). By detecting human gestures, it can help machines better understand human instructions, and then complete corresponding auxiliary tasks. For example, in a smart home environment, you can control the switch of the air conditioner and switch the TV screen through gestures; in the process of smart driving, you can also control some functions inside the car through gestures, so that the driver can focus more on the road. itself, reducing the occurrence of traffic accidents. [0003] At present, the research on ge...

AI Technical Summary

Problems solved by technology

[0006] In order to solve the problems of expensive equipment, limited interaction scenarios, and large amount of calculation required by traditional gesture recognition methods, the present invention considers combining Efficient Channel Attention (ECA) and Single Shot MultiBox Detector (Single Shot MultiBox Detector, SSD) improves the dual-stream algorithm, and then establishes a more general vision-based gesture recognition model

Images