Hand motion recognition method based on depth image and color image

Abstract

The invention discloses a hand motion recognition method based on a depth image and a color image. The method comprises the following steps: taking 36 types of gestures of an ASL sign language libraryas templates, obtaining gesture data through a Kinect sensor, and building a gesture database under the depth and color backgrounds; a regression-based target detection algorithm SSD is used as a research basis; under a Tensorflow deep learning framework, transfer learning is carried out on a selected target detection model by utilizing a gesture database self-built based on color and depth backgrounds respectively to obtain two types of network models capable of carrying out recognition detection on hand movement under the depth and color backgrounds. A hand motion recognition detection network framework with detection results fused under depth and color backgrounds is utilized, a non-maximum suppression algorithm is improved, and finally the effectiveness of hand motion recognition detection of the proposed network framework is obtained. According to the invention, the problems of missing detection and false detection of the target are avoided, the gesture recognition rate is improved, and single-hand recognition and double-hand recognition can be realized.

Description

technical field [0001] The invention relates to the technical field of image processing and intelligent interaction, in particular to a hand movement recognition method based on depth images and color images. Background technique [0002] With the development of machine vision and artificial intelligence related disciplines, human-computer interaction technology has gradually become an important research direction. The randomness and variability of hand shapes make it impossible for the current gesture recognition technology to replace traditional interactive devices in practical applications to complete human-computer interaction. This shows that gesture recognition detection interaction technology and some other computer vision-based Recognition technology still needs to continue to explore and research. Compared to color data, which provides appearance and texture information and is sensitive to light changes, depth data provides more shape information, sharp edges, and ...

AI Technical Summary

Problems solved by technology

The randomness and variability of hand shapes make the current gesture recognition technology unable to replace traditional interactive devices in practical applications to complete human-computer interaction. This shows that gesture recognition detection interaction technology and some other computer vision-based Recognition technology still needs to continue to explore and research

Images