The invention discloses a
hand motion recognition method based on a depth image and a
color image. The method comprises the following steps: taking 36 types of gestures of an ASL sign language libraryas templates, obtaining gesture data through a Kinect sensor, and building a gesture
database under the depth and color backgrounds; a regression-based target detection
algorithm SSD is used as a research basis; under a Tensorflow
deep learning framework, transfer learning is carried out on a selected target detection model by utilizing a gesture
database self-built based on color and depth backgrounds respectively to obtain two types of network models capable of carrying out recognition detection on hand movement under the depth and color backgrounds. A
hand motion recognition detection network framework with detection results fused under depth and color backgrounds is utilized, a non-maximum suppression
algorithm is improved, and finally the effectiveness of
hand motion recognition detection of the proposed network framework is obtained. According to the invention, the problems of missing detection and
false detection of the target are avoided, the
gesture recognition rate is improved, and single-hand recognition and double-hand recognition can be realized.