Sign language recognition and skeleton generation method based on RNN

Abstract

The invention discloses a sign language recognition and skeleton generation method based on RNN, and the method specifically comprises the following steps: step 1, collecting a skeleton frame sequenceof a Chinese sign language through employing a Kinect RGB-D data set, inputting the skeleton frame sequence into an RNN hidden layer for twice coding reconstruction, and calculating and outputting aChinese sign language semantic label; step 2, according to Chinese sign language semantics, generating a skeleton sequence with the same probability density distribution as the encoded probability density distribution in the step 1, and decoding the skeleton sequence through a secondary probability model; and step 3, inputting the skeleton sequence decoded in the step 2 into the step 1 for identification, calculating the loss amount of generated data and real data, returning errors, continuously updating system parameters and minimizing a loss function, thereby finally obtaining a skeleton identification and generation framework for identification and generation of Chinese sign language. When CSL semanteme is given, various Chinese sign language skeleton sequences with different styles canbe automatically identified and drawn by using the method, and communication between deaf people and ordinary people is facilitated.

Description

technical field [0001] The invention belongs to the technical field of sign language recognition methods, and relates to a method for sign language recognition and skeleton generation based on RNN. Background technique [0002] Sign language recognition is a technology that can convert sign language information into voice, text, and read or display. The automatic recognition and generation of Chinese sign language (CSL, Chinese signal language) is a key technology for two-way communication between deaf people and ordinary people. Most previous studies have focused on CSL recognition. However, CSL recognition is only one aspect of communication between deaf-mute and ordinary people. Another challenging task is to teach the machine to automatically draw and generate CSL, so that the thoughts of ordinary people can be translated into sign language to express to the deaf-mute. [0003] In the prior art, there are mainly the following methods for sign language recognition: [...

AI Technical Summary

Problems solved by technology

[0011] The purpose of the present invention is to provide a method for sign language recognition and skeleton generation based on RNN, which solves the problem that the data collection workload of the sign language recognition method in the prior art is large, and the recognition and skeleton generation methods do not have global optimization.

Images