Sign language recognition method and system

Abstract

The embodiment of the invention provides a sign language recognition method and system, and the method comprises the steps: collecting a sign language video in a target region; extracting feature representation of the sign language video by using a preset hierarchical long-short-term memory network, and modeling a context relationship among words in the sign language video by using a preset long-short-term memory neural network to obtain text semantic features; and connecting the feature representation of the sign language video and the text semantic features in a preset recurrent neural network converter, and fitting an optimal path to realize alignment of the feature representation of the sign language video and the text semantic features. According to the method, tThe robustness of signlanguage recognition in practical application is effectively improved, the ambiguity of translated sentences is reduced, the recognized result better conforms to the habit of a real language, and recognition of sign language videos of any length is supported.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a sign language recognition method and a sign language recognition system. Background technique [0002] Sign language is the main tool for deaf people to communicate. For most people who have not systematically learned this visual language, it is very difficult to master this language and use it for communication. In order to promote communication between the deaf and dumb people and society and others, it is necessary and urgent to invent video-based sign language recognition technology. According to the difficulty of recognition and the length of the video, sign language recognition can be divided into word-level recognition and sentence-level recognition. For word-level sign language recognition, the sign language video is converted into a word category, so this task can be regarded as a classification task. Different from word-level sign language recognition, sente...

AI Technical Summary

Problems solved by technology

But this is not true in reality

Second, the CTC algorithm cannot solve the case where the input length is shorter than the output length, because the CTC algorithm can only predict one word at each moment, but in actual use, the length of the sign language video is not fixed, and there may be sign language video frames There are more words than the corresponding text, but the CTC algorithm cannot solve this red case

However, the encoder-decoder structure will reduce the accuracy for longer sign language video recognition

After analysis, it can be obtained that a longer video is encoded into a fixed-length feature vector and decoded sequentially. This feature vector cannot represent the information of the video well.

Images