Device and method for sign language intelligent interaction capable of real-time identification of gestures

Abstract

The present invention discloses a device for sign language intelligent interaction capable of real-time identification of gestures. The device comprises a network television, a processor, a display and a camera. The processor comprises an obtaining unit, a processing unit, an edition unit and a decoding unit. The obtaining unit is configured to obtain audio information of the network television and viewers' sign language gestures' image information and depth information; the processing unit is configured to convert the audio information to text information; the edition unit is configured to distinguish different sound sources of the audio information and employ a virtual character sign language edition technology to edit corresponding virtual character sign language animations according tothe sound sources; and the decoding unit is configured to perform decoding processing of the image information and the depth information and convert the image information and the depth information after decoding to the text information. The device and the method for sign language intelligent interaction capable of real-time identification of gestures can help the deaf-mute with understanding of television programs and also can help the deaf-mute with communication with the television programs.

Description

technical field [0001] The invention relates to the field of artificial intelligence, in particular to a sign language intelligent interactive device for real-time gesture recognition and an interactive method thereof. Background technique [0002] As a visual language, sign language mainly relies on the movement of hands and arms, supplemented by appropriate facial expressions and mouth shapes to express meaning. It is an important way for deaf-mute people to communicate with the outside world. [0003] At present, there is a huge population of deaf-mute people in our country. Compared with TV programs, the programs that deaf-mute people can watch are relatively single. Only in some news programs, there will be sign language hosts to help the disabled understand the news content. In terms of entertainment programs, There is a lack of devices on the market to help deaf people understand the content of the program. [0004] In recent years, although sign language editing syn...

AI Technical Summary

Problems solved by technology

[0003] At present, there is a huge population of deaf-mute people in our country. Compared with TV programs, the programs that deaf-mute people can watch are relatively single. Only in some news programs, there will be sign language hosts to help the disabled understand the news content. In terms of entertainment programs, There is a lack of devices on the market to help deaf people understand the content of the program

[0004] In recent years, although sign language editing synthesis technology and virtual human synthesis technology have been relatively mature, the application of multi-virtual human synthesis technology for sign language expression is still very small

Images