Sign language recognition and conversion system and method based on deep learning and big data

Abstract

The invention discloses a sign language recognition and conversion system and method based on deep learning and big data. The system comprises an image acquisition module, an image recognition module,an information matching module, a content arrangement module, a text output module and a voice output module. The method includes: collecting a human body image sequence; extracting face key point coordinates and hand key point coordinates in each frame of image of the human body image sequence; searching natural language morphemes most matched with the face key point coordinates and the hand keypoint coordinates in a sign language action database, and calculating matching values; filtering the natural language morphemes according to the repetition condition and the matching value between the adjacent morphemes; converting the reserved natural language morphemes into characters and displaying the characters on a screen; and searching voices corresponding to the characters according to the character language database, and playing the voices. According to the system, the sign language image sequence can be conveniently and quickly converted into characters and voice of other languagesto be output, the meaning of the sign language can be understood more easily, and the communication efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of sign language recognition, in particular to a sign language recognition and conversion system and method based on deep learning and big data. Background technique [0002] There are a large number of deaf-mute people in our country who use sign language to communicate. For those who do not know sign language, it is very difficult to understand the meaning of sign language. Sign language is composed of gestures, arm movements, and facial expressions, and has a complete ability to express. People who master sign language can communicate efficiently, but people who don't understand sign language often have difficulty understanding the meaning expressed in sign language. This situation hinders communication and communication between people. Contents of the invention [0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. [0004] Therefore...

AI Technical Summary

Problems solved by technology

People who master sign language can communicate efficiently, but people who don't understand sign language often have difficulty understanding the meaning expressed in sign language. This situation hinders communication and communication between people

Images