Sign language recognition and conversion system and method based on deep learning and big data

A deep learning and big data technology, applied in the field of sign language recognition, can solve problems such as hindering communication and communication between people and difficult to understand the meaning of sign language, and achieve the effect of improving communication efficiency.

Active Publication Date: 2020-08-25
TSINGHUA UNIV
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

People who master sign language can communicate efficiently, but people who don't understand sign language often have difficulty understanding the meaning expressed in sign language. This situation hinders communication and communication between people

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Sign language recognition and conversion system and method based on deep learning and big data
  • Sign language recognition and conversion system and method based on deep learning and big data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0041] The following describes the sign language recognition and conversion system and method based on deep learning and big data according to the embodiments of the present invention with reference to the accompanying drawings.

[0042] Firstly, the sign language recognition and conversion system based on deep learning and big data according to the embodiment of the present invention will be described with reference to the accompanying drawings.

[0043] figure 1 It is a schematic structural diagram of a sign language recogniti...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a sign language recognition and conversion system and method based on deep learning and big data. The system comprises an image acquisition module, an image recognition module,an information matching module, a content arrangement module, a text output module and a voice output module. The method includes: collecting a human body image sequence; extracting face key point coordinates and hand key point coordinates in each frame of image of the human body image sequence; searching natural language morphemes most matched with the face key point coordinates and the hand keypoint coordinates in a sign language action database, and calculating matching values; filtering the natural language morphemes according to the repetition condition and the matching value between the adjacent morphemes; converting the reserved natural language morphemes into characters and displaying the characters on a screen; and searching voices corresponding to the characters according to the character language database, and playing the voices. According to the system, the sign language image sequence can be conveniently and quickly converted into characters and voice of other languagesto be output, the meaning of the sign language can be understood more easily, and the communication efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of sign language recognition, in particular to a sign language recognition and conversion system and method based on deep learning and big data. Background technique [0002] There are a large number of deaf-mute people in our country who use sign language to communicate. For those who do not know sign language, it is very difficult to understand the meaning of sign language. Sign language is composed of gestures, arm movements, and facial expressions, and has a complete ability to express. People who master sign language can communicate efficiently, but people who don't understand sign language often have difficulty understanding the meaning expressed in sign language. This situation hinders communication and communication between people. Contents of the invention [0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. [0004] Therefore...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06F3/01G06F3/16
CPCG06F3/017G06F3/167G06V40/161G06V40/28
Inventor 徐枫周玉枭杨东
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap