Mutually translating system and method of sign language and speech

Abstract

The invention discloses a mutually translating system of sign language and speech, a gesture image collecting module 101 is used for collecting the video data of gestures, an input image preprocessing module 102 is used for image preprocessing, an image characteristic extracting module 103 is adopted for image characteristic extraction of the video data after image preprocessing and then outputs 56-dimension characteristic vectors, the 56-dimension characteristic vectors are used for constructing a sign language model 104, a continuous and dynamic sign language recognizing module 105 is used for recognizing the sign language model 104, and recognition results are output and translated into Chinese speech through a Chinese sounding module 106; voice signals collected by a voice signal collecting device are input in a speech recognition programming interface of Microsoft Speech SDK 5.1 and converted into characters to be output; three-dimensional models and three-dimensional animation are established through three-dimensional modeling software; the information of the three-dimensional models and the three-dimensional animation is output into .x formatted files through a Panda plug-in; and DirectX 3D is utilized to load the .x formatted three-dimensional models and the three-dimensional animation and then output sign language animation.

Description

Technical field: [0001] The invention belongs to the application field of image pattern recognition, and in particular relates to the application of a method for image and voice mutual transformation in image processing and feature extraction. Background technique: [0002] The research on the Sign Language & Speech Intertranslating System not only helps to improve and enhance the living, learning and working conditions of the deaf-mute, and provides them with better services, but also can be applied to computer-assisted dumb language teaching, Bilingual broadcasting of TV programs, research on virtual humans, special effects processing in film production, animation production, medical research, game entertainment and many other aspects. [0003] From the perspective of sign language input devices, sign language recognition systems are mainly divided into data glove-based recognition systems and visual (image)-based sign language recognition systems. [0004] The vision-bas...

AI Technical Summary

Problems solved by technology

The neural network method has classification characteristics and anti-interference, but because of its weak ability to deal with time series, it is currently widely used in static sign language recognition

Images