Sign language interpreting, displaying and sound producing system based on electromyographic signals and motion sensors

Abstract

The invention relates to a sign language interpreting, displaying and sound producing system based on electromyographic signals and motion sensors. The sign language interpreting, displaying and sound producing system comprises a gesture recognition subsystem and a semantic displaying and sound producing subsystem. The gesture recognition subsystem comprises the multi-axial motion sensors and a multi-channel muscle current acquisition and analysis module, the gesture recognition subsystem is put on the left arm and the right arm of a user, and the original surface electromyogram signals of the user and motion information of the arms of the user are obtained; gestures are differentiated by processing the electromyogram signals and data of the motion sensors. The displaying and sound producing subsystem comprises a semantic analyzer, a voice control module, a loudspeaker, a displaying module, a storage module, a communication module and the like. According to the sign language interpreting, displaying and sound producing system, by the adoption of the mode recognition technology based on the electromyographic signals of the double arms and the data of the motion sensors, the gesture recognition accuracy rate is increased; through the combination of the semantic displaying and sound producing subsystem, interpreting from commonly-used sign language to voice or text is achieved, and the efficiency of direct communication between people with language disorders and normal people is improved.

Description

technical field [0001] The invention relates to a real-time translation system from commonly used sign language to natural language text or voice. The realization principle is based on the multi-channel surface muscle current signals of the left and right arms of the user and the detection of the movement and rotation posture of the arms to distinguish and recognize sign language actions, and through The display sound system automatically adjusts the context semantics, and finally presents it in the form of text or voice. Background technique [0002] There is a huge base of hearing-impaired and language-impaired groups. The main way for the group to communicate with the outside world is sign language. However, due to the low chances of the general population to receive sign language training, it is very difficult for language-impaired groups to communicate with the general population. In terms of providing real-time sign language recognition and translation for hearing-impa...

AI Technical Summary

Problems solved by technology

This kind of solution can monitor the myoelectric signal of the arm in real time through the equipment worn on the arm, but according to its principle, it can only detect the relative posture of the hand, but cannot detect the movement posture of the arm and hand in three-dimensional space, and the Sign language movements are precisely the tight combination of these two postures, so this method is very difficult to use in sign language translation, and there are huge technical defects

Images