Electronic skin and multi-sensor fusion-based sign language identification system

Abstract

The invention provides an electronic skin and multi-sensor fusion-based sign language identification system. A front-end sensor module is used for collecting finger motion information, wrist motion information and arm motion information to form hand motion information; an internal integrated processor module performs decoding translation on the received hand motion information; and a sign languagetranslation result display module is used for display. A communication bridge is established for communication between deaf-mutes and normal persons; a signal collection device only uses electronic skin and a wristband, seen from appearance, so that the interference on a user is smaller, and the signal collection device is suitable to be worn for a long time and has the characteristics of flexibility, low cost and the like; a wearable scheme is adopted, so that the influence of external environment illumination and background complexity is avoided; the front-end sensor module captures a signal more directly and more precisely; in the aspect of translation result display, a T shirt capable of displaying texts is introduced, so that convenience is provided for a scene inconvenient for speaking; and the system also can be used in the fields of virtual reality and the like.

Description

Technical field [0001] The invention relates to the technical field of human-computer interaction, in particular to a sign language recognition system based on electronic skin and multi-sensor fusion. Background technique [0002] Sign language is a way of simulating images or syllables through gesture changes, providing a way for deaf-mute people to communicate. Due to the relatively low percentage of control in the normal population, there are barriers to communication between deaf-mute people and normal people. [0003] Through the sign language recognition system, the translation of sign language actions into text or speech that normal people can understand can facilitate the communication between deaf-mute people and normal people, which has significant needs and social significance. [0004] The literature search of the prior art found that: [0005] The application number is CN201010606052.5, the patent publication number is CN102063825A, and the patent name is a data glove-ba...

AI Technical Summary

Problems solved by technology

The disadvantage is that the muscle electrical signal is a time-varying non-stationary signal, which is easily affected by external changes such as sweating, and has limited ability to decode a large number of different gestures.

[0008] Sign language recognition is difficult to realize due to the characteristics of many gestures and high similarity between some actions. The sign language recognition device represented by the traditional single electronic skin has the disadvantages of not being able to recognize wrist information, and this information is very important for sign language recognition. has a vital role

Images