A flexible
liquid metal sensor and a voiced / unvoiced
speech recognition system. The flexible
liquid metal sensor attached to the
throat is configured to detect a
hybrid-
modal signal consisting of laryngeal low-frequency
muscle movements and acoustic vibrations during voiced or unvoiced speech of a person. By designing discrete and continuous sensor layouts to accurately capture the frequency of laryngeal vibrations during
phonation and
spatial distribution characteristics of
muscle movements, pressure information in different directions and at different positions is measured. Laryngeal
data acquisition and filtering are achieved by means of a
signal acquisition and analysis
system, so as to construct a relational dataset between laryngeal acoustic-vibration
hybrid-
modal signals and expressed speech
semantics. By establishing a deep neural network training model based on time-frequency features to
train the relational dataset, the content expressed by a person during voiced or unvoiced speech can be recognized, thereby meeting the requirements of persons with speech impairments and the general
population for conveying the expressed content in environments where speech is impaired.