Chinese initial and final visualization method based on combination feature

Abstract

The invention relates to a Chinese initial and final visualization method based on a combination feature, which comprises the steps of: pre-processing a voice signal; calculating the frame number of the pre-processed voice signal as a length feature, representing a resonance strength feature by correlation of a frequency domain peak amplitude and an average amplitude to obtain a resonant peak feature value of each frame signal, and calculating robust feature parameters WPTC1-WPTC20 and PMUSIC-MFCC1-PMUSIC-MFCC12; respectively encoding image width information and image length information by the length feature and the resonance strength feature; encoding the main color information by the resonant peak feature; enabling 32 feature parameters to serve as input of a neural network and the output of the neural network to be corresponding pattern information, wherein the output corresponds to 23 initials and 24 finals sequentially; and fusing the width, length, main color and pattern information in an image and displaying the image on a display screen. The Chinese initial and final visualization method has the advantages that the Chinese initial and final visualization method based on the combination feature is helpful for deaf-mutes for speech training to establish and improve auditory perception and form correct speed reflection so as to recover the speed function of the deaf-mutes.

Description

technical field The invention relates to a method for visualizing Chinese initials and finals, in particular to a method for visualizing Chinese initials and finals based on combination features. Background technique Speech is the acoustic performance of language, the most natural, effective and convenient means for human to exchange information, and also a kind of support for human thinking. For deaf-mute people, language communication has become a difficult thing to achieve. Some deaf-mute people cannot speak because their auditory organs have been damaged and they cannot collect voice information to the brain. Studies have shown that the human auditory system and visual system are two different and complementary information systems. The visual system is a highly parallel information receiving and processing system. Millions of cone cells on the retina of the human eyeball pass through The fibrous nerve tissue is connected with the brain to form a highly parallel channel...

AI Technical Summary

Problems solved by technology

In 1947, R.K.Potter and G.A.Kopp et al. proposed a visualization method—the spectrogram, and then different speech research experts began to study and improve this speech visualization method. For example, in 1976, L.C.Stewart et al. proposed a chromatogram And in 1984, G.M.Kuhn et al. proposed a real-time spectrogram system for training deaf people, and P.E.Stern in 1986, F.Plante in 1998 and R.Steinberg in 2008 also proposed many spectrogram improvements. method, but the displayed spectrogram is very professional, and it is difficult to distinguish memory

Especially for the same person with different voices, or even the same voice for the same person, it may cause changes in the spectrogram, and its robustness is even worse for voice signals recorded in different environments

In addition, some scholars have realized speech visualization through the movement changes of human vocal organs and facial expressions, and effectively analyzed the human pronunciation process. However, in terms of speech intelligibility, it is still difficult to achieve the desired effect. Except for a few experts, it is difficult for people to directly perceive speech sounds directly by observing the movement of vocal organs and changes in facial expressions

Images