LSTM-based method for inverse interpretation of independent speaker speech pronunciation

Abstract

The invention relates to the test system and a method thereof and particularly relates to an LSTM-based method for inverse interpretation of independent speaker speech pronunciation. The method is specifically characterized in that (1), firstly, four designated audio signals and synchronized trajectory signals are collected, and data at six points of an upper lip (UL), a lower lip (LL), a lower incisor (LI), a tongue tip (TP), a tongue body (TB), a tongue dorsum (TD) are collected through mounting a sensor; and (2), after the step (1), a nose bridge (RF) is selected as a reference point, and asensor is further placed at the reference point for data collection. The method is advantaged in that firstly, the speech pronunciation trajectory of speakers not appearing in a training set is predicted; secondly, input characteristics are changed, better and more appropriate acoustic characteristics are selected as network input, and RMSE and correlation coefficients are improved; and thirdly,discontinuous and unsmooth characteristics of trajectory collection are overcome.

Description

technical field [0001] The present invention relates to a test system and a method thereof, more specifically to a method for inversely solving independent speaker voice pronunciation based on LSTM. Background technique [0002] The inverse solution of speech pronunciation is to obtain the quasi-solution model by collecting the trajectory data of the pronunciation organs and synchronous audio, and training the neural network model, trying to infer the position of the vocal tract pronunciation device from the sound speech signal. The system is able to predict the position of the vocalizer from the acoustic signal. The system can be applied in the following aspects: in speech recognition, pronunciation information can improve the performance of the recognition system; in speech synthesis, it can improve the quality of speech and modify the characteristics of synthesized speech; in character animation, it can be used to automate movies or videos Facial animation for virtual ch...

AI Technical Summary

Problems solved by technology

It is not only time-consuming to collect, but also cannot be practically applied

[0005] Second, the feature selection effect is not good enough, the root mean square error (Root mean-squared error, RMSE) is about 2-5mm and the correlation coefficient r is about 0.7, and it is predicting the trajectory of known speakers, not unknown speakers Prediction of people's trajectories

and there is still a bias in predicting pronunciation trajectories

[0006] Third, the network does not have timing

Long-term large-scale data collection will also have unstable data fluctuation ranges

Images