Model training method, method for synthesizing speaking expression and related device

Abstract

The embodiment of the invention discloses a model training method for synthesizing speaking expressions. Expression characteristics, acoustic characteristics and text characteristics are obtained according to videos containing face action expressions of speakers and corresponding voices. Because the acoustic feature and the text feature are obtained according to the same video, the time interval and the duration of the pronunciation element identified by the text feature are determined according to the acoustic feature. A first corresponding relation is determined according to the time interval and duration of the pronunciation element identified by the text feature and the expression feature, and an expression model is trained according to the first corresponding relation. The expressionmodel can determine different sub-expression characteristics for the same pronunciation element with different durations in the text characteristics; the change patterns of the speaking expressions are added, the speaking expressions generated according to the target expression characteristics is determined by the expression model. The speaking expressions have different change patterns for the same pronunciation element, and therefore the situation that the speaking expressions are excessively unnatural in change is improved to a certain degree.

Description

Technical field [0001] This application relates to the field of data processing, in particular to a model training method for synthesizing speech expressions, a method for synthesizing speech expressions, and related devices. Background technique [0002] With the development of computer technology, human-computer interaction has become more common, but it is mostly pure voice interaction. For example, the interactive device can determine the reply content according to the text or voice input by the user, and play a virtual sound synthesized according to the reply content. [0003] The user immersion brought by this type of human-computer interaction is difficult to meet the current user interaction needs. In order to improve user immersion, virtual objects with the ability to change expressions, such as changing mouth shapes, emerge as interactive objects with the user. This kind of virtual objects can be avatars such as cartoons and virtual people. When interacting with users, in...

AI Technical Summary

Problems solved by technology

[0004] At present, what kind of expression this kind of virtual object makes is mainly determined according to the currently played pronunciation elements. As a result, when the virtual voice is played, the expression change style of the virtual object is limited, and the expression change is too unnatural. The reality is not good, and it is difficult to improve the user's immersion

Images