Chinese and English mixed speech synthesis method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of language processing, and provides a Chinese and English mixed speech synthesis method and device, electronic equipment and a storage medium. The speech synthesis method comprises the following steps: regularizing an initial text containing a Chinese text and an English text, converting the Chinese text into pinyin with tones, and converting the English text into words; aligning the regularized text with the corresponding initial audio to obtain an aligned text with a pause rhythm; carrying out phoneme conversion on the aligned text, and respectively converting pinyin and words in the aligned text into corresponding CMU phonemes; converting each CMU phoneme into a phoneme vector, inputting the phoneme vector into an acoustic model, and obtaining a Mel spectrum feature corresponding to the initial text; and inputting the Mel spectrum features into a vocoder to synthesize a target audio. By converting Chinese and English into unified CMU phonemes, Chinese and English pronunciations are mapped to the same pronunciation space, and the synthesis effect of the Chinese and English mixed speech is effectively improved.

Description

technical field [0001] The invention relates to the technical field of language processing, in particular to a Chinese-English mixed speech synthesis method, device, electronic equipment and storage medium. Background technique [0002] Large-scale online travel service companies have a large number of users who need services. Using speech synthesis technology, combined with speech recognition, dialogue management, natural language understanding, and natural language generation to build an outbound robot, it can save labor costs and serve users efficiently. Among them, the broadcast effect of speech synthesis plays a vital role in enabling users to obtain a better service experience. [0003] With the continuous expansion of the tourism business, a large number of overseas services and overseas users need to be connected, and a large amount of mixed Chinese and English information needs to be broadcast. Based on this, a Chinese-English mixed speech synthesis model (hereinaf...

AI Technical Summary

Problems solved by technology

Although the characters sent to the model in Chinese and English are the same, the phonemes sent to the model are not uniform when the two voices pronounce the same sound, which leads to the model being affected by the speaker, and the synthesis effect is flawed

Images