Sound replication method and related apparatus

Abstract

The application discloses a sound replication method and related device, and relates to the technical field of speech synthesis. In one aspect, a hybrid expert sound replication model is adopted, each expert sub-model corresponds to only one acoustic category, and all the expert sub-models adopt a lightweight structure. Therefore, the end-side device only needs to activate a single expert sub-model matched with target acoustic category information for reasoning, which greatly reduces the model parameter quantity and computing cost, and thus successfully runs on the low-resource end-side device. In another aspect, the target speaker personalized acoustic features pre-delivered by the cloud and locally stored are directly used in the replication process. The features accurately depict the details of the speaker's timbre, accent, prosody, etc., and make up for the insufficient expression ability of the lightweight model. Therefore, the scheme can generate a highly similar replicated voice to the target speaker on the low-resource end-side device without relying on real-time calculation of the cloud large model, and takes into account the feasibility of end-side deployment and the high fidelity of the synthesized sound quality.

Images