A speech synthesis method, apparatus, electronic device, and storage medium

Abstract

The present application relates to a kind of speech synthesis method, device, electronic equipment and storage medium, the method comprises: obtaining text voice pair containing tone word;Using text voice pair containing tone word to train to obtain duration model, acoustic model and neural network vocoder model;Obtain multiple sentences containing tone word, determine N template voice of each tone word according to multiple sentences containing tone word, acoustic model and neural network vocoder model;Obtain the text of the speech to be synthesized, parse the text of the speech to be synthesized to obtain phoneme sequence;According to the modified phoneme sequence of pre-set repetition rule;According to the phoneme sequence of modified, duration model, acoustic model and neural network vocoder model obtain preliminary synthesized speech;Insert template voice in the mute part of preliminary synthesized speech to obtain final synthesized speech.The present application is according to the actual speaking tone of person, processes synthesized speech, to obtain more real synthesized speech.

Description

Technical Field

[0001] This invention relates to the field of speech synthesis technology, and more specifically to a speech synthesis method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, synthesized speech is produced by using a trained model to synthesize corresponding speech based on input text. While the synthesized speech has high sound quality and naturalness, it lacks human-likeness and is noticeably different from normal human speech. Summary of the Invention

[0003] This invention provides a speech synthesis method, apparatus, electronic device, and storage medium that can solve the above-mentioned technical problems.

[0004] In a first aspect, embodiments of the present invention provide a speech synthesis method, comprising:

[0005] Obtain text-to-speech pairs containing modal particles;

[0006] The duration model, acoustic model, and neural network vocoder model were obtained by training text-to-speech pairs containing interjections.

[0007] Obtain multiple sentences containing modal particles, and determine N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model;

[0008] Obtain the text of the speech to be synthesized, and parse the text of the speech to be synthesized to obtain the phoneme sequence;

[0009] Modify the phoneme sequence according to the preset repetition rules;

[0010] Preliminary synthesized speech was obtained based on the modified phoneme sequence, duration model, acoustic model, and neural network vocoder model;

[0011] The template speech is inserted into the silence portion of the initial synthesized speech to obtain the final synthesized speech.

[0012] Furthermore, in the aforementioned speech synthesis method, a duration model is obtained by training text-to-speech pairs containing interjections, including:

[0013] The text is analyzed to obtain the phoneme sequence corresponding to each sentence;

[0014] Force alignment is performed on the phoneme sequence and speech to obtain the duration of each phoneme;

[0015] A duration model is trained by using phoneme sequences as input and phoneme durations as output.

[0016] Furthermore, in the aforementioned speech synthesis method, an acoustic model is obtained by training text-speech pairs containing interjections, including:

[0017] The text is analyzed to obtain the phoneme sequence corresponding to each sentence;

[0018] Force alignment is performed on the phoneme sequence and speech to obtain the duration of each phoneme, and acoustic parameters are extracted from the language.

[0019] The acoustic model is trained using phoneme sequences and phoneme durations as inputs and acoustic parameters as outputs.

[0020] Furthermore, in the aforementioned speech synthesis method, a neural network vocoder model is trained using text-to-speech pairs containing interjections, including:

[0021] Extracting acoustic parameters and waveforms from language;

[0022] A neural network vocoder model is trained using acoustic parameters as input and waveforms as output.

[0023] Furthermore, in the aforementioned speech synthesis method, N template speech words for each tone word are determined based on multiple sentences containing tone words, an acoustic model, and a neural network vocoder model, including:

[0024] Acoustic parameters of multiple sentences containing modal particles are obtained using an acoustic model;

[0025] The speech of multiple sentences containing interjections is obtained through a neural network vocoder;

[0026] Extract speech segments of the modal particles from the speech of multiple sentences containing modal particles;

[0027] Extract the fundamental frequency sequence from all speech segments and sample the fundamental frequency sequence into 2N points;

[0028] For each modal particle, K-Means clustering is performed to obtain N categories;

[0029] The speech corresponding to the sample closest to each class center point is saved as template speech, thus obtaining N template speech for each tone word.

[0030] Furthermore, in the aforementioned speech synthesis method, the preset repetition rules include:

[0031] Repetition with preset probability; repetition after modifying the tone with preset probability; repetition after unclear pronunciation due to pronunciation being too fast with preset probability.

[0032] Furthermore, in the aforementioned speech synthesis method, obtaining preliminary synthesized speech based on the modified phoneme sequence, the duration model, the acoustic model, and the neural network vocoder model includes:

[0033] The modified phoneme sequence is fed into the duration model to obtain the phoneme duration;

[0034] The duration of repeated phonemes is processed when pronunciation is too fast and unclear within the preset repetition rules;

[0035] The modified phoneme sequence and the processed phoneme duration are fed into the acoustic model to obtain acoustic parameters.

[0036] The acoustic parameters are fed into a neural network vocoder to obtain preliminary synthesized speech.

[0037] Secondly, embodiments of the present invention also provide a speech synthesis device, comprising:

[0038] Acquisition module: used to acquire text-to-speech pairs containing interjections;

[0039] Training module: used to train the duration model, acoustic model, and neural network vocoder model using text-to-speech pairs containing interjections;

[0040] Determining Module: Used to acquire multiple sentences containing modal particles, and determine N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model;

[0041] Parsing module: used to acquire the text of the speech to be synthesized, and parse the text of the speech to be synthesized to obtain a phoneme sequence;

[0042] Modification module: Used to modify the phoneme sequence according to preset repetition rules;

[0043] The first obtaining module is used to obtain preliminary synthesized speech based on the modified phoneme sequence, the duration model, the acoustic model, and the neural network vocoder model.

[0044] The second module is used to insert template speech into the silence portion of the initial synthesized speech to obtain the final synthesized speech.

[0045] Thirdly, embodiments of the present invention also provide an electronic device, including: a processor and a memory;

[0046] The processor executes any of the above speech synthesis methods by calling programs or instructions stored in memory.

[0047] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a program or instructions that cause a computer to execute any of the above-mentioned speech synthesis methods.

[0048] The beneficial effects of this invention are as follows: This invention acquires text-speech pairs containing interjections; uses these text-speech pairs to train a duration model, an acoustic model, and a neural network vocoder model; acquires multiple sentences containing interjections, and determines N template speech for each interjection based on these sentences, the acoustic model, and the neural network vocoder model; acquires the text of the speech to be synthesized, and parses the text to obtain a phoneme sequence; modifies the phoneme sequence according to a preset repetition rule; obtains preliminary synthesized speech based on the modified phoneme sequence, duration model, acoustic model, and neural network vocoder model; and inserts template speech into the silent parts of the preliminary synthesized speech to obtain the final synthesized speech. This invention processes the synthesized speech according to the actual tone of human speech, thereby obtaining more realistic synthesized speech. Attached Figure Description

[0049] Figure 1 A speech synthesis method provided in this embodiment of the invention Figure 1 ;

[0050] Figure 2 A speech synthesis method provided in this embodiment of the invention Figure 2 ;

[0051] Figure 3 A speech synthesis method provided in this embodiment of the invention Figure 3 ;

[0052] Figure 4 A speech synthesis method provided in this embodiment of the invention Figure 4 ;

[0053] Figure 5 A speech synthesis method provided in this embodiment of the invention Figure 5 ;

[0054] Figure 6 A speech synthesis method provided in this embodiment of the invention Figure 6 ;

[0055] Figure 7 A diagram of a speech synthesis device provided in an embodiment of the present invention;

[0056] Figure 8 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0057] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0058] Figure 1 A speech synthesis method provided in this embodiment of the invention Figure 1 .

[0059] In a first aspect, embodiments of the present invention provide a speech synthesis method, combined with Figure 1 It includes seven steps, S101 to S107:

[0060] S101: Obtain text-to-speech pairs containing interjections.

[0061] Specifically, in this embodiment of the invention, the speech pairs containing interjections can be obtained through pre-recording by a large number of users, and the interjections can be "um," "ah," "this," etc.

[0062] S102: Train the duration model, acoustic model, and neural network vocoder model using text-to-speech pairs containing interjections.

[0063] Specifically, in this embodiment of the invention, a duration model, an acoustic model, and a neural network vocoder model are trained using text-to-speech pairs containing interjections such as "um," "ah," and "this." The specific training method is described in detail below.

[0064] S103: Obtain multiple sentences containing modal particles, and determine N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model.

[0065] Specifically, in this embodiment of the invention, the sentence can be a pre-recorded sentence. The method of obtaining multiple sentences containing modal particles from the sentence and determining N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model is described in detail below.

[0066] S104: Obtain the text of the speech to be synthesized, and parse the text of the speech to be synthesized to obtain the phoneme sequence.

[0067] Specifically, in this embodiment of the invention, the text to be synthesized speech can be any piece of text, and the phoneme sequence corresponding to the text to be synthesized speech is obtained by parsing the text to be synthesized speech.

[0068] S105: Modify the phoneme sequence according to the preset repetition rules.

[0069] Specifically, in this embodiment of the invention, in order to simulate the phenomenon of repetition caused by hesitation or mispronunciation when a person is speaking, the phoneme sequence is modified according to a preset repetition rule. The preset repetition rule includes: 1) repetition with a certain probability; 2) repetition after modifying the tone with a certain probability; 3) repetition after unclear pronunciation due to speaking too fast with a certain probability.

[0070] S106: The preliminary synthesized speech is obtained based on the modified phoneme sequence, duration model, acoustic model, and neural network vocoder model.

[0071] Specifically, in this embodiment of the invention, the method for obtaining preliminary synthesized speech based on the modified phoneme sequence, duration model, acoustic model, and neural network vocoder model is described in detail below.

[0072] S107: Insert template speech into the silence part of the initial synthesized speech to obtain the final synthesized speech.

[0073] Specifically, in this embodiment of the invention, in the silence part of the speech, a template speech of an interjection is inserted with a certain probability. The type of interjection inserted and which template speech is inserted are randomly selected. The speech after inserting the template speech is the final synthesized speech.

[0074] In some embodiments, different modulation factors can be set for different interjections based on the speaker's own characteristics.

[0075] Figure 2 A speech synthesis method provided in this embodiment of the invention Figure 2 .

[0076] Furthermore, in the aforementioned speech synthesis method, a duration model is obtained by training text-to-speech pairs containing interjections, combined with... Figure 2 It includes three steps: S201 to S203.

[0077] S201: Analyze the text to obtain the phoneme sequence corresponding to each sentence;

[0078] S202: Perform force alignment on the phoneme sequence and speech to obtain the duration of each phoneme;

[0079] S203: Train a duration model using phoneme sequences as input and phoneme durations as output.

[0080] Figure 3 A speech synthesis method provided in this embodiment of the invention Figure 3 .

[0081] Furthermore, in the aforementioned speech synthesis method, an acoustic model is obtained by training text-to-speech pairs containing interjections, combined with... Figure 3It includes three steps, S301 to S303:

[0082] S301: Analyze the text to obtain the phoneme sequence corresponding to each sentence;

[0083] S302: Perform force alignment on the phoneme sequence and speech to obtain the duration of each phoneme and extract acoustic parameters from the language;

[0084] S303: Train an acoustic model using phoneme sequences and phoneme durations as inputs and acoustic parameters as outputs.

[0085] Figure 4 A speech synthesis method provided in this embodiment of the invention Figure 4 .

[0086] Furthermore, in the aforementioned speech synthesis method, a neural network vocoder model is trained using text-to-speech pairs containing interjections, including two steps S401 to S402:

[0087] S401: Extract acoustic parameters and waveforms from speech;

[0088] S402: Train a neural network vocoder model using acoustic parameters as input and waveform as output.

[0089] Figure 5 A speech synthesis method provided in this embodiment of the invention Figure 5 .

[0090] Furthermore, in the aforementioned speech synthesis method, N template speech words for each tone word are determined based on multiple sentences containing tone words, an acoustic model, and a neural network vocoder model, and then combined with... Figure 5 ,include:

[0091] S501: Use an acoustic model to obtain the acoustic parameters of multiple sentences containing modal particles;

[0092] Specifically, in this embodiment of the invention, the multiple sentences containing modal particles are sentences containing modal particles such as "um," "ah," and "this." An acoustic model is used to generate GTA (Ground Truth Aligned) for the multiple sentences containing modal particles to obtain acoustic parameters.

[0093] S502: Obtain the speech of multiple sentences containing interjections through a neural network vocoder.

[0094] Specifically, in this embodiment of the invention, multiple sentences containing interjections are fed into a neural network vocoder to obtain speech that retains the original speech rhythm and has a sound quality between synthesized speech and natural speech.

[0095] S503: Extract speech segments of modal particles from the speech of multiple sentences containing modal particles.

[0096] Specifically, in this embodiment of the invention, speech segments containing interjections such as "um," "ah," and "this" are extracted from speech that retains the original speech rhythm and has a sound quality between synthesized speech and natural speech.

[0097] S504: Extract the fundamental frequency sequence from all speech segments and sample the fundamental frequency sequence into 2N points.

[0098] Specifically, in this embodiment of the invention, the YIN algorithm is used to extract the fundamental frequency sequence from all speech segments and sample the fundamental frequency sequence into 2N points, such as 10 points.

[0099] S505: Perform K-Means clustering on each modal particle to obtain N categories.

[0100] Specifically, in this embodiment of the invention, K-Means clustering is performed according to different modal particles, and each modal particle obtains N categories, such as 5 categories.

[0101] S506: Save the speech corresponding to the sample closest to each class center point as template speech to obtain N template speech for each interjection.

[0102] Specifically, in this embodiment of the invention, the speech corresponding to the sample closest to each class center point is saved as template speech, and each interjection has N template speech, such as 5 template speech.

[0103] Furthermore, in the aforementioned speech synthesis method, the preset repetition rules include:

[0104] Repetition with preset probability; repetition after modifying the tone with preset probability; repetition after unclear pronunciation due to pronunciation being too fast with preset probability.

[0105] Figure 6 A speech synthesis method provided in this embodiment of the invention Figure 6 .

[0106] Furthermore, in the aforementioned speech synthesis method, preliminary synthesized speech is obtained based on the modified phoneme sequence, the duration model, the acoustic model, and the neural network vocoder model, combined with... Figure 6 It includes four steps, S601 to S604:

[0107] S601: Input the modified phoneme sequence into the duration model to obtain the phoneme duration;

[0108] S602: Process the duration of the repeated corresponding phoneme after unclear pronunciation due to excessively fast pronunciation within the preset repetition rules.

[0109] Specifically, in this embodiment of the invention, the duration of the repeated corresponding phoneme after unclear pronunciation due to excessively fast pronunciation within the preset repetition rules can be halved.

[0110] S603: Input the modified phoneme sequence and the processed phoneme duration into the acoustic model to obtain acoustic parameters;

[0111] S604: The acoustic parameters are fed into the neural network vocoder to obtain the initial synthesized speech.

[0112] Figure 7 A diagram of a speech synthesis device provided in an embodiment of the present invention.

[0113] Secondly, embodiments of the present invention also provide a speech synthesis device, combined with Figure 7 ,include:

[0114] Acquisition module 701: Used to acquire text-to-speech pairs containing interjections.

[0115] Specifically, in this embodiment of the invention, the acquisition module 701 acquires text-to-speech pairs containing interjections. The text-to-speech pairs containing interjections can be obtained through pre-recording by a large number of users. Interjections can be "um," "ah," "this," etc.

[0116] Training module 702: Used to train the duration model, acoustic model, and neural network vocoder model using text-to-speech pairs containing interjections.

[0117] Specifically, in this embodiment of the invention, the training module 702 uses text speech pairs containing interjections such as "um," "ah," and "this" to train a duration model, an acoustic model, and a neural network vocoder model.

[0118] Determining module 703: used to acquire multiple sentences containing modal particles, and determine N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model.

[0119] Specifically, in this embodiment of the invention, the sentence can be a pre-recorded sentence. Multiple sentences containing modal particles are obtained from the sentence. The determining module 703 determines N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model.

[0120] Parsing module 704: used to obtain the text of the speech to be synthesized, and to parse the text of the speech to be synthesized to obtain a phoneme sequence.

[0121] Specifically, in this embodiment of the invention, the text to be synthesized speech can be any piece of text, and the parsing module 704 parses the text to be synthesized speech to obtain the phoneme sequence corresponding to the text to be synthesized speech.

[0122] Modification module 705: Used to modify the phoneme sequence according to preset repetition rules.

[0123] Specifically, in this embodiment of the invention, in order to simulate the phenomenon of repetition caused by hesitation or mispronunciation when a person is speaking, the modification module 705 modifies the phoneme sequence according to preset repetition rules. The preset repetition rules include: 1) repetition with a certain probability; 2) repetition after modifying the tone with a certain probability; 3) repetition after unclear pronunciation due to speaking too fast with a certain probability.

[0124] First module 706: used to obtain preliminary synthesized speech based on the modified phoneme sequence, the duration model, the acoustic model and the neural network vocoder model.

[0125] Specifically, in this embodiment of the invention, the method by which the first obtaining module 706 obtains preliminary synthesized speech based on the modified phoneme sequence, duration model, acoustic model, and neural network vocoder model is described in detail below.

[0126] The second module 707 is used to insert template speech into the silence portion of the initial synthesized speech to obtain the final synthesized speech.

[0127] Specifically, in this embodiment of the invention, the second obtaining module 707 is used to insert template speech into the silent part of the preliminary synthesized speech to obtain the final synthesized speech. In the silent part of the speech, template speech of interjections is inserted with a certain probability. The type of interjection and which template speech is inserted are randomly selected. The speech after inserting the template speech is the final synthesized speech.

[0128] In some embodiments, different modulation factors can be set for different interjections based on the speaker's own characteristics.

[0129] Thirdly, embodiments of the present invention also provide an electronic device, including: a processor and a memory;

[0130] The processor executes any of the above speech synthesis methods by calling programs or instructions stored in memory.

[0131] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a program or instructions that cause a computer to execute any of the above-mentioned speech synthesis methods.

[0132] Figure 8 This is a schematic block diagram of an electronic device provided in an embodiment of this disclosure.

[0133] like Figure 8 As shown, the electronic device includes at least one processor 801, at least one memory 802, and at least one communication interface 803. The various components in the electronic device are coupled together via a bus system 804. The communication interface 803 is used for information transmission with external devices. It is understood that the bus system 804 is used to implement communication between these components. In addition to a data bus, the bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 8 The general labeled all buses as Bus System 804.

[0134] It is understood that the memory 802 in this embodiment may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0135] In some implementations, memory 802 stores elements such as executable units or data structures, or subsets thereof, or extended sets thereof: operating systems and applications.

[0136] The operating system includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application programs include various applications, such as media players and browsers, used to implement various application functions. A program implementing any method in the speech synthesis method provided in this embodiment of the invention can be included in the application programs.

[0137] In this embodiment of the invention, the processor 801 executes the steps of various embodiments of the speech synthesis method provided in this embodiment of the invention by calling the program or instructions stored in the memory 802, specifically, the program or instructions stored in the application program.

[0138] Obtain text-to-speech pairs containing modal particles;

[0139] The duration model, acoustic model, and neural network vocoder model were obtained by training text-to-speech pairs containing interjections.

[0140] Obtain multiple sentences containing modal particles, and determine N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model;

[0141] Obtain the text of the speech to be synthesized, and parse the text of the speech to be synthesized to obtain the phoneme sequence;

[0142] Modify the phoneme sequence according to the preset repetition rules;

[0143] Preliminary synthesized speech was obtained based on the modified phoneme sequence, duration model, acoustic model, and neural network vocoder model;

[0144] The template speech is inserted into the silence portion of the initial synthesized speech to obtain the final synthesized speech.

[0145] Any method in the speech synthesis method provided in this embodiment of the invention can be applied to, or implemented by, the processor 801. The processor 801 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuits in the hardware of the processor 801 or by instructions in software form. The processor 801 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor.

[0146] The steps of any method in the speech synthesis method provided in this embodiment of the invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software units in the decoding processor. The software units can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 802, and processor 801 reads the information in memory 802 and combines it with its hardware to complete the steps of the method.

[0147] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention and form different embodiments.

[0148] Those skilled in the art will understand that the descriptions of the various embodiments have different focuses, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0149] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention. All such modifications and variations fall within the scope defined by the appended claims. The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0150] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A speech synthesis method, characterized in that, include: Obtain text-to-speech pairs containing modal particles; The duration model, acoustic model, and neural network vocoder model are trained using the text-speech pairs containing interjections. Obtain multiple sentences containing modal particles, and determine N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model; Obtain the text of the speech to be synthesized, and parse the text of the speech to be synthesized to obtain a phoneme sequence; Modify the phoneme sequence according to the preset repetition rules; The preliminary synthesized speech is obtained based on the modified phoneme sequence, the duration model, the acoustic model, and the neural network vocoder model; The template speech is inserted into the silence portion of the initial synthesized speech to obtain the final synthesized speech.

2. The speech synthesis method according to claim 1, characterized in that, The process of training a duration model using the text-speech pair containing interjections includes: The text is analyzed to obtain the phoneme sequence corresponding to each sentence; Force alignment is performed on the phoneme sequence and speech to obtain the duration of each phoneme; The duration model is trained by using the phoneme sequence as input and the duration of the phonemes as output.

3. The speech synthesis method according to claim 1, characterized in that, The process of training an acoustic model using the text-speech pair containing interjections includes: The text is analyzed to obtain the phoneme sequence corresponding to each sentence; Force alignment is performed on the phoneme sequence and speech to obtain the duration of each phoneme, and acoustic parameters are extracted from the language. The acoustic model is trained using the phoneme sequence and the phoneme duration as inputs and the acoustic parameters as outputs.

4. The speech synthesis method according to claim 1, characterized in that, The neural network vocoder model is trained using the text-speech pairs containing interjections. Extracting acoustic parameters and waveforms from language; The neural network vocoder model is trained using the acoustic parameters as input and the waveform as output.

5. The speech synthesis method according to claim 1, characterized in that, The step of determining N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model includes: The acoustic model is used to obtain the acoustic parameters of multiple sentences containing modal particles; The neural network vocoder obtains the speech of multiple sentences containing interjections; Extract speech segments of the modal particles from the speech of the multiple sentences containing modal particles; Extract the fundamental frequency sequence from all speech segments and sample the fundamental frequency sequence into 2N points; For each modal particle, K-Means clustering is performed to obtain N categories; The speech corresponding to the sample closest to each class center point is saved as template speech, thus obtaining N template speech for each tone word.

6. The speech synthesis method according to claim 1, characterized in that, The preset repetition rules include: Repetition with preset probability; repetition after modifying the tone with preset probability; repetition after unclear pronunciation due to pronunciation being too fast with preset probability.

7. The speech synthesis method according to claim 1, characterized in that, The process of obtaining preliminary synthesized speech based on the modified phoneme sequence, the duration model, the acoustic model, and the neural network vocoder model includes: The modified phoneme sequence is fed into the duration model to obtain the phoneme duration; The duration of repeated phonemes is processed when pronunciation is too fast and unclear within the preset repetition rules; The modified phoneme sequence and the processed phoneme duration are fed into the acoustic model to obtain acoustic parameters. The acoustic parameters are fed into the neural network vocoder to obtain preliminary synthesized speech.

8. A speech synthesis device, characterized in that, include: Acquisition module: used to acquire text-to-speech pairs containing interjections; Training module: used to train the duration model, acoustic model, and neural network vocoder model using the text-speech pairs containing interjections; Determination module: used to acquire multiple sentences containing modal particles, and determine N template speech for each modal particle based on the multiple sentences containing modal particles, the acoustic model, and the neural network vocoder model; Parsing module: used to acquire the text of the speech to be synthesized, and parse the text of the speech to be synthesized to obtain a phoneme sequence; Modification module: Used to modify the phoneme sequence according to preset repetition rules; The first obtaining module is used to obtain preliminary synthesized speech based on the modified phoneme sequence, the duration model, the acoustic model, and the neural network vocoder model. The second obtaining module is used to insert the template speech into the silence portion of the preliminary synthesized speech to obtain the final synthesized speech.

9. An electronic device, characterized in that, include: Processor and memory; The processor executes a speech synthesis method as described in any one of claims 1 to 7 by calling programs or instructions stored in the memory.

10. A computer-readable storage medium, characterized in that, A non-transitory computer-readable storage medium stores a program or instructions that cause a computer to perform a speech synthesis method as described in any one of claims 1 to 7.

Images