Audio labeling method, speech synthesis model training method, device, and medium
By acquiring the speech data of the target speaker for word-level recognition and modifying the pre-annotated file to train the speech synthesis model, the problem of poor speech synthesis effect is solved, achieving more efficient speech synthesis effect and reduced cost.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- AISPEECH CO LTD
- Filing Date
- 2022-12-29
- Publication Date
- 2026-05-29
AI Technical Summary
Existing speech synthesis models often suffer from poor performance during training because the annotation files obtained from text sequence annotations are inconsistent with the speaker's speech characteristics.
By acquiring the speech data of the target speaker, word-level recognition is performed to obtain speech annotation information. Based on this information, the pre-annotation file is modified to match the speech data in order to train the speech synthesis model.
It improves the speech synthesis effect of the speech synthesis model, solves the problem of differences in results among different annotators, reduces annotation costs, and ensures that the annotation files are completely consistent with the speech data.
Smart Images
Figure CN116013250B_ABST
Abstract
Description
Technical Field
[0001] This application relates to an audio annotation method, a training method for a speech synthesis model, an apparatus, and a medium, belonging to the field of computer technology. Background Technology
[0002] Speech synthesis technology refers to the technology of converting text information generated by a computer or input from an external source into fluent speech data output that users can understand. Generally, speech synthesis technology is based on a speech synthesis model, which typically includes a text model and an acoustic model. During speech synthesis, the text model performs text segmentation, text normalization (TN), word segmentation, part-of-speech (PoS) tagging, prosody prediction, and grapheme-to-phoneme (G2P) conversion on the text sequence, transforming the text into a corresponding phoneme sequence. This sequence is then fed into the acoustic model to obtain an acoustic sequence corresponding to the text sequence. Finally, a vocoder converts the acoustic sequence into the final audio.
[0003] Before using a speech synthesis model, it needs to be trained. The text model is trained using a text sequence and its corresponding annotation file. The annotation file includes the part-of-speech tag, prosody, and phonemes of the text sequence.
[0004] However, the audio of the same text sequence spoken by different people may have different personal characteristics, and the annotation file obtained based on the text sequence annotation may not be consistent with the speaker's speaking characteristics. This will lead to the problem that the speech synthesis model trained in the end has poor speech synthesis effect. Summary of the Invention
[0005] This application provides an audio annotation method, a training method for a speech synthesis model, an apparatus, and a medium, which can solve the problem of poor speech synthesis performance in speech synthesis models trained based on annotation files obtained from text sequence annotation. This application provides the following technical solutions:
[0006] In a first aspect, an audio annotation method for speech synthesis is provided, the method comprising:
[0007] Obtain pre-annotated files of text samples, which are used as input to the speech synthesis model during training, so that the speech synthesis model can generate audio prediction results corresponding to the text samples;
[0008] Obtain the speech data corresponding to the text sample spoken by the target speaker;
[0009] The speech data is subjected to character-level recognition to obtain the speech annotation information corresponding to the speech data;
[0010] The pre-annotated file is modified based on the speech annotation information to obtain an annotation file consistent with the speech data, and the speech synthesis model is trained using the annotation file.
[0011] Optionally, the pre-annotated file includes phoneme information and prosodic information of the text sample;
[0012] Accordingly, the step of performing character-level recognition on the speech data to obtain the speech annotation information corresponding to the speech data includes:
[0013] The phoneme sequence of each character in the speech data and the pause duration between adjacent characters are identified to obtain the speech prosody of the target speaker;
[0014] Accordingly, modifying the pre-annotated file based on the speech annotation information to obtain an annotation file consistent with the speech data includes:
[0015] The phoneme sequence and the phoneme information are fused to modify data in the phoneme information that differs from the phoneme sequence; the speech prosody and the prosody information are fused to modify data in the prosody information that differs from the speech prosody.
[0016] Optionally, the step of identifying the phoneme sequence of each character in the speech data and the pause duration between adjacent characters includes:
[0017] The phoneme sequence was identified using a pre-trained Automatic Speech Recognition (ASR) model;
[0018] The start and end times of each character are determined based on forced alignment technology;
[0019] The pause duration of the current character is determined based on the end time of the current character and the start time of the next character.
[0020] Optionally, the forced alignment technique is the forced alignment technique in the ASR model, or the forced alignment technique based on the Gaussian Mixture Model (GMM) model.
[0021] Optionally, the step of performing character-level recognition on the speech data to obtain the speech annotation information corresponding to the speech data includes:
[0022] Determine the signal-to-noise ratio of the speech data;
[0023] When the signal-to-noise ratio meets the preset requirements, the speech data is subjected to word-level recognition to obtain the speech annotation information corresponding to the speech data.
[0024] Optionally, the step of obtaining the pre-annotated file of the text sample includes:
[0025] The phoneme sequence of the text sample is determined to obtain the phoneme information of the text sample;
[0026] The text sample is segmented into words to determine the part-of-speech tag for each word.
[0027] The text sample is input into a pre-trained prosodic prediction model to obtain the prosodic information of the text sample.
[0028] Optionally, after modifying the pre-annotated file based on the speech annotation information to obtain an annotation file consistent with the speech data, the method further includes:
[0029] Display the annotation file;
[0030] Upon receiving a modification operation applied to the annotation file, the annotation file is modified based on the modification operation;
[0031] Save the modified annotation file.
[0032] Secondly, a training method for a speech synthesis model is provided, the method comprising:
[0033] The process involves obtaining a text sample, speech data corresponding to the text sample emitted by the target speaker, and an annotation file for the speech data, wherein the annotation file is obtained based on the aforementioned audio annotation method for speech synthesis.
[0034] The text sample is input into the text model in the speech synthesis model to obtain the phoneme prediction sequence and prosody prediction result output by the text model;
[0035] The phoneme prediction sequence and the prosody prediction result are input into the acoustic model and vocoder in the speech synthesis model to obtain the audio synthesis result.
[0036] The text model is trained based on the differences between the phoneme prediction sequence and the phoneme annotation results in the annotation file, and the differences between the prosody prediction results and the prosody annotation results in the annotation file.
[0037] The acoustic model and the vocoder are trained based on the difference between the audio synthesis result and the speech data.
[0038] Thirdly, an electronic device is provided, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement the method provided in any of the preceding aspects.
[0039] Fourthly, a computer-readable storage medium is provided, wherein a program is stored therein, which, when executed by a processor, is used to implement the method provided in any of the preceding aspects.
[0040] The beneficial effects of this application include at least the following: obtaining pre-annotated files of text samples, which are used as input to the speech synthesis model during training, so that the speech synthesis model can generate audio prediction results corresponding to the text samples; obtaining speech data corresponding to the text samples spoken by the target speaker; performing character-level recognition on the speech data to obtain speech annotation information corresponding to the speech data; modifying the pre-annotated file based on the speech annotation information to obtain an annotation file consistent with the speech data, so as to use the annotation file to train the speech synthesis model; solving the problem of poor speech synthesis effect of the speech synthesis model trained based on the annotation file obtained from text sequence annotation; since the pre-annotated file can be modified using the speech annotation information of the target speaker, the annotation file can have the speech characteristics of the target speaker, thus improving the speech synthesis effect. At the same time, it can also solve the problem of differences in annotation results from different annotators and differentiate annotation files at different prosodic levels.
[0041] Meanwhile, the automatic annotation method provided in this embodiment does not require manual intervention, which can reduce annotation costs.
[0042] In addition, by identifying the phoneme sequence, start time, and end time of each character, and determining the pause duration based on the start and end times to obtain the prosody of each character, it can be ensured that the features of each character match the speech data of the target speaker, thereby ensuring that the annotation file is completely consistent with the speech data.
[0043] In addition, by performing character-level recognition on the speech data when the signal-to-noise ratio meets the preset requirements, the speech annotation information corresponding to the speech data can be obtained, which can avoid the problem of noise affecting the speech synthesis effect and improve the speech synthesis effect of the speech synthesis model.
[0044] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description
[0045] Figure 1 This is a flowchart of an audio annotation method for speech synthesis provided in one embodiment of this application;
[0046] Figure 2 This is a flowchart of an audio annotation method for speech synthesis provided in another embodiment of this application;
[0047] Figure 3 This is a schematic diagram comparing the speech synthesis effect based on manual annotation and the speech synthesis effect based on automatic annotation, according to one embodiment of this application.
[0048] Figure 4 This is a block diagram of an electronic device provided in one embodiment of this application. Detailed Implementation
[0049] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.
[0050] Generally, the same text spoken by different people may have strong personal characteristics, and may sound completely different to the same user. In order for speech synthesis models to have this one-to-many mapping capability, when annotating the text, it is necessary to annotate the content that is completely consistent with the audio. For example, the annotated text's pinyin, prosody, and parts of speech should be completely consistent with the pinyin, prosody, and parts of speech of the audio.
[0051] For the purposes mentioned above, traditional annotation methods in the field of speech synthesis are generally manual. However, manual annotation has at least the following problems:
[0052] 1. Long annotation cycle; 2. High experience requirements for annotators, requiring professionals with expertise in linguistics to perform annotations; 3. Difficulty in standardizing annotation results, as different annotators have different understandings of linguistics, leading to discrepancies in annotation results and making it difficult to standardize; at the same time, inaccurate control of prosodic levels can result in significant differences in the annotation results for the same prosodic duration in different audio recordings; 4. High annotation costs, resulting in high annotation expenses.
[0053] Based on the above problems, some current training methods for speech synthesis models can automatically annotate text samples. For example, the invention patent with publication number CN 111968619 A, entitled "Method and Apparatus for Controlling the Pronunciation of Speech Synthesis," discloses the following annotation method, which specifically includes: performing rule-based processing on the text to be synthesized, filtering out illegal characters, performing word segmentation and part-of-speech tagging on legal input, inputting the extracted comprehensive linguistic features into a prosodic prediction model to obtain pause level annotations, and converting Chinese characters into pinyin tags.
[0054] However, while the above method can achieve automatic annotation, it annotates the synthesized text, and the resulting annotation information cannot guarantee a match with the speech characteristics of the target speaker expected by the user.
[0055] Based on this, this application provides an audio annotation method for speech synthesis, which obtains speech annotation information by recognizing the speech data of the target speaker; using the speech annotation information to modify the pre-annotated file of the text sample can make the final annotation file conform to the speaking characteristics of the target speaker, thereby improving the speech synthesis effect of the speech synthesis model.
[0056] The audio annotation method for speech synthesis provided in this application is described below. Optionally, this application uses the audio annotation method for speech synthesis provided in various embodiments in an electronic device as an example for illustration. The electronic device is a terminal or a server. The terminal can be a mobile phone, computer, tablet computer, or other device with computing capabilities. This embodiment does not limit the type of electronic device.
[0057] Figure 1 This is a flowchart of an audio annotation method for speech synthesis provided in one embodiment of this application. The method includes at least the following steps:
[0058] Step 101: Obtain the pre-annotated file of the text sample. The text sample is used as input to the speech synthesis model when training the speech synthesis model, so that the speech synthesis model can generate the audio prediction result corresponding to the text sample.
[0059] In one example, obtaining a pre-annotated file of a text sample includes: determining the phoneme sequence of the text sample to obtain the phoneme information of the text sample; performing word segmentation on the text sample to determine the part-of-speech of each word; and inputting the text sample into a pre-trained prosodic prediction model to obtain the prosodic information of the text sample.
[0060] Therefore, pre-annotated files should at least include phoneme and prosodic information of the text samples. Pre-annotated files may also include other annotation information of the text samples required during the training of the speech synthesis model, such as part-of-speech tags.
[0061] Optionally, the electronic device can determine the phoneme sequence of the text sample through regularization, or based on the semantic, syntactic, or other features of the text sample. This embodiment does not limit the method of determining the phoneme sequence. The prosody prediction model is based on a neural network.
[0062] The pre-annotated file for the same text sample is stored in the electronic device in advance. For different target speakers, the pre-annotated file corresponding to the same text sample can be reused.
[0063] Step 102: Obtain the speech data corresponding to the text sample issued by the target speaker.
[0064] The target speaker can be one or at least two. The voice data can be emitted by the target speaker according to the text sample, or the text sample can be obtained by recognizing the voice data emitted by the target speaker. This embodiment does not limit the acquisition method of the text sample and the corresponding voice data.
[0065] Step 103: Perform character-level recognition on the speech data to obtain the speech annotation information corresponding to the speech data.
[0066] Specifically, the phoneme sequence of each character in the speech data and the pause duration between adjacent characters are identified to obtain the speech prosody of the target speaker.
[0067] In one example, identifying the phoneme sequence of each word in speech data and the pause duration between adjacent words includes: identifying the phoneme sequence using a pre-trained Automatic Speech Recognition (ASR) model; determining the start and end times of each word based on an alignment technique; and determining the pause duration of the current word based on the end time of the current word and the start time of the next word.
[0068] In traditional audio recognition methods, electronic devices typically first acquire the start and end times of the audio to obtain the target audio within the time period from the start to the end time. Then, feature extraction is performed on the target audio, and the extracted features are input into an audio recognition model or classification model to obtain the text label corresponding to the target audio. The text label is then compared with the target text. Finally, the target audio that meets the preset conditions is saved.
[0069] In the above method, the target text and target audio are considered to be a complete match when the text tag and target text meet a certain matching ratio (i.e., a preset condition). This may lead to different matching thresholds being needed to label the text of the target audio in different datasets. On the other hand, the target audio obtained by the above method may also contain parts of a sentence that do not match the target text.
[0070] In this embodiment, the pre-trained ASR model is used to identify the phoneme sequence of each character, which can avoid the problem of mismatch in some parts of a sentence.
[0071] In one example, speech prosody is represented by the pause duration and its corresponding pause level. For instance, there are four pause levels, from shortest to longest: first duration, second duration, third duration, and fourth duration (time unit: milliseconds). The pause duration is compared with the duration corresponding to each pause level to obtain the corresponding pause level.
[0072] Optionally, the forced alignment technique is a forced alignment technique in the ASR model, such as the Connectionist Temporal Classification (CTC) based on neural networks in the ASR model that identifies the start and end times of each word. Alternatively, the forced alignment technique is a forced alignment technique based on the Gaussian Mixed Model (GMM) model.
[0073] Optionally, performing character-level recognition on the speech data to obtain speech annotation information corresponding to the speech data further includes: determining the signal-to-noise ratio of the speech data; and, if the signal-to-noise ratio meets preset requirements, performing character-level recognition on the speech data to obtain speech annotation information corresponding to the speech data.
[0074] Because speech synthesis requires high audio quality, noisy audio can severely impact the final synthesis result. Therefore, by controlling the signal-to-noise ratio (SNR) and discarding speech data that does not meet the SNR requirements, the quality of speech synthesis can be guaranteed.
[0075] Step 104: Modify the pre-annotated file based on the speech annotation information to obtain an annotation file consistent with the speech data, so as to use the annotation file to train the speech synthesis model.
[0076] In one example, the pre-annotated file is modified based on the speech annotation information to obtain an annotation file consistent with the speech data. This includes: fusing the phoneme sequence and phoneme information to modify data in the phoneme information that differs from the phoneme sequence; and fusing the speech prosody and prosodic information to modify data in the prosodic information that differs from the speech prosody.
[0077] Optionally, to improve the accuracy of the annotation file, after modifying the pre-annotated file based on the speech annotation information to obtain an annotation file consistent with the speech data, the process further includes: displaying the annotation file; modifying the annotation file based on the modification operation when a modification operation is received; and saving the modified annotation file. At this point, the annotation file can be reviewed and corrected to further improve its accuracy.
[0078] To better understand the audio annotation method provided in this application, an example is given below. (Reference) Figure 2The method includes at least the following steps:
[0079] Step 21: Obtain the speech data corresponding to the text sample delivered by the target speaker;
[0080] Step 22: Detect the signal-to-noise ratio of the speech data;
[0081] Step 23: Determine whether the signal-to-noise ratio meets the preset requirements. If it does, proceed to step 24. If it does not, discard the voice data.
[0082] Step 24: Perform ASR recognition on the speech data to obtain the phoneme sequence and timestamp of each character;
[0083] The timestamp includes the start and end times of each word;
[0084] Step 25: Fuse the phoneme information and phoneme sequence in the pre-labeled file of the text sample, and fuse the prosodic information and the speech prosodic indicated by the timestamp in the pre-labeled file;
[0085] Step 26: Generate a new annotation file; the process ends.
[0086] In summary, the audio annotation method for speech synthesis provided in this embodiment obtains a pre-annotated file of text samples, which is used as input to the speech synthesis model during training, allowing the model to generate audio prediction results corresponding to the text samples. It then acquires speech data corresponding to the text samples spoken by the target speaker; performs character-level recognition on the speech data to obtain speech annotation information; and modifies the pre-annotated file based on the speech annotation information to obtain an annotation file consistent with the speech data, which is then used to train the speech synthesis model. This method solves the problem of poor speech synthesis performance in speech synthesis models trained based on annotation files obtained from text sequence annotation. Since the pre-annotated file can be modified using the target speaker's speech annotation information, the annotation file can possess the speech characteristics of the target speaker, improving the speech synthesis effect. Furthermore, it addresses the issue of differences in annotation results from different annotators and differentiates annotation files at different prosodic levels. (Reference) Figure 3 The left side shows the audio data synthesized from manually annotated files, while the right side shows the audio data synthesized from annotated files obtained using the automatic annotation method provided in this embodiment. Figure 3 It can be seen that in long speech, the prosodic differences of automatic annotation are more obvious. From the perspective of hearing, the synthesized speech with automatic annotation is closer to the pronunciation of real people. At the same time, in the mean opinion score (MOS) evaluation of speech synthesis, the effect of automatic annotation is greater than or equal to that of manual annotation.
[0087] Meanwhile, the automatic annotation method provided in this embodiment does not require manual intervention, which can reduce annotation costs.
[0088] In addition, by identifying the phoneme sequence, start time, and end time of each character, and determining the pause duration based on the start and end times to obtain the prosody of each character, it can be ensured that the features of each character match the speech data of the target speaker, thereby ensuring that the annotation file is completely consistent with the speech data.
[0089] In addition, by performing character-level recognition on the speech data when the signal-to-noise ratio meets the preset requirements, the speech annotation information corresponding to the speech data can be obtained, which can avoid the problem of noise affecting the speech synthesis effect and improve the speech synthesis effect of the speech synthesis model.
[0090] Optionally, based on the above embodiments, this embodiment also provides a training method for a speech synthesis model. The method includes: acquiring text samples, speech data corresponding to the text samples emitted by a target speaker, and a labeling file for the speech data, wherein the labeling file is obtained based on the audio labeling method for speech synthesis described in the above embodiments; inputting the text samples into a text model in the speech synthesis model to obtain a phoneme prediction sequence and prosody prediction result output by the text model; inputting the phoneme prediction sequence and prosody prediction result into an acoustic model and a vocoder in the speech synthesis model to obtain an audio synthesis result; training the text model based on the differences between the phoneme prediction sequence and the phoneme labeling results in the labeling file, and the differences between the prosody prediction result and the prosody labeling results in the labeling file; and training the acoustic model and vocoder based on the differences between the audio synthesis result and the speech data.
[0091] In this embodiment, training the speech synthesis model with the annotation file obtained in the above embodiment enables the speech synthesis model to learn the speaking features of the target speaker, thereby improving the speech synthesis effect.
[0092] Figure 4 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 401 and a memory 402.
[0093] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.
[0094] Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 402 is used to store at least one instruction, which is executed by processor 401 to implement the audio annotation or speech synthesis model training method provided in the method embodiments of this application.
[0095] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 401, memory 402, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuitry, a touch display screen, audio circuitry, and a power supply.
[0096] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.
[0097] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the audio annotation or training method for speech synthesis in the above-described method embodiments.
[0098] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the audio annotation or training method for speech synthesis of the above-described method embodiments.
[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0100] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. An audio annotation method for speech synthesis, characterized in that, The method includes: Obtain a pre-annotated file of a text sample, which is used as input to the speech synthesis model during training so that the speech synthesis model can generate an audio prediction result corresponding to the text sample. The pre-annotated file includes the phoneme information and prosodic information of the text sample. Obtain the speech data corresponding to the text sample spoken by the target speaker; The speech data is subjected to character-level recognition to obtain the speech annotation information corresponding to the speech data; Based on the speech annotation information, the pre-annotated file is modified to obtain an annotation file consistent with the speech data, so as to use the annotation file to train the speech synthesis model; The step of performing character-level recognition on the speech data to obtain the speech annotation information corresponding to the speech data includes: The phoneme sequence of each character in the speech data and the pause duration between adjacent characters are identified to obtain the speech prosody of the target speaker.
2. The method according to claim 1, characterized in that, The step of modifying the pre-annotated file based on the speech annotation information to obtain an annotation file consistent with the speech data includes: The phoneme sequence and the phoneme information are fused to modify data in the phoneme information that differs from the phoneme sequence; the speech prosody and the prosody information are fused to modify data in the prosody information that differs from the speech prosody.
3. The method according to claim 2, characterized in that, The process of identifying the phoneme sequence of each character in the speech data and the pause duration between adjacent characters includes: The phoneme sequence was identified using a pre-trained Automatic Speech Recognition (ASR) model; The start and end times of each character are determined based on forced alignment technology; The pause duration of the current character is determined based on the end time of the current character and the start time of the next character.
4. The method according to claim 3, characterized in that, The forced alignment technique is either the forced alignment technique in the ASR model or the forced alignment technique based on the Gaussian Mixture Model (GMM).
5. The method according to claim 1, characterized in that, The step of performing character-level recognition on the speech data to obtain the speech annotation information corresponding to the speech data includes: Determine the signal-to-noise ratio of the speech data; When the signal-to-noise ratio meets the preset requirements, the speech data is subjected to word-level recognition to obtain the speech annotation information corresponding to the speech data.
6. The method according to claim 1, characterized in that, The pre-annotated file for obtaining text samples includes: The phoneme sequence of the text sample is determined to obtain the phoneme information of the text sample; The text sample is segmented into words to determine the part-of-speech tag for each word. The text sample is input into a pre-trained prosodic prediction model to obtain the prosodic information of the text sample.
7. The method according to any one of claims 1 to 6, characterized in that, After modifying the pre-annotated file based on the speech annotation information to obtain an annotation file consistent with the speech data, the process further includes: Display the annotation file; Upon receiving a modification operation applied to the annotation file, the annotation file is modified based on the modification operation; Save the modified annotation file.
8. A training method for a speech synthesis model, characterized in that, The method includes: The method involves acquiring a text sample, speech data corresponding to the text sample emitted by the target speaker, and an annotation file for the speech data, wherein the annotation file is obtained based on the audio annotation method for speech synthesis as described in any one of claims 1 to 7. The text sample is input into the text model in the speech synthesis model to obtain the phoneme prediction sequence and prosody prediction result output by the text model; The phoneme prediction sequence and the prosody prediction result are input into the acoustic model and vocoder in the speech synthesis model to obtain the audio synthesis result. The text model is trained based on the differences between the phoneme prediction sequence and the phoneme annotation results in the annotation file, and the differences between the prosody prediction results and the prosody annotation results in the annotation file. The acoustic model and the vocoder are trained based on the difference between the audio synthesis result and the speech data.
9. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program that is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement the method as described in any one of claims 1 to 8.